LLM Pricing Page Tutorial

This document explains how the LLM pricing comparison page works and how to update it.

Overview

The LLM pricing page (/llm-apis/) displays and compares pricing for AI language models from various providers. The page dynamically loads model data and organizes it by provider, with customizable ordering and filtering.

Key Files

/assets/js/llm-apis-fetcher.js - Main controller that fetches data from GitHub and manages display logic, ordering, and filtering
/assets/js/llm-apis-data.js - Local reference copy of model pricing data (NOT loaded by the page - see Data Architecture below)
/assets/js/llm-apis-logos.js - Provider logo definitions
/_pages/llm-apis.md - The page template
/update_llm_data.sh - Script to update local reference data

Data Architecture

How Data is Loaded

The pricing page uses a live fetch architecture:

Primary Data Source: The page fetches data directly from LiteLLM’s GitHub repository:
```
https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json
```
This ensures the page always shows the latest models and pricing.
Local Reference Copy: The llm-apis-data.js file serves as:
- A local reference to track changes over time
- A way to see new models via git diff assets/js/llm-apis-data.js
- NOTE: This file is NOT loaded by default - it’s for development use only
Optional Fallback: The fetcher now checks for llmApiData variable if GitHub fetch fails:
- To enable fallback: Add <script src="/assets/js/llm-apis-data.js"></script> before fetcher script
- Without this script tag, the page shows an error message on fetch failure
- This is intentionally optional to prefer live data

Updating the Local Reference

To update the local reference data:

./update_llm_data.sh

To see what models were added/changed:

git diff assets/js/llm-apis-data.js

This is useful for:

Tracking when new models like GPT-5, Claude Opus 4.1, etc. are added
Comparing pricing changes over time
Having a backup of the data structure

How Model Display Works

1. Model Priority Ordering

Models are displayed in a specific order defined in modelsPriorityOrder:

const modelsPriorityOrder = {
  "openai": [
    'o3-pro',
    'o3',
    'o4-mini',
    'codex-mini-latest',
    'o4-mini-deep-research',
    // ... more models
  ],
  "anthropic": [
    'claude-4-opus-20250514',
    'claude-4-sonnet-20250514',
    // ... more models
  ],
  // ... other providers
};

To update model order: Simply rearrange the model names in the array for the relevant provider.

2. Model Synonyms

Many models have multiple names across different providers. Synonyms are grouped together:

const modelsSynonyms = {
  "anthropic": [
    ['claude-4-opus-20250514', 'claude-opus-4-20250514', 'anthropic.claude-opus-4-20250514-v1:0', 'claude-opus-4', 'claude-opus-4@20250514'],
    // Each array groups all names that refer to the same model
  ],
};

To add synonyms: Add the alternative names to the appropriate array. The first name in each array is used as the primary display name.

3. Model Filtering/Exclusions

There are three ways to exclude models from display:

a) Exclude models containing specific text (`excludedModelsInclude`)

const excludedModelsInclude = [
  'gpt-4-vision',     // Excludes any model containing "gpt-4-vision"
  'grok-beta',        // Excludes any model containing "grok-beta"
  'gemini-gemma-2',   // Excludes any model containing "gemini-gemma-2"
];

b) Exclude models starting with specific text (`excludedModelsStartWith`)

const excludedModelsStartWith = [
  'ft:',              // Excludes fine-tuned models
  'gemini-2.0-flash', // Excludes all gemini-2.0-flash variants
  'gemini-1.5-flash', // Excludes all gemini-1.5-flash variants
];

c) Exclude specific model names (`excludedModelsExactMatch`)

const excludedModelsExactMatch = [
  'o1-preview',
  'gpt-4',
  'gemini-pro',
  // Exact matches only
];

Common Update Tasks

Adding a New Model to Display

The model should already exist in llm-apis-data.js
Add it to modelsPriorityOrder under the appropriate provider
If it has alternative names, add them to modelsSynonyms

Removing/Hiding a Model

Choose the appropriate exclusion method:

Model name contains unwanted text → Add to excludedModelsInclude
Model name starts with unwanted prefix → Add to excludedModelsStartWith
Specific model name → Add to excludedModelsExactMatch

Reordering Models

Simply rearrange the order in the modelsPriorityOrder array for that provider.

Adding Model Aliases

Add a new array or update existing array in modelsSynonyms with all name variants.

Example: Recent Updates

Here’s what we did in the last update:

Prioritized new models:
- Put o3-pro first for OpenAI
- Added claude-4-opus-20250514 and claude-4-sonnet-20250514 for Anthropic
- Set gemini-2.5-pro and gemini-2.5-flash as primary Google models
Excluded outdated models:
- Added gemini-2.0-flash to excludedModelsStartWith
- Added grok-beta variants to excludedModelsInclude
- Added gpt-4-vision to excludedModelsInclude
Added synonyms:
- Linked o4-mini-deep-research with o4-mini-deep-research-2025-06-26
- Added all Claude 4 variants (Bedrock, Vertex AI, APAC versions)

Testing Changes

Start Jekyll server: bundle exec jekyll serve
Visit http://127.0.0.1:4000/llm-apis/
Verify:
- Models appear in correct order
- Excluded models don’t appear
- Synonyms are properly grouped
- No JavaScript errors in console

Tips

When adding exclusions, consider if you need partial match (contains), prefix match (starts with), or exact match
Model names are case-sensitive
The first model in a synonym group is used as the display name
Check browser console for any JavaScript errors after changes
Some models may have provider prefixes (e.g., xai/grok-3) - include the full name

Troubleshooting

Models not appearing: Check if they’re in exclusion lists
Wrong order: Verify modelsPriorityOrder has correct spelling
Synonyms not grouping: Ensure exact name matches in synonym arrays
Page not updating: Hard refresh (Cmd+Shift+R) or restart Jekyll

Remember: The display logic is in llm-apis-fetcher.js, not the data file!

Younesse Kaddar