LLM_Token_Index // Daily_Benchmark
Daily cost, latency, and reliability benchmark across 6 LLM models from 3 providers via OpenRouter. Quality-adjusted pricing shows the true cost of usable output.
Last updated: 2026-03-20v0.2.0Basket: 0.2.0
Index Cost
$6.36/ 1M tokens
-1.2% vs prev
Latency p50
1667ms
-25.6% vs prev
Latency p95
24565ms
+14.4% vs prev
Error Rate
0.0%
Cost Trend
Model Comparison
| Model | Provider | Quality | $/1M Tokens | p50 ms | p95 ms | Errors |
|---|---|---|---|---|---|---|
| google/gemini-2.5-flash-lite | 100% | $0.41 | 768 | 10871 | 0.0% | |
| openai/gpt-4o-mini | openai | 97% | $0.68 | 1702 | 20734 | 0.0% |
| google/gemini-3-flash-preview | 97% | $3.31 | 1436 | 5411 | 0.0% | |
| anthropic/claude-haiku-4.5 | anthropic | 91% | $5.74 | 2124 | 12481 | 0.0% |
| openai/gpt-4o | openai | 100% | $11.09 | 3836 | 21920 | 0.0% |
| anthropic/claude-sonnet-4.6 | anthropic | 91% | $16.94 | 1621 | 45191 | 0.0% |
Sensitivity Analysis
How quality-adjustment scoring impacts effective pricing. Models with lower quality get penalized with higher effective costs.
| Model | Adjusted $/1M | Unadjusted $/1M | Delta |
|---|---|---|---|
| google/gemini-2.5-flash-lite | $0.41 | $0.41 | 0.0% |
| openai/gpt-4o-mini | $0.68 | $0.66 | +3.2% |
| google/gemini-3-flash-preview | $3.31 | $3.21 | +3.2% |
| anthropic/claude-haiku-4.5 | $5.74 | $5.21 | +10.3% |
| openai/gpt-4o | $11.09 | $11.09 | 0.0% |
| anthropic/claude-sonnet-4.6 | $16.94 | $15.35 | +10.3% |
Provider Breakdown
anthropic
Cost/1M$11.34
Latency p502061ms
Latency p9540218ms
Error Rate0.0%
Models2/2 valid
Weight33%
google
Cost/1M$1.86
Latency p501293ms
Latency p959365ms
Error Rate0.0%
Models2/2 valid
Weight33%
openai
Cost/1M$5.88
Latency p502141ms
Latency p9521087ms
Error Rate0.0%
Models2/2 valid
Weight33%