Benchmarking Costs AI News & Updates

Reasoning AI Models Drive Up Benchmarking Costs Eight-Fold

April 10, 2025 13:30

AI reasoning models like OpenAI's o1 are substantially more expensive to benchmark than their non-reasoning counterparts, costing up to $2,767 to evaluate across seven popular AI benchmarks compared to just $108 for non-...