Topic

Benchmarks

15 stories on this topic, newest first.

  1. Models

    Claude Fable 5.2 for builders: when to route past Opus 5.5, and what it costs

    Fable 5.2 costs $8/$40 per million tokens, twice Opus 5.5. What Anthropic's data says it buys, plus fast mode, advisor rules and the preserved-thinking break.

    29 min read

  2. Models

    Gemini 4 Carbon goes public, and its introductory price ends 30 November

    Google's strongest model costs $2/$10 per million tokens until 30 November, then $4/$20. How to budget the switch, what /swarm does, and the #1 claim today.

    25 min read

  3. Tools

    Copilot Fusion hands GitHub the model choice. Copilot access makes the budget the control

    What GitHub's Fusion router and cross-repo projects change for engineering teams, how the benchmark claim checks out, and what credit-only Copilot access costs.

    27 min read

  4. Models

    Muse Ember wants the subagent slot. Its list price matches Haiku and Luna

    Meta's smallest Muse model lists at $0.10 in and $0.50 out, like its rivals. Cache rates, context tiers and a max-effort default decide what an executor costs.

    24 min read

  5. ToolsExplainer

    Is LiteLLM written in Rust? What the codebase and migration tracker show

    LiteLLM Rust, checked: the repo is 3.11% Rust, the migration tracker shows 24% of Major APIs, and Chat Completions still runs on Python.

    20 min read

  6. ToolsAnalysis

    The '4.5x more requests per dollar' claim is a model, not a measurement

    LiteLLM cost claims for its Rust gateway rest on an assumed 4 vCPU / 16 GB instance. What the 4.5x, 5.8x and 4.4x ratios measure, and what is missing.

    24 min read

  7. ToolsExplainer

    Peak RSS vs memory at rest: reading LiteLLM's '9x less memory' figure

    LiteLLM's homepage labels its memory figure 'memory at rest', but the data is peak RSS. What the files show, why Bifrost sat near 282 MB, and how to read RSS.

    20 min read

  8. ToolsComparison

    LiteLLM Rust vs LiteLLM Python vs Bifrost: what the data means for teams running LiteLLM today

    In LiteLLM's own sweep the Python LiteLLM proxy adds 546 ms of p99 at c=64 and serves 179.8 RPS, and main:latest still runs chat routes on Python.

    23 min read

  9. ToolsExplainer

    Throughput vs latency: why LiteLLM Rust and Bifrost tie at 2,722 vs 2,719 RPS

    Throughput vs latency for LLM gateways: why 2,722 vs 2,719 requests per second is a tie, how RPS and p99 interact, and how to read a throughput tile.

    22 min read

  10. ModelsExplainer

    Claude Sonnet 5.5: a practical guide to price, effort, benchmarks and migration

    Anthropic's mid-tier model costs $2 in and $10 out per million tokens with a 1M context. What changed from Sonnet 5, when to pick it over Opus 5.5 or Haiku 5.5, and how to migrate.

    24 min read

  11. ModelsComparison

    Best LLM for coding in October 2026: ten models ranked on the evidence

    Opus 5.5, GPT-6.1 Sol, Gemini 4 Argon and seven more ranked on Terminal-Bench 4.0, DeepSWE and cost per task, with each score's harness and source.

    23 min read

  12. ResearchExplainer

    What is an agent harness? The loop, tools and context around the model

    An agent harness is everything around the model: the loop, tools, context, memory, permissions and sandbox. Why it moves benchmark scores as much as the model.

    24 min read

  13. ModelsComparison

    Top 10 AI models in October 2026, ranked by what they are good at

    Claude Opus 5.5, GPT-6.1 Sol, Gemini 3.8 Flash, Kimi K3 and six more, ranked on independent evals, coding, context, price and speed, with every figure sourced and dated.

    25 min read

  14. ModelsExplainer

    How to read a model card without being fooled: the 2026 field guide

    System cards now run past 200 pages. We take five current cards apart section by section, show how one benchmark is reported five different ways, and set out a ten-step reading protocol.

    30 min read

  15. IndustryEditorial

    Benchmarks became a marketing channel. Read the footnotes, then run your own

    Launch tables report the same benchmark under different attempts, tools and thinking budgets. How measurement became collateral, what independent evaluators find, and how to run your own.

    21 min read