Model Inference Pricing: Examples & Companies

What is model inference pricing?

Model inference pricing is the billing structure a vendor applies when a customer calls an AI model to generate output — whether that is text tokens from an LLM, an image from a diffusion model, a speech clip from a TTS model, or an embedding vector from an embedding model. “Inference” refers to the forward pass through a trained model: you send an input, the model generates an output, you pay for the compute consumed.

Fifty-five companies in the UsagePricing corpus tag model inference as a primary use case, making it the largest single use-case category. They include frontier model labs (Anthropic, OpenAI, Google), open-model inference platforms (Groq, Fireworks, Together, DeepInfra, Novita), managed serving platforms (Baseten, Anyscale, Modal, Replicate), and API-first specialists (Cohere, Mistral, DeepSeek).

The dominant billing units

Inference pricing uses three primary units, each reflecting a different cost driver:

Tokens (input + output, separately priced) — the standard for LLM text APIs. Input tokens are typically cheaper than output tokens because generation is more compute-intensive than encoding. Frontier examples: OpenAI GPT-5.5 at $5/$30 per 1M input/output, Anthropic Claude 3.5 Sonnet at $3/$15, DeepSeek V3 at $0.27/$1.10.

GPU-hours / compute seconds — the standard for dedicated deployments, fine-tuned models, and GPU-cloud platforms. Baseten, Anyscale, Modal, RunPod, and Replicate all meter GPU time directly. On-demand vs reserved rates differ by 20-40%.

API calls / requests — the standard for image generation (per image), embedding (per query), and some search APIs. Fal.ai, Clipdrop, Replicate (some models), and Browserbase use per-request metering.

Key structural discounts

Batch processing (~50% off): Asynchronous, latency-tolerant workloads earn roughly half off the synchronous rate across Anthropic, OpenAI, Google, Fireworks, Groq, Mistral, and Together. If any meaningful portion of your workload is async, batch API can halve the token bill with no model change.

Cached-input discounts (50-80% off): Nine corpus vendors publish a discounted rate for repeated input context. Anthropic charges $3.75/1M for cached Claude 3.5 Sonnet input vs $15/1M uncached (75% off); OpenAI offers 50% off cached input; Google 75% off Gemini cached input. Workloads with stable system prompts or large RAG context benefit most.

Volume tiers: Most inference vendors offer volume discounts at a monthly spend or token threshold, usually unlocking at $1k-$10k/month of usage.

Free tiers and onboarding

88% of pure-usage inference vendors offer a free tier — the highest free-tier rate in the corpus. Free tiers typically provide 200K-1M tokens or $5-$10 in credits, sufficient to test models before committing. Exceptions include DeepInfra (no free tier) and some dedicated-GPU platforms.

Pricing transparency

Inference providers are the most transparent segment in the corpus: nearly all publish per-token or per-call rate cards publicly. The exceptions are vendors adding dedicated deployment or enterprise commitment tiers, where rates are sales-quoted. Gated pricing in inference is a signal of either (a) a new product not yet publicly priced or (b) dedicated capacity that requires utilization negotiation.

What to watch

Token prices continue to fall at each model generation — GPT-4o launched 50% cheaper than GPT-4 Turbo; Anthropic rebased Opus ~3x inside the 4.x line; DeepSeek V3 launched at frontier-class performance for $0.27/1M input. Re-baseline your cost model at least twice a year and evaluate whether a cheaper new model meets your quality bar before assuming current rates are fixed.