Groq pricing

About

Groq is a Mountain View-based AI inference company founded in April 2016 by Jonathan Ross — the original engineer behind Google’s Tensor Processing Unit (TPU). The product is GroqCloud, an inference API built on Groq’s proprietary LPU (Language Processing Unit) silicon. The LPU was designed exclusively for inference workloads: deterministic execution, on-die SRAM (no HBM), and architectural choices that prioritize single-stream throughput over GPU-style parallel batch efficiency. The result is published throughput of 800+ tokens/second on Llama 3 8B and 500+ TPS on 120B-class models, with per-token pricing that holds competitive with GPU-based serverless inference.

By 2026 Groq serves a mix of latency-sensitive production AI customers (voice assistants, real-time agentic systems, customer-facing chatbots) and high-throughput batch workloads. The company raised a $640M Series D in August 2024 led by BlackRock at $2.8B valuation, with Tiger Global, Samsung Catalyst, KDDI, and others participating; a 2025 round led by Saudi PIF brought valuation to $6.9B. The Saudi partnership was paired with a strategic commitment to build Groq’s largest data center in Saudi Arabia, signaling the company’s intent to compete on global inference capacity, not just per-token economics.

Groq competes with Fireworks AI, Together AI, Baseten, and Replicate for the managed-inference market, plus first-party providers (OpenAI, Anthropic) for general-purpose API customers. Its differentiation is bespoke LPU silicon (the only commercial inference platform built on inference-specific hardware), TPU-pioneer founder credibility (Jonathan Ross designed the original TPU), and the highest published throughput-per-token in the category — making Groq the canonical choice for latency-critical AI workloads where time-to-first-token matters more than raw cost.

Pricing summary : How Groq’s LPU + per-token + tools stack works

Groq runs a per-token serverless inference API priced by model, with throughput (TPS) published alongside each rate so customers can evaluate the latency-cost trade-off in a single view. The model catalog spans Llama (3.1 8B Instant at $0.05/$0.08 input/output per 1M tokens, 3.3 70B Versatile at $0.59/$0.79, 4 Scout at $0.11/$0.34), GPT OSS (20B at $0.075/$0.30, 120B at $0.15/$0.60), Qwen3 (32B at $0.29/$0.59), Mixtral, and Whisper for audio transcription. Audio is priced per hour transcribed with a 10-second minimum per request.

Cached input tokens get a 50% discount on standard rates, and the Batch API offers a 50% discount for asynchronous workloads. Built-in agentic tools have transparent per-use pricing: web search at $5–$8 per 1,000 requests, website visits at $1 per 1,000 requests, code execution at $0.18/hour. Free tier provides rate-capped access for evaluation; Enterprise plans add custom SLAs, dedicated LPU capacity reservations, volume discount commits, and VPC deployment.

This single-SKU plus per-tool structure — token / hour-audio / per-use-tool — is one of the most legible usage-based rate cards in AI inference middleware. The marketing positioning explicitly emphasizes “linear and predictable” pricing as a core value proposition.

What makes this different: Groq’s LPU silicon delivers throughput that GPU-based competitors cannot match at the same price point. Llama 3.1 8B at 840 TPS is roughly 4–10× faster than Llama 8B on Fireworks or Together at comparable prices — and the latency advantage compounds for multi-turn agentic workflows where time-to-first-token determines user experience.

Pricing by product

Serverless per-token inference (with published TPS)

Audio transcription (per hour, 10-second minimum)

Built-in agentic tools

Discount mechanics

Sales motions across products: PLG / self-serve for Free and Developer tier; sales-led for Enterprise annual commits, dedicated LPU capacity, and VPC deployments. All prices accessed 2026-05-29 from groq.com/pricing.

Hidden costs : What Groq customers actually pay beyond the per-token rate

Archetype A: Voice assistant startup on Llama 3.1 8B Instant

A latency-critical voice assistant startup serving ~200K requests/day at average 1K input + 250 output tokens, using Whisper Turbo for upstream audio transcription:

For latency-critical voice workloads, Groq’s per-token rates plus Whisper Turbo deliver one of the lowest published total costs in the category. The same workload on a comparable GPU-based platform would typically cost 2–4× more — and would not match the latency.

Archetype B: Mid-market team running mixed Llama 70B + GPT OSS 120B with agentic tools

A 50-person team building an internal AI assistant using Llama 3.3 70B Versatile for general queries and GPT OSS 120B for reasoning-heavy tasks, with frequent agentic tool usage:

Built-in tool usage dominates the bill at this scale — and the transparency of the tool rate card lets finance teams forecast tool spend accurately. Most platforms either bundle tools into opaque tier pricing or force customers onto third-party tool providers, making forecasting harder.

Want to estimate your own Groq bill? Use the Groq pricing calculator to model token spend by model, audio transcription hours, and built-in tool usage.

Pricing evolution : Groq’s pricing history from LPU prototype to commercial inference category

Cadence

Tracked range: 2016 Q2–2026 Q1. Quarters not listed above were verified stable (0 price changes, 0 SKU additions).

Notable changes

• 2023-03-22 — GroqCloud public launch with Llama 2 + Mistral; LPU silicon as the structural differentiator.
• 2024-02-19 — Llama 3 day-one support at 800+ TPS for the 8B variant; established Groq as the throughput leader.
• 2025-03-04 — Whisper Large v3 ($0.111/hr) and Whisper Large v3 Turbo ($0.04/hr) for audio transcription with 10-second minimum billing.
• 2025-05-13 — Batch API and cached input 50% discounts launched; brought Groq to parity with Fireworks and Anthropic on discount mechanics.
• 2026-01-22 — Built-in tools per-use pricing published: web search $5–$8/1K, website visits $1/1K, code execution $0.18/hour.

What’s unique : Groq’s distinctive pricing mechanics

1. Throughput (TPS) published alongside per-token rates. Groq is one of the only inference platforms that publishes throughput (tokens per second) next to every per-token rate. For latency-critical workloads (voice agents, real-time interactive UIs), this transparency lets customers evaluate the latency-cost trade-off in a single view rather than running benchmark calls. Most competitors hide TPS behind documentation pages or marketing claims.

2. Bespoke LPU silicon as the structural cost moat. The LPU is the only commercial inference platform built on inference-specific silicon (rather than repurposed GPU). This delivers throughput that GPU-based competitors cannot match at comparable prices — making Groq’s $0.05/$0.08 Llama 3.1 8B rate genuinely hard to replicate without similar silicon investment.

3. Per-use agentic tool pricing in line-item form. Web search ($5–$8/1K), website visits ($1/1K), and code execution ($0.18/hour) are priced individually rather than bundled into opaque tiers. For finance teams forecasting agent-loop costs, the transparent tool rate card reduces budget uncertainty in a way that bundled-tools competitors cannot match.

4. Two-variant audio transcription (Whisper Large v3 vs Turbo). The 2.8× price spread ($0.111/hr vs $0.04/hr) between Whisper Large v3 and Whisper Large v3 Turbo lets customers pick accuracy-vs-speed explicitly. For high-volume transcription workloads (call centers, podcast processing), the Turbo rate is meaningfully cheaper than first-party Whisper API rates while preserving acceptable accuracy.

5. Cached input + Batch API discount mechanics in standard rate card. Both discounts apply at 50% off base rates, bringing Groq to parity with Fireworks, OpenAI, and Anthropic. The discount mechanics are the same across all eligible models — making them legible to customers without requiring per-model discount lookup. This discount-as-default architecture is becoming canonical for token-priced inference.

Strengths & weaknesses

Billing UX : Groq’s account controls and payment experience

• Self-serve signup — Sign up at console.groq.com with email; free tier access enabled immediately, no credit card required.
• Per-token usage console — Real-time view of per-model token consumption (input, output, cached) and per-request latency metrics.
• TPS transparency in console — Per-request TPS shown alongside latency, letting developers verify throughput SLAs in real time.
• Spend alerts — Configurable email alerts at $X spend per period; hard spend caps available on Developer credit-card plans.
• Payment methods — Credit card on self-serve Developer; wire transfer, invoice billing, and AWS/GCP Marketplace on Enterprise.
• Cached input transparency — API responses include cached input token count separately, letting customers verify cache-hit rates per request.
• Annual commit pricing — Enterprise customers receive volume discounts in exchange for annual usage commitments and dedicated LPU capacity.
• Multi-region availability — US regions standard; international expansion (including Saudi Arabia data center) underway.
• Audit logging + RBAC — Workspace-level RBAC on Pro+; SOC 2 audit-log exports on Enterprise via webhook or S3 delivery.

Strategic wins : Why Groq’s pricing decisions worked

1. Bespoke LPU silicon as the structural cost moat

By investing in inference-specific silicon rather than repurposing GPU, Groq created a structural cost advantage that GPU-based competitors cannot replicate without comparable silicon investment. The LPU’s 800+ TPS on Llama 3 8B at $0.05/$0.08 is not just a marketing claim — it is the canonical proof that inference-specific silicon outperforms training-derived GPU silicon for sustained inference workloads.

2. TPS published alongside per-token rates removed evaluation friction

By publishing throughput (TPS) next to every per-token rate, Groq made the latency-cost trade-off legible without benchmark calls. For latency-critical workloads (voice agents, interactive UIs), this transparency converts more self-serve customers and reduces sales-led overhead. Most competitors hide TPS behind benchmark blog posts or marketing claims.

3. TPU-pioneer founder credibility as the silicon-trust anchor

Jonathan Ross’s role as the original Google TPU engineer gives Groq unusual credibility on silicon architecture — making the LPU’s throughput claims believable in a way that pure-engineering teams cannot replicate. For enterprise procurement leaders evaluating bespoke-silicon inference, the founder credentials act as a trust multiplier that distinguishes Groq from generic inference middleware.

4. Per-use agentic tool pricing preempted bundled-tool competitor lock-in

By publishing per-use rates for web search, website visits, and code execution as line items on the rate card, Groq let customers forecast agent-loop costs without negotiating contracts or accepting opaque tool bundling. This transparent agentic pricing is a meaningful competitive advantage as agentic workflows scale and tool usage becomes the dominant cost line.

Areas to improve : Gaps in Groq’s pricing approach

1. Cached input + Batch API discounts should stack

Fireworks lets cached input (50%) and Batch API (50%) discounts stack — making batched cached workloads land at 25% of standard. Groq’s discounts do not stack — customers must pick one. Adding stacking would close a meaningful competitive gap with Fireworks for RAG and agent-loop workloads at scale.

2. No published fine-tuning SKU

Customers wanting custom-trained models must use Fireworks, Together, Anyscale, or first-party model providers. Adding a fine-tuning SKU (even at limited model coverage initially) would let Groq capture the model-customization workload that currently goes elsewhere. The LPU’s deterministic execution may make fine-tuning architecturally awkward, but customers want the option.

3. Free tier rate caps may be too restrictive

Free tier rate limits are tight enough that meaningful evaluation requires a credit card and Developer-tier upgrade. Loosening free-tier caps (or offering a $5–$10 trial credit) would let self-serve developers reach proof-of-value without paid commitment and likely accelerate self-serve revenue.

4. Audio transcription only via Whisper

Whisper is the dominant open-source ASR model, but customers wanting Deepgram-comparable accuracy or specialized vertical models (medical, legal) must use other providers. Adding a second ASR model option (or a transcription quality tier above Whisper) would broaden the audio TAM and reduce vendor sprawl for audio-heavy workloads.

Key takeaways

1. Bespoke silicon as the structural moat. Groq’s LPU is the only commercial inference platform built on inference-specific silicon. The throughput advantage (800+ TPS on Llama 3 8B) is genuinely hard to replicate without comparable silicon investment — a durable competitive moat that pure-software optimization cannot match.

2. Throughput transparency converts latency-critical buyers. Publishing TPS alongside per-token rates makes the latency-cost trade-off legible for voice-agent, interactive-UI, and real-time agentic workloads where time-to-first-token determines user experience. Inference platforms targeting latency-sensitive workloads should publish TPS on the rate card.

3. Per-use agentic tool pricing is the next transparency frontier. As agent loops scale, web search and code execution costs become the dominant bill line. Platforms that publish per-tool rates in line-item form win finance-team trust over competitors that bundle tools into opaque tiers.

4. Two-variant audio transcription (Whisper Large v3 vs Turbo) captures both quality-sensitive and cost-sensitive workloads with a single SKU surface. The 2.8× price spread between variants is wide enough to matter and narrow enough to be a real choice rather than a trick.

5. TPU-pioneer founder credibility is the most durable silicon-platform trust anchor. Jonathan Ross’s role as the original Google TPU engineer gives Groq unusual credibility on architectural claims — distinguishing it from generic inference middleware and converting silicon-skeptical enterprise procurement leaders.

UBP implications

1. Throughput (TPS) as a published metric on token-priced rate cards lets usage-based platforms compete on latency-cost rather than just absolute per-token price. Inference platforms that hide TPS behind documentation lose latency-sensitive buyers who cannot self-qualify.

2. Per-use line-item tool pricing is becoming the canonical structure for agentic UBP. Bundled-tool pricing creates forecasting uncertainty that finance teams reject; transparent per-tool rates win procurement trust.

3. Bespoke silicon as a UBP moat is the structural cost advantage that pure-software platforms cannot replicate. Groq’s LPU is the canonical example — but the precedent suggests that other inference-specific silicon investments (Cerebras, Tenstorrent, Etched) will reshape competitive dynamics over the coming decade.

Sources

• Groq pricing page (accessed 2026-05-29)
• Groq docs — models catalog (accessed 2026-05-29)
• Groq blog — Llama 3 launch (accessed 2026-05-29)
• Groq blog — Series D announcement (accessed 2026-05-29)
• Groq Saudi Arabia partnership (accessed 2026-05-29)
• Related infra blueprint — Cerebras
• Related infra blueprint — Fireworks AI
• Blueprint corpus index

Bottom line

Groq priced its inference API around three structural ideas: bespoke LPU silicon that delivers throughput GPU-based competitors cannot match (800+ TPS on Llama 3 8B at $0.05/$0.08), TPU-pioneer founder credibility (Jonathan Ross designed the original Google TPU) that justifies the silicon investment, and transparent per-use agentic tool pricing (web search at $5–$8/1K, website visits at $1/1K, code execution at $0.18/hour) that makes Groq one of the most legible commercial inference platforms in the market. Marketing positioning explicitly emphasizes “linear and predictable” pricing — and the rate card structure makes the claim genuinely accurate.

For AI engineering teams running latency-critical voice agents, real-time interactive UIs, and high-throughput batch workloads, Groq delivers throughput and per-token economics that GPU-based platforms cannot match. The remaining gaps (no fine-tuning SKU, audio limited to Whisper, restrictive free tier, non-stacking discounts) are TAM-expansion problems rather than structural pricing flaws.

Compare with peers via the blueprint corpus, or model your own spend with the Groq pricing calculator.

Pricing timeline : Major events on a vertical axis

Each milestone below corresponds to a public pricing change, product launch, or material adjustment. Major events use a filled marker; minor adjustments use a faded one.

Built-In Tools Pricing: Web Search, Visits, Code Execution

Jan 2026

Groq published per-use pricing for built-in agentic tools: web search at $5–$8 per 1,000 requests, website visits at $1 per 1,000 requests, code execution at $0.18/hour. Made Groq one of the few platforms with transparent line-item billing on agentic tool usage.

Saudi PIF Investment at $6.9B Valuation

Aug 2025

Groq raised a Saudi PIF-led round at a $6.9B post-money valuation, with the funding earmarked for international data center expansion and additional LPU production. The Saudi investment was paired with a strategic partnership to build Groq's largest data center in Saudi Arabia.

Batch API + Cached Input Discounts

May 2025

Groq launched a Batch API at 50% discount for asynchronous workloads and added cached input token discounts at 50% off standard rates. Brought Groq into parity with Fireworks, OpenAI, and Anthropic on standard inference discount mechanics.

Whisper Large v3 + Turbo Variants at Per-Hour Pricing

Mar 2025

Groq added Whisper Large v3 ($0.111/hour transcribed) and Whisper Large v3 Turbo ($0.04/hour transcribed) for audio transcription. Per-hour billing with 10-second minimum per request established the audio SKU structure that remains canonical.

Series D ($640M) at $2.8B Valuation

Aug 2024

Groq raised a $640M Series D led by BlackRock at a $2.8B post-money valuation. Tiger Global, Samsung Catalyst, KDDI, and others participated. The round funded LPU silicon production scaling and the GroqCloud Enterprise tier launch.

Llama 3 Day-One Support at Industry-Leading TPS

Feb 2024

Groq launched Llama 3 support on day one with throughput exceeding 800 tokens/second for the 8B variant — the fastest published inference rate for any commercial Llama 3 endpoint at launch. Per-token pricing held competitive with serverless GPU competitors.

GroqCloud Public Launch

Mar 2023

Groq launched GroqCloud — its inference API — with Llama 2 and Mistral models at competitive per-token rates. The launch positioning emphasized speed-of-inference (hundreds of TPS) rather than absolute cheapness, marketing LPU silicon as a structural advantage over GPU-based inference middleware.

Groq Founded

Apr 2016

Jonathan Ross (original engineer behind Google's TPU) founded Groq to build bespoke inference silicon. The company spent 2016–2022 designing the LPU (Language Processing Unit) architecture before launching GroqCloud — the commercial inference API — in early 2023.