Groq

Very fast LLM inference on custom LPU hardware.

Hand-reviewed by the Toolglade team·Updated Aug 2026

coding#inference#llm-api#low-latency#open-source

Free plan Claimed API

Toolglade’s take

Groq is the go-to when raw inference speed matters: agents, voice, and interactive chat feel noticeably snappier on its LPU hardware, and per-token prices are competitive. The trade-off is scope. You serve models from Groq's supported catalog rather than arbitrary custom weights, and the exact model lineup shifts over time. Corporate turbulence around the 2026 Nvidia deal and a down round is worth noting for those making long-term platform bets, though day-to-day service has stayed reliable. A strong default for latency-sensitive apps on popular open models.

About Groq

Groq is an inference cloud running on custom LPU chips that deliver very high tokens-per-second on supported open models. It exposes an OpenAI-compatible API with a free tier, per-token paid pricing, and batch discounts. It is best for latency-sensitive workloads that fit its curated model catalog rather than custom-model hosting.

Groq designs custom Language Processing Unit (LPU) hardware optimized for low-latency LLM inference and sells access through GroqCloud, an OpenAI-compatible API. Its headline advantage is speed: on supported open models such as Llama, it typically delivers much higher tokens-per-second than general-purpose GPU inference, which makes it attractive for chat, agents, and real-time applications. The service focuses on serving a curated set of popular open-weight models rather than letting you upload arbitrary custom models. Developers get a free tier with rate limits, a paid developer tier with higher limits and discounts, and batch pricing for large jobs. Pricing is quoted per million tokens and varies by model. Groq's business has been eventful. It raised large rounds through 2025, then a 2026 licensing arrangement with Nvidia reshaped the company, followed by a down round. For users, the practical picture is stable: a fast, low-cost inference API for a defined model catalog, best suited to latency-sensitive workloads that fit the supported models.

TL;DR

Groq is an inference cloud built on custom LPU chips that deliver very high tokens-per-second on popular open models via an OpenAI-compatible API. It offers a free tier and competitive per-token pricing, making it a strong choice for latency-sensitive chat, agents, and real-time apps. The main limits are its curated model catalog and lack of custom-model hosting. Its 2026 corporate changes, including an Nvidia licensing deal and a down round, are worth noting. Day-to-day the service has remained reliable.

Company overview

Groq was founded in 2016 by Jonathan Ross, who previously helped create Google's Tensor Processing Unit. The company designs LPU hardware and operates GroqCloud, an inference service for open models.

In 2025 and 2026 Groq raised large financing rounds and entered a significant licensing arrangement with Nvidia in 2026, which was followed by a down-round financing. Despite the corporate changes, the inference service has continued operating.

Product features

GroqCloud provides an OpenAI-compatible API for a curated catalog of open-weight models, with a free tier, a paid Developer tier, and batch pricing. Its defining feature is inference speed driven by LPU hardware.

The platform targets latency-sensitive use cases such as chat, agents, and voice, where fast token generation improves user experience. It focuses on text generation rather than broad multimodal workloads.

Target market

Developers and teams building latency-sensitive AI applications, including chat assistants, agents, and real-time or voice products, who are comfortable using open-weight models from Groq's supported catalog.

Buyer personas

End users

Developers integrating fast LLM inference into chat, agent, and voice products.

Buyers

Engineering leaders choosing an inference provider for latency-sensitive workloads.

Key influencers

AI infrastructure engineers and open-model advocates.

Ideal customer profile

A product team whose app depends on fast, low-cost inference on popular open models and that does not need custom-model hosting.

Funding & performance

Groq raised a reported $640 million Series D in 2024 at around a $2.8 billion valuation, and additional financing in 2025 was reported at a valuation near $6.9 billion. In 2026, following a licensing arrangement with Nvidia, a further round of roughly $350 million was reported at about a $3.5 billion valuation, described as a down round. Treat specific figures as reported, and verify with the company.

Pros & cons

Pros

Exceptional inference speed on supported models
Competitive per-token pricing
OpenAI-compatible API is easy to adopt
Free tier with no credit card
Good fit for agents and real-time apps
Batch discounts for large jobs

Cons

Limited to a curated catalog of open models
No hosting of arbitrary custom weights
Model lineup changes over time
Corporate turbulence in 2026 (Nvidia deal, down round)
Free-tier rate limits are modest
Text-focused; limited multimodal support

Pricing plans

Free

$0 / month

Access to supported models via API and playground
No credit card required
Rate limits (~30 RPM)
Community support

Developer

Usage-based

~10x higher rate limits
Per-token pricing from ~$0.05/1M input
On-demand discounts
Higher daily request caps

Batch

Discounted per token

Discounted rates for large jobs
Asynchronous processing
Suited to bulk inference

Key features

API

Multi-language

Integrations

OpenAI-compatible API, LangChain, LlamaIndex, Vercel AI SDK

Input types

text

Output types

text

Best For

Low-latency chat, AI agents, Real-time apps, High-throughput inference

Compare key features

View all alternatives →

Feature

Groq

Together AI

Fireworks AI

Pricing

Freemium

Free plan

Yes

Free trial

Yes

API

Yes

Team support

Yes

Frequently asked questions

What makes Groq fast?+

It runs inference on custom Language Processing Unit (LPU) hardware designed for LLM serving, which delivers much higher tokens-per-second than typical GPU inference on supported models.

Can I run my own model on Groq?+

Generally no. Groq serves a curated set of popular open-weight models rather than arbitrary custom weights.

Is there a free tier?+

Yes. GroqCloud has a free tier with rate limits and no credit card required, plus a paid Developer tier with higher limits.

Is Groq's API compatible with OpenAI?+

Yes. It exposes an OpenAI-compatible endpoint, so most existing OpenAI client code works with minimal changes.

How is Groq priced?+

Per million tokens, with rates varying by model, plus batch discounts for large jobs.

Reviews (0)

Loading reviews…

Compare

Compare Groq with other AI tools

Side-by-side pages for pricing, features, and best-fit use cases.

All comparisons →

Compare side by side

From the blog

All articles →

Article

Groq vs Together AI vs Fireworks: Best LLM Inference API in 2026

Groq, Together AI, and Fireworks all serve open models via API — but they optimize for different things. Here's how to choose on speed, catalog, and cost.

Read →