LLM endpoint latency test from US, Europe and India

Test an endpoint

Fill it in, check it, then start the test: a Quick check from your balance ($10), or any test on a plan. You can do all of it here; the report opens when it is ready.

Your endpointAddressHTTP(S), WebSocket (wss://) or gRPC (grpc://, grpcs://). A raw IP and port are fine: http://203.0.113.7:8000/v1/chat/completions

APIModel (as the endpoint names it)

API key (optional for your own server)Sealed while the test runs, deleted after it. Never shown or kept in this browser.Sent as

Other headers

What to sendPromptEach request gets a prefix of its own, so no cache answers it.Longest answer (tokens)

Advanced

Your own request (JSON, instead of the prompt)More of the probe's options (JSON)gRPC services, custom WebSockets and other APIs: see the supported APIs.

Fill it in now; it is kept here while you log in.

Loading the form…

An example report

A vLLM endpoint serving Qwen3-8B on one GPU in the US, tested from the three regions, with a load test from US East. The model takes about 110 ms to start answering wherever you are; from India, reaching it adds 600 ms.

Fastest from US East (Virginia), slowest from India (Mumbai).

Each region, one user at a time
From	Reaching it	First token (p50, bar to p95)	Per token	Tokens/s	Whole answer (p50)	Errors	Within targets
US East (Virginia)	11 ms	117 ms · p95 179 ms	7 ms	135	1.59 s	none	100%
Europe Central (Frankfurt)	268 ms	374 ms · p95 431 ms	8 ms	133	1.88 s	none	100%
India (Mumbai)	612 ms	721 ms · p95 1.24 s	8 ms	131	2.24 s	none	80%

Within targets: the first token in 1.00 s or less and each later token in 50 ms or less. Reaching it: DNS, connecting and TLS from that region.

US East (Virginia)11 ms reaching it · 106 ms the endpoint

Europe Central (Frankfurt)268 ms reaching it · 106 ms the endpoint

India (Mumbai)612 ms reaching it · 109 ms the endpoint

US East (Virginia) under load

Users at once	First answer (p50, bar to p95)	Tokens/s each	Tokens/s in all	Within targets	Errors
1	117 ms · p95 179 ms	135	122	100% · 0.6/s	none
4	141 ms · p95 298 ms	123	455	100% · 2.3/s	none
16 most capacity	236 ms · p95 702 ms	86	1290	96% · 6.4/s	none
32	1.18 s · p95 2.84 s	52	1460	41% · 4.1/s	2 of 96

Its most capacity here: 6.4 answers a second within the targets, at 16 users at once.

What it measures

Reaching it

DNS, connecting and TLS from each region: how far your endpoint is from people there.

First token

From the request to the first token of the answer (an empty first chunk does not count), and the first answer token after a model's thinking.

Speed

Time per token after the first, tokens per second for each user and in all, and the longest pause in the stream.

Under load

The same with 1, 4, 16 and 32 users at once: where it slows down, where errors and rate limits start.

Within targets

How many answers start within a second and keep up 50 ms per token: the capacity that matters to users.

Voice and speech

Time to the first audio and audio speed; for speech to text, time to the first words, the wait for the final transcript after the speaker stops, and the word error rate.

Every request gets a prompt of its own (no cached answers), token counts come from your endpoint when it reports them, and the first requests are reported apart as the cold start. p50 and p95; p99 from 100 requests.

Works with

Text over HTTP(S)

OpenAI chat, completions and Responses (vLLM, TGI, Ollama, Together, Groq, Fireworks, OpenRouter, Azure OpenAI and gateways), Anthropic, Google Gemini, Ollama, Hugging Face TGI, Cohere, NVIDIA Triton, or your own JSON.

Voice

OpenAI Realtime and Gemini Live over WebSocket, NVIDIA Riva text to speech over gRPC, or your own WebSocket: time to the first audio, and how fast the audio comes.

Speech to text

OpenAI-style /audio/transcriptions (OpenAI, Groq, vLLM Whisper), Deepgram, Cloudflare Workers AI, AssemblyAI, OpenAI Realtime transcription and NVIDIA Riva: your recording, sent as a caller would, with the word error rate.

gRPC

NVIDIA Triton (vLLM and TensorRT-LLM backends) and any gRPC service: its schema is read from the server's reflection, or from the descriptor set you give.

Runs and plans

Test	What it does	Without a plan	On a plan
Quick check	100 requests, one at a time, from one region	$10 from your balance	1 run
Global check	100 requests, one at a time, from every region your plan has	—	3 runs
Load test	1, 4, 16 and 32 users at once (Tier 1: up to 16)	—	2 runs a region

Test	What it does	Without a plan	On a plan
Quick check	100 requests, one at a time, from one region	$10 from your balance	1 run
Global check	100 requests, one at a time, from every region your plan has	—	3 runs
Load test	1, 4, 16 and 32 users at once (Tier 1: up to 16)	—	2 runs a region

A Quick check needs no plan: it is paid from your balance when it starts, and not refunded. Other tests use your Model benchmark plan's runs, the same runs as GPU benchmarks. Tier 1 tests from one region, Tier 2 from up to three, Enterprise from more. If a plan test fails on GridPup's side, its runs are given back.

Questions

What can I test?

Any AI endpoint you run or use: your own vLLM, TGI, Ollama or Triton server (a raw IP and port are fine), or an API like OpenAI, Anthropic, Gemini, Together, Groq, Cloudflare or Deepgram, over HTTP(S), WebSocket or gRPC.

Where do the tests run from?

From AWS regions: US East (Virginia), Europe (Frankfurt) and India (Mumbai) now, with more to come. Each region sends real requests to your endpoint and measures what people there see.

What does it cost?

A Quick check (100 requests from one region) is $10 from your balance, with no plan. Global checks and load tests use the runs of a Model benchmark plan: a global check 3 runs, a load test 2 runs for each region, and a Quick check 1 run. Tier 1 tests from one region, Tier 2 from up to three, Enterprise from more. Checking an endpoint before a test needs a plan or that much in your balance (it is not charged); a test paid from your balance is not refunded.

Do I pay for the tokens?

Your provider bills the tokens of the requests, as for any use of your key; the report shows how many tokens were used. Your own server costs you nothing extra.

Why do I have to show an endpoint is mine?

So nobody can use GridPup to load-test someone else's server. Put a token we give you at /.well-known/gridpup.txt on the endpoint (or in a DNS record). Known API providers do not need it: your key and their limits apply.

Is my API key safe?

It is encrypted while the test runs, sent only to the regions running it, and deleted when the test ends. It never appears in the report or in logs.

Do I need to log in?

You can read this page and fill in the test without an account. Checking an endpoint needs a login and a plan, or at least $10 in your balance. A Quick check is paid from your balance ($10); global checks and load tests need a Model benchmark plan.