Menu

Sort providers by cost, latency, or throughput on AI Gateway - Vercel
📰
0

Sort providers by cost, latency, or throughput on AI Gateway - Vercel

Vercel News·Walter Korman·4 days ago
#guLoaMHp
#vercel#sort#providers#cost#gateway#provider
Reading 0:00
15s threshold

You can now sort the providers behind a model by cost, time to first token (TTFT), or throughput (TPS) in AI Gateway . The default provider order blends provider reliability, quality of model output, cost, and speed of response. You can now use sort for explicit control over ranking criteria. For models with many providers and noticeable cost or speed variation, you can use sort to optimize on your dimension of choice. Ranking is computed at request time, so newly added providers, price changes, and shifts in observed latency or throughput flow through automatically without any code changes.…

Continue reading — create a free account

Join HashtagPLUS to read full articles, follow hashtags, vote, and join the conversation.

Read More