How to use it
- Pick a model, then enter how many items the job has and the average input and output tokens per item.
- Set the batch discount your provider offers. 50% is common for major providers.
- Add the share of input that is identical for every item, and a retry allowance.
- Compare the standard cost with the batch cost and the saving.
What a batch job is
A batch job sends many independent requests that do not need an instant answer: tagging product listings, translating a catalogue, extracting fields from invoices, scoring sales leads or summarising call transcripts. Because nobody is waiting on each result, providers let you submit the whole set at once and return results within a set period, often up to 24 hours. In exchange, they charge less per token.
How the calculation works
The calculator multiplies items by average tokens to get the total input and output. Any input you mark as shared, such as the same instructions sent with every item, is billed at the cached rate when the model has one. The retry allowance adds extra requests for items that fail or need a second attempt. The batch discount is then applied to the total. The standard figure shows the same job sent as normal real-time requests.
Worked example
A real estate agency wants to classify 80,000 old enquiries by property type, budget and urgency. Each request sends 1,500 tokens: 1,000 tokens of instructions and examples, plus a 500-token enquiry. Each reply is a 60-token JSON object. On a model priced at $1 input and $5 output, the standard cost is about $120 input plus $24 output, $144 in total. With a 50 percent batch discount it drops to about $72, and caching the shared instructions brings it lower still.
Tips for cheaper, more reliable batch jobs
- Ask for structured output. A short JSON reply is cheaper than prose and easier to load into a spreadsheet. Check it with the JSON validator.
- Use a smaller model first. Classification and extraction often work well on budget models. Try one on 200 items and spot-check the results.
- Put shared text first. Caching only works on the beginning of a prompt, so place fixed instructions before the part that changes.
- Plan for failures. Some items will come back malformed or empty. A 2 to 5 percent retry allowance is a sensible starting point.
Limits of this estimate
Batch discounts, whether they stack with caching, and maximum batch sizes differ by provider and change over time. Some providers do not offer a batch mode for every model. Confirm the current terms in your provider's documentation, then run a small pilot and scale the real figure.
Questions
How much do batch APIs save?
Major providers such as OpenAI, Anthropic and Google have offered around 50% off standard prices for batch jobs. Terms change, so check your provider's current batch documentation and enter the discount you actually get.
How long do batch jobs take?
Providers typically promise results within 24 hours, and many jobs finish much sooner. Use real-time requests when a person is waiting for the answer.
Does caching combine with the batch discount?
With some providers it does, with others it does not. This calculator applies both when you enter both, so set the shared share to zero if your provider does not combine them.
What is the retry allowance for?
Some requests fail, time out or return output you cannot use, and you pay for those attempts too. The allowance adds that share of extra requests to the total.