AI Cost Optimization Workflow: Cache, Route, and Batch

1/ Cache the part of your prompt that never changes. If you send the same system prompt or knowledge base on every call, you pay full price to re-read text that never moved. Anthropic lets you cache that stable prefix. Cache reads cost 10% of normal input. That's 90% off the repeating part of every call. 2/ But caching only works if the prefix is identical. Change one word, even a single timestamp, and you pay full price again. Fix: put the static stuff first (instructions, tools, docs), and the changing stuff last. Order matters more than people think. 3/ Stop resending the entire conversation. Most apps resend the whole chat history every turn. Your cost climbs with every message. Summarize old turns into a short running summary. Keep only the last few messages word for word. A 40-message thread can run on 5 lines plus the last 3 turns. 4/ Retrieve, don't stuff. Pasting your whole knowledge base 'just in case' is how bills explode. Pull only the 3 to 5 chunks that actually match the question. You go from 50,000 tokens of context to about 2,000, and the answers get sharper. 5/ Use the smallest model that can do the job. Most requests don't need your biggest model. Send classification, routing, and short replies to a cheaper model like Haiku. Save the expensive model for the hard 10%. 6/ Cap the output. Output tokens usually cost more than input. Set max_tokens and ask for the exact format you want. 'Answer in one sentence' is a cost lever, not just a style choice. 7/ Cache the answer, not just the prompt. If 500 users ask the same FAQ, you don't need 500 model calls. Store common answers and serve them straight from your own cache. The cheapest token is the one you never send. 8/ Batch everything that isn't real-time. Nightly reports, tagging a backlog, evals, model migrations. None of it needs an instant reply. Anthropic's Batch API is 50% off input and output, and most batches finish in under an hour. Stack it with prompt caching and input savings can reach around 95%. 9/ Measure before you optimize. Count tokens before you send. Log usage per feature. You'll almost always find one bloated prompt eating most of the bill. Kill that one first. Closer: Unlimited access was never the point. Getting the same result for a tenth of the tokens is. That's the whole game.