My LLM Bill Dropped 68% When I Stopped Doing These Four Things

Three months ago my API bill for a side project hit $310 in a month. Revenue was around $500. That ratio was not going to work, so I spent a week digging into where the tokens were actually going. After four changes, the bill settled at about $100 a month for the same traffic. Here is what I stopped doing.

1. Sending the whole history every time

My chat feature resent the full conversation on every turn. A twenty-message thread meant the model was reading the first message twenty times. I switched to keeping the last six turns verbatim and replacing older turns with a short running summary.

context = summary_of_older_turns + last_6_turns + new_message

Average input tokens per request dropped by roughly half. Users did not notice, because almost nobody refers back to message three in a twenty-message conversation.

2. Using the biggest model for everything

I had defaulted to the most capable model for every task, including trivial ones like classifying whether a message was a question or a complaint. I split my calls into tiers:

• Small, fast model: classification, extraction, formatting.
• Mid-tier model: most summaries and replies.
• Top model: only the rare complex requests, routed by a simple rule.

About 60 percent of my calls turned out to be small-model work. That alone was the biggest saving.

3. Ignoring caching

My system prompt was around 1,800 tokens: instructions, examples, tone guidelines. It was identical on every call. Once I restructured requests so the static prefix came first and enabled prompt caching, those repeated tokens became much cheaper.

I also added a plain response cache for identical inputs. In my case, about 9 percent of summary requests were exact repeats, usually someone refreshing a page.

4. Writing prompts like essays

My original prompt had paragraphs of polite explanation. I rewrote it as short, direct rules plus two examples instead of five. It went from 1,800 to about 900 tokens and, surprisingly, outputs got slightly more consistent.

How I measured before changing anything

Before touching code, I logged every call for a week with these fields:

• Feature name
• Model used
• Input and output tokens
• Latency

Then I simply sorted by total cost per feature. One feature I barely thought about was responsible for 41 percent of spend. Without the log, I would have optimised the wrong thing.

The takeaway

Most AI cost problems are not model problems. They are plumbing problems.

If your bill is creeping up, do not start by switching providers. Start by logging, then look for repeated context, oversized models on small tasks, and prompts that could be half as long. It is boring work. It also paid for my hosting for the next year.

Cover: Photo by Duskfall Crew on Unsplash.