AI Dev Notes

You ran a few tests, maybe built a small prototype, and then your OpenAI or Anthropic bill arrived — and it was way higher than you expected. Sound familiar? Understanding exactly how tokens are counted, how costs are calculated, and where the hidden waste lives is the single most impactful skill for anyone building with LLM APIs in 2026. This post walks you through the mechanics of tokenization, shows you how to measure usage precisely, and gives you battle-tested strategies to cut costs without sacrificing response quality. Table of Contents What Is a Token, Really? How Costs Are Calculated Counting Tokens Before You Send Where Tokens Hide: The Usual Suspects Strategies to Reduce Token Usage Prompt Compression Techniques Caching and Batching Monitoring and Budgeting in Production Putting It All Together 🔤 What Is a Token, Really? A token is not a word, and it is not a character — it sits somewhere in betw...

A Telegram bot that forgets every message the moment it replies is barely more useful than a search bar. If you're forwarding user messages to OpenAI's Chat Completions API, you need to maintain a per-user message history array — otherwise the model has zero context, and multi-turn conversations are impossible. This is a common gap in beginner implementations, and fixing it cleanly requires understanding both where to store state and how to structure the messages payload. 🧠 Why the Bot Forgets OpenAI's Chat Completions API is stateless. Every request you send must include the full conversation history in the messages array. If you only send the latest user message, the model treats it as the first message in a brand-new conversation. Your bot isn't broken — it's just not accumulating history before each API call. The fix has two parts: Maintain an in-memory (or persistent) list of message dicts per user Appen...