Implement caching for tool calls

I know a lot of AI Agent Frameworks. I don't know many Agents in production. Why?? Because building frameworks is easy. (hot take :D) Peel back the abstraction, look at the code, and it’s always the same pattern. I don’t think it has a name yet. For now it’s the “double loop”. If you know a better name, please comment! Here’s how it works: >> The outer loop is the “turn loop”. The turn loop produces a single Turn (duh!). It cycles through three steps: get input from the user, enrich it with context, roll out the agent. >> The inner loop is the “trace loop”. The trace loop produces Traces. It cycles through two steps: get output from the LLM, call the requested tool. Stack both loops, wrap them as OSS, season with some docs. Congratulations. You’ve built your own AI agent framework. Except I don’t think you should; we already have too many of those. Instead, I would use this pattern for following: >> Choosing the right framework for my next project. The double-loop is the same across frameworks. Context engineering is not. That’s where frameworks differ. Do we query the RAG database after each user input (dynamic context) or as a tool? Is memory injected at the beginning, or loaded dynamically? What comes out-of-the-box? What do I need to configure myself? You will very quickly know if a framework fits your needs. >> Improve the robustness of an agent. Check your app logs. Can you rebuild Traces? Do you capture full Turns? If every trace calls the same tool, that tool should really be dynamic context. How many Traces fail if you re-run the same Turn 1000 times? A pattern will emerge. Fix it and robustness goes up. Building around this double-loop gives you a fast feedback loop. I’m just scratching the surface, but this post is already too long. If you’re interested in the full-length talk please find me at PyData Boston 2025. I would love to chat!