Improve surgical LLM context management in orchestration

Today's Docker report says that for 48% of organizations building agents, orchestration complexity is their #1 challenge. I've spent the last two years building orchestration for conversational agents, so this really resonated with me. Let me try to break it down. One thing I've found is that the core challenge isn't really about building a workflow graph. That part is usually straightforward. Where I've seen things get genuinely tricky is in managing the LLM's context surgically - controlling exactly what the model is aware of across every single invocation. Where the magic starts happening is when every invocation is "just right" - no more, no less - otherwise consistency starts falling apart. That's been our experience, at least. Let me try to make this concrete with an example. Say a customer sends a message complaining about a defective product, asking about returns, and sounding frustrated - all in one message. Now, you have 200 guidelines for the many different edge cases you've actually observed. Which of them should be get into the LLM's attention right now? All of them? But now multiply that by every turn. That's every slight shift of topic. A new compound question that partially references one from 3 turns ago. Every "actually, never mind, I want to ask about something else." This is totally normal for real-world interactions. But if you cram all 200 rules into the agent's context at every turn you get an agent in "shuffle" mode: - The agent becomes inconsistent, bypassing instructions on almost every turn. - Or it hyper-activates instructions out of context, trying to do everything at once. That's where the "mess" and "complexity" begin. Trying to root-cause that behavior, let alone debug and fix it so the agent responds consistently, can feel virtually impossible. This is the orchestration problem that caught me off guard early on, and from what I can tell, it catches a lot of teams off guard too - usually right when they hit real-world production requirements. LMK if your experience matches ours. 👍 https://lnkd.in/ezyT7rQp