As your AI brokers transfer from prototype to manufacturing, the challenges shift from getting them to work to holding them quick and environment friendly. In Half 1 of this sequence, we walked via debugging two frequent agent failures: infinite loops and gear invocation errors. These situations handled brokers that have been damaged. On this put up, we sort out a unique problem: brokers that work appropriately however carry out poorly. Gradual response occasions and unbounded reminiscence progress are the commonest operational points that floor after you resolve the preliminary debugging issues. They don’t set off error alerts, however they erode consumer belief and improve prices over time.
Utilizing AgentCore Observability, a functionality of Amazon Bedrock AgentCore, and Amazon CloudWatch, you’ll discover ways to determine efficiency bottlenecks throughout your agent’s execution path and diagnose reminiscence points in long-running classes. Additionally, you will implement monitoring practices that catch degradation earlier than customers discover it. For extra data and greatest practices, overview AgentCore Evaluations, a functionality of Amazon Bedrock AgentCore, and AgentCore Insights.
You want an AWS account with Amazon Bedrock AgentCore entry, CloudWatch Transaction Search enabled, and a deployed agent. See Half 1 for full setup particulars.
State of affairs 3: Efficiency bottlenecks
Brokers expertise efficiency bottlenecks once they work appropriately however reply too slowly. You count on sub-second responses however expertise multi-second delays. Brokers full duties efficiently, however the latency makes them impractical for interactive use circumstances. This situation is especially difficult as a result of sluggish is subjective. What’s acceptable for a batch processing agent is unacceptable for a customer support chatbot. You have to set up efficiency budgets in your particular use case, then systematically determine which parts violate these budgets.
Signs to observe for
Efficiency degradation usually manifests step by step. Brokers would possibly begin with acceptable 2-second response occasions, however as you add options, combine extra instruments, or accumulate extra reminiscence, latency creeps to five seconds, then 10, then turns into unusable. P95 response occasions exceed your thresholds, customers abandon classes, however error charges keep low. The agent works appropriately however responds too slowly.
Determine 1 — Session particulars exhibiting a number of traces with persistently excessive latency. The three invocations took 7.5-8.2 seconds (common span latency), demonstrating a systemic efficiency bottleneck moderately than occasional slowness. This sample signifies the agent’s structure wants optimization.
To seek out bottlenecks, begin by figuring out high-latency requests. Question CloudWatch for agent invocations that exceed your efficiency price range:
This question returns agent invocations that took longer than 3 seconds (modify the brink primarily based in your necessities), sorted by latency. Decide a consultant high-latency request and word its RequestId.
Subsequent, analyze the request timeline to grasp the place time is spent:
The question reveals you the sequence of operations inside the request and the way lengthy every took. Search for operations that devour disproportionate time. Frequent culprits embrace reminiscence retrieval operations, software invocations, token technology, and sequential operations that would run in parallel.
Determine 2 — OpenTelemetry hint timeline exhibiting 17 spans throughout three sequential execute_event_loop_cycle operations. The instruments (customer_lookup, order_history) execute one after one other moderately than in parallel, with every cycle ready for the earlier one to finish. This sequential sample compounds latency throughout every software invocation.
Examine reminiscence retrieval latency particularly:
Reminiscence retrieval ought to full in beneath 200 milliseconds, representing the purpose the place customers start perceiving noticeable delays in interactive functions. Greater latency suggests inefficient reminiscence group.
Look at software invocation latency to determine sluggish integrations:
The question identifies which instruments contribute most to general latency. A single sluggish software can bottleneck your entire agent workflow.
Root trigger evaluation
Efficiency bottlenecks usually stem from three root causes. Gradual software execution happens when exterior instruments take seconds to reply due to poor optimization, overload, or community points. Latencies compound with sequential calls, so a 2-second software referred to as 3 times turns into a 6-second bottleneck. Extreme token technology is an element as a result of basis fashions (FMs) produce tokens sequentially, which means a 500-token response takes 5x longer than a 100-token one, impacting each latency and price. Lastly, sequential processing, performing unbiased operations one by one as an alternative of in parallel, will increase each price and compute occasions.
The repair
For sluggish software execution, implement caching, connection pooling, and correct database indexing to scale back response occasions. Set timeout limits and think about quicker software alternate options. If a software persistently lags, profile it independently. The difficulty may be community latency, chilly begins, or useful resource competition moderately than the software’s logic itself.
For reminiscence retrieval, substitute single giant namespaces with topic-specific partitions, corresponding to preferences, historical past, and area information, to scale back search area. Summarize previous conversations into compact entries moderately than storing them verbatim, and set dimension limits per namespace, corresponding to 100 preferences, 50 current messages, or 500 area information.
For token technology, optimize prompts to encourage temporary, direct solutions of two–3 sentences except extra element is requested. Add express size constraints and monitor token utilization with alerts for unexpectedly lengthy responses.
For sequential processing, run unbiased software calls in parallel. Sequential calls totaling 4.5s (2s + 1.5s + 1s) drop to solely 2s when parallelized, usually slicing latency by 50 % or extra with minimal effort. To confirm your optimizations, re-run the latency question from the figuring out bottlenecks part. Affirm that P95 response occasions now fall inside your efficiency price range and that the hint timeline reveals parallel execution the place anticipated.
State of affairs 4: Reminiscence points in long-running classes
Brokers expertise reminiscence points in long-running classes once they keep classes the place reminiscence utilization grows unbounded. Brokers accumulate context, and finally, brokers hit token limits, lose essential context, or exhaust accessible reminiscence. Classes fail unexpectedly, and also you lose dialog state. This situation is especially problematic for brokers that help prolonged workflows, corresponding to customer support classes, analysis assistants, or monitoring brokers. With out correct reminiscence administration, these use circumstances develop into impractical.
Signs to observe for
When abnormally lengthy agent classes exist, token utilization grows linearly with session period. Reminiscence retrieval latency will increase over time as reminiscence shops develop. Classes terminate unexpectedly with out-of-memory errors or context window exceeded errors.
Determine 3 — Session particulars for the memorygrowth_Agent exhibiting 6 traces, 15.7K complete tokens consumed, and a median hint latency of three,757 ms inside a single session. Token utilization grows with every successive invocation, demonstrating unbounded context accumulation. In manufacturing classes spanning hours, this sample results in context window exhaustion and sudden session failures.
To determine long-running classes with excessive reminiscence utilization:
The question returns classes lasting longer than one hour (3600 seconds), sorted by reminiscence dimension. Decide a session with unusually excessive reminiscence utilization and word its SessionId.
Look at reminiscence extraction patterns to confirm consolidation is happening:
Determine 4 — CloudWatch Logs Insights exhibiting 209 memory-related log entries concentrated round 18:20. The spike in reminiscence operations signifies the agent storing data with out consolidation. Every invocation provides new reminiscence entries (via add_conversation_note and add_user_context instruments) with out pruning or summarizing previous information, demonstrating the unbounded progress sample.
Reminiscence extraction ought to happen usually all through the session. In the event you see gaps the place no extraction occurs for prolonged intervals, the agent isn’t consolidating recollections correctly.
Examine for reminiscence extraction failures:
Failed reminiscence extraction stops brokers from consolidating context, inflicting unbounded progress. Frequent failure causes embrace token limits exceeded throughout summarization, invalid reminiscence codecs that may’t be processed, community timeouts when writing to reminiscence storage, and permission errors that block reminiscence updates.
Analyze reminiscence namespace group:
Recollections are distributed throughout namespaces. Poor namespace group can result in inefficient reminiscence retrieval and consolidation. In the event you see a single namespace containing hundreds of recollections, that’s a crimson flag indicating your agent wants higher reminiscence group.
Root trigger evaluation
Reminiscence points usually stem from misconfigured settings, filter on inbuilt datetime metadata of the report. For a deeper understanding of how AgentCore reminiscence works, see AgentCore reminiscence.
The repair
To resolve reminiscence points, confirm your reminiscence methods embrace a consolidation configuration so AgentCore reminiscence merges and summarizes information over time moderately than accumulating them indefinitely. Set up information utilizing namespace templates in your technique definitions to ensure retrieval stays scoped to related context. Set eventExpiryDuration to manage how lengthy uncooked occasions persist (between 7–one year). For implementation particulars, see AgentCore reminiscence implementation.
Having coated the 4 failure situations throughout each components, we now flip to manufacturing greatest practices that cease these points earlier than they happen. The troubleshooting workflows we’ve coated enable you reply when issues go unsuitable, however a well-architected observability technique helps you proactively catch points. The mixing of AgentCore with CloudWatch supplies the inspiration for this proactive strategy, supplying you with real-time visibility into agent well being and efficiency.
Activate complete instrumentation for manufacturing brokers, reminiscence programs, and gateways. Configure CloudWatch logs, CloudWatch metrics, and OpenTelemetry traces.
Configure CloudWatch alarms for essential metrics. Set thresholds for error charges (5 %), ninety fifth percentile (P95) latency (3 seconds), and token utilization per session. Don’t anticipate customers to report issues. Let CloudWatch provide you with a warning when metrics exceed acceptable thresholds.
Create operational dashboards. Construct a main dashboard exhibiting complete invocations (final 24 hours), error charge (present in comparison with baseline), P50/P95/P99 latency, token utilization tendencies, and energetic classes depend. Create per-agent dashboards exhibiting agent-specific invocation patterns, software utilization breakdown, reminiscence consumption tendencies, error varieties distribution, and price per session. Assessment these dashboards day by day to identify tendencies earlier than they develop into issues.
Spend money on observability infrastructure earlier than you want it. Construct monitoring into your improvement course of from day one. Share dashboards with product managers and stakeholders so everybody understands agent efficiency. When somebody diagnoses a tough manufacturing concern, share the strategy with the group to construct institutional information about frequent failure patterns.
To watch software accuracy at scale, you should utilize Amazon Bedrock AgentCore Evaluators to repeatedly and routinely assess agent conduct. As a substitute of manually reviewing traces after failures happen, Evaluators examine agent classes in actual time, scoring them towards predefined high quality standards. Amazon Bedrock AgentCore Insights (preview) builds off Evaluators and supplies triage evaluation. Insights can present failure evaluation, consumer intent extraction, and execution abstract.
After testing the optimization strategies on this put up, clear up assets to keep away from pointless prices.
For CloudWatch assets, delete take a look at CloudWatch dashboards created throughout debugging, take away CloudWatch alarms arrange for testing functions, and think about archiving or deleting previous CloudWatch log teams if now not wanted.
For AgentCore assets, should you created take a look at brokers particularly for this tutorial, delete them via the AgentCore console. Take away any take a look at reminiscence namespaces created throughout reminiscence optimization testing. Delete any non permanent software integrations used for efficiency testing.
For price optimization, overview your Amazon CloudWatch Logs retention settings and modify primarily based in your compliance necessities. Think about using CloudWatch Logs information safety to chop storage prices for older logs.
To delete CloudWatch log teams:
To take away CloudWatch alarms:
Efficiency bottlenecks and reminiscence points symbolize the commonest operational challenges for manufacturing brokers past the debugging situations coated in Half 1. With systematic prognosis utilizing CloudWatch traces and metrics, you’ll be able to determine whether or not latency stems from sluggish instruments, inefficient reminiscence retrieval, extreme token technology, or sequential processing. For long-running classes, monitoring reminiscence progress patterns and implementing consolidation methods retains brokers secure over prolonged interactions.
Subsequent steps
Able to put these strategies into follow? Begin by turning on AgentCore Observability in your manufacturing brokers should you haven’t already. This one-time setup supplies the inspiration for all the pieces we’ve coated. Arrange CloudWatch alarms for essential metrics. Create troubleshooting runbooks in your particular brokers and workflows, documenting the queries you run, the thresholds that point out issues, and the fixes you implement.
Apply debugging in non-production environments. Intentionally introduce failures and follow diagnosing them utilizing the workflows we’ve coated. Construct muscle reminiscence for troubleshooting earlier than you want it in manufacturing. Share this data together with your group to assist affirm everybody who operates your brokers understands these debugging strategies and is aware of the way to use the observability options of AgentCore.
Manufacturing brokers usually fail for preventable causes. With the best observability instruments, systematic troubleshooting approaches, and a tradition that treats each concern as an opportunity to enhance, you’ll be able to catch issues early, resolve them rapidly, and construct brokers that earn lasting belief.
Study extra
For extra details about AgentCore and observability options, go to the AgentCore documentation. To get began with AgentCore, go to the AgentCore console. For extra CloudWatch monitoring greatest practices, see the CloudWatch Person Information.
In regards to the authors
