Scalable voice agent design with Amazon Nova Sonic: multi-agent, instruments, and session segmentation

Design patterns for scalable voice brokers matter for organizations that have to ship quick, pure, and dependable voice experiences. Many groups face challenges like excessive latency, managing real-time audio, and coordinating a number of brokers in complicated workflows.

On this submit, you’ll discover ways to use Amazon Nova Sonic, Amazon Bedrock AgentCore, and Strands BidiAgent to construct scalable, maintainable voice brokers that deal with these challenges effectively, leading to extra responsive and clever buyer interactions.

We’ll discover three widespread architectural patterns for voice brokers, highlighting their trade-offs and finest practices for minimizing latency.

The constructing blocks

Earlier than diving deeper into the structure patterns, right here’s a fast overview of the three key elements used because the pattern resolution on this submit.

Amazon Nova Sonic is a basis mannequin that creates pure, human-like speech-to-speech conversations for generative AI functions. Customers can work together with AI by way of voice in actual time, with capabilities for understanding tone, pure conversational stream, and performing actions.

Amazon Bedrock AgentCore Runtime is a serverless internet hosting setting for AI brokers. You package deal your agent as a container, deploy to AgentCore Runtime, and it handles scaling, session isolation, and billing. For voice brokers, it supplies bidirectional WebSocket streaming with SigV4 auth, microVM-level session isolation to keep away from noisy-neighbor latency spikes, AgentCore Gateway for shared software internet hosting utilizing the Mannequin Context Protocol (MCP) open supply protocol, persistent reminiscence throughout periods, and telemetry for voice-specific metrics like time-to-first-audio.

Strands Brokers is an open supply framework for constructing AI brokers. Its BidiAgent class is one integration choice between Nova Sonic and your utility. It manages the bidirectional stream lifecycle, routes software calls, and handles session administration, simplifying the voice agent utility by way of the mannequin SDK interface.

Three integration patterns: software, agent-as-tool (sub-agent), and session segmentation

As an alternative of constructing one omnipotent agent, trendy voice techniques are more and more composed of tool-driven brokers, sub-agents appearing as instruments and session segmentation methods that isolate prompts, reminiscence, and permissions. These patterns enable groups to decompose giant assistants into smaller, specialised, and reusable elements whereas sustaining clear safety boundaries.

Earlier than working the samples within the following sections, set up Python and the required dependencies, together with strands-agents and boto3, and ensure your IAM setup has the mandatory permissions for the required companies. For the total instance, consult with the GitHub repository.

Sample 1: AgentCore Gateway – software choice for low latency

A software name is when a voice agent sends enter to an exterior perform or service, which processes it and returns output. It lets the agent carry out duties like querying a database or triggering a service shortly and securely, with out further reasoning steps.

With AgentCore Gateway, you expose your present enterprise logic as instruments, discrete capabilities that Nova Sonic can name straight throughout a dialog. The voice mannequin selects which software to invoke, passes parameters, will get a outcome, and speaks it again. There’s no intermediate reasoning layer between the mannequin and the software.

AgentCore Gateway hosts MCP servers as managed endpoints. MCP is the protocol, AgentCore Gateway is the AWS function that runs them. The voice agent connects through Gateway ARNs.

# Nova Sonic calls instruments straight through AgentCore Gateway
mannequin = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=[
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/auth-tools",
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/banking-tools",
        "arn:aws:bedrock-agentcore:us-east-1:123456789012:gateway/mortgage-tools",
    ],
)

When a consumer says “What’s my account stability?”, Nova Sonic:

1. Understands the intent from speech.
2. Selects get_account_balance from the accessible MCP instruments.
3. Calls the software with the fitting parameters.
4. Speaks the outcome again.

Commerce-off: Nova Sonic makes all the choices. If a software name requires multi-step validation, conditional logic, or chaining a number of operations collectively, that reasoning burden falls completely on the voice mannequin’s system immediate. For easy instruments that is advantageous. For complicated workflows, it will get brittle.

Sample 2: Sub-agent – further reasoning with decoupled brokers

With the sub-agent or agent-as-tool sample, your present enterprise logic runs in autonomous brokers, every with its personal mannequin, system immediate, instruments, and reasoning capabilities. The voice orchestrator delegates entire duties to those sub-agents as a substitute of calling particular person instruments.

There are numerous methods to connect with a sub-agent out of your voice agent. Agent-to-Agent (A2A) and Strands Agent-as-Software are two widespread approaches:

• Native agent-as-tool: The sub-agent runs in-process, wrapped as a @software perform utilizing the Brokers as Instruments sample in Strands. That is probably the most simple method with no community hop and no separate deployment. The trade-off is that the sub-agent shares the identical course of and scales with the orchestrator.
• Distant agent through A2A protocol: The sub-agent is deployed as an unbiased A2A server on AgentCore Runtime (or a distant server) and invoked over the community. A2A is an open protocol for agent-to-agent communication. As MCP connects brokers to instruments, A2A connects brokers to different brokers. Because the AWS weblog on A2A protocol help in AgentCore Runtime explains, brokers constructed with totally different frameworks (Strands, OpenAI, LangGraph, Google ADK) can share context and reasoning in a typical format. This supplies full deployment independence and cross-framework interoperability.

Strands Brokers has built-in help for each protocols, MCP for software entry and A2A for agent-to-agent communication. For a hands-on walkthrough, see the group information on Agent Collaboration: Strands Brokers, MCP, and the Agent2Agent Protocol.

Right here’s the native agent-as-tool method, every sub-agent is a @software wrapping a full Strands Agent:

# sub_agents.py — Outline sub-agents as Strands instruments utilizing the Brokers-as-Instruments sample
from strands import Agent, software
from strands.fashions import BedrockModel

# Every sub-agent is a full Strands Agent wrapped as a @software
# The BidiAgent orchestrator calls these through Nova Sonic's software use

@software
def authenticate_customer(account_id: str, date_of_birth: str) -> str:
    """Authenticate a buyer utilizing their account ID and date of start.
    Handles the total verification stream together with id checks and retry logic.
    Returns authentication standing and token."""
    auth_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're an authentication agent. Confirm the client's id
        utilizing the supplied account ID and date of start. Name verify_identity to test
        credentials. Return a transparent auth standing in 1-2 sentences.""",
        instruments=[verify_identity, check_account_exists],  # Sub-agent's personal instruments
    )
    outcome = auth_agent(f"Authenticate account {account_id}, DOB: {date_of_birth}")
    return str(outcome)


@software
def handle_banking_inquiry(question: str, auth_token: str) -> str:
    """Deal with banking questions — balances, transactions, transfers.
    Validates permissions and returns a conversational abstract."""
    banking_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're a banking agent. Use the supplied instruments to reply
        the client's question. Summarize ends in 2-3 pure sentences.
        Don't return uncooked JSON.""",
        instruments=[get_account_balance, get_recent_transactions, transfer_funds],
    )
    outcome = banking_agent(question)
    return str(outcome)


@software
def handle_mortgage_inquiry(question: str) -> str:
    """Deal with mortgage questions — charges, calculations, eligibility, utility standing.
    Performs its personal calculations and reasoning."""
    mortgage_agent = Agent(
        mannequin=BedrockModel(model_id="amazon.nova-lite-v1:0"),
        system_prompt="""You're a mortgage specialist. Assist with charge inquiries,
        fee calculations, and eligibility assessments. Maintain responses concise
        and conversational — this will likely be spoken aloud.""",
        instruments=[get_mortgage_rates, calculate_payment, check_eligibility],
    )
    outcome = mortgage_agent(question)
    return str(outcome)

The voice orchestrator then makes use of BidiAgent with these sub-agent instruments:

# voice_orchestrator.py — BidiAgent with sub-agents as instruments
from strands.experimental.bidi.agent import BidiAgent
from strands.experimental.bidi.fashions.nova_sonic import BidiNovaSonicModel
from sub_agents import authenticate_customer, handle_banking_inquiry, handle_mortgage_inquiry

mannequin = BidiNovaSonicModel(
    area="us-east-1",
    model_id="amazon.nova-2-sonic-v1:0",
    provider_config={"audio": {"voice": "tiffany", "input_sample_rate": 16000, "output_sample_rate": 16000}},
)

agent = BidiAgent(
    mannequin=mannequin,
    instruments=[authenticate_customer, handle_banking_inquiry, handle_mortgage_inquiry],
    system_prompt="""You're a banking voice assistant. Route buyer requests to the
    applicable specialist. All the time authenticate earlier than accessing account knowledge.
    Maintain your individual responses transient — the sub-agents deal with the small print.""",
)

await agent.run(inputs=[ws_input], outputs=[ws_output])

The sub-agent does its personal considering. Nova Sonic doesn’t have to orchestrate the person steps. It delegates and speaks the outcome.

Commerce-off: Every sub-agent name provides latency: the sub-agent’s personal mannequin inference plus its software calls. In a voice dialog, this implies longer silence whereas the sub-agent causes. The AWS weblog on multi-agent voice assistants recommends beginning with smaller, environment friendly fashions like Amazon Nova 2 Lite for sub-agents to cut back latency whereas nonetheless dealing with specialised duties successfully.

Amazon Nova 2 Sonic helps asynchronous software calling, so the dialog continues naturally whereas instruments run within the background. It retains accepting enter, can run a number of instruments in parallel, and gracefully adapts if the consumer modifications their request mid-process, delivering all outcomes whereas specializing in what’s nonetheless related.

Sample 3: Session segmentation for ultra-low latency

There’s a 3rd method value contemplating. It doesn’t map neatly to the MCP or sub-agent patterns, however is purpose-built for voice situations the place latency is the overriding concern.

As an alternative of delegating exterior instruments or sub-agents, you section the dialog into logical phases, every with its personal Nova Sonic session, system immediate, and gear set. When the dialog transitions from one section to the following (for instance, from authentication to account inquiry), you shut the present session and open a brand new one with a distinct immediate and instruments, throughout the similar WebSocket connection. Every sub-voice-agent can use its personal MCP gateways, instruments, and even sub-agents — the variations that it operates with a centered immediate and minimal software floor, lowering reasoning overhead and latency.

Consider a banking voice assistant with three dialog phases: authentication, account administration, and mortgage inquiry. Reasonably than loading one large system immediate with each software, you run every section as a centered Nova Sonic session:

# Part 1: Authentication
auth_session = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=["arn:...gateway/auth-tools"],  # Solely auth instruments
)
auth_agent = BidiAgent(
    mannequin=auth_session,
    instruments=[],
    system_prompt="""You're an authentication assistant. 
    Accumulate the consumer's account ID and date of start. 
    Name verify_identity to authenticate. 
    As soon as verified, say 'You are all set' and cease.""",
)
# Run till authentication completes
await auth_agent.run(inputs=[ws_input], outputs=[ws_output])

# Part 2: Account administration (new session, new immediate, new instruments)
banking_session = BidiNovaSonicModel(
    model_id="amazon.nova-2-sonic-v1:0",
    mcp_gateway_arn=["arn:...gateway/banking-tools"],  # Solely banking instruments
)
banking_agent = BidiAgent(
    mannequin=banking_session,
    instruments=[],
    system_prompt="""You're a banking assistant. The consumer is already authenticated.
    Assist with stability inquiries, transactions, and transfers.
    Maintain responses to at least one or two sentences.""",
)
await banking_agent.run(inputs=[ws_input], outputs=[ws_output])

Every section will get a clear Nova Sonic session with:

• A centered system immediate: Shorter, extra particular, much less room for the mannequin to get confused.
• Solely the related instruments: through MCP gateways, native instruments, or each. The mannequin doesn’t waste reasoning cycles selecting between 15 instruments when it solely wants 3.
• Optionally its personal sub-agents: a section that requires deeper reasoning can use Sample 2 internally, whereas less complicated phases keep tool-only.
• The earlier session context might be handed into the brand new session as chat historical past, so the general dialog retains continuity.

In comparison with software, sub-agent, and session segmentation patterns

Latency finest practices for voice brokers

Latency is a key consideration when constructing voice versus textual content brokers. Listed below are sensible strategies to maintain response instances quick and responsive:

Begin with small fashions for sub-agents. Your voice orchestrator makes use of Nova Sonic for the dialog, however sub-agents don’t want a big mannequin. Begin with Amazon Nova 2 Lite or Nova 2 Micro. They’re quick, value optimized, and deal with most specialised duties effectively. You’ll be able to at all times improve a particular sub-agent to a bigger mannequin if high quality requires it, however default to small.

Design stateful sub-agents with caching. A stateless sub-agent that hits a database or API on each name provides latency each time. As an alternative, design sub-agents to cache outcomes from knowledge sources (APIs, AWS Lambda capabilities, databases) inside a session. If the banking sub-agent fetches account particulars as soon as, it ought to maintain that knowledge in reminiscence and serve subsequent questions (stability, transactions, abstract) from cache reasonably than making repeated backend calls.

Prefetch knowledge after authentication. That is particularly helpful for contact heart situations. After a buyer authenticates, you already know who they’re. Don’t look ahead to them to ask earlier than pulling their knowledge. Instantly fetch account balances, current transactions, pending alerts, and mortgage standing within the background. When the client asks “What’s my stability?”, the reply is already in reminiscence.

Parallelize unbiased software calls. If the consumer asks “Give me an summary of my accounts”, don’t name get_checking_balance, then get_savings_balance, then get_credit_card_balance sequentially. Use concurrent execution so three calls occur directly. Strands helps this natively. The agent’s software executor runs unbiased calls in parallel by default.

Use filler phrases to masks software latency. When a software name or sub-agent delegation is unavoidable, instruct the voice mannequin to talk a short filler whereas ready: “Let me test that for you…” or “One second whereas I look that up…” This retains the dialog feeling alive as a substitute of dropping into silence.

Reduce software rely per session. Software choice will get slower because the variety of accessible instruments grows. In case your agent has 15 instruments however a typical dialog solely makes use of 3 to 4, think about the session segmentation sample to load solely the related instruments per section.

Clear up

After you end testing the pattern, keep in mind to scrub up the sources you created to keep away from pointless prices. Observe the repository directions to cease companies and delete any deployed infrastructure.

Conclusion

Migrating a textual content chatbot to a voice assistant isn’t a simple wrapper job. The interplay mannequin is basically totally different, from response design to latency budgets to turn-taking conduct. However with a well-structured multi-agent structure and Amazon Bedrock AgentCore, the enterprise logic layer stays intact.

The sub-agents you’ve already constructed are your greatest asset. Reuse them.

For a working instance of a Strands BidiAgent voice assistant deployed on AgentCore Runtime with WebSocket streaming, see the AgentCore bidirectional streaming pattern.

Subsequent steps

Subsequent, you possibly can lengthen the pattern to suit your personal use case, combine your corporation instruments, refine prompts for voice interactions, and check the agent in real-world situations to arrange for manufacturing deployment. To be taught extra about voice brokers on AWS, go to:

In regards to the authors