Superior fine-tuning strategies for multi-agent orchestration: Patterns from Amazon at scale

Our work with giant enterprise clients and Amazon groups has revealed that top stakes use circumstances proceed to learn considerably from superior giant language mannequin (LLM) fine-tuning and post-training strategies. On this put up, we present you ways fine-tuning enabled a 33% discount in harmful treatment errors (Amazon Pharmacy), engineering 80% human effort discount (Amazon World Engineering Providers), and content material high quality assessments bettering 77% to 96% accuracy (Amazon A+). These aren’t hypothetical projections—they’re manufacturing outcomes from Amazon groups. Whereas many use circumstances could be successfully addressed by way of immediate engineering, Retrieval Augmented Era (RAG) techniques, and switch key agent deployment,, our work with Amazon and enormous enterprise accounts reveals a constant sample: One in 4 high-stakes purposes—the place affected person security, operational effectivity, or buyer belief are on the road—demand superior fine-tuning and post-training strategies to attain production-grade efficiency.

This put up particulars the strategies behind these outcomes: from foundational strategies like Supervised High quality-Tuning (SFT) (instruction tuning), and Proximal Coverage Optimization (PPO), to Direct Choice Optimization (DPO) for human alignment, to cutting-edge reasoning optimizations resembling Grouped-based Reinforcement Studying from Coverage Optimization (GRPO), Direct Benefit Coverage Optimization (DAPO), and Group Sequence Coverage Optimization (GSPO) purpose-built for agentic techniques. We stroll by way of the technical evolution of every method, study real-world implementations at Amazon, current a reference structure on Amazon Net Providers (AWS), and supply a choice framework for choosing the correct approach primarily based in your use case necessities.

The continued relevance of fine-tuning within the agentic AI

Regardless of the rising capabilities of basis fashions and agent frameworks, roughly one in every of 4 enterprise use circumstances nonetheless require superior fine-tuning to attain the mandatory efficiency ranges. These are sometimes situations the place the stakes are excessive from income or buyer belief views, domain-specific data is crucial, enterprise integration at scale is required, governance and management are paramount, enterprise course of integration is complicated, or multi-modal help is required. Organizations pursuing these use circumstances have reported larger conversion to manufacturing, better return on funding (ROI), and as much as 3-fold year-over-year development when superior fine-tuning is appropriately utilized.

Evolution of LLM fine-tuning strategies for agentic AI

The evolution of generative AI has seen a number of key developments in mannequin customization and efficiency optimization strategies. Beginning with SFT, which makes use of labeled information to show fashions to comply with particular directions, the sphere established its basis however confronted limitations in optimizing complicated reasoning. To deal with these limitations, reinforcement studying (RL) refines the SFT course of with a reward-based system that gives higher adaptability and alignment with human choice. Amongst a number of RL algorithms, a big leap comes with PPO, which consists of a workflow with a price (critic) community and a coverage community. The workflow accommodates a reinforcement studying coverage to regulate the LLM weights primarily based on the steerage of a reward mannequin. PPO scales effectively in complicated environments, although it has challenges with stability and configuration complexity.

DPO emerged as a breakthrough in early 2024, addressing PPO’s stability points by eliminating the express reward mannequin and as an alternative working instantly with choice information that features most well-liked and rejected responses for given prompts. DPO optimizes the LLM weights by evaluating the popular and rejected responses, permitting the LLM to study and regulate its conduct accordingly. This simplified method gained widespread adoption, with main language fashions incorporating DPO into their coaching pipelines to attain higher efficiency and extra dependable outputs. Different alternate options together with Odds Ratio Coverage Optimization (ORPO), Relative Choice Optimization (RPO), Id choice optimization (IPO), Kahneman-Tversky Optimization (KTO), they’re all RL strategies for human choice alignment. By incorporating comparative and identity-based choice constructions, and grounding optimization in behavioral economics, these strategies are computationally environment friendly, interpretable, and aligned with precise human decision-making processes.

As agent-based purposes gained prominence in 2025, we noticed rising calls for for customizing the reasoning mannequin in brokers, to encode domain-specific constraints, security tips, and reasoning patterns that align with brokers’ supposed features (process planning, software use, or multi-step drawback fixing). The target is to enhance brokers’ efficiency in sustaining coherent plans, avoiding logical contradictions, and making acceptable choices for the area particular use circumstances. To fulfill these wants, GRPO was launched to boost reasoning capabilities and have become notably notable for its implementation in DeepSeek-V1.

The core innovation of GRPO lies in its group-based comparability method: somewhat than evaluating particular person responses in opposition to a set reference, GRPO generates teams of responses and evaluates every in opposition to the typical rating of the group, rewarding these performing above common whereas penalizing these beneath. This relative comparability mechanism creates a aggressive dynamic that encourages the mannequin to provide higher-quality reasoning. GRPO is especially efficient for bettering chain-of-thought (CoT) reasoning, which is the important basis for agent planning and complicated process decomposition. By optimizing on the group stage, GRPO captures the inherent variability in reasoning processes and trains the mannequin to persistently outperform its personal common efficiency.

Some complicated agent duties would possibly require extra fine-grained and crisp corrections inside lengthy reasoning chains, DAPO addresses these use circumstances by constructing upon GRPO sequence-level rewards, using a better clip ratio (roughly 30% larger than GRPO) to encourage extra various and exploratory pondering processes, implementing dynamic sampling to get rid of much less significant samples and enhance general coaching effectivity, making use of token-level coverage gradient loss to supply extra granular suggestions on prolonged reasoning chains somewhat than treating total sequences as monolithic items, and incorporating overlong reward shaping to discourage excessively verbose responses that waste computational sources. Moreover, when the agentic use circumstances require lengthy textual content outputs within the Combination-of-Specialists (MoE) mannequin coaching, GSPO helps these situations by shifting the optimization from GRPO’s token-level significance weights to the sequence stage. With these enhancements, the brand new strategies (DAPO and GSPO) allow extra environment friendly and complicated agent reasoning and planning technique, whereas sustaining computational effectivity and acceptable suggestions decision of GRPO.

Actual-world purposes at Amazon

Utilizing the fine-tuning strategies described within the earlier sections, the post-trained LLMs play two essential roles in agentic AI techniques. First is within the improvement of specialised tool-using elements and sub-agents inside the broader agent structure. These fine-tuned fashions act as area consultants, every optimized for particular features. By incorporating domain-specific data and constraints throughout the fine-tuning course of, these specialised elements can obtain considerably larger accuracy and reliability of their designated duties in comparison with general-purpose fashions. The second key utility is to function the core reasoning engine, the place the muse fashions are particularly tuned to excel at planning, logical reasoning, and decision-making, for brokers in a extremely particular area. The intention is to enhance the mannequin’s skill to keep up coherent plans and make logically sound choices—important capabilities for any agent system. This twin method, combining a fine-tuned reasoning core with specialised sub-components, was rising as a promising structure in Amazon for evolving from LLM-driven purposes to agentic techniques, and constructing extra succesful and dependable generative AI purposes. The next desk depicts multi-agent AI orchestration with of superior fine-tuning approach examples.

Amazon Healthcare Providers (AHS) started its journey with generative AI with a big problem two years in the past, when the crew tackled customer support effectivity by way of a RAG-based Q&A system. Preliminary makes an attempt utilizing conventional RAG with basis fashions yielded disappointing outcomes, with accuracy hovering between 60 and 70%. The breakthrough got here once they fine-tuned the embedding mannequin particularly for pharmaceutical area data, resulted in a big enchancment to 90% accuracy and an 11% discount in buyer help contacts. In treatment security, treatment path errors can pose severe security dangers and value as much as $3.5 billion yearly to right. By fine-tuning a mannequin with 1000’s of expert-annotated examples, Amazon Pharmacy created an agent part that validates treatment instructions utilizing pharmacy logic and security tips. This diminished near-miss occasions by 33%, as indicated of their Nature Drugs publication. In 2025, AHS is increasing their AI capabilities and rework these separate LLM-driven purposes right into a holistic multi-agent system to boost affected person expertise. These particular person purposes pushed by fine-tuned fashions play a vital function within the general agentic structure, serving as area skilled instruments to handle particular mission-critical features in pharmaceutical companies.

The Amazon World Engineering Providers (GES) crew, liable for overseeing tons of of Amazon achievement facilities worldwide, launched into an formidable journey to make use of generative AI of their operations. Their preliminary foray into this know-how targeted on creating a complicated Q&A system designed to help engineers in effectively accessing related design info from huge data repositories. The crew’s method was fine-tuning a basis mannequin utilizing SFT, which resulted in a big enchancment in accuracy (measured by semantic similarity rating) from 0.64 to 0.81. To raised align with the suggestions from the subject material consultants (SMEs), the crew additional refined the mannequin utilizing PPO incorporating the human suggestions information, which boosted the LLM-judge scores from 3.9 to 4.2 out of 5, a outstanding achievement that translated to a considerable 80% discount within the effort required from the area consultants. Just like the Amazon Pharmacy case, these fine-tuned specialised fashions will proceed to perform as area skilled instruments inside the broader agentic AI system.

In 2025, the GES crew ventured into uncharted territory by making use of agentic AI techniques to optimize their enterprise course of. LLM fine-tuning methodologies represent a important mechanism for enhancing the reasoning capabilities in AI brokers, enabling efficient decomposition of complicated goals into executable motion sequences that align with predefined behavioral constraints and goal-oriented outcomes. It additionally serves as important structure part in facilitating specialised process execution and optimizing for task-specific efficiency metrics.

Amazon A+ Content material powers wealthy product pages throughout tons of of tens of millions of annual submissions. The A+ crew wanted to guage content material high quality at scale—assessing cohesiveness, consistency, and relevancy, not simply surface-level defects. Content material high quality instantly impacts conversion and model belief, making this a high-stakes utility.

Following the architectural sample seen in Amazon Pharmacy and World Engineering Providers, the crew constructed a specialised analysis agent powered by a fine-tuned mannequin. They utilized feature-based fine-tuning to Nova Lite on Amazon SageMaker—coaching a light-weight classifier on imaginative and prescient language mannequin (VLM)-extracted options somewhat than updating full mannequin parameters. This method, enhanced by expert-crafted rubric prompts, improved classification accuracy from 77% to 96%. The consequence: an AI agent that evaluates tens of millions of content material submissions and delivers actionable suggestions. This demonstrates a key precept from our maturity framework—approach complexity ought to match process necessities. The A+ use case, whereas high-stakes and working at large scale, is essentially a classification process well-suited to those strategies. Not each agent part requires GRPO or DAPO; deciding on the correct approach for every drawback is what delivers environment friendly, production-grade techniques.

Reference structure for superior AI orchestration utilizing fine-tuning

Though fine-tuned fashions serve various functions throughout completely different domains and use circumstances in an agentic AI system, the anatomy of an agent stays largely constant and could be encompassed in part groupings, as proven within the following structure diagram.

This modular method adopts various AWS generative AI companies, together with Amazon Bedrock AgentCore, Amazon SageMaker, and Amazon Bedrock, that maintains construction of key groupings that make up an agent whereas offering varied choices inside every group to enhance an AI agent.

1. LLM customization for AI brokers

Builders can use varied AWS companies to fine-tune and post-train the LLMs for an AI agent utilizing the strategies mentioned within the earlier part. When you use LLMs on Amazon Bedrock in your brokers, you should use a number of mannequin customization approaches to fine-tune your fashions. Distillation and SFT by way of parameter-efficient fine-tuning (PEFT) with low-rank adaptation (LoRA) can be utilized to handle easy customization duties. For superior fine-tuning, Continued Pre-training (CPT) extends a basis mannequin’s data by coaching on domain-specific corpora (medical literature, authorized paperwork, or proprietary technical content material), embedding specialised vocabulary and area reasoning patterns instantly into mannequin weights. Reinforcement fine-tuning (RFT), launched at re:Invent 2025, teaches fashions to grasp what makes a top quality response with out giant quantities of pre-labeled coaching information. There are two approaches supported for RFT: Reinforcement Studying with Verifiable Rewards (RLVR) makes use of rule-based graders for goal duties like code technology or math reasoning, whereas Reinforcement Studying from AI Suggestions (RLAIF) makes use of AI-based judges for subjective duties like instruction following or content material moderation.

When you require deeper management over mannequin customization infrastructure in your AI brokers, Amazon SageMaker AI supplies a complete platform for customized mannequin improvement and fine-tuning. Amazon SageMaker JumpStart accelerates the customization journey by providing pre-built options with one-click deployment of well-liked basis fashions (Llama, Mistral, Falcon, and others) and end-to-end fine-tuning notebooks that deal with information preparation, coaching configuration, and deployment workflows. Amazon SageMaker Coaching jobs present managed infrastructure for executing customized fine-tuning workflows, routinely provisioning GPU cases, managing coaching execution, and dealing with cleanup after completion. This method fits most fine-tuning situations the place normal occasion configurations present adequate compute energy and coaching completes reliably inside the job period limits. You need to use SageMaker Coaching jobs with customized Docker containers and code dependencies housing any machine studying (ML) framework, coaching library, or optimization approach, enabling experimentation with rising strategies past managed choices.

At re:Invent 2025, Amazon SageMaker HyperPod launched two capabilities for large-scale mannequin customization: Checkpointless coaching reduces checkpoint-restart cycles, shortening restoration time from hours to minutes. Elastic coaching routinely scales workloads to make use of idle capability and yields sources when higher-priority workloads peak. These options construct on the core strengths of HyperPod—resilient distributed coaching clusters with automated fault restoration for multi-week jobs spanning 1000’s of GPUs. HyperPod helps NVIDIA NeMo and AWS Neuronx frameworks, and is right when coaching scale, period, or reliability necessities exceed what job-based infrastructure can economically present.

In SageMaker AI, for builders who need to customise fashions with out managing infrastructure, Amazon SageMaker AI serverless customization, launched at re:Invent 2025, supplies a completely managed, UI- and SDK-driven expertise for mannequin fine-tuning. This functionality supplies infrastructure administration—SageMaker routinely selects and provisions acceptable compute sources (P5, P4de, P4d, and G5 cases) primarily based on mannequin dimension and coaching necessities. By the SageMaker Studio UI, you’ll be able to customise well-liked fashions (Amazon Nova, Llama, DeepSeek, GPT-OSS, and Qwen) utilizing superior strategies together with SFT, DPO, RLVR, and RLAIF. You may as well run the identical serverless customization utilizing SageMaker Python SDK in your Jupyter pocket book. The serverless method supplies pay-per-token pricing, automated useful resource cleanup, built-in MLflow experiment monitoring, and seamless deployment to each Amazon Bedrock and SageMaker endpoints.

If you must customise Amazon Nova fashions in your agentic workflow, you are able to do it by way of recipes and prepare them on SageMaker AI. It supplies end-to-end customization workflow together with mannequin coaching, analysis, and deployment for inference. with better flexibility and management to fine-tune the Nova fashions, optimize hyperparameters with precision, and implement strategies resembling LoRA PEFT, full-rank SFT, DPO, RFT, CPT, PPO, and so forth. For the Nova fashions on Amazon Bedrock, you can even prepare your Nova fashions by SFT and RFT with reasoning content material to seize intermediate pondering steps or use reward-based optimization when precise right solutions are tough to outline. In case you have extra superior agentic use circumstances that require deeper mannequin customization, you should use Amazon Nova Forge—launched at re:Invent 2025—to construct your individual frontier fashions from early mannequin checkpoints, mix your datasets with Amazon Nova-curated coaching information, and host your customized fashions securely on AWS.

2. AI agent improvement environments and SDKs

The event surroundings is the place builders writer, take a look at, and iterate on agent logic earlier than deployment. Builders use built-in improvement environments (IDEs) resembling SageMaker AI Studio (Jupyter Notebooks in comparison with code editors), Amazon Kiro, or IDEs on native machines like PyCharm. Agent logic is carried out utilizing specialised SDKs and frameworks that summary orchestration complexity—Strands supplies a Python framework purpose-built for multi-agent techniques, providing declarative agent definitions, built-in state administration, and native AWS service integrations that deal with the low-level particulars of LLM API calls, software invocation protocols, error restoration, and dialog administration. With these improvement instruments dealing with the low-level particulars of LLM API calls, builders can deal with enterprise logic somewhat than infrastructure design and upkeep.

3. AI agent deployment and operation

After your AI agent improvement is accomplished and able to deploy in manufacturing, you should use Amazon Bedrock AgentCore to deal with agent execution, reminiscence, safety, and gear integration with out requiring infrastructure administration. Bedrock AgentCore supplies a set of built-in companies, together with:

1. 1. AgentCore Runtime provides purpose-built environments that summary away infrastructure administration, whereas container-based alternate options (SageMaker AI jobs, AWS Lambda, Amazon Elastic Kubernetes Service (Amazon EKS), and Amazon Elastic Container Service (Amazon ECS)) present extra management for customized necessities. Basically, the runtime is the place your fastidiously crafted agent code meets actual customers and delivers enterprise worth at scale.
   2. AgentCore Reminiscence offers your AI brokers the power to recollect previous interactions, enabling them to supply extra clever, context-aware, and customized conversations. It supplies an easy and highly effective approach to deal with each short-term context and long-term data retention with out the necessity to construct or handle complicated infrastructure.
   3. With AgentCore Gateway, builders can construct, deploy, uncover, and hook up with instruments at scale, offering observability into software utilization patterns, error dealing with for failed invocations, and integration with id techniques for accessing instruments on behalf of customers (utilizing OAuth or API keys). Groups can replace software backends, add new capabilities, or modify authentication necessities with out redeploying brokers as a result of the gateway structure decouples software implementation from agent logic—sustaining flexibility as enterprise necessities evolve.
   4. AgentCore Observability helps you hint, debug, and monitor agent efficiency in manufacturing environments. It supplies real-time visibility into agent operational efficiency by way of entry to dashboards powered by Amazon CloudWatch and telemetry for key metrics resembling session rely, latency, period, token utilization, and error charges, utilizing the OpenTelemetry (OTEL) protocol normal.

4. LLM and AI agent analysis

When your fine-tuned LLM pushed AI brokers are operating in manufacturing, it’s necessary to guage and monitor your fashions and brokers repeatedly to make sure top quality and efficiency. Many enterprise use circumstances require customized analysis standards that encode area experience and enterprise guidelines. For the Amazon Pharmacy treatment path validation course of, analysis standards embody: drug-drug interplay detection accuracy (share of recognized contraindications appropriately recognized), dosage calculation precision (right dosing changes for age, weight, and renal perform), near-miss prevention fee (discount in treatment errors that might trigger affected person hurt), FDA labeling compliance (adherence to permitted utilization, warnings, and contraindications), and pharmacist override fee (share of agent suggestions accepted with out modification by licensed pharmacists).

In your fashions on Amazon Bedrock, you should use Amazon Bedrock evaluations to generate predefined metrics and human evaluation workflows. For superior situations, you should use SageMaker Coaching jobs to fine-tune specialised choose fashions on domain-specific analysis datasets. For holistic AI agent analysis, AgentCore Evaluations, launched at re:Invent 2025, supplies automated evaluation instruments to measure your agent or instruments efficiency on finishing particular duties, dealing with edge circumstances, and sustaining consistency throughout completely different inputs and contexts.

Determination information and beneficial phased method

Now that you simply perceive the technical evolution of superior fine-tuning strategies—from SFT to PPO, DPO, GRPO, DAPO and GSPO—the important query turns into when and why it is best to use them. Our expertise exhibits that organizations utilizing a phased maturity method obtain 70–85% manufacturing conversion charges (in comparison with the 30–40% trade common) and 3-fold year-over-year ROI development. The 12–18 month journey from preliminary agent deployment to superior reasoning capabilities delivers incremental enterprise worth at every part. The secret’s letting your use case necessities, obtainable information, and measured efficiency information development—not technical sophistication for its personal sake.

The maturity path progresses by way of 4 phases (proven within the following desk). Strategic endurance on this development builds reusable infrastructure, collects high quality coaching information, and validates ROI earlier than main investments. As our examples reveal, aligning technical sophistication with human and enterprise wants delivers transformative outcomes and sustainable aggressive benefits in your most important AI purposes.

Conclusion

Whereas brokers have remodeled how we construct AI techniques, superior fine-tuning stays a important part for enterprises looking for aggressive benefit in high-stakes domains. By understanding the evolution of strategies like PPO, DPO, GRPO, DAPO and GSPO, and making use of them strategically inside agent architectures, organizations can obtain vital enhancements in accuracy, effectivity, and security. The true-world examples from Amazon reveal –that the mixture of agentic workflows with fastidiously fine-tuned fashions delivers dramatic enterprise outcomes.

AWS continues to speed up these capabilities with a number of key launches at re:Invent 2025. Reinforcement fine-tuning (RFT) on Amazon Bedrock now allows fashions to study high quality responses by way of RLVR for goal duties and RLAIF for subjective evaluations—with out requiring giant quantities of pre-labeled information. Amazon SageMaker AI Serverless Customization eliminates infrastructure administration for fine-tuning, supporting SFT, DPO, and RLVR strategies with pay-per-token pricing. For big-scale coaching, Amazon SageMaker HyperPod launched checkpointless coaching and elastic scaling to scale back restoration time and optimize useful resource utilization. Amazon Nova Forge empowers enterprises to construct customized frontier fashions from early checkpoints, mixing proprietary datasets with Amazon-curated coaching information. Lastly, AgentCore Analysis supplies automated evaluation instruments to measure agent efficiency on process completion, edge circumstances, and consistency—closing the loop on production-grade agentic AI techniques.

As you consider your generative AI technique, use the choice information and phased maturity method outlined on this put up to establish the place superior fine-tuning can tip the scales from adequate to transformative. Use the reference structure as a baseline to construction your agentic AI techniques, and use the capabilities launched at re:Invent 2025 to speed up your journey from preliminary agent deployment to production-grade outcomes.

Concerning the authors

Yunfei Bai is a Principal Options Architect at AWS. With a background in AI/ML, information science, and analytics, Yunfei helps clients undertake AWS companies to ship enterprise outcomes. He designs AI/ML and information analytics options that overcome complicated technical challenges and drive strategic goals. Yunfei has a PhD in Digital and Electrical Engineering. Outdoors of labor, Yunfei enjoys studying and music.

Kristine Pearce is a Principal Worldwide Generative AI GTM Specialist at AWS, targeted on SageMaker AI mannequin customization, optimization, and inference at scale. She combines her MBA, BS Industrial Engineering background, and human-centered design experience to deliver strategic depth and behavioral science to AI-enabled transformation. Outdoors work, she channels her creativity by way of artwork.

Harsh Asnani is a Worldwide Generative AI Specialist Options Architect at AWS specializing in ML idea, MLOPs, and manufacturing generative AI frameworks. His background is in utilized information science with a deal with operationalizing AI workloads within the cloud at scale.

Sung-Ching Lin is a Principal Engineer at Amazon Pharmacy, the place he leads the design and adoption of AI/ML techniques to enhance buyer expertise and operational effectivity. He focuses on constructing scalable, agent-based architectures, ML analysis frameworks, and production-ready AI options in regulated healthcare domains.

Elad Dwek is a Senior AI Enterprise Developer at Amazon, working inside World Engineering, Upkeep, and Sustainability. He companions with stakeholders from enterprise and tech aspect to establish alternatives the place AI can improve enterprise challenges or fully rework processes, driving innovation from prototyping to manufacturing. With a background in development and bodily engineering, he focuses on change administration, know-how adoption, and constructing scalable, transferable options that ship steady enchancment throughout industries. Outdoors of labor, he enjoys touring all over the world together with his household.

Carrie Track is a Senior Program Supervisor at Amazon, engaged on AI-powered content material high quality and buyer expertise initiatives. She companions with utilized science, engineering, and UX groups to translate generative AI and machine studying insights into scalable, customer-facing options. Her work focuses on bettering content material high quality and streamlining the buying expertise on product element pages.