How PDI constructed an enterprise-grade RAG system for AI purposes with AWS

PDI Applied sciences is a world chief within the comfort retail and petroleum wholesale industries. They assist companies across the globe improve effectivity and profitability by securely connecting their knowledge and operations. With 40 years of expertise, PDI Applied sciences assists clients in all points of their enterprise, from understanding shopper conduct to simplifying expertise ecosystems throughout the availability chain.

Enterprises face a major problem of creating their information bases accessible, searchable, and usable by AI techniques. Inner groups at PDI Applied sciences have been combating info scattered throughout disparate techniques together with web sites, Confluence pages, SharePoint websites, and varied different knowledge sources. To deal with this, PDI Applied sciences constructed PDI Intelligence Question (PDIQ), an AI assistant that provides staff entry to firm information by way of an easy-to-use chat interface. This resolution is powered by a customized Retrieval Augmented Technology (RAG) system, constructed on Amazon Internet Providers (AWS) utilizing serverless applied sciences. Constructing PDIQ required addressing the next key challenges:

• Robotically extracting content material from numerous sources with totally different authentication necessities
• Needing the pliability to pick, apply, and interchange probably the most appropriate giant language mannequin (LLM) for numerous processing necessities
• Processing and indexing content material for semantic search and contextual retrieval
• Making a information basis that allows correct, related AI responses
• Repeatedly refreshing info by way of scheduled crawling
• Supporting enterprise-specific context in AI interactions

On this submit, we stroll by way of the PDIQ course of move and structure, specializing in the implementation particulars and the enterprise outcomes it has helped PDI obtain.

Resolution structure

On this part, we discover PDIQ’s complete end-to-end design. We study the info ingestion pipeline from preliminary processing by way of storage to consumer search capabilities, in addition to the zero-trust safety framework that protects key consumer personas all through their platform interactions. The structure consists of those parts:

1. Scheduler – Amazon EventBridge maintains and executes the crawler scheduler.
2. Crawlers – AWS Lambda invokes crawlers which can be executed as duties by Amazon Elastic Container Service (Amazon ECS).
3. Amazon DynamoDB – Persists crawler configurations and different metadata similar to Amazon Easy Storage Service (Amazon S3) picture location and captions.
4. Amazon S3 – All supply paperwork are saved in Amazon S3. Amazon S3 occasions set off the downstream move for each object that’s created or deleted.
5. Amazon Easy Notification Service (Amazon SNS) – Receives notification from Amazon S3 occasions.
6. Amazon Easy Queue Service (Amazon SQS) – Subscribed to Amazon SNS to carry the incoming requests in a queue.
7. AWS Lambda – Handles the enterprise logic for chunking, summarizing, and producing vector embeddings.
8. Amazon Bedrock – Gives API entry to basis fashions (FMs) utilized by PDIQ:
9. Amazon Aurora PostgreSQL-Suitable Version – Shops vector embeddings.

The next diagram is the answer structure.

Subsequent, we overview how PDIQ implements a zero-trust safety mannequin with role-based entry management for 2 key personas:

• Directors configure information bases and crawlers by way of Amazon Cognito consumer teams built-in with enterprise single sign-on. Crawler credentials are encrypted at relaxation utilizing AWS Key Administration Service (AWS KMS) and solely accessible inside remoted execution environments.
• Finish customers entry information bases primarily based on group permissions validated on the utility layer. Customers can belong to a number of teams (similar to human sources or compliance) and swap contexts to question role-appropriate datasets.

Course of move

On this part, we overview the end-to-end course of move. We break it down by sections to dive deeper into every step and clarify the performance.

Crawlers

Crawlers are configured by Administrator to gather knowledge from quite a lot of sources that PDI depends on. Crawlers hydrate the info into the information base in order that this info might be retrieved by finish customers. PDIQ at the moment helps the next crawler configurations:

• Internet crawler – Through the use of Puppeteer for headless browser automation, the crawler converts HTML net pages to markdown format utilizing turndown. By following the embedded hyperlinks on the web site, the crawler can seize full context and relationships between pages. Moreover, the crawler downloads property similar to PDFs and pictures whereas preserving the unique reference and gives customers configuration choices similar to fee limiting.
• Confluence crawler – This crawler makes use of Confluence REST API with authenticated entry to extract web page content material, attachments, and embedded photos. It preserves web page hierarchy and relationships, handles particular Confluence parts similar to data packing containers, notes, and lots of extra.
• Azure DevOps crawler – PDI makes use of Azure DevOps to handle its code base, observe commits, and keep challenge documentation in a centralized repository. PDIQ makes use of Azure DevOps REST API with OAuth or private entry token (PAT) authentication to extract this info. Azure DevOps crawler preserves challenge hierarchy, dash relationships, and backlog construction additionally maps work merchandise relationships (similar to dad or mum/little one or linked gadgets), thereby offering a whole view of the dataset.
• SharePoint crawler – It makes use of Microsoft Graph API with OAuth authentication to extract doc libraries, lists, pages, and file content material. The crawler processes MS Workplace paperwork (Phrase, Excel, PowerPoint) into searchable textual content and maintains doc model historical past and permission metadata.

By constructing separate crawler configurations, PDIQ gives straightforward extensibility into the platform to configure further crawlers on demand. It additionally gives the pliability to administrator customers to configure the settings for his or her respective crawlers (similar to frequency, depth, or fee limits).

The next determine exhibits the PDIQ UI to configure the information base.

The next determine exhibits the PDI UI to configure your crawler (similar to Confluence).

The next determine exhibits the PDIQ UI to schedule crawlers.

Dealing with photos

Information crawled is saved in Amazon S3 with correct metadata tags. If the supply is in HTML format, the duty converts the content material into markdown (.md) information. For these markdown information, there may be an extra optimization step carried out to switch the photographs within the doc with the Amazon S3 reference location. Key advantages of this strategy embrace:

• PDI can use S3 object keys to uniquely reference every picture, thereby optimizing the synchronization course of to detect modifications in supply knowledge
• You may optimize storage by changing photos with captions and avoiding the necessity to retailer duplicate photos
• It offers the flexibility to make the content material of the photographs searchable and relatable to the textual content content material within the doc
• Seamlessly inject unique photos when rendering a response to consumer inquiry

The next is a pattern markdown file the place photos are changed with the S3 file location:

![image-20230113-074652](https:// amzn-s3-demo-bucket.s3.amazonaws.com/kb/123/file/attachments/12133171243_image-20230113-074652.png)

Doc processing

That is probably the most important step of the method. The important thing goal of this step is to generate vector embeddings in order that they can be utilized for similarity matching and efficient retrieval primarily based on consumer inquiry. The method follows a number of steps, beginning with picture captioning, then doc chunking, abstract era, and embedding era. To caption the photographs, PDIQ scans the markdown information to find picture tags . For every of those photos, PDIQ scans and generates a picture caption that explains the content material of the picture. This caption will get injected again into the markdown file, subsequent to the tag, thereby enriching the doc content material. This strategy gives improved contextual searchability. PDIQ enhances content material discovery by embedding insights extracted from photos immediately into the unique markdown information. This strategy ensures that picture content material turns into a part of the searchable textual content, enabling richer and extra correct context retrieval throughout search and evaluation. The strategy additionally saves prices. To keep away from pointless LLM inference requires very same photos, PDIQ shops picture metadata (file location and generated captions) in Amazon DynamoDB. This step permits environment friendly reuse of beforehand generated captions, eliminating the necessity for repeated caption era calls to LLM.

The next is an instance of a picture caption immediate:

You're a skilled picture captioning assistant. Your job is to offer clear, factual, and goal descriptions of photos. Concentrate on describing seen parts, objects, and scenes in a impartial and acceptable method.

The next is a snippet of markdown file that comprises the picture tag, LLM-generated caption, and the corresponding S3 file location:

![image-20230818-114454: The image displays a security tip notification on a computer screen. The notification is titled "Security tip" and advises the user to use generated passwords to keep their accounts safe. The suggested password, "2m5oFX#g&tLRMhN3," is shown in a green box. Below the suggested password, there is a section labeled "Very Strong," indicating the strength of the password. The password length is set to 16 characters, and it includes lowercase letters, uppercase letters, numbers, and symbols. There is also a "Dismiss" button to close the notification. Below the password section, there is a link to "See password history." The bottom of the image shows navigation icons for "Vault," "Generator," "Alerts," and "Account." The "Generator" icon is highlighted in red.]
(https:// amzn-s3-demo-bucket.s3.amazonaws.com/kb/ABC/file/attachments/12133171243_image-20230818-114454.png)

Now that markdown information are injected with picture captions, the subsequent step is to interrupt the unique doc into chunks that match into the context window of the embeddings mannequin. PDIQ makes use of Amazon Titan Textual content Embeddings V2 mannequin to generate vectors and shops them in Aurora PostgreSQL-Suitable Serverless. Based mostly on inside accuracy testing and chunking finest practices from AWS, PDIQ performs chunking as follows:

• 70% of the tokens for content material
• 10% overlap between chunks
• 20% for abstract tokens

Utilizing the doc chunking logic from the earlier step, the doc is transformed into vector embeddings. The method contains:

1. Calculate chunk parameters – Decide the scale and whole variety of chunks required for the doc primarily based on the 70% calculation.
2. Generate doc abstract – Use Amazon Nova Lite to create a abstract of your entire doc, constrained by the 20% token allocation. This abstract is reused throughout all chunks to offer constant context.
3. Chunk and prepend abstract – Break up the doc into overlapping chunks (10%), with the abstract prepended on the high.
4. Generate embeddings – Use Amazon Titan Textual content Embeddings V2 to generate vector embeddings for every chunk (abstract plus content material), which is then saved within the vector retailer.

By designing a personalized strategy to generate a abstract part atop of all chunks, PDIQ ensures that when a specific chunk is matched primarily based on similarity search, the LLM has entry to your entire abstract of the doc and never solely the chunk that matched. This strategy enriches finish consumer expertise leading to a rise of approval fee for accuracy from 60% to 79%.

The next is an instance of a summarization immediate:

You're a specialised doc summarization assistant with experience in enterprise and technical content material.

Your job is to create concise, information-rich summaries that:
Protect all quantifiable knowledge (numbers, percentages, metrics, dates, monetary figures)
Spotlight key enterprise terminology and domain-specific ideas
Extract necessary entities (folks, organizations, merchandise, places)
Establish important relationships between ideas
Keep factual accuracy with out including interpretations
Concentrate on extracting info that might be most beneficial for:
Answering particular enterprise questions
Supporting data-driven determination making
Enabling exact info retrieval in a RAG system
The abstract must be complete but concise, prioritizing particular info over normal descriptions.
Embrace any tables, lists, or structured knowledge in a format that preserves their relationships.
Guarantee all technical phrases, acronyms, and specialised vocabulary are preserved precisely as written.

The next is an instance of abstract textual content, accessible on every chunk:

### Abstract: PLC Person Creation Course of and Password Reset
**Doc Overview:**
This doc offers directions for creating new customers and resetting passwords 
**Key Directions:**

  {Shortened for Weblog illustration} 


This abstract captures the important steps, necessities, and entities concerned within the PLC consumer creation and password reset course of utilizing Jenkins.
---

Chunk 1 has a abstract on the high adopted by particulars from the supply:

{Abstract Textual content from above}
This abstract captures the important steps, necessities, and entities concerned within the PLC consumer creation and password reset course of utilizing Jenkins.

title: 2. PLC Person Creation Course of and Password Reset

![image-20230818-114454: The image displays a security tip notification on a computer screen. The notification is titled "Security tip" and advises the user to use generated passwords to keep their accounts safe. The suggested password, "2m5oFX#g&tLRMhN3," is shown in a green box. Below the suggested password, there is a section labeled "Very Strong," indicating the strength of the password. The password length is set to 16 characters, and it includes lowercase letters, uppercase letters, numbers, and symbols. There is also a "Dismiss" button to close the notification. Below the password section, there is a link to "See password history." The bottom of the image shows navigation icons for "Vault," "Generator," "Alerts," and "Account." The "Generator" icon is highlighted in red.](https:// amzn-s3-demo-bucket.s3.amazonaws.com/kb/123/file/attachments/12133171243_image-20230818-114454.png)

Chunk 2 has a abstract on the high, adopted by continuation of particulars from the supply:

{Abstract Textual content from above}
This abstract captures the important steps, necessities, and entities concerned within the PLC consumer creation and password reset course of utilizing Jenkins.
---
Maintains a menu with choices similar to 

![image-20230904-061307:  - The generated text has been blocked by our content filters.](https:// amzn-s3-demo-bucket.s3.amazonaws.com/kb/123/file/attachments/12133171243_image-20230904-061307.png)

PDIQ scans every doc chunk and generates vector embeddings. This knowledge is saved in Aurora PostgreSQL database with key attributes, together with a singular information base ID, corresponding embeddings attribute, unique textual content (abstract plus chunk plus picture caption), and a JSON binary object that features metadata fields for extensibility. To maintain the information base in sync, PDI implements the next steps:

• Add – These are internet new supply objects that must be ingested. PDIQ implements the doc processing move described beforehand.
• Replace – If PDIQ determines the identical object is current, it compares the hash key worth from the supply with the hash worth from the JSON object.
• Delete – If PDIQ determines {that a} particular supply doc not exists, it triggers a delete operation on the S3 bucket (s3:ObjectRemoved:*), which leads to a cleanup job, deleting the information akin to the important thing worth within the Aurora desk.

PDI makes use of Amazon Nova Professional to retrieve probably the most related doc and generates a response by following these key steps:

• Utilizing similarity search, retrieves probably the most related doc chunks, which embrace abstract, chunk knowledge, picture caption, and picture hyperlink.
• For the matching chunk, retrieve your entire doc.
• LLM then replaces the picture hyperlink with the precise picture from Amazon S3.
• LLM generates a response primarily based on the info retrieved and the preconfigured system immediate.

The next is a snippet of system immediate:

Assist assistant specializing in PDI's Logistics(PLC) platform, serving to employees analysis and resolve assist instances in Salesforce. You'll help with discovering options, summarizing case info, and recommending acceptable subsequent steps for decision.

Skilled, clear, technical when wanted whereas sustaining accessible language.

Decision Course of:
Response Format template:
Deal with Confidential Data:

Outcomes and subsequent steps

By constructing this personalized RAG resolution on AWS, PDI realized the next advantages:

• Versatile configuration choices permit knowledge ingestion at consumer-preferred frequencies.
• Scalable design permits future ingestion from further supply techniques by way of simply configurable crawlers.
• Helps crawler configuration utilizing a number of authentication strategies, together with username and password, secret key-value pairs, and API keys.
• Customizable metadata fields allow superior filtering and enhance question efficiency.
• Dynamic token administration helps PDI intelligently steadiness tokens between content material and summaries, enhancing consumer responses.
• Consolidates numerous supply knowledge codecs right into a unified structure for streamlined storage and retrieval.

PDIQ offers key enterprise outcomes that embrace:

• Improved effectivity and determination charges – The device empowers PDI assist groups to resolve buyer queries considerably quicker, typically automating routine points and offering rapid, exact responses. This has led to much less buyer ready on case decision and extra productive brokers.
• Excessive buyer satisfaction and loyalty – By delivering correct, related, and personalised solutions grounded in reside documentation and firm information, PDIQ elevated buyer satisfaction scores (CSAT), internet promoter scores (NPS), and general loyalty. Prospects really feel heard and supported, strengthening PDI model relationships.
• Price discount – PDIQ handles the majority of repetitive queries, permitting restricted assist employees to deal with expert-level instances, which improves productiveness and morale. Moreover, PDIQ is constructed on serverless structure, which mechanically scales whereas minimizing operational overhead and price.
• Enterprise flexibility – A single platform can serve totally different enterprise models, who can curate the content material by configuring their respective knowledge sources.
• Incremental worth – Every new content material supply provides measurable worth with out system redesign.

PDI continues to reinforce the appliance with a number of deliberate enhancements within the pipeline, together with:

• Construct further crawler configuration for brand new knowledge sources (for instance, GitHub).
• Construct agentic implementation for PDIQ to be built-in into bigger advanced enterprise processes.
• Enhanced doc understanding with desk extraction and construction preservation.
• Multilingual assist for international operations.
• Improved relevance rating with hybrid retrieval methods.
• Capacity to invoke PDIQ primarily based on occasions (for instance, supply commits).

Conclusion

PDIQ service has remodeled how customers entry and use enterprise information at PDI Applied sciences. Through the use of Amazon serverless providers, PDIQ can mechanically scale with demand, cut back operational overhead, and optimize prices. The answer’s distinctive strategy to doc processing, together with the dynamic token administration and the customized picture captioning system, represents vital technical innovation in enterprise RAG techniques. The structure efficiently balances efficiency, price, and scalability whereas sustaining safety and authentication necessities. As PDI Applied sciences proceed to develop PDIQ’s capabilities, they’re excited to see how this structure can adapt to new sources, codecs, and use instances.

In regards to the authors

Samit Kumbhani is an Amazon Internet Providers (AWS) Senior Options Architect within the New York Metropolis space with over 18 years of expertise. He at the moment companions with impartial software program distributors (ISVs) to construct extremely scalable, modern, and safe cloud options. Outdoors of labor, Samit enjoys enjoying cricket, touring, and biking.

Jhorlin De Armas is an Architect II at PDI Applied sciences, the place he leads the design of AI-driven platforms on Amazon Internet Providers (AWS). Since becoming a member of PDI in 2024, he has architected a compositional AI service that allows configurable assistants, brokers, information bases, and guardrails utilizing Amazon Bedrock, Aurora Serverless, AWS Lambda, and DynamoDB. With over 18 years of expertise constructing enterprise software program, Jhorlin makes a speciality of cloud-centered architectures, serverless platforms, and AI/ML options.

David Mbonu is a Sr. Options Architect at Amazon Internet Providers (AWS), serving to horizontal enterprise utility ISV clients construct and deploy transformational options on AWS. David has over 27 years of expertise in enterprise options structure and system engineering throughout software program, FinTech, and public cloud firms. His latest pursuits embrace AI/ML, knowledge technique, observability, resiliency, and safety. David and his household reside in Sugar Hill, GA.