If you happen to’re utilizing Retrieval-Augmented Technology (RAG) for advanced analytical duties that span tons of of paperwork, corresponding to monetary due diligence or regulatory compliance evaluations, you’ve doubtless hit its ceiling. Similarity search surfaces related fragments however typically misses cross-document connections. This publish reveals you the right way to tackle that hole utilizing task-aware information compression (TAKC), a way that pre-compresses whole information bases into task-specific representations deployed on AWS. You’ll be able to deploy an entire open-source implementation in your individual account.
Activity-aware information compression
Contemplate a personal fairness agency evaluating a $500 million acquisition of a producing firm. The due diligence workforce should analyze monetary statements spanning 12 subsidiaries and 5 years. In addition they face 200+ provider contracts, environmental compliance studies from 8 amenities, and 50+ authorized instances. When an analyst asks about consolidated monetary dangers given present provider phrases and pending litigation, RAG’s similarity search can’t floor that response. A whole lot of paperwork maintain the related data, and the connections between them share no lexical similarity.
TAKC addresses this kind of downside by utilizing an LLM to supply shorter, task-focused summaries of paperwork, with totally different summaries for various duties.
Completely different duties require totally different data from the identical doc. An annual report compressed for monetary evaluation wants income figures, margins, and money movement information. The identical report compressed for a compliance evaluate wants regulatory citations and violation histories. Generic summarization makes an attempt to cowl all the things, which dilutes the data density for any particular use case. TAKC compresses paperwork via the lens of a particular process, preserving what issues and discarding the remainder. The Ingestion pipeline part reveals how the compression immediate specifies precisely what data to protect. For manufacturing deployments, retailer task-type prompts in a versioned configuration (corresponding to AWS Programs Supervisor Parameter Retailer or a devoted Amazon Easy Storage Service (Amazon S3) prefix) in order that immediate modifications are auditable and recompression could be triggered when prompts are up to date.
The system compresses paperwork offline, as soon as per doc per process sort. At question time, the system retrieves a pre-compressed illustration slightly than the unique doc. The system then solutions questions utilizing the compressed model as an alternative of the complete doc. If the compressed illustration lacks ample element, the question complexity analyzer routes the query to a decrease compression tier that retains extra context.
TAKC offers entry to the whole information base in compressed kind, not simply the top-k chunks {that a} similarity search returns. The system preserves connections between paperwork as a result of the compression sees paperwork collectively. It additionally produces totally different compressed outputs for various duties from the identical supply materials. The monetary compression of a 10-Ok submitting seems to be nothing just like the authorized danger compression of that very same submitting. The compression reduces token rely by 8x to 64x whereas focusing on task-relevant data for retention.
Multi-rate compression
Completely different queries require totally different ranges of constancy. A query corresponding to “What was Q3 income?” wants far much less context than a request to investigate relationships between provider cost phrases and quarterly money movement throughout subsidiaries.
TAKC addresses this by sustaining 4 compression tiers for every process sort. On the lightest compression (8x), the system retains roughly 87.5% much less context. It preserves sufficient element for multi-step reasoning and cross-document synthesis. On the medium tier (16x), context discount reaches roughly 93.8%. This stage fits analytical queries with reasonable complexity. The excessive tier (32x) reduces context by roughly 96.9% and serves factual lookups and well-defined questions. On the extremely tier (64x), discount reaches roughly 98.4%. This stage is suitable for classification duties and key phrase lookups.
A question complexity analyzer routes incoming inquiries to the suitable tier based mostly on alerts like question size, query sort, and presence of analytical language. Easy factual questions hit the ultra-compressed cache whereas advanced analytical questions use the calmly compressed cache. This occurs transparently to the consumer.
Most enterprise queries are lookups that may be served from the upper compression tiers at minimal value. The rare advanced queries devour bigger context budgets solely when wanted. This tier-based routing enhances current RAG optimizations like metadata filtering and question reformatting, which might slim the doc set earlier than compression is utilized. To validate compression high quality, evaluate LLM responses at every tier in opposition to responses generated from the complete uncompressed doc. The reference implementation consists of take a look at scripts that run this comparability to your particular process sorts and paperwork.
Structure on AWS
The implementation runs on AWS as two decoupled serverless pipelines: one for ingestion and one for queries. Determine 1 illustrates each pipelines.
TAKC structure. The pipeline movement handles information ingestion and compression. The consumer movement handles authenticated queries
We selected AWS Lambda for compute as a result of every perform invocation is short-lived and event-driven. The workload processes information in bursts throughout ingestion and handles variable question masses between bursts, making serverless a pure match.
We selected Amazon API Gateway to show the question interface as a REST endpoint. For caching, we selected Amazon ElastiCache Serverless for reads on composite keys (takc:{process}:{charge}) with out shard administration. Amazon Cognito handles JWT issuance and token refresh with out customized auth code, decreasing the implementation floor space.
Ingestion pipeline
When a doc lands in Amazon S3 beneath a task-type prefix (for instance, raw-data/monetary/), an S3 occasion notification triggers an AWS Lambda perform. This perform chunks the doc into 256-token segments with 50-token overlap to stop data loss at boundaries. It then invokes a compression Lambda perform for every chunk asynchronously, enabling parallel processing. For big-scale ingestion, configure reserved concurrency on the compression perform and place an Amazon Easy Queue Service (Amazon SQS) queue between the chunking and compression steps to deal with throttling gracefully.
The second perform calls Amazon Bedrock to compress the chunks in any respect 4 compression tiers. Every compression name features a task-aware immediate that tells the mannequin what data to protect:
TASK: Monetary evaluation. Protect income metrics, margins, money movement, debt obligations, and monetary danger indicators.
COMPRESSION TARGET: Cut back to roughly 1/16 of unique size.
INSTRUCTIONS:
- Deal with info and relationships related to the duty
- Protect numerical information and metrics
- Preserve entities and their attributes
- Preserve causal relationships and dependencies
- Take away redundant or irrelevant dataThe mannequin is aware of what to protect as a result of the immediate specifies what issues for the duty. That distinction is what makes this task-aware slightly than generic compression. The system shops compressed outputs in Amazon ElastiCache Serverless with keys like takc:monetary:medium, and backs them as much as S3 for sturdiness. The Redis OSS information mannequin helps the hierarchical key construction wanted for multi-rate cache lookups. Cache entries use a 24-hour TTL and are backed as much as S3. If a cache entry is evicted or expires, the question perform falls again to the S3 backup and re-populates the cache on learn.
Question pipeline
A consumer authenticates via Amazon Cognito, receives a JWT, and sends a question via Amazon API Gateway. AWS WAF fronts the API for charge limiting and menace safety. A Lambda perform analyzes question complexity utilizing heuristics (key phrase alerts and question size), retrieves the suitable compressed cache from Amazon ElastiCache Serverless, and sends the compressed context plus question to Amazon Bedrock for inference. When routing confidence is low, the system defaults to the medium compression tier as a secure fallback.
The upper-cost Bedrock compression calls occur as soon as throughout ingestion. The question path is a cache lookup plus inference on compressed context.
The stack makes use of Amazon Bedrock (Anthropic Claude 3 Haiku, Claude 3 Sonnet, and Amazon Titan Textual content) for compression and inference. The mannequin choice is configurable via CDK context values with out code modifications. AWS Lambda (Python 3.12+) handles information processing and question logic. Amazon ElastiCache Serverless shops the compressed cache, whereas Amazon S3 holds uncooked information, chunks, and cache backups (KMS-encrypted). Amazon API Gateway exposes the REST endpoints, and Amazon Cognito offers JWT-based authentication. AWS WAF fronts the API with charge limiting and managed safety guidelines. Amazon CloudWatch offers monitoring and metrics, and AWS Key Administration Service (AWS KMS) manages encryption keys.
Outline the infrastructure as a single AWS Cloud Growth Equipment (AWS CDK) stack and deploy it with a single command. AWS CDK offers repeatable deployments and allows you to customise parameters like compression chunk dimension, Lambda reminiscence allocation, and ElastiCache storage limits via context values. For manufacturing deployments, think about separating stateful assets (S3, ElastiCache, Cognito) into their very own stack to scale back blast radius and allow unbiased lifecycle administration of compute and storage layers.
Value comparability
Enter token utilization for a 100,000-token information base queried 1,000 occasions per day (output tokens are excluded as a result of response size is unbiased of context dimension):
| Strategy | Enter tokens per question | Every day enter tokens | Relative enter value |
| Full context | 100,000 | 100,000,000 | 100% |
| RAG (top-10 chunks) | ~10,000 | 10,000,000 | 10% |
| TAKC Mild (8×) | ~12,500 | 12,500,000 | 12.5% |
| TAKC Medium (16×) | ~6,250 | 6,250,000 | 6.25% |
| TAKC Excessive (32×) | ~3,125 | 3,125,000 | 3.1% |
| TAKC Extremely (64×) | ~1,563 | 1,563,000 | 1.6% |
The token financial savings from compression are derived instantly from the compression ratios. Precise financial savings rely in your particular Bedrock mannequin pricing and question patterns.
TAKC requires upfront compression value via one-time Bedrock calls throughout ingestion. This value amortizes for information bases that change occasionally and get queried repeatedly. For information bases that change hourly, RAG’s per-query retrieval mannequin could also be extra sensible.
When to decide on TAKC, RAG, or each
This desk summarizes when every strategy is the higher match, based mostly in your workload traits:
| Issue | Favors TAKC | Favors RAG |
| Question sort | Cross-document reasoning, synthesis | Slender factual lookups |
| Information base stability | Modifications each day or much less | Modifications hourly or extra |
| Activity predictability | Properly-defined process sorts | Unpredictable question patterns |
| Protection requirement | Should think about full corpus | Only some paperwork are related |
| Supply attribution | Not required | Required (consumer must see supply) |
| Token price range | Tight | Versatile |
In observe, a manufacturing system typically advantages from each. RAG handles fast lookups effectively. TAKC handles the analytical queries the place retrieval-based approaches miss connections. A question complexity analyzer can route between them. Use RAG when customers must hint responses again to particular supply paperwork. For regulated workloads that require each cross-document reasoning and auditability, mix TAKC with RAG: use TAKC for the analytical response and RAG to retrieve the supporting supply paperwork for the audit path.
Getting began
Conditions
To deploy this reference implementation, verify that you’ve got the next:
Deploy and take a look at
Clone the repository aws-samples/sample-bedrock-takc-compression and deploy the CDK stack:
After deployment completes, take a look at the pipeline:
- Add a doc to the S3 bucket:
- Wait 2-3 minutes for the ingestion pipeline to chunk and compress the doc in any respect 4 compression tiers.
- Question the API endpoint:
The system handles chunking, multi-rate compression, caching, and question routing with out extra configuration.
Clear up
To keep away from ongoing prices, destroy the CDK stack if you’re carried out testing. First, empty the S3 bucket as a result of CDK can’t delete buckets that comprise objects:
Then destroy the stack:
This removes the Lambda capabilities, API Gateway, Amazon ElastiCache Serverless cache, Amazon Cognito consumer pool, WAF net ACL, and Amazon CloudWatch alarms. The KMS key’s retained with a 30-day pending deletion window. To schedule its deletion instantly, run the next command:
Conclusion
Advanced analytical duties that span tons of of paperwork require greater than fragment retrieval. TAKC gives an strategy constructed for this. Compress the complete information base offline via the lens of particular duties, cache these representations at a number of constancy ranges, and match the fitting compression tier to every question’s complexity.
The AWS implementation makes use of Amazon Bedrock for compression and inference, Amazon ElastiCache Serverless for caching, and a totally serverless structure that scales with demand. Entry the reference implementation, CDK infrastructure, and deployment scripts at aws-samples/sample-bedrock-takc-compression.
To get began, deploy the CDK stack with your individual paperwork and see how the system responds to totally different question sorts. In case your workload includes cross-document reasoning over steady information bases, TAKC can cut back token prices whereas bettering response high quality.
References
In regards to the authors
