Azure Storage prospects more and more function at large scale, with tens of millions and even billions of things distributed throughout a number of storage accounts. As the dimensions of the info will increase, managing the info introduces a distinct set of challenges.
In a current episode of Azure Storage Discuss, I sat down with Shashank, a Product Supervisor on the Azure Storage Actions crew, to debate how Azure Storage Actions helps prospects automate widespread information administration duties with out writing customized code or managing infrastructure.
This submit summarizes the important thing ideas, eventualities, and learnings from that dialog. Take heed to the complete dialog under.
As storage estates develop, prospects usually have to:
- Apply retention or immutability insurance policies for compliance
- Shield delicate or vital information from modification
- Optimize storage prices by tiering occasionally accessed information
- Add or clear up metadata (blob index tags) for discovery and downstream processing
Immediately, many purchasers deal with these wants by writing customized scripts or sustaining inside tooling. This method requires vital engineering effort, ongoing upkeep, cautious credential dealing with, and in depth testing, particularly when working throughout tens of millions of merchandise throughout a number of storage accounts.
These challenges change into extra pronounced as information estates sprawl throughout areas and subscriptions.
Azure Storage Actions is a completely managed, serverless automation platform designed to carry out routine information administration operations at scale for:
- Azure Blob Storage
- Azure Knowledge Lake Storage
It permits prospects to outline condition-based logic and apply native storage operations akin to tagging, tiering, deletion, or immutability, throughout giant datasets with out deploying or managing servers. Azure Storage Actions is constructed round two major ideas:
A storage job is an Azure Useful resource Supervisor (ARM) useful resource that defines:
- The circumstances used to guage blobs (for instance, file identify, dimension, timestamps, or index tags)
- The actions to take when circumstances are met (akin to altering tiers, including immutability, or modifying tags)
The duty definition is created as soon as and centrally managed.
A job project applies a storage job to a number of storage accounts. This permits the identical logic to be reused with out redefining it for every account.
Every project can:
- Run as soon as (for cleanup or one-off processing)
- Run on a recurring schedule
- Be scoped utilizing container filters or excluded prefixes
Throughout the episode, Shashank demonstrated a real-world situation involving a storage account utilized by a authorized crew.
- Determine PDF information tagged as vital
- Apply a time-based immutability coverage to forestall tampering
- Transfer these information from the Sizzling tier to the Archive tier to cut back storage prices
- Add a brand new tag indicating the info is protected
- Transfer all different blobs to the Cool tier for value effectivity
With out Storage Actions, this is able to usually require:
- Writing scripts to iterate by way of blobs
- Dealing with credentials and permissions
- Testing logic on pattern information
- Scaling execution safely throughout giant datasets
- Sustaining and rerunning the scripts over time
With Storage Actions, the administrator:
- Defines circumstances primarily based on file extension and index tags
- Chains a number of actions (immutability, tiering, tagging)
- Makes use of a built-in preview functionality to validate which blobs match the circumstances
- Executes the duty with out provisioning infrastructure
All the workflow is authored declaratively within the Azure portal and executed by the platform.
Azure Storage Actions offers built-in observability:
- Preview circumstances enable prospects to validate logic in opposition to a subset of blobs earlier than execution
- Azure Monitor metrics monitor job runs, focused objects, and profitable operations
- Execution studies are generated as CSV information for every run, detailing:
- Blobs processed
- Actions carried out
- Execution standing for audit functions
This makes Storage Actions appropriate for eventualities the place traceability and assessment are vital.
Shashank shared a number of examples of how prospects are utilizing Azure Storage Actions at this time:
- Monetary companies: Making use of immutability and retention insurance policies to name recordings for compliance
- Airways: Value optimization by tiering or cleansing up blobs primarily based on creation time or dimension
- Manufacturing: One-time processing to reset or take away blob index tags on IoT-generated information
These eventualities vary from recurring automation to one-off operational duties.
Azure Storage Actions is on the market in over 40 public Azure areas.
To study extra, try:
For questions or suggestions, the crew might be reached at: storageactions@microsoft.com
