TechOps Examples
Hey — It's Govardhana MK 👋
Welcome to another technical edition.
Every Tuesday – You’ll receive a free edition with a byte-size use case, remote job opportunities, top news, tools, and articles.
Every Thursday and Saturday – You’ll receive a special edition with a deep dive use case, remote job opportunities, and articles.
Staying behind on tech trends can be a career killer.
But let’s face it, no one has hours to spare every week trying to stay updated.
That’s why over 150,000 engineers at companies like Google, Meta, and Apple read The Code twice a week.
Curated tech news that shapes your career - Filtered from thousands of sources so you know what's coming 6 months early.
Practical resources you can use immediately - Real tutorials and tools that solve actual engineering problems.
Research papers and insights decoded - Practical complex tech break downs.
👀 Remote Jobs
Tether is hiring a DevOps Engineer
Remote Location: Worldwide
Weekday AI is hiring a DevSecOps Engineer
Remote Location: India
Powered by: Jobsurface.com
📚 Resources
Looking to promote your company, product, service, or event to 54,000+ Cloud Native Professionals? Let's work together. Advertise With Us
🧠 DEEP DIVE USE CASE
How Cloud Engineers Optimize Amazon S3 for High Performance Workloads
I see too often many cloud practitioners jumping straight into S3 performance tweaks without understanding storage classes. Picking the wrong class silently drains performance and money.
Here’s what each storage class actually offers:
S3 Express One Zone
Sub millisecond latency, over 10 GBps per prefix, single AZ, best for high performance compute or AI pipelines.S3 Standard
99.99 percent availability, multi AZ, millisecond latency, suitable for frequently accessed data.S3 Standard-Infrequent Access
99.9% availability, multi AZ, 30 day minimum storage, per GB retrieval charge, ideal for infrequently accessed but critical data.S3 One Zone-Infrequent Access
Same as Standard-IA but in one AZ, lower cost, 99.5% availability, use for secondary backups or recreatable data.S3 Glacier Instant Retrieval
Millisecond access, archival grade pricing, multi AZ, for active archives like medical or legal data.S3 Glacier Flexible Retrieval
Access in minutes or hours, cheaper than Instant, with options for expedited, standard, and bulk retrieval.S3 Glacier Deep Archive
Lowest cost, 12 to 48 hour retrieval, for long-term archives and compliance data.
On top of this, use S3 Intelligent Tiering to automatically move data across Frequent, Infrequent, Archive, and Deep Archive tiers based on actual access, with no retrieval fees and only pay per objects monitored.
With this basic understanding, let us look into the prime aspects of S3 performance optimization, such as:
Multi Part Upload
S3 Transfer Acceleration
S3 Select and Glacier Select
Pre signed URLs
1. Multi Part Upload
In production, uploading anything over 100 MB without multi part upload is asking for trouble. We’ve seen S3 PUT operations fail silently on large monolithic uploads, especially over unstable or throttled VPC endpoints.
In that case, always use the Multi Part Upload API. It breaks the file into parts and uploads them in parallel, improving speed, fault tolerance, and resource efficiency.
Field tested recommendations:
Set part size between 25 MB and 100 MB
Limit total parts to under 10,000 per object
Use 5 to 10 concurrent threads for uploads from EC2
Pre calculate part count for files over 100 GB
Monitor CompleteMultipartUpload metrics. Spikes indicate backend retries or IOPS saturation.
In VPC or low bandwidth setups, avoid aggressive concurrency. Adjust part size and threads based on network. Use --checksum-algorithm SHA256 if consistency matters.
2. S3 Transfer Acceleration
A common misconception is that enabling Transfer Acceleration will always make S3 uploads faster. In reality, it helps only when data is uploaded over long geographic distances.
For same region or intra VPC uploads, it often adds no benefit and just increases cost.
🔴 Get my DevOps & Kubernetes ebooks! (free for Premium Club and Personal Tier newsletter subscribers)
Upgrade to Paid to read the rest.
Become a paying subscriber to get access to this post and other subscriber-only content.
UpgradePaid subscriptions get you:
- Access to archive of 250+ use cases
- Deep Dive use case editions (Thursdays and Saturdays)
- Access to Private Discord Community
- Invitations to monthly Zoom calls for use case discussions and industry leaders meetups
- Quarterly 1:1 'Ask Me Anything' power session
