TechOps Examples
Hey — It's Govardhana MK 👋
Welcome to another technical edition.
Every Tuesday – You’ll receive a free edition with a byte-size use case, remote job opportunities, top news, tools, and articles.
Every Thursday and Saturday – You’ll receive a special edition with a deep dive use case, remote job opportunities and articles.
Top engineers at Anthropic and OpenAI say AI now writes 100% of their code.
If you're not using AI, you're spending 40 hours doing what they do in 4.
These 100+ Claude Code hacks fix that and help you ship 10x faster.
Sign up for The Code and get:
100+ Claude Code hacks used by top engineers — free
The Code newsletter — learn the latest AI tools, tips, and skills to code faster with AI in 5 minutes a day
🛠 TOOL OF THE DAY
gitdiagram - Turn any GitHub repository into an interactive diagram for visualization.
🧠 USE CASE
Steps to Prevent and Recover Lost Terraform State Files
In my recent LinkedIn post, I wrote:
Among the responses, Rob Larsen shared this experience
This pain is all too familiar for many of us in the DevOps world, and I couldn't walk away with just a comment.
Here's my take on it...
What Can You Do to Prevent This?
1. Enable State File Versioning in Remote Backends
Use an S3 bucket with versioning enabled.
Example S3 backend configuration:
terraform {
backend "s3" {
bucket = "tf-state-bucket"
key = "path/to/terraform.tfstate"
region = "us-east-1"
encrypt = true
dynamodb_table = "terraform-lock-table"
}
}
Versioning ensures that you can recover from accidental deletions or corruptions by restoring a previous state.
2. Lock the State File
Always use locking mechanisms (e.g., DynamoDB for S3 backend). This prevents simultaneous writes and ensures data integrity.
Add dynamodb_table to your backend config for distributed locks.
3. Take Regular Backups
Automate state file backups using lifecycle policies or scripts.
aws s3 cp s3://tf-state-bucket/path/to/terraform.tfstate /
s3://backup-bucket/path/to/backup-$(date +%Y-%m-%d).tfstate
4. Use Workspaces Wisely
Avoid overloading a single workspace with multiple environments (e.g., dev, prod).
Keep state files separated for better manageability.
terraform workspace new dev
terraform workspace new prod
5. Access Control and Least Privilege
Limit who can delete state files. Use IAM policies to enforce.
{
"Effect": "Deny",
"Action": "s3:DeleteObject",
"Resource": "arn:aws:s3:::tf-state-bucket/*",
"Condition": {
"BoolIfExists": {"aws:MultiFactorAuthPresent": false}
}
}
Despite the best intentions and efforts, things can still go wrong, as highlighted in Rob Larsen's comment.
How to Approach If This Happens
1. Restore a Previous Version from Backend
aws s3api copy-object
--copy-source tf-state-bucket/path/to/terraform.tfstate?versionId=123456789
--bucket tf-state-bucket
--key path/to/terraform.tfstate
2. Recreate the State File with Terraformer (If Completely Lost)
terraformer import aws --resources=ec2,s3 --regions=us-east-1 --profile=default
terraform plan -refresh-only
terraform apply
aws: Specifies the provider.
--resources=ec2,s3: Defines the resource types to import (e.g., EC2 instances and S3 buckets).
--regions=us-east-1: Targets the AWS region.
--profile=default: (Optional) Specifies the AWS CLI profile to use.
Terraformer automates resource discovery and imports them into Terraform state while generating the corresponding .tf configuration files.
Validate and apply carefully.
Looking to promote your company, product, service, or event to 30,000+ TechOps Professionals? Let's work together.
