Ask a team where their Terraform state lives and you will hear one of three answers: someone's laptop, a shared S3 bucket with no locking, or — if you are lucky — a remote backend with locking configured.
The first two are how infrastructure goes missing.
State is the source of truth, not the code
Your .tf files describe intent. The state file describes reality: which resources exist, their IDs, and the attributes the provider computed. Terraform reconciles the two on every plan.
Lose the state and Terraform does not know what it created. It will either refuse to act or, worse, recreate resources that already exist — which is how you end up with two load balancers and one bill.
The state file is a database. It deserves the same respect as your production one: backups, locking, and exactly one writer at a time.
Remote backend plus locking, or nothing
The minimum viable setup is a remote backend with state locking. On Terraform with AWS that is an S3 bucket plus a DynamoDB lock table, or an S3 bucket with native locking where available:
terraform {
backend "s3" {
bucket = "acme-tf-state"
key = "prod/network/terraform.tfstate"
region = "eu-west-1"
encrypt = true
dynamodb_table = "acme-tf-locks"
}
}
The lock table is not optional polish. Two concurrent apply runs against the same state produce a corrupted state file, and a corrupted state file is a weekend.
Drift is a feature request, not an insult
Drift — reality diverging from state — happens because someone changed something in the console. It is not a moral failure. It is a signal that the console was easier than the pipeline.
Handle it the way you handle any divergence between two sources of truth:
- Detect it on a schedule (
terraform planin CI, nightly). - Decide deliberately: import the change into code, or revert it.
- Never
refreshyour way out of a surprise you do not understand.
The teams that are good at this are not the ones with zero drift. They are the ones who notice drift within a day instead of during the next incident.
Conversation
0 commentsNo comments yet. Start the conversation below.