Terraform State Management: Remote Backends, Locking, and Best Practices

Terraform state is the heart of how Terraform tracks your infrastructure. Understanding state management is critical for team collaboration, security, and disaster recovery. This guide covers local and remote state, state locking, importing resources, and state manipulation commands.

📑 Table of Contents

What is Terraform State?

Terraform state is a JSON file that maps your configuration to real-world resources. It contains:

Resource IDs and attributes
Metadata about resources
Dependencies between resources
Provider configurations

Why State Matters

Resource Tracking – Maps config to real infrastructure
Performance – Caches resource attributes
Dependencies – Determines creation/destruction order
Collaboration – Enables team workflows

Local State

By default, Terraform stores state locally in terraform.tfstate.

# Example terraform.tfstate structure
{
  "version": 4,
  "terraform_version": "1.7.0",
  "resources": [
    {
      "type": "aws_instance",
      "name": "web",
      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
      "instances": [
        {
          "attributes": {
            "id": "i-1234567890abcdef0",
            "ami": "ami-0c55b159cbfafe1f0",
            "instance_type": "t2.micro",
            "public_ip": "54.123.45.67"
          }
        }
      ]
    }
  ]
}

Problems with Local State

Not suitable for team collaboration
Risk of data loss
No locking mechanism
Sensitive data stored locally

Remote State Backends

Remote backends store state in a shared location, enabling team collaboration.

S3 Backend (AWS)

# Create S3 bucket and DynamoDB table first
# Then configure backend:

terraform {
  backend "s3" {
    bucket         = "my-terraform-state-bucket"
    key            = "prod/infrastructure/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-locks"
  }
}

Setting Up S3 Backend Infrastructure

# bootstrap/main.tf - Run this first with local state

provider "aws" {
  region = "us-east-1"
}

resource "aws_s3_bucket" "terraform_state" {
  bucket = "my-terraform-state-bucket"
  
  lifecycle {
    prevent_destroy = true
  }
}

resource "aws_s3_bucket_versioning" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_server_side_encryption_configuration" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

resource "aws_s3_bucket_public_access_block" "terraform_state" {
  bucket = aws_s3_bucket.terraform_state.id
  
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_dynamodb_table" "terraform_locks" {
  name         = "terraform-state-locks"
  billing_mode = "PAY_PER_REQUEST"
  hash_key     = "LockID"
  
  attribute {
    name = "LockID"
    type = "S"
  }
}

Azure Storage Backend

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "tfstatestorage"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

Google Cloud Storage Backend

terraform {
  backend "gcs" {
    bucket = "my-terraform-state"
    prefix = "prod/infrastructure"
  }
}

Terraform Cloud Backend

terraform {
  cloud {
    organization = "my-organization"
    
    workspaces {
      name = "my-workspace"
    }
  }
}

State Locking

State locking prevents concurrent operations that could corrupt state.

How Locking Works

Before any write operation, Terraform acquires a lock
Other operations wait or fail
After completion, the lock is released

DynamoDB Locking (S3 Backend)

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"  # Enables locking
    encrypt        = true
  }
}

Force Unlock

# Only use if lock is stuck due to crash
terraform force-unlock LOCK_ID

# Example
terraform force-unlock 12345678-1234-1234-1234-123456789012

Migrating State

Local to Remote

# 1. Add backend configuration
terraform {
  backend "s3" {
    bucket = "my-terraform-state"
    key    = "terraform.tfstate"
    region = "us-east-1"
  }
}

# 2. Initialize with migration
terraform init -migrate-state

# Answer yes to migrate existing state

Remote to Remote

# 1. Update backend configuration
# 2. Run init with migration
terraform init -migrate-state -force-copy

Remote to Local

# 1. Remove or comment out backend block
# 2. Run init
terraform init -migrate-state

State Commands

View State

# Show full state
terraform show

# Show specific resource
terraform state show aws_instance.web

# List all resources
terraform state list

# List with filtering
terraform state list aws_instance.*

Move Resources

# Rename resource in state
terraform state mv aws_instance.old aws_instance.new

# Move to different module
terraform state mv aws_instance.web module.compute.aws_instance.web

# Move module
terraform state mv module.old_name module.new_name

Remove Resources

# Remove from state (does not destroy resource)
terraform state rm aws_instance.web

# Useful when:
# - Resource was manually deleted
# - Want to stop managing a resource
# - Moving resource to different state

Import Resources

# Import existing resource into state
terraform import aws_instance.web i-1234567890abcdef0

# Import with module path
terraform import module.compute.aws_instance.web i-1234567890abcdef0

# Import examples by resource type
terraform import aws_vpc.main vpc-12345678
terraform import aws_s3_bucket.data my-bucket-name
terraform import aws_security_group.web sg-12345678

Pull and Push State

# Download state to local file
terraform state pull > state.json

# Push local state to remote (dangerous!)
terraform state push state.json

Working with Multiple State Files

Workspaces

# Create workspaces
terraform workspace new dev
terraform workspace new staging
terraform workspace new prod

# List workspaces
terraform workspace list

# Select workspace
terraform workspace select prod

# Delete workspace
terraform workspace delete dev

# Each workspace has separate state file

Separate Backend Keys

# dev/backend.tf
terraform {
  backend "s3" {
    bucket = "terraform-state"
    key    = "dev/terraform.tfstate"
    region = "us-east-1"
  }
}

# prod/backend.tf
terraform {
  backend "s3" {
    bucket = "terraform-state"
    key    = "prod/terraform.tfstate"
    region = "us-east-1"
  }
}

Remote State Data Source

Access outputs from other Terraform configurations:

# Read outputs from another state file
data "terraform_remote_state" "vpc" {
  backend = "s3"
  
  config = {
    bucket = "terraform-state"
    key    = "vpc/terraform.tfstate"
    region = "us-east-1"
  }
}

# Use the outputs
resource "aws_instance" "web" {
  subnet_id = data.terraform_remote_state.vpc.outputs.public_subnet_ids[0]
  vpc_security_group_ids = [
    data.terraform_remote_state.vpc.outputs.web_security_group_id
  ]
}

State Refresh

# Sync state with real infrastructure
terraform refresh

# Or use plan/apply with refresh
terraform plan -refresh=true  # default
terraform plan -refresh=false # skip refresh

Handling State Conflicts

Scenario: Resource Drift

# Someone manually changed a resource
# Terraform plan shows differences

# Option 1: Apply to restore desired state
terraform apply

# Option 2: Accept manual changes
terraform apply -refresh-only

Scenario: Corrupted State

# Use backup if available
cp terraform.tfstate.backup terraform.tfstate

# Or pull previous version from S3
aws s3api list-object-versions --bucket my-terraform-state --prefix terraform.tfstate

# Download specific version
aws s3api get-object --bucket my-terraform-state --key terraform.tfstate --version-id VERSION_ID terraform.tfstate

Security Best Practices

Enable encryption – Always encrypt state at rest
Restrict access – Use IAM policies for state bucket
Enable versioning – Keep history for recovery
Use state locking – Prevent concurrent modifications
Never commit state – Add to .gitignore
Audit access – Enable logging on state bucket

# .gitignore
*.tfstate
*.tfstate.*
.terraform/
*.tfvars  # If contains secrets

IAM Policy for State Bucket

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject"
      ],
      "Resource": "arn:aws:s3:::my-terraform-state/*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::my-terraform-state"
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:*:*:table/terraform-locks"
    }
  ]
}

Disaster Recovery

State Backup Strategy

# Enable S3 versioning for automatic backups
resource "aws_s3_bucket_versioning" "state" {
  bucket = aws_s3_bucket.state.id
  
  versioning_configuration {
    status = "Enabled"
  }
}

# Set lifecycle policy
resource "aws_s3_bucket_lifecycle_configuration" "state" {
  bucket = aws_s3_bucket.state.id
  
  rule {
    id     = "state-versions"
    status = "Enabled"
    
    noncurrent_version_expiration {
      noncurrent_days = 90
    }
  }
}

Recovery from Lost State

# 1. Create new state file
terraform init

# 2. Import all resources
terraform import aws_vpc.main vpc-12345
terraform import aws_subnet.public subnet-12345
# ... import all resources

# 3. Verify with plan
terraform plan
# Should show no changes if imports are complete

Conclusion

Proper state management is essential for production Terraform workflows. Always use remote backends for team environments, enable state locking, and follow security best practices. Regular backups and understanding state commands will help you recover from issues quickly.

Was this article helpful?