DevOps cheat sheet

aws sts get-caller-identity

Who am I? The ARN must say user/, never root

chmod 600 .env

Only you can read secrets

chmod 400 key.pem

SSH refuses looser key files

ps aux | grep node

Is the app running?

df -h

Disk full? Check this first

Watch out: Root user: MFA on, no keys, never for daily work. Closing the terminal kills npm start; use PM2.

• Ports: 22 SSH, 80 HTTP, 443 HTTPS, 5432 Postgres, 6379 Redis. Port 3000 is never public.
• Security Group: inbound denied by default, stateful (replies return by themselves).
• SSH source is your /32, never 0.0.0.0/0. A DB rule's source is the app's group.

ps aux | grep node
sudo ss -tulpn | grep 3000
nc -zv SERVER_IP 3000

• TTL is how long resolvers cache. Lower it to 60 to 300 before a move, raise it after. '24 to 48 hours' is a myth.
• *.yourapp.com serves every tenant subdomain, not the bare domain. The app reads Host.
• Migrate with weighted records (10, 50, 100). Copy MX, SPF, DKIM too.

dig NAME +short

What IP does it resolve to?

curl --resolve H:443:IP URL

Test the new stack before DNS

• User = person, group = department, policy = rule card, role = temporary key card for machines.
• Everything is denied by default. An explicit Deny always wins.
• Attach policies to groups, not users. People use Identity Center; machines use roles.

{
  "Effect": "Allow",
  "Action": [
    "s3:PutObject",
    "s3:GetObject"
  ],
  "Resource":
    "arn:aws:s3:::myapp-pdfs/*"
}

Watch out: Long-lived keys in a .env are the costliest mistake. Never commit them. Set a budget alert first.

• AWS keeps 5 addresses per subnet. Private ranges: 10/8, 172.16/12, 192.168/16. Peered VPCs must not overlap.
• Public subnet = route 0.0.0.0/0 to the Internet Gateway, plus a public IP. Private = only the local route.

Watch out: A NAT Gateway costs about $40 a month idle plus $0.056/GB. Use the free S3 gateway endpoint instead.

ssh -i key.pem ubuntu@IP

Ubuntu user is always ubuntu

pm2 start pnpm --name myapp -- start

Run under PM2

pm2 startup && pm2 save

Survive a reboot

• Stop keeps disk and Elastic IP (about $6/month). Terminate deletes for good; release the IP yourself.
• NEXT_PUBLIC_* is baked in at build time: rebuild. Add swap for next build.

• RDS: private subnet group, public access No, db-sg only from web-sg. App uses appuser, never dbadmin.
• Point-in-time restore makes a new instance. Multi-AZ copies a DROP TABLE too; only backups help.
• Connections: servers x pool <= 80% of max_connections (about 100 on micro). Fix: smaller pool, bigger box, RDS Proxy or PgBouncer.

• Redis: cache-aside with a TTL (plus jitter), tenant ID in every key, revocable sessions, INCR rate limits.
• Eviction: allkeys-lru for a cache, noeviction for queues. Never the only copy; fail open.

• Order layers: copy lockfile and install first, COPY . . last. Multi-stage build, standalone output, non-root USER, HOSTNAME=0.0.0.0.
• .dockerignore holds .env*, .git. Tag by commit SHA, never latest. Secrets at run time, not in layers.

docker run -d --name myapp \
  --restart unless-stopped \
  --env-file ~/myapp.env \
  -p 127.0.0.1:3000:3000 \
  --memory 1g myapp:a1b2c3d

Tip: Run sudo nginx -t before every systemctl reload nginx. Use 127.0.0.1, not localhost, in proxy_pass.

• Files go in S3, the key goes in the database. Key shape: tenants/ID/uploads/UUID; never raw user names.
• Block Public Access on. App reads and writes through its instance role.

1. Browser asks the API to upload.
2. API checks login, returns a presigned PUT URL (5 minutes).
3. Browser PUTs straight to S3 with the same Content-Type.
4. API saves the key. Needs bucket CORS for your origins, never *.

Watch out: The CloudFront certificate must live in us-east-1. Keep the bucket private with OAC and a policy on AWS:SourceArn.

• Stateless first: sessions to cookie or Redis, files to S3, cron to one scheduler.
• Policy: target tracking on CPU (about 50%). Scale out fast, in slowly.
• Keep the health check shallow: a DB check can fail every server at once.
• Deploy: new tag in SSM, then instance refresh (min healthy 100%, max 110%). Rollback is the old tag.

Watch out: Random 502s: Node keep-alive (5 s) is shorter than the ALB idle timeout (60 s). Set KEEP_ALIVE_TIMEOUT=65000. 503 = no targets, 504 = slower than 60 s.

1. Push: lint, types, tests.
2. Build the image, tag it with the SHA, push to ECR.
3. Approval gate on the production environment.
4. Migrations through SSM, then instance refresh.

• OIDC, no stored keys: the trust policy sub pins repo:ORG/app:ref:refs/heads/main; the job needs id-token: write.
• Deploys use concurrency with cancel-in-progress: false. Pin actions to a SHA.

• Terraform loop: init, plan, read it, apply. State lives in a versioned, locked S3 bucket and holds secrets.
• Use for_each not count; prevent_destroy on data; ignore_changes on desired_capacity.

• Alarms: 5xx >= 5 (2 of 3 min), p95 > 1 s, healthy hosts < 1, no requests for 10 min, RDS free storage < 5 GB.
• EC2 shows no memory or disk without the CloudWatch agent. Docker awslogs ships logs off the server.
• Alert on symptoms; page for outages. Every alarm links a runbook. Log JSON with requestId, tenantId.

Watch out: Log ingestion costs about $0.50 to $0.70 per GB. Use 30-day retention, info level, no health-check lines.

• AWS secures the cloud; you secure what is in it. Only the ALB takes 0.0.0.0/0 (80, 443). Never 22, 3389, 5432, 6379, 3000.
• Session Manager replaces SSH (AmazonSSMManagedInstanceCore, close 22). IMDSv2 required, EBS encryption default.
• Secrets in SSM SecureString or Secrets Manager. Turn on CloudTrail and GuardDuty.
• Backups 3-2-1. A backup is real only after a restore drill; the time it takes is your RTO.

1. Leaked key: aws iam update-access-key --status Inactive (do not delete yet).
2. Attach an explicit Deny-all to that identity.
3. Read CloudTrail by key ID, in every region.
4. Remove unknown instances, users, roles; rotate what it could reach.