S3 and CloudFront: Static Assets and CDN

Concept

This lesson covers two tools that work as a pair. Each one solves a different problem:

Why files do not belong on the server

On Vercel you may have used its blob storage, or you may have put images in /public (the Next.js folder for plain files that are served as they are). On your own server the easy way is a folder such as /uploads. This works until one of these things happens:

• You run two servers. A customer uploads a PDF to server A. The next request goes to server B, which has never seen the file. The file “vanished”. This is the stateless rule from Lesson 8, applied to files. (A stateless server keeps no data of its own between requests.)
• A server is replaced. Auto Scaling (Lesson 12) is an AWS service that adds and removes servers as traffic changes. It removes servers when traffic drops. Every upload on a removed server disappears with its disk.
• The disk fills. EBS is the virtual hard disk of an EC2 server. A 20 GB disk is full after a few thousand good photos, and making it bigger is extra work.
• Your CPU serves files. Every image request keeps a Node or Nginx worker busy. That worker should be running your app logic.
• Distance. A user in Delhi loading images from a server in Mumbai is fine. A user in London is much slower, and you cannot make light travel faster.

S3 gives you storage with no practical limit and no disk to manage. It keeps copies of your data in at least three Availability Zones (separate groups of data centres). It has 99.999999999% (eleven nines) durability, which means a very small chance of ever losing an object. The price is about $0.025 per GB per month in Mumbai.

How S3 thinks: buckets, keys, objects

S3 looks like a folder tree in the console, but it is not a file system. It is a giant key-value store. That means you give it a name (the key), and it gives you back the file (the value), like a huge dictionary:

Two facts help you avoid real bugs. First, objects are immutable. You replace a whole object. You cannot edit part of it. Second, S3 has strong read-after-write consistency. Right after an upload, any read shows the new file.

Private by default — and keeping it that way

Many famous cloud data leaks came from public S3 buckets. AWS has since made the safe choice the default. You should keep it that way.

Who can read a private bucket, then?

• Your app, through its IAM role. A role is a set of permissions that a server can use without any stored password (next section).
• CloudFront, through a bucket policy (a permission rule attached to the bucket) that names the distribution. This is called Origin Access Control.
• One user for a short time, through a presigned URL (a temporary link, explained below).

Nobody else can read it, and the open internet cannot. That is what “private” means.

Create a bucket and use it

Go to S3 → Create bucket. Bucket names are shared by every AWS account in the world, so add something random, such as myapp-uploads-7f3a. Choose the region ap-south-1. Leave Block all public access switched on. Leave default encryption as it is. Both are already the safe choice. That is all.

From your laptop, the AWS CLI (the command-line tool for AWS) can copy files in and out, list a prefix, sync a whole folder, and delete. Try one command now. It shows the “private by default” idea at work:

Letting the app read and write it (no keys)

Lesson 5 created myapp-pdfs-rw, a policy that allows exactly get and put on one bucket. Now make a policy for your new bucket and attach it to the EC2 role. Notice the two different resources: the bucket itself, and the objects inside it. An ARN (Amazon Resource Name) is the unique ID of an AWS thing.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListOnlyThisBucket",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::myapp-uploads-7f3a"
    },
    {
      "Sid": "ReadWriteObjects",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject", "s3:DeleteObject"],
      "Resource": "arn:aws:s3:::myapp-uploads-7f3a/*"
    }
  ]
}

Create this as a policy (myapp-uploads-rw) and attach it to myapp-ec2-role. You do not need to restart anything. The server gets its temporary credentials from AWS by itself, and a policy change takes effect quickly, usually within seconds. On the server, with no keys set up, listing your bucket now works. Listing any other bucket is denied.

Presigned URLs — uploads that skip your server

Here is a common beginner upload flow. The browser sends a 40 MB video to your Next.js API, and the API forwards it to S3. This keeps your server busy for the whole upload and uses double the bandwidth. It also hits body-size limits (Nginx 413, and 4.5 MB on Vercel functions).

The better pattern uses a presigned URL. This is a web link with a built-in signature (a proof of permission). It allows one action on one object for a short time. Your server only signs a permission slip, and the browser uploads straight to S3.

1. Browser asks your API: “I want to upload photo.jpg (image/jpeg).”
2. Your API checks that the user is logged in and allowed. It makes a key and returns a presigned PUT URL that works for a few minutes.
3. The browser sends the file straight to S3 with a PUT request (the HTTP method for “store this”), using that URL.
4. Your API stores the key in the database.

import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";
import { randomUUID } from "node:crypto";
import { NextResponse } from "next/server";

// No credentials: the SDK (the AWS code library) finds the EC2 role by itself (Lesson 5)
const s3 = new S3Client({ region: "ap-south-1" });

const ALLOWED = ["image/jpeg", "image/png", "image/webp", "application/pdf"];

export async function POST(req: Request) {
  const user = await requireUser(req);                       // your own auth check — never skip this
  const { contentType } = await req.json();

  if (!ALLOWED.includes(contentType)) {
    return NextResponse.json({ error: "File type not allowed" }, { status: 400 });
  }

  const key = `tenants/${user.tenantId}/uploads/${randomUUID()}`;   // server decides the key

  const url = await getSignedUrl(
    s3,
    new PutObjectCommand({
      Bucket: process.env.UPLOAD_BUCKET!,
      Key: key,
      ContentType: contentType,              // the upload MUST send exactly this header
    }),
    { expiresIn: 300 },                      // 5 minutes
  );

  return NextResponse.json({ url, key });
}

async function upload(file: File) {
  const res = await fetch("/api/upload-url", {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({ contentType: file.type }),
  });
  const { url, key } = await res.json();

  const put = await fetch(url, {
    method: "PUT",
    headers: { "Content-Type": file.type },   // must match what was signed
    body: file,
  });
  if (!put.ok) throw new Error("Upload failed");
  return key;                                  // save this in your database
}

CORS, the error every browser upload hits

When the browser calls S3 directly, the browser's same-origin rule applies. An origin is the protocol, domain and port together, such as https://yourapp.com. A page from one origin may not use another origin (such as *.s3.amazonaws.com) unless that server agrees. CORS (Cross-Origin Resource Sharing) is the set of rules a server uses to say yes. Before the PUT, the browser sends a small check request (a preflight) to ask S3 whether it is allowed. Your first try will show “blocked by CORS policy”. The fix is a CORS configuration on the bucket:

{
  "CORSRules": [
    {
      "AllowedOrigins": ["https://yourapp.com", "https://*.yourapp.com", "http://localhost:3000"],
      "AllowedMethods": ["PUT", "GET", "HEAD"],
      "AllowedHeaders": ["Content-Type"],
      "ExposeHeaders": ["ETag"],
      "MaxAgeSeconds": 3000
    }
  ]
}

Paste it in the bucket's Permissions → CORS. List only origins that you own. Never use * for a bucket that accepts uploads. Remove the localhost origin from the production bucket when you no longer need it.

Versioning, lifecycle and storage classes

Versioning — the undo button

Versioning is a bucket setting that keeps every old version of an object. When you delete a file, S3 only adds a “delete marker” (a hidden note that makes the file look deleted). You can restore an overwritten or deleted file. This is the cheapest insurance for user data. It is also part of Lesson 18's backup story.

Both are in the console. Versioning is under Properties → Bucket versioning. A lifecycle rule is an automatic rule that acts on objects as they get older. Make one under Management → Create lifecycle rule with the three actions below.

This is what the lifecycle rule does:

• Old versions of files are deleted after 30 days. Without this, versioning grows forever.
• Half-finished uploads are cleaned up after 7 days. They are a common hidden cost.
• Files that nobody touches for 90 days move to a cheaper storage class automatically.

Storage classes

Start with Standard. Add Intelligent-Tiering when you have a lot of data and do not know how often it is read. Do not make this complicated until storage is a big part of your bill. The prices in the table are rough and differ by region. Some classes also have a minimum storage time (for example 30 days for Standard-IA), so check the current pricing page.

What a CDN is and why it is fast

Latency is the delay before data arrives. It depends on distance (data cannot travel faster than light) and on the number of hops (steps between machines). A file stored in Mumbai takes about 20 ms to reach a user in Mumbai. It can take 100–300 ms to reach London or São Paulo. A web page needs dozens of files, so the delays add up. A Content Delivery Network (CDN) fixes this. It keeps copies of your files in hundreds of small data centres (edge locations) near users.

A cache is a saved copy that is kept for quick reuse. The origin is the original server (here, S3). The first visitor in a city causes a cache miss: the edge location has no copy, so it fetches the file from your origin. Everyone after that gets a cache hit: the edge answers from its copy. Your origin sees only a small part of the traffic. This is also why a CDN protects your server from traffic spikes.

CloudFront in front of S3, step by step

The flow is: users → CloudFront → (only on a miss) → private S3 bucket. A distribution is one CloudFront setup. The bucket allows only that one distribution to read it. This works through Origin Access Control (OAC), a CloudFront setting that signs CloudFront's requests to S3. The console is the easiest way to create a distribution the first time:

1. Create the distribution

   Go to CloudFront → Create distribution. For Origin domain, pick your S3 bucket. For Origin access, choose Origin access control settings (recommended) and create a new OAC (signing: sign requests). Under Viewer protocol policy, choose Redirect HTTP to HTTPS. Turn on compression. Pick the cache policy CachingOptimized. Then click Create.

2. Apply the bucket policy CloudFront shows you

   After you create the distribution, the console shows a ready-made policy. It looks like this. Read it carefully. It says “only this distribution may read objects”:

   bucket policy for OACCopy

   {
     "Version": "2012-10-17",
     "Statement": [{
       "Sid": "AllowCloudFrontServicePrincipalReadOnly",
       "Effect": "Allow",
       "Principal": { "Service": "cloudfront.amazonaws.com" },
       "Action": "s3:GetObject",
       "Resource": "arn:aws:s3:::myapp-uploads-7f3a/*",
       "Condition": {
         "StringEquals": {
           "AWS:SourceArn": "arn:aws:cloudfront::123456789012:distribution/E1ABCDEF2GHIJK"
         }
       }
     }]
   }

   The Condition is the important part. Without it, any CloudFront distribution in the world could read your bucket. Paste the policy in the bucket's Permissions → Bucket policy.

3. Test the two paths

   Upload a test file. Then compare the direct S3 URL with the CloudFront URL:

   Direct S3 is denied, CloudFront worksCopy all

   • curl -I https://myapp-uploads-7f3a.s3.ap-south-1.amazonaws.com/assets/logo.pngCopy

     Direct to S3 you get 403 Forbidden. This is correct, because the bucket is private

   • curl -I https://d1234abcd.cloudfront.net/assets/logo.pngCopy

     Through CloudFront you get 200 OK. Run it twice. The second answer says x-cache: Hit from cloudfront, which means it came from the edge

4. Use your own domain: assets.yourapp.com

   The random name d1234abcd.cloudfront.net is ugly, and it ties you to CloudFront. To use assets.yourapp.com instead:

   • Request an ACM certificate for assets.yourapp.com in us-east-1 (N. Virginia). ACM is AWS Certificate Manager. CloudFront is a global service, and it reads certificates only from that one region. This is a common mistake, because everything else in this course is in Mumbai.
   • Add the name as an Alternate domain name on the distribution, and select the certificate.
   • In Route 53 (the AWS DNS service, Lesson 13), create an alias A record from assets.yourapp.com to the distribution.

Cache control, cache busting and invalidation

A cache has one hard problem: when the original file changes, how do the copies find out? There are three ways to handle this. The best way is to avoid the problem. Two terms first. The TTL (time to live) is how long a copy may be kept. The max-age setting in the Cache-Control header sets the TTL in seconds.

Next.js already does the third way for you. The file /_next/static/chunks/main-8f3a2c.js has a content hash in its name (a hash is a short code made from the file's content, so it changes when the content changes), so it can use Cache-Control: public, max-age=31536000, immutable. (immutable tells the cache that the file will never change.) For your own uploads, use the same trick. Never overwrite a key. Upload to a new UUID key and save the new key in the database.

Using it from Next.js

There are two common ways to use the CDN. First, serve your built /_next/static files from it. You do this with assetPrefix, a setting that puts your CDN address in front of those file URLs. Next.js writes the new URLs into the HTML, so the files load from the edge instead of your server:

import type { NextConfig } from "next";

const nextConfig: NextConfig = {
  output: "standalone",
  // Production only, so local development works without a CDN
  assetPrefix: process.env.NODE_ENV === "production" ? "https://assets.yourapp.com" : undefined,
  images: {
    remotePatterns: [{ protocol: "https", hostname: "assets.yourapp.com" }],
  },
};

export default nextConfig;

Then upload the static build output as part of your deploy. This step belongs in the CI pipeline of Lesson 14. (CI/CD is a robot that checks, builds and ships your code.) Only the .next/static folder goes to the CDN:

aws s3 sync .next/static s3://myapp-uploads-7f3a/_next/static \
  --cache-control "public, max-age=31536000, immutable"

Do not upload the rest of the .next folder, because it holds your server code. Also note that assetPrefix does not cover files in the public folder. If you want those on the CDN, you must add the prefix yourself.

What this costs

For a small company's image and file traffic, this is usually a few dollars a month. It is often cheaper than serving the same files from EC2, because data that leaves EC2 is billed, but data from S3 to CloudFront is free. Prices change, so check the current pricing pages.

Troubleshooting table

Interview corner

Practice task before Lesson 12

1. Create a private bucket with encryption and Block Public Access. Upload a file and prove the plain object URL returns 403 while a presign URL works.
2. Attach the permission policy to myapp-ec2-role. Then confirm, from the server with no keys, that you can list this bucket but not another one.
3. Create a CloudFront distribution with OAC. Confirm direct-to-S3 is 403 and the CloudFront URL returns Miss then Hit.
4. Build the /api/upload-url route and a small page that uploads a file to S3. You will hit a CORS error first — fix it with the bucket CORS config.
5. Turn on versioning. Overwrite a file, then restore the earlier version in the console.
6. Re-upload a changed logo.png to the same key and observe the CDN serving the stale copy. Solve it two ways: an invalidation, and by renaming to logo-v2.png. Which would you do in production?

Conclusion

S3 stores files safely and cheaply. CloudFront delivers them quickly. Together they take the heaviest and most repeated work away from your servers. This makes the servers easy to replace.

• Files go in S3, records in RDS. The database keeps the key, not the file.
• Private by default: Block Public Access on the account; access through the IAM role, CloudFront OAC or a presigned URL — never a public bucket.
• Presigned URLs let the browser upload directly, which removes size limits and server load. Remember CORS.
• Version your file names, so a CDN can cache them forever and you never need to invalidate. Lifecycle rules keep the bill small.
• The ACM certificate for CloudFront must be in us-east-1. Remember this one, because it surprises many people.

Your data is now in RDS and your files are in S3, so your app servers are finally stateless. This must be true before the next lesson: running many servers behind a load balancer.

End of Lesson 11. Next: Lesson 12 — ALB and Auto Scaling: Scaling to 1000+ Concurrent Users.