AWS S3 — Object Storage, Storage Classes, Security, and Replication

What is S3?

Amazon S3 (Simple Storage Service) is AWS’s object storage service. It stores files — called objects — in containers called buckets. S3 is not a filesystem; it has no folders, no hierarchy. The slash in images/photo.jpg is just part of the key name — it looks like a folder but it is not.

S3 is globally unique by bucket name, regionally stored, and built for 99.999999999% (eleven nines) durability. AWS achieves this by automatically replicating every object across at least 3 Availability Zones.

---

Buckets and Objects

Buckets

A bucket is a container for objects. Each bucket has a globally unique name — no two buckets in all of AWS can share a name. Buckets are created in a specific region; objects stay in that region unless you replicate them.

Naming rules: 3–63 characters, lowercase letters/numbers/hyphens only, no dots (causes SSL issues), must start with a letter or number.

# Create a bucket
aws s3api create-bucket \
  --bucket my-unique-bucket-name \
  --region eu-west-1 \
  --create-bucket-configuration LocationConstraint=eu-west-1

# List buckets
aws s3api list-buckets --output table

# High-level S3 commands (simpler syntax)
aws s3 ls                                        # list all buckets
aws s3 ls s3://my-bucket/                        # list objects in bucket
aws s3 cp file.txt s3://my-bucket/               # upload
aws s3 cp s3://my-bucket/file.txt .              # download
aws s3 sync ./local-dir s3://my-bucket/prefix/   # sync directory
aws s3 rm s3://my-bucket/file.txt                # delete object

📸 SCREENSHOT: S3 → Create Bucket. Show the bucket name field, region dropdown, and the Block Public Access settings section below it.

Objects

An object is a file + metadata. Maximum object size is 5 TB. For objects larger than 100 MB, use multipart upload — splits the file into parts, uploads in parallel, assembles on S3.

# Upload with multipart (automatic for large files via CLI)
aws s3 cp large-file.zip s3://my-bucket/ \
  --storage-class STANDARD_IA \
  --metadata "project=alpha,version=1.2"

# Low-level multipart upload
aws s3api create-multipart-upload --bucket my-bucket --key large-file.zip
# ... upload parts ...
aws s3api complete-multipart-upload --bucket my-bucket --key large-file.zip \
  --multipart-upload file://parts.json

---

Storage Classes

S3 has multiple storage classes — choose based on access frequency and retrieval speed requirements.

Class	Use Case	Availability	Min Duration	Retrieval
S3 Standard	Frequently accessed data	99.99%	None	Milliseconds
S3 Intelligent-Tiering	Unknown access patterns	99.9%	None	Milliseconds
S3 Standard-IA	Infrequent access, rapid retrieval	99.9%	30 days	Milliseconds
S3 One Zone-IA	Infrequent, can recreate if AZ fails	99.5%	30 days	Milliseconds
S3 Glacier Instant Retrieval	Archive, accessed ~quarterly	99.9%	90 days	Milliseconds
S3 Glacier Flexible Retrieval	Archive, 1–5 minute retrieval	99.99%	90 days	1–5 min / 3–5 hrs
S3 Glacier Deep Archive	Compliance archive, 12hr retrieval	99.99%	180 days	12 hours

S3 Intelligent-Tiering monitors access patterns and automatically moves objects between tiers — useful when you don’t know how often files will be accessed.

---

Lifecycle Policies

Lifecycle policies automatically transition objects between storage classes or expire (delete) them after a set number of days. This is the primary way to control S3 costs — move old data to cheaper tiers.

📸 SCREENSHOT: S3 → Bucket → Management → Lifecycle Rules → Create Lifecycle Rule. Show a rule with two transitions: Standard → Standard-IA at 30 days, then Standard-IA → Glacier at 90 days, and expiration at 365 days.

aws s3api put-bucket-lifecycle-configuration \
  --bucket my-bucket \
  --lifecycle-configuration '{
    "Rules": [{
      "ID": "archive-old-objects",
      "Status": "Enabled",
      "Filter": {"Prefix": "logs/"},
      "Transitions": [
        {"Days": 30, "StorageClass": "STANDARD_IA"},
        {"Days": 90, "StorageClass": "GLACIER"}
      ],
      "Expiration": {"Days": 365}
    }]
  }'

---

Versioning

Versioning keeps every version of every object in a bucket. When you overwrite or delete an object, the old version is preserved. Deletion creates a delete marker rather than removing the object — the object is still recoverable.

# Enable versioning
aws s3api put-bucket-versioning \
  --bucket my-bucket \
  --versioning-configuration Status=Enabled

# List object versions
aws s3api list-object-versions --bucket my-bucket --prefix myfile.txt

# Restore a previous version (copy it back as the current version)
aws s3api copy-object \
  --bucket my-bucket \
  --copy-source "my-bucket/myfile.txt?versionId=abc123" \
  --key myfile.txt

Versioning cannot be disabled once enabled — only suspended. Use lifecycle policies to expire old versions and control costs (non-current version expiration).

---

Security

Block Public Access

The Block Public Access setting is the master switch that prevents any public access regardless of bucket policies or ACLs. Enable it on every bucket that should not be public — which is most of them.

📸 SCREENSHOT: S3 → Bucket → Permissions → Block Public Access. Show all four checkboxes enabled (Block all public access).

aws s3api put-public-access-block \
  --bucket my-bucket \
  --public-access-block-configuration \
    BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

Bucket Policies

Bucket policies are JSON IAM policies attached to the bucket. They control who can access the bucket and what they can do.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"AWS": "arn:aws:iam::123456789012:role/app-role"},
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::my-bucket/*"
    },
    {
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": ["arn:aws:s3:::my-bucket", "arn:aws:s3:::my-bucket/*"],
      "Condition": {
        "Bool": {"aws:SecureTransport": "false"}
      }
    }
  ]
}

The second statement enforces HTTPS — any unencrypted HTTP request is denied. This is a security best practice for all buckets.

ACLs

Object ACLs are a legacy access control mechanism. AWS recommends disabling ACLs and using bucket policies instead. When you create a new bucket, the default is “ACLs disabled” (Object Ownership = Bucket owner enforced).

---

Encryption

All objects uploaded to S3 are encrypted by default since January 2023.

Type	Key Managed By	Use Case
SSE-S3	AWS (S3 managed keys)	Default encryption — zero admin overhead
SSE-KMS	AWS KMS (you choose the key)	Audit trail, key rotation, cross-account
SSE-C	You (provide key per request)	Bring your own key, not stored in AWS
DSSE-KMS	AWS KMS (dual-layer)	Compliance requiring two independent encryption layers

# Upload with SSE-KMS
aws s3 cp file.txt s3://my-bucket/ \
  --sse aws:kms \
  --sse-kms-key-id arn:aws:kms:eu-west-1:123456789012:key/mrk-abc123

# Set default encryption on bucket
aws s3api put-bucket-encryption \
  --bucket my-bucket \
  --server-side-encryption-configuration '{
    "Rules": [{
      "ApplyServerSideEncryptionByDefault": {
        "SSEAlgorithm": "aws:kms",
        "KMSMasterKeyID": "arn:aws:kms:...:key/mrk-abc123"
      },
      "BucketKeyEnabled": true
    }]
  }'

Bucket Key reduces the number of KMS API calls (and therefore KMS costs) by generating a short-lived bucket-level key that encrypts object keys locally, rather than calling KMS for every object.

---

Pre-Signed URLs

A pre-signed URL grants temporary access to a private S3 object without requiring AWS credentials. The URL contains a signature valid for a set duration (up to 7 days). Use for sharing private files, browser uploads, or one-time download links.

# Generate a pre-signed download URL (valid for 1 hour)
aws s3 presign s3://my-bucket/private-report.pdf --expires-in 3600

# Pre-signed upload URL (requires SDK — CLI only supports GET)
# Python example:
import boto3
s3 = boto3.client('s3')
url = s3.generate_presigned_url(
    'put_object',
    Params={'Bucket': 'my-bucket', 'Key': 'uploads/file.pdf'},
    ExpiresIn=3600
)

---

Static Website Hosting

S3 can serve a static website directly — HTML, CSS, JS, images. No server required. Pair with CloudFront for HTTPS, custom domains, and global caching.

# Enable static website hosting
aws s3api put-bucket-website \
  --bucket my-bucket \
  --website-configuration '{
    "IndexDocument": {"Suffix": "index.html"},
    "ErrorDocument": {"Key": "404.html"}
  }'

# The website endpoint (HTTP only)
# http://my-bucket.s3-website-eu-west-1.amazonaws.com

---

Replication

Cross-Region Replication (CRR)

Automatically copies objects to a bucket in a different region. Use for disaster recovery, geographic distribution, or compliance requirements.

Same-Region Replication (SRR)

Copies objects to another bucket in the same region. Use for log aggregation, test/prod account separation, or compliance data copies.

Requirements for both: versioning must be enabled on source and destination buckets.

aws s3api put-bucket-replication \
  --bucket source-bucket \
  --replication-configuration '{
    "Role": "arn:aws:iam::123456789012:role/s3-replication-role",
    "Rules": [{
      "Status": "Enabled",
      "Filter": {"Prefix": ""},
      "Destination": {
        "Bucket": "arn:aws:s3:::destination-bucket",
        "StorageClass": "STANDARD_IA"
      }
    }]
  }'

Replication only copies new objects after the rule is created. To replicate existing objects, use S3 Batch Replication.

---

Event Notifications

S3 can trigger events when objects are created, deleted, or restored. Destinations: Lambda, SQS, SNS, or EventBridge.

📸 SCREENSHOT: S3 → Bucket → Properties → Event Notifications → Create Event Notification. Show the event types (PUT, DELETE), prefix/suffix filters, and the Lambda function destination.

aws s3api put-bucket-notification-configuration \
  --bucket my-bucket \
  --notification-configuration '{
    "LambdaFunctionConfigurations": [{
      "LambdaFunctionArn": "arn:aws:lambda:...:function:process-upload",
      "Events": ["s3:ObjectCreated:*"],
      "Filter": {
        "Key": {
          "FilterRules": [
            {"Name": "prefix", "Value": "uploads/"},
            {"Name": "suffix", "Value": ".jpg"}
          ]
        }
      }
    }]
  }'

---

S3 Object Lock

Object Lock prevents objects from being deleted or overwritten for a set period. Used for WORM (Write Once Read Many) compliance — financial records, audit logs, backups.

Mode	Effect
Governance	Prevents most users from deleting — but users with s3:BypassGovernanceRetention permission can override
Compliance	No one — including root — can delete the object during the retention period

# Enable Object Lock when creating bucket (cannot enable later)
aws s3api create-bucket \
  --bucket compliance-archive \
  --object-lock-enabled-for-bucket \
  --region eu-west-1 \
  --create-bucket-configuration LocationConstraint=eu-west-1

# Lock a specific object version
aws s3api put-object-retention \
  --bucket compliance-archive \
  --key audit-log-2026.csv \
  --retention '{"Mode": "COMPLIANCE", "RetainUntilDate": "2033-01-01T00:00:00Z"}'

---

Performance

S3 scales automatically — no provisioning needed. Per prefix, S3 supports:

• 3,500 PUT/COPY/POST/DELETE requests per second
• 5,500 GET/HEAD requests per second

To scale beyond these limits, use multiple key prefixes — each prefix has its own performance quota.

S3 Transfer Acceleration uses CloudFront edge locations to accelerate uploads from distant locations. Uploads go to the nearest edge location over optimised AWS backbone routing.

# Enable Transfer Acceleration
aws s3api put-bucket-accelerate-configuration \
  --bucket my-bucket \
  --accelerate-configuration Status=Enabled

# Use accelerated endpoint
aws s3 cp large-file.zip s3://my-bucket/ \
  --endpoint-url https://my-bucket.s3-accelerate.amazonaws.com

---

Quick Reference

# Buckets
aws s3 ls
aws s3api list-buckets --query 'Buckets[*].[Name,CreationDate]' --output table

# Objects
aws s3 ls s3://my-bucket/ --recursive --human-readable
aws s3 cp s3://my-bucket/file.txt .
aws s3 rm s3://my-bucket/file.txt
aws s3 rm s3://my-bucket/prefix/ --recursive

# Sync (upload only changed files)
aws s3 sync ./dist s3://my-bucket/ --delete

# Bucket size and object count
aws s3api list-objects-v2 --bucket my-bucket \
  --query '[length(Contents), sum(Contents[*].Size)]'

# Get bucket policy
aws s3api get-bucket-policy --bucket my-bucket

# Check encryption settings
aws s3api get-bucket-encryption --bucket my-bucket

# Get versioning status
aws s3api get-bucket-versioning --bucket my-bucket