Designing With S3

Designing with Amazon S3

Security & Design

Amazon S3 Default Security Configurations

Default Security Posture

Note

Security by Default: S3 buckets and objects created are private and protected by default.

• Private by Default: “S3 buckets and objects created are private and protected by default”
• Default Encryption: “S3 buckets have encryption configured by default”
• Encryption Method: “Server-side encryption with Amazon S3 managed keys (SSE-S3) is the default encryption”

Data Sharing Principles

When use cases must share Amazon S3 data:

• Access Management: “Manage and control the data access”
• Least Privilege: “Follow the principle of least privilege”

Protection Through Encryption

“When your objective is to protect digital data, data encryption is an essential tool. Data encryption takes data that is legible and encodes it. Encrypted data is unreadable to anyone who does not have access to the secret key that can be used to decode it.”

• Attack Protection: “Even if an attacker gains access to your data, they cannot make sense of it”
• Key Management: “Optionally, use AWS Key Management Service (AWS KMS) to manage secret keys”

Encrypting Objects in Amazon S3

Encryption Overview

“Encryption encodes data with a secret key, which makes it unreadable without a key.”

Server-Side Encryption

Default Configuration

SSE-S3 Automatic

• All S3 buckets have encryption configured by default
• All new objects uploaded are automatically encrypted at rest
• SSE-S3 is default encryption configuration for every bucket

Process

Encryption Workflow

• Amazon S3 encrypts objects before saving to disk
• Amazon S3 decrypts objects when you download them
• Enable by selecting default encryption option on bucket

Alternative Server-Side Encryption Options

• SSE-KMS: Server-side encryption with AWS Key Management Service (AWS KMS) keys
• DSSE-KMS: Dual-layer server-side encryption with AWS KMS keys
• SSE-C: Server-side encryption with customer-provided keys

Client-Side Encryption

• Process: “Encrypt data on the client side and upload the encrypted data to Amazon S3”
• Management: “In this case, you manage the encryption process”
• Benefits: Provides additional security by managing encryption process, keys, and related tools
• Risk Reduction: “Can reduce risk by encrypting the data with a key that is stored in a different mechanism than the mechanism that stores the data itself”

Amazon S3 Tools for Protecting Buckets and Objects

Block Public Access

Overrides all other policies

• Makes buckets inaccessible to the public
• Provides straightforward method for avoiding unintended exposure
• Enable for all buckets that shouldn’t be publicly accessible

IAM Policies

User Authentication

• Authenticates users by using IAM
• Specify users or roles that can access specific buckets and objects
• Define granular access controls

Bucket Policies

Resource-Based Access

• Defines access based on specific written rules
• Use when user or system cannot authenticate using IAM
• Can grant access across AWS accounts or public access
• Deny statements restrict access even with identity-based permissions

Access Control Lists

Legacy Access Control

• Sets rules for access to buckets and objects
• Less commonly used (ACLs predate IAM)
• Bucket policies are preferred method for controlling access

Advanced Protection Tools

Amazon S3 Access Points

• Purpose: “Named network endpoints that are attached to buckets”
• Function: “You can use access endpoints to perform S3 object operations”
• Scalability: “Customers with shared datasets can scale access for many applications by creating individualized access points”
• Customization: “Names and permissions that are customized for each application”

Preassigned URLs

• Function: “Grant time-limited access to others with temporary URLs”
• Use Case: Temporary access without permanent permissions

AWS Trusted Advisor

• Feature: “Provides a bucket permission check feature”
• Purpose: “Useful tool for discovering if any of the buckets in your account have permissions that grant global access”

Three General Approaches to Configuring Access

Option 1: Default Security (Recommended)

Tip

Default Security: By default, all S3 buckets and objects stored in them are private (protected).

• Access: Only account administrator and AWS account root user have access
• Permissions: Resource owner can grant specific access permissions to others
• Principle: Anyone without permissions will not have access

Option 2: Controlled Access (Common)

• User A: Granted access to objects in bucket
• User B: Denied access to objects
• Configuration: Bucket owner configures using access control tools
• Use Cases: Most common scenario for business applications

Option 3: Public Access (Not Recommended)

• Access: Anyone can publicly access objects stored in bucket
• Risk: S3 security settings have been disabled
• Use Case: Static website hosting (specific use case)
• Caution: “For most Amazon S3 use cases, you would not want to grant public access”

Danger

Security Warning: Most use cases do not require public access. Never grant public access to buckets that hold sensitive data.

Considerations When Choosing a Region

Data Privacy Laws and Regulatory Compliance

• Legal Requirements: “Data that you store on AWS is subject to the laws of the country and locality where it is stored”
• Jurisdictional Rules: “Some laws dictate that if you are operating your business in their jurisdiction, you cannot store that data anywhere else”
• Compliance Standards: Standards like HIPAA have strict guidelines on how and where data can be stored
• Governance: “Can you meet your governance obligation?”

Proximity of Users to Data

• Latency Impact: “Small differences in latency can impact customer experience”
• User Expectations: “Customers expect responsive environments, and as time passes and technology becomes more powerful, those expectations also rise”
• Best Practice: “Choose the Region closest to your users”

Availability of Services and Features

• Service Distribution: “Not all AWS services are available in all Regions”
• Expansion: “Services expand to new Regions regularly”
• Cross-Region Use: “Use some services cross-Region but at increased latency”
• Launch Strategy: “Services are released when they are ready. Service availability is then expanded as soon as possible”

Cost-Effectiveness

• Regional Pricing: “Costs vary by Region”
• Data Transfer: “Some services such as Amazon S3 have costs for transferring data out”
• Replication Costs: “Consider the cost-effectiveness of replicating the entire environment in another Region”
• Decision Factor: “In cases where the latency, compliance, and service availability differences between Regions are minimal, you might be able to save by using the lower-cost Region”

Multi-Region Considerations

Benefits

Global Distribution Advantages

• Optimize customer experience by replicating environment in multiple Regions
• Distribute load across multiple environments
• Potential cost reduction per environment due to load distribution
• AWS flexibility allows scaling existing environment down to mitigate costs

Challenges

Multi-Region Complexity

• Two environments to manage
• Not all components scale down enough to mitigate all costs
• May need single storage source of truth in one Region
• Secondary Region communication increases latency and cost

Amazon S3 Inventory

Inventory Overview

• Purpose: “Use Amazon S3 Inventory to help manage your storage”
• Compliance: “Use it to audit and report on the replication and encryption status of your objects for business, compliance, and regulatory needs”
• Performance: “Speed up business workflows and big data jobs by using Amazon S3 Inventory”
• Alternative: “Provide a scheduled alternative to the Amazon S3 synchronous List API operations”

Inventory Features

• Output Formats: Provides CSV, Apache ORC files, or Apache Parquet output files
• Frequency: List objects and metadata on daily or weekly basis
• Scope: For S3 bucket or objects with shared prefix
• Scheduling: Weekly inventory generates every Sunday (UTC time zone) after initial report

Configuration Options

When configuring inventory list, specify:

• Metadata: What object metadata to include in the inventory
• Versions: Whether to list all object versions or only current versions
• Output Location: Where to store the inventory list file output
• Frequency: Whether to generate inventory on daily or weekly basis
• Encryption: Whether to encrypt the inventory list file

Query Capabilities

“You can query Amazon S3 Inventory with standard SQL queries by using Amazon Athena, Amazon Redshift Spectrum, and other tools, such as Presto, Apache Hive, and Apache Spark.”

Terminology

• Source Bucket: “The bucket that the inventory lists objects for”
• Destination Bucket: “The bucket where the inventory list file is stored”

Amazon S3 Costs

Pay-for-Use Model

“With Amazon S3, you pay for only what you use. There is no minimum fee.”

Storage Costs

Object Storage

Monthly Storage Charges

• Gigabytes of objects stored per month
• Different pricing for each Region and storage class
• Rate depends on object size, storage duration, and storage class

Monitoring Costs

S3 Intelligent-Tiering

• Monthly monitoring and automation charge for each object
• Monitor access patterns and move objects between access tiers
• No retrieval charges or additional tiering charges

Request Costs

“There are ingest charges for each request when using PUT, COPY, POST, or LIST requests or when using lifecycle rules to move data into any Amazon S3 storage class.”

Data Transfer Costs

No Charge for Data Transferred

• Internet Outbound: Out to internet for first 100 GB per month
• Internet Inbound: In from the internet
• Same Region: Between S3 buckets or to any service in same AWS Region
• AWS Services: From S3 bucket to any AWS service within same Region
• CloudFront: Out to CloudFront

Encryption Costs

• SSE-S3: “Amazon S3 automatically applies SSE-S3 as base layer of encryption to all new objects added to Amazon S3 at no additional cost”
• SSE-C: “Does not incur any additional Amazon S3 charges”
• SSE-KMS: “You pay AWS KMS charges to generate or retrieve the data key used for encryption and decryption”
• DSSE-KMS: Additional encryption fee for each gigabyte for second layer of encryption and decryption

AWS Free Tier

New AWS customers receive:

• Storage: 5 GB of Amazon S3 storage in S3 Standard storage class
• Requests: 20,000 GET requests; 2,000 PUT, COPY, POST, or LIST requests
• Data Transfer: 100 GB of data transfer out each month
• Calculation: Usage calculated monthly across all AWS Regions (except AWS GovCloud)
• Rollover: Unused monthly usage will not roll over

Activity: Designing with Amazon S3

Business Scenario

The café owner wants to make documents more accessible to employees. They want to house content that only employees can access:

• Cooking instructions
• Vendor information
• Payroll
• Compliance training

Solution: Controlled Access Implementation

“Because you do not want to set this up for public access, setting up controlled access meets the customer’s need.”

Implementation Steps

1. Folder Structure: Create a folder for each type of content (cooking, vendors, payroll, and compliance)
2. IAM Policies: Create IAM policies (group or individual) to allow or deny users access to specific information
3. Employee Levels: Base access on employee level at the café
4. Access Control: Implement principle of least privilege for each user group

This approach ensures that sensitive business information remains secure while providing employees with access to the resources they need for their roles.

Designing with Amazon S3 requires careful consideration of security configurations, encryption options, access control mechanisms, regional placement, and cost optimization to create secure, performant, and cost-effective storage solutions.