What is S3 lifecycle policy?
AWS S3 lifecycle policy consists of rules applied to a group of objects
- Two types of actions in a lifecycle policy:
- Transition actions: Move objects to another storage class after a certain period.
- Expiration actions: Delete objects after a specified time.
- Lifecycle policies can:
- Reduce storage costs by transitioning objects to cheaper storage classes.
- Improve performance by moving frequently accessed objects to Standard class and less accessed
objects to Standard IA or Glacier. - Reduce data loss risk by setting expiration rules for unnecessary objects.
Policies are applied daily, and transition/expiration dates are rounded to the next day’s midnight UTC.
Lifecycle policies can be created using the S3 console, AWS CLI, or AWS SDKs.
Note: Every bucket has 1 lifecycle configuration, which can have up to 1000 rules.
How to create AWS S3 lifecycle policy using Management console
How to create AWS S3 lifecycle policy using Boto3
This snippet code shows you how to use boto3 to add lifecycle rules to S3 bucket
What is S3 end point
AWS S3 endpoint is a URL used to access S3 from a VPC, improving performance and security.
- Two types of S3 endpoints:
- Gateway endpoints (created via AWS Management Console, CLI, or SDKs) available in all regions,
supporting IPv4 and IPv6. - Interface endpoints (created via AWS PrivateLink) available in select regions, supporting only IPv4.
- Gateway endpoints (created via AWS Management Console, CLI, or SDKs) available in all regions,
- Add the S3 endpoint to VPC’s route table and use S3 API to access buckets and objects.
- Benefits of S3 endpoints:
- Improved performance by routing traffic directly to S3.
- Increased security by bypassing the public internet.
- Simplified routing with a single endpoint for all S3 traffic.
- Limitations of S3 endpoints:
- Not available in all regions (interface endpoints limited).
- Only usable with VPCs, not on-premises networks.
- Potential additional costs, such as bandwidth charges for interface endpoints.
Default access controls allow authorized users to access data in non-public buckets.
S3 access controls are used to restrict access to objects within the bucket.
- Two main methods for access controls:
- S3 bucket policies: Control access to individual objects.
- IAM controls: Limit users, groups, and resources accessing buckets.
- Attribute-based control models, such as using tags, can also be used.
- It’s recommended to choose one method of access control and avoid mixing them to simplify troubleshooting
permission issues.
S3 access control
- Access Control Lists (ACLs) can be used for object-level restrictions at the bucket or object access level.
- IAM policies are preferred by many for controlling access to S3 buckets.
- IAM policies provide control over entitlement at the bucket and folder levels.
- IAM policies allow the construction of complex conditions based on tags, VPC-id, source IP address, and
other factors. - Cross-account access can be established by setting up a cross-account access role.
- This enables users or resources in one account to access objects in another account.
S3 encryption
By default, objects are stored unencrypted, but compliance requirements may necessitate encryption.
S3 provides encryption at rest for stored objects.
- Two encryption options available:
- Server-side encryption: Encryption is handled by S3 using S3-managed keys (SSE-S3), AWS Key
Management Service (SSE-KMS), or a customer-provided key (SSE-C). - Client-side encryption: Encryption is performed by the client before uploading objects to S3, using
client-side encryption libraries and a customer-provided key.
- Server-side encryption: Encryption is handled by S3 using S3-managed keys (SSE-S3), AWS Key
How to use AWS S3 event with other services?
Common scenarios for notifications include object uploads and deletions.
- Three AWS services can receive S3 event notifications:
- AWS Lambda
- Amazon Simple Queue Service (SQS)
- Amazon Simple Notification Service (SNS)
- Notifications can be configured for new object additions or overwrites.
- AWS Lambda can process notifications through Lambda functions.
To use S3 events, grant the necessary permissions to the S3 principle for requested services. This
includes publishing to SNS queues or SQS topics and invoking Lambda.