Update an Existing Amazon S3 Object Using Java

1. Overview

Amazon Simple Storage Service (Amazon S3) is a widely used storage service that provides scalable, secure, and durable object storage. There are use cases when we need to update an Existing Amazon S3 Object. In S3, objects are immutable, meaning we cannot modify the content of an object directly. However, we can overwrite the object with new content, effectively “updating” it.

In this tutorial, we learn how to replace the existing file content with newer content for the same AWS S3 path using AWS Java SDK.

2. Prerequisites

First, we need to ensure the AWS SDK Maven dependency package is incorporated into the project:

<dependency>
    <groupId>com.amazonaws</groupId>
    <artifactId>aws-java-sdk</artifactId>
    <version>1.12.530</version>
</dependency>

The latest version of the package can be found online.

We also need to set up AWS credentials. We can do this using:

Environment Variables: AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
AWS Credentials File: Stored at ~/.aws/credentials
IAM Roles: Assign IAM roles with necessary S3 permissions when running the app on an Amazon EC2 instance

For a comprehensive guide on credentials setup, let’s refer to the Official AWS Documentation

3. Steps to Update an S3 Object

3.1. Initialize the S3 Client

The @PostConstruct annotation is from the javax.annotation package and is used to execute the method after the bean’s properties have been set by the container (Spring, in this case). Below is the code for initializing the S3 client:

@PostConstruct
private void init(){
    AWSCredentials credentials = new BasicAWSCredentials(
      "AWS AccessKey",
      "AWS secretKey"
    );
    this.amazonS3 = AmazonS3ClientBuilder.standard()
      .withRegion(Regions.fromName("us-east-1"))
      .withCredentials(new AWSStaticCredentialsProvider(credentials))
      .build();
}

3.2. Upload New Object to S3

Now we can upload the file to the S3 bucket in the method uploadDocument(), using AWS Java SDK:

ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentType(multipartFile.getContentType());
Map<String, String> attributes = new HashMap<>();
attributes.put("document-content-size", String.valueOf(multipartFile.getSize()));
metadata.setUserMetadata(attributes);
InputStream documentStream = multipartFile.getInputStream();
PutObjectResult putObjectResult = this.awsS3Client.putObject(new PutObjectRequest(s3bucket, key, documentStream, metadata));

Below is the code snippet to invoke the above code:

public String uploadFile(MultipartFile multipartFile) throws Exception {
    String key = "/documents/" + multipartFile.getOriginalFilename();
    return this.uploadDocument(this.awsS3Bucket, key, multipartFile);
}

3.3. Upload (Overwrite) the Object

Since objects in S3 are immutable, “updating” an object involves overwriting the object with new content. So for updating, we need to call the same uploadDocument() method with the same set of parameters that were used for adding the document:

public String updateFile(MultipartFile multipartFile, String key) throws Exception {
    return this.uploadDocument(this.awsS3Bucket, key, multipartFile);
}

This code will replace the existing object with the new content provided. If the object with the given key doesn’t exist, S3 will create a new object.

3.4. (Optional) Verify the Update

We might want to verify that the object was successfully updated. One way to do this is by retrieving the object’s metadata and checking the LastModified date or by computing the object’s checksum and comparing it against the expected value.

S3Object s3Object = this.awsS3Client.getObject(s3bucket, key);
logger.info("Last Modified: " + s3Object.getObjectMetadata().getLastModified());

4. Important Considerations

We also need to remember that in S3, overwriting an object is effectively a PUT operation, which may incur costs. Always be aware of the cost implications when performing operations on S3.

If the bucket has versioning enabled, overwriting an object will not delete the old version. Instead, we’ll have multiple versions of the object. Each version has a unique ID; we can retrieve any previous version if required.

Finally, if metadata is associated with the object, be aware that overwriting the object will replace the old metadata with the new one provided during the PUT operation. We must explicitly set it in the request if we want to retain the old metadata.

5. Conclusion

While we cannot directly modify an S3 object’s content, overwriting the object with new content is straightforward using the AWS SDK for Java. Always keep best practices in mind, such as not hardcoding credentials and being aware of the cost implications of operations. With these steps, we can confidently manage and update S3 objects using Java.

The full implementation of this article can be found over on GitHub.

Persistence

REST

Security