Code that deletes all files of the specified prefix in AWS S3 (Java)

I had a requirement like that in my work, so I implemented it.

It was unexpectedly troublesome to think that I could write in one or two lines ... The library uses Guava and lombok. (Unfortunately my workplace is Java 7) I think there is a better way, but it worked for the time being, so make a note

code

S3ObjectDelete.java


@RequiredArgsConstructor
public class S3ObjectDelete {
    private AmazonS3Client client = new AmazonS3Client();

    private final String bucket;

    public static void main(String[] args) {
        val hoge = new S3ObjectDelete("Bucket name");
        hoge.delete("Prefix");
    }

    public void delete(String prefix) {
        //It seems that the maximum number of items that can be deleted at the same time is 1000, so divide and process
        for (val keys : Lists.partition(keys(prefix), 1000)) {
            String[] array = keys.toArray(new String[keys.size()]);
            client.deleteObjects(new DeleteObjectsRequest(bucket).withKeys(array));
        }
    }

    List<String> keys(String prefix) {
        ObjectListing objects = client.listObjects(bucket, prefix);
        val f = new S3Object2StringKey();
        List<String> keys = new ArrayList<>(Lists.transform(objects.getObjectSummaries(), f));

        //It seems that the default number of items that can be acquired at one time is 1000, so loop and acquire all
        while (objects.isTruncated()) {
            objects = client.listNextBatchOfObjects(objects);
            keys.addAll(Lists.transform(objects.getObjectSummaries(), f));
        }
        return keys;
    }
}

public static class S3Object2StringKey implements Function<S3ObjectSummary, String> {
    @Override
    public String apply(S3ObjectSummary input) {
        return input.getKey();
    }
}

Postscript

** Amazon S3 Client ** has been deprecated Seems to recommend using AmazonS3ClientBuilder This article was helpful.

Recommended Posts

Code that deletes all files of the specified prefix in AWS S3 (Java)
[java tool] A tool that deletes files under the specified path by extension
Import files of the same hierarchy in Java
Guess the character code in Java
Deleting AWS S3 Objects in Java
Renamed folders in AWS S3 (Java)
Examine the system information of AWS Lambda operating environment in Java
Find out the list of fonts available in AWS Lambda + Java
Sample code that uses the Mustache template engine JMustache in Java
Get the result of POST in Java
Get a list of S3 files with ListObjectsV2Request (AWS SDK for Java)
This and that of the implementation of date judgment within the period in Java
Sample program that returns the hash value of a file in Java
All same hash code string in Java
The story of writing Java in Emacs
[Java] Where is the implementation class of annotation that exists in Bean Validation?
Validate the identity token of a user authenticated with AWS Cognito in Java
Sample code to get the values of major SQL types in Java + MySQL 8.0
The story of low-level string comparison in Java
[Java] Handling of JavaBeans in the method chain
The story of making ordinary Othello in Java
About the idea of anonymous classes in Java
[Java] Access the signed URL of s3 (signed version 2)
The story of learning Java in the first programming
Feel the passage of time even in Java
First touch of the Files class (or Java 8)
I wrote a test code (Junit & mockit) for the code that calls the AWS API (Java)
Get the URL of the HTTP redirect destination in Java
[Java] Get the file in the jar regardless of the environment
Change the storage quality of JPEG images in Java
Code that only displays the built-in camera in Processing
Differences in code when using the length system in Java
Summarize the additional elements of the Optional class in Java 9
The story that .java is also built in Unity 2018
A program (Java) that outputs the sum of odd and even numbers in an array
Shout Java at the heart of technology-Themes and elemental technologies that Java engineers should pursue in 2017-
Sample code to get the values of major SQL types in Java + Oracle Database 12c
Create a database of all the books that have been circulating in Japan in the last century
[Java] It seems that `0 <hoge <10` cannot be written in the conditional expression of the ʻif` statement.
Was done in the base year of the Java calendar week
A quick explanation of the five types of static in Java
Get the next business day after the specified date in JAVA
Solved the problem that all the data in the table was displayed
[Java] [Java EE] [Glassfish] (Continued) Style that doubts the cache of Glassfish
Count the number of digits after the decimal point in Java
Things to be aware of when writing code in Java
Correct the character code in Java and read from the URL
A program that counts the number of words in a List
How to derive the last day of the month in Java
[AWS / S3] After all, how do you upload multiple files?
[Java] Delete the specified number of characters from the end of StringBuilder
Static function to check if the RGB error of BufferdImage is within the specified ratio in Java
Let's implement the condition that the circumference and the inside of the Ougi shape are included in Java [Part 2]
Let's implement the condition that the circumference and the inside of the Ougi shape are included in Java [Part 1]
If you do not call shutdownNow when the transfer is completed in the Java SDK of AWS S3, threads will continue to remain