Search AWS S3 resources with Spring Cloud AWS Resource Handling feature

I can operate resources on S3 with the SDK of AWS, but it seems that there is no function to specify the resource name and search. You can use Spring Cloud AWS Resource Handling to search for resources that match a specified Ant pattern.

Image of screen after completion ↓

Verification environment

・ Spring Boot (Thymeleaf) 1.4.3

Required library

pom.xml

	...
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-thymeleaf</artifactId>
	</dependency>
	<dependency>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-web</artifactId>
	</dependency>
	<dependency>
	  <groupId>org.springframework.cloud</groupId>
	  <artifactId>spring-cloud-aws-context</artifactId>
	  <version>1.2.1.RELEASE</version>
	</dependency>
	<dependency>
	    <groupId>org.springframework.cloud</groupId>
	    <artifactId>spring-cloud-aws-autoconfigure</artifactId>
	    <version>1.2.1.RELEASE</version>
	</dependency>
	<dependency>
	  <groupId>com.amazonaws</groupId>
	  <artifactId>aws-java-sdk</artifactId>
	  <version>1.11.163</version>
	</dependency>
	...

Settings for connecting to AWS S3

Specify the access key, secret key, and region in application.properties. Also, since the access is from the local, specify the automatic region detection as "false".

application.properties

cloud.aws.credentials.accessKey=AKIAXXXXXXXXXA3Q
cloud.aws.credentials.secretKey=T9JBXXXXXXXXXXXXXXXXXXXXGfLAQ
cloud.aws.region.static=ap-northeast-1
cloud.aws.region.auto=false

Create an AWS config class to read the above settings.

AWSConfig

@Configuration
public class AWSConfig {

	@Value("${cloud.aws.credentials.accessKey}")
	private String accessKey;

	@Value("${cloud.aws.credentials.secretKey}")
	private String secretKey;

	@Value("${cloud.aws.region.static}")
	private String region;

	@Bean
	public BasicAWSCredentials basicAWSCredentials() {
		return new BasicAWSCredentials(accessKey, secretKey);
	}

	@Bean
	public AmazonS3 amazonS3Client(AWSCredentials awsCredentials) {
		return AmazonS3ClientBuilder.standard().withRegion(Regions.fromName(region)).build();
	}
}

By creating the above, you will be able to get an instance of ResourceLoader and Amazon S3 from the DI container.

Application layer

Create a normal Controller class and a form class for screen display.

AwsS3Controller

	@RequestMapping(value="/aws/s3/search", method=RequestMethod.POST)
	public String search(Model model, S3FileForm fileForm) throws IOException {
		
		S3FileForm s3FileForm = storageService.search(fileForm.getFileName());
		model.addAttribute("s3FileForm", s3FileForm);
		
		return "/aws/s3/downloadFromS3";
	}

S3FileForm

public class S3FileForm implements Serializable {
	
	private String fileName;
	
	private String downloadKey;
	
	private String[] checkedItems;
	
	private List<S3File> fileList;

	//...get set omitted
}

S3File

public class S3File implements Serializable {

	private String bucketName;
	
	private String key;
	
	private String contentType;
	
	private Date lastModifiedDate;

	//...get set omitted
}

Domain layer

Use ResourcePatternResolver in the service class to search for the resource name that matches the Ant pattern s3: // cinpo / ** / * test and get the target resource list. In addition, create an Amazon S3 object with the resource name and get key information such as the last modified date. You can use s3: //**/*.txt to search for all accessible buckets.

S3StorageService

/**
 *AWS S3 Operations Service
 * @author
 *
 */
@Service
public class S3StorageService {
	
	private final String BUCKET_NAME = "cinpo";
	
	@Autowired
	private ResourceLoader resourceLoader;
	
	@Autowired
 	private ResourcePatternResolver resourcePatternResolver;
	
	@Autowired
	private AmazonS3 amazonS3;
	
	/**
	 *Resource search
	 * 
	 * @param fileName
	 */
	public S3FileForm search(String fileName) {
		
		//Screen display form
		S3FileForm s3FileForm = new S3FileForm();

			Resource[] resources = null;
			try {
				//Ant pattern (s3://cinpo/**/*test.*) To get the target list.
				resources = this.resourcePatternResolver.getResources("s3://" + BUCKET_NAME + "/**/*" + fileName + ".*");
			} catch (IOException e) {
				throw new RuntimeException(e);
			}
			
			List<S3File> s3FileList = new ArrayList<S3File>();
			
			//Get S3Object for each resource.
			for (Resource resource : resources) {
				S3Object s3Object = amazonS3.getObject(new GetObjectRequest("s3.cinpo", resource.getFilename()));
				
				//Screen display item settings
				S3File s3File = new S3File();
				s3File.setBucketName(s3Object.getBucketName());
				s3File.setKey(s3Object.getKey());
				s3File.setContentType(s3Object.getObjectMetadata().getContentType());
				s3File.setLastModifiedDate(s3Object.getObjectMetadata().getLastModified());
				
				s3FileList.add(s3File);
			}
			s3FileForm.setFileList(s3FileList);
		
		return s3FileForm;
	}
}

HTML Finally, create a screen for searching and listing.

downloadFromS3

<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml"
      xmlns:th="http://www.thymeleaf.org">
      
	<head>
		<meta charset="UTF-8" />
		<link th:substituteby="common/header :: common_header"/>
		<title>File download from S3</title>
	</head>
	<body>
		<h1>File Download</h1>
		<p><a href="/index" th:href="@{/index}">Back to home</a></p>
		<form action="#" th:action="@{/aws/s3/search}" th:object="${s3FileForm}" method="post">
			<table>
				<tr>
					<td width="100px"><label for="fileName">search key</label>:</td>
					<td><input type="text" id="fileName" th:field="*{fileName}" size="30px" /></td>
				</tr>
				<tr>
					<td colspan="2"><input type="submit" value="Search" /></td>
				</tr>
			</table>
		</form>
		<div th:if="${s3FileForm.fileList != null}">
			<form action="#" th:action="@{/aws/s3/download}" th:object="${s3FileForm}" method="post">
				<table border="1">
					<tr>
						<th width="20px">Download</th>
						<th width="10px">Check</th>
						<th>Bucket</th>
						<th>Key</th>
						<th>Content Type</th>
						<th>Last Modified Date</th>
					</tr>
					<tr th:each="file:${s3FileForm.fileList}">
						<td>
							<button type="submit" name="downloadKey" th:field="*{downloadKey}" th:value="${file.key}">Download</button>
						</td>
						<td><input type="checkbox" th:field="*{checkedItems}" th:value="${file.key}" /></td>
						<td th:text="${file.bucketName}"></td>
						<td th:text="${file.key}"></td>
						<td th:text="${file.contentType}"></td>
						<td th:text="${file.lastModifiedDate}"></td>
					</tr>
				</table>
				<div><input type="submit" value="Download Selected" /></div>
			</form>
		</div>
	</body>
</html>

Reference side: Spring Cloud AWS