Don't forget to release it when you get the object from S3!

environment

phenomenon

The following error occurred when retrieving an object from S3.

org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for connection from pool

Code in question

public InputStream getFile(String bucketName,String prefix){
  try{
    GetObjectRequest request = new GetObjectRequest(bucketName, prefix);
    S3Object object = s3client.getObject(request); //s3client has been generated as a singleton instance variable
    return object.getObjectContent();
  } catch(AmazonServiceException e){
    throw e;
  }
}

Cause

Because the release process was not performed when the S3Object was acquired. It has an HTTP connection pool size inside S3 and defaults to 50 </ b>. If this is exceeded, the above error will occur.

Solution

Part 1 When using the wrapper class of the S3 client

public InputStream getFile(String bucketName,String prefix){
  GetObjectRequest request = new GetObjectRequest(bucketName, prefix);
  try(S3Object object = s3client.getObject(request)){
    //Expand to memory. Remove the S3Object reference here
    byte[] file = IOUtils.toByteArray(object.getObjectContent());
    // byte[]Return as InputStream from
    return new ByteArrayInputStream(file);
  } catch(AmazonServiceException e){
    throw e;
  } catch(IOException e){
    throw new RuntimeException(e);
  }
}

Note that it is temporarily expanded in memory. </ font> </ b> InputStream cannot be returned as it is because of the release process.

Part 2 When the caller releases

public static main(String[] args){
  try(S3Object object = getFile("hoge","fuga")){
~processing~
  }catch (IOException e) {
    throw new RuntimeException(e);
  }
}

public S3Object getFile(String bucketName,String prefix){
  try{
    GetObjectRequest request = new GetObjectRequest(bucketName, prefix);
    return s3client.getObject(request);
  } catch(AmazonServiceException e){
    throw e;
  }
}

It is necessary to perform release processing for each call, but InputStream can be handled.

Recommended Posts