[PYTHON] [AWS S3] Confirmation of the existence of folders on S3

Overview

A folder on S3 [^ 1] using the boto3 library, which is an API wrapper for Python to AWS. Or I wrote a function to check if the file exists.

Motivation

There was a process that I wanted to distinguish by the presence or absence of a folder on S3, but on the net it is a process that uses boto2 and a function that is not currently implemented, or it can only be used for "files" on S3 There were only a few methods, and there was almost no existence judgment for "folders".

So I made it myself.

Code example

import boto3
from botocore.exceptions import ClientError

client = boto3.client('s3')
bucket_name = 'hoge'
key = "fuga/piyo/bar/" # /I think it's a good idea to put up to. It may not be necessary. I'm not sure.

result = client.list_objects(Bucket=bucket_name, Prefix=key)

#Next is Kimo. If the above path does not exist, some of the returned results will be
#There is no key called Contents. Existence can be determined by using this.
if "Contents" in result:
    exists = True
else:
    exists = False

[^ 1]: Strictly not a folder. On S3, everything is managed by a combination of keys (like paths) and values (contents of files), like a dictionary in Python. In S3, the key happens to be represented by a / delimiter like a path, and in the browser, the AWS viewer just interprets it nicely with a / delimiter.

Recommended Posts

[AWS S3] Confirmation of the existence of folders on S3
Existence from the viewpoint of Python
Investigate the effect of outliers on correlation
Check the existence of the file with python
Post the subject of Gmail on twitter
[AWS] Judge the existence of Next Token and take all the values [boto3]
Display the graph of tensorBoard on jupyter
Change the order of PostgreSQL on Heroku
Prepare the environment of Chainer on EC2 spot instance with AWS Lambda
Upload data to s3 of aws with a command and update it, and delete the used data (on the way)
Change the Key of Object on S3 from normal date format to Hive format
[Linux] Review of commands for deploying on AWS
The behavior of signal () depends on the compile options
[2020July] Check the UDID of the iPad on Linux
Use the latest version of PyCharm on Ubuntu
Calculate the probability of outliers on a boxplot
At the time of python update on ubuntu
Change the resolution of Ubuntu running on VirtualBox
Jupyter on AWS
Install the latest version of CMake on Ubuntu 18.04.4 LTS
Maybe I overestimated the impact of ShellShock on CGI
[AWS CLI] Enable the AWS CLI on Red Hat Enterprise Linux 8
Try to estimate the number of likes on Twitter
Drawing on Jupyter using the plot function of pandas
Looking back on the transition of the Qiita Advent calendar
Support for Python 2.7 runtime on AWS Lambda (as of 2020.1)
Tweet the triple forecast of the boat race on Twitter
Check for the existence of BigQuery tables in Java
Scroll Japanese on the LED of RaspberryPi Sense HAT
A Study on Visualization of the Scope of Prediction Models
Create a shape on the trajectory of an object
Announcing the availability of Java 11 LTS on Amazon Linux 2