[S3] CRUD with S3 using Python [Python]

Get the key you need to access

  1. Go to the AWS security credentials page.
  2. Click "User" in the access control tag
  3. Press "Add User", enter the user name and check for programmatic access. If there is no group, create it and select Amazon S3 Full Access.
  4. Add tags as they are and go to the next step without entering anything
  5. "Create user" completes the creation. Make a note of the CSV download and access key ID and secret access key.

installation of boto3

pip install boto3

CRUD Specify the access key and secret key used here by environment variables. It will be a big deal if someone knows it. region is the name of the region you are using. Example) Ohio us-east-2

Create (Save File)

def create_img_s3(path, img):#Save image
    s3 = boto3.client('s3', aws_access_key_id=accesskey, aws_secret_access_key=secretkey, region_name=region)
    img.thumbnail((900, 1200), Image.ANTIALIAS)
    out = BytesIO()
    img.save(out, "PNG")

    s3.put_object(Bucket=bucket_name, Key=path, Body=out.getvalue())

    url = "https://"+bucket_name+".s3-"+region+".amazonaws.com/"+path
    return url

def create_csv_s3(path, dataframe):#Convert from DataFrame to CSV and save as CSV
    out2 = StringIO()
    dataframe.to_csv(out2, encoding='utf_8_sig')
    s3 = boto3.client('s3', aws_access_key_id=accesskey, aws_secret_access_key=secretkey, region_name=region)
    s3.put_object(Bucket=bucket_name, Key=path, Body=out2.getvalue().encode("utf-8_sig"))

    url = "https://" + bucket_name + ".s3-" + region + ".amazonaws.com/" + path
    return url

Read

def readImg_s3(imgpath):
    s3 = boto3.client('s3', aws_access_key_id=accesskey, aws_secret_access_key=secretkey, region_name=region)
    obj = s3.Object(Bucket=bucket_name, Key=imgpath)
    print(obj)
    print(obj["Body"])

def readDir_s3(dirpath):#Get a list of data in a specific folder
    s3 = boto3.client('s3', aws_access_key_id=accesskey, aws_secret_access_key=secretkey, region_name=region)
    res = s3.list_objects_v2(Bucket=bucket_name, Prefix=dirpath, Delimiter='/')
    urls = []
    for data in res["Contents"]:
        data_url = data["Key"]
        if dirpath != data_url:
            url = "https://" + bucket_name + ".s3-" + region + ".amazonaws.com/" + data_url
            urls.append(url)

    return urls

Recommended Posts

[S3] CRUD with S3 using Python [Python]
Using Quaternion with Python ~ numpy-quaternion ~
S3 operation with python boto3
[Python] Using OpenCV with Python (Basic)
Using OpenCV with Python @Mac
Send using Python with Gmail
Complement python with emacs using company-jedi
Harmonic mean with Python Harmonic mean (using SciPy)
Using Rstan from Python with PypeR
[Python] Using OpenCV with Python (Image transformation)
[Python] Using OpenCV with Python (Edge Detection)
Notes on using rstrip with python.
When using MeCab with virtualenv python
Precautions when using six with Python 2.5
[AWS] Using ini files with Lambda [Python]
FizzBuzz with Python3
Scraping with Python
Statistics with python
Try mathematical formulas using Σ with python
Behind the flyer: Using Docker with Python
Scraping with Python
Python with Go
Implementation of CRUD using REST API with Python + Django Rest framework + igGrid
Using Python and MeCab with Azure Databricks
Socket communication using socketserver with python now
Twilio with Python
Integrate with Python
Start using Python
Try using Python with Google Cloud Functions
Play with 2016-Python
AES256 with python
Check stock prices with slackbot using python
Working with OpenStack using the Python SDK
Tested with Python
python starts with ()
Tips for using python + caffe with TSUBAME
with syntax (Python)
CRUD with Django
Bingo with python
Zundokokiyoshi with python
Connect to s3 with AWS Lambda Python
WebApi creation with Python (CRUD creation) For beginners
I'm using tox and Python 3.3 with Travis-CI
Scraping using Python
S3 server-side encryption SSE with Python boto3
Excel with Python
Microcomputer with Python
Cast with python
[Personal memo] julia --Using Python library with julia using PyCall
Debug with VS Code using boost python numpy
I tried using mecab with python2.7, ruby2.3, php7
[Python] Summary of S3 file operations with boto3
Export RDS snapshot to S3 with Lambda (Python)
Recent ranking creation using Qiita API with Python
Manipulate objects using Blender 2.8's low-level Python API
Upload images to S3 with GUI using tkinter
What are you using when testing with Python?
Serial communication with Python
Zip, unzip with python
Django 1.11 started with Python3.6
Python with eclipse + PyDev.