I decided to participate in a project that uses AWS for the first time in a few years. As a memo, it was necessary to update the information such as the release of Version 2 of the AWS CLI.

For AWS CLI Version 2, see here.

What is explained in this article

--Installing and configuring AWS CLI Version 2 --How to import csv file for authentication and explanation of existing problems --Other --About the Osaka region that will be updated in 2021 --About the column structure of AWS authentication csv --How to install AWS CLI Version 2 in individual environment of pyenv

Work environment

cat /etc/issue
> Ubuntu 18.04.5 LTS

python -V
> Python 3.8.6

pip --version
> pip 20.2.1 from /home/dev-user/.pyenv/versions/3.8.6/envs/sandbox-awscli/lib/python3.8/site-packages/pip (python 3.8)

(Reference: Construction procedure with Pyenv)

The work done in pyenv to create the above environment on Ubuntu 18.04 is as follows.

# Python3.Get 8 environment
pyenv install 3.8.6

#3 for work.8 Make a copy of the environment
pyenv virtualenv 3.8.6 sandbox-awscli

#Apply the environment copied to the current directory
pyenv local sandbox-awscli

AWS CLI (Version 2) installation

A simple pip install awscli will install the old version 1. Install with Official Method.

#Download the installer
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"

#Answer the installer
unzip awscliv2.zip

#Perform installation
# (/usr/local/Will be installed in bin)
sudo ./aws/install

#Verification
aws --version
> aws-cli/2.1.14 Python/3.7.3 Linux/5.4.0-58-generic exe/x86_64.ubuntu.18 prompt/off

(Reference) Install by specifying the copy destination, such as for general users

As an example of not using sudo in the README, the following command describes how to install and add the path (~/.local/bin) to the environment variable.

# (Option description)
#   -i, --install-dir <path>Specify the copy destination directory of the executable file.(default: /usr/local/aws-cli
#   -b, --bin-dir <path>Specify the directory where you want to create the symlink to the executable.(default: /usr/loca/bin)
./install -i ~/.local/aws-cli -b ~/.local/bin

(Reference) Install in individual environment of pyenv

When installing in the individual environment of pyenv (sandbox-awscli in this case), it will be as follows.

./aws/install -i ~/.pyenv/versions/sandbox-awscli/bin/aws-cli -b ~/.pyenv/versions/sandbox-awscli/bin

In my environment, the aws command didn't work unless I reread the path in source after execution.

AWS user verification

Check the information of the user to use on the AWS console. ・ Is the group policy suitable for the purpose?

Check the region

Check the recent situation.

Region list

https://docs.aws.amazon.com/ja_jp/AWSEC2/latest/UserGuide/using-regions-availability-zones.html#concepts-regions

Tokyo region

Basically this

area	Region name
Asia-Pacific(Tokyo)	ap-northeast-1

About Osaka Region (ap-northeast-3)

Extending AWS Osaka Local Region to Full Region

By early 2021, Osaka's local region has expanded to a full AWS region with three Availability Zones. Like all other AWS Regions, Availability Zones are separated by their own power, cooling system, and physical security. (As of December 28, 2020)

Prepare the authentication file (.csv) for configuration

As a new feature from AWS CLI Version 2, you can read and set the authentication information from the csv file (because it is a workflow familiar to GCP users, private is a nice update for GCP main body)

However, it is good when creating a new user and using it, but when using an existing user, it took a lot of work, so I will explain it later.

For new users

You can download it from the csv download button that is normally displayed on the creation completion screen.

The csv file has the following column structure.

User name	Password	Access key ID	Secret access key	Console login link
USERNAME		XXX	YYY	https://ZZZ.signin.aws.amazon.com/console

The contents of the file (new_user_credentials.csv) are as follows:

User name,Password,Access key ID,Secret access key,Console login link
USER NAME,,ACCESS KEY,SECRET KEY,https://USER ID.signin.aws.amazon.com/console

For existing users

*** The csv file downloaded from the "Create Access Key" completion screen as an existing user does not include columns such as User Name required at the time of import. *** ***

Therefore, you need to add it manually after downloading it by following the steps below.

Download csv for authentication

*** "AWS Console"-> "IAM"-> "Authentication"-> "Create Access Key" *** And it is displayed on the creation completion screen *** "Download .csv file" *** Please download from.

The csv file has the following column structure.

Access key ID	Secret access key
XXX	YYY

If you try to import as it is, the following error will occur.

aws configure import --csv file://yuuki-sandbox_accessKeys.csv --profile-prefix prof-
> Expected header "User Name" not found

Add column

I couldn't find a good way, so if I want to use import as an existing user, I decided to copy the contents of csv of the new user mentioned above and rewrite the necessary parts.

*** If you know a good way, I would appreciate it if you could tell me! *** ***

AWS CLI settings

Import csv for authentication.

*** When imported without options, a profile is automatically created with the username stored in the User Name column of csv and given the credentials. *** ***

--profile can also be set in the import command, but be aware that the above will be prioritized and ignored.

--profile-prefix There is an option called , which can be a profile named .

# --profile-According to the prefix option, prof-<USER NAME>A profile named is created and the credentials are imported here.
aws configure import --csv file://new_credentials.csv --profile-prefix prof-
> Successfully imported 1 profile(s)

#Confirmation of profile addition
aws configure list-profiles
> <Profile name>

Other settings

Set the rules.

Region settings

aws configure set region ap-northeast-1 --profile <Profile name>

Output type setting

aws configure set output <Output type> --profile <Profile name>

The output types that can be selected are as follows (as of December 28, 2020)

Output type	Explanation
json	It is output in JSON string format.
yaml	It is output in YAML string format. (Available only with AWS CLI version 2.)
yaml-stream	The output is streamed and formatted as a YAML string. Streaming can speed up the processing of large data types. (Available only with AWS CLI version 2.)
text	It is output in the form of a multi-line tab-delimited string value. This is useful for passing output to a text processor such as grep, sed, or awk.
table	The string that forms the cell border+\|-Is output in tabular format using. Information is usually displayed in a "easy-to-understand" format that is easier to read than other formats, but it is not useful programmatically.

Change default profile

#For Bourne shell
export AWS_DEFAULT_PROFILE=<Profile name>

#For fish shell
set -x AWS_DEFAULT_PROFILE <Profile name>

If necessary, add it to .bashrc etc.

Verification

I will briefly display the information to confirm that the authentication has passed.

Try to display the bucket list of s3

Required access policy example: AmazonS3ReadOnlyAccess

aws s3 ls
> 2020-12-28 17:36:44 yuuki-sample-bucket

Try to display the IAM user list

Required access policy example: IAMReadOnlyAccess

aws iam list-users
> ~~
> - Arn: arn:aws:iam::XXX:user/yuuki-sandbox
>   CreateDate: '2020-12-28T02:16:22+00:00'
>   Path: /
>   UserId: YYY
>   UserName: yuuki-sandbox

Finally

Please point out any mistakes or missing work.

[PYTHON] [End of 2020] A memo to start using AWS CLI (Version 2)