[GO] Python basics basic course CSV processing (functions and classes, part 1 CSV is read and written)

This article is for Python beginners at Django Girls Japan and elv, This is a material for study sessions. In addition, since the author is also a beginner, we apologize for any problems.

Namer environment: Windows 10 python3.5

This page has the basic steps to read CSV and spit it out in another format This page explains step by step. First of all, before creating a class, let's write the process from the beginning to the end.

When you execute the file created this time, it will be like this

This time, the read CSV (addresslist.csv) is converted to the following format. Export as a separate CSV file (allcodelist.csv).

Address and phone number are appropriate information

addresslist.csv 6.JPG

allcodelist.csv 5.JPG

Let's create a folder

First of all, preparation. Create a new folder in any location. On the command line (Click here for how to open the command line [http://qiita.com/sandream/items/a0f69e69b561192517a2) When you reach the location where you want to create the folder, enter ** mkdir folder name **. This time, create a folder called address.

mkdir address

Preparation 2

Write the following in a text editor in the created folder, Save the python file.

# -*- coding: utf-8 -*-
#!/usr/bin/env python

Please include the above at the beginning of the file. (The explanation of the above contents is omitted here.) Give the file a name and save it with a .py extension. (Character code is utf-8.) There are various text editors, but the article author uses ATOM.

Read CSV

Open CSV data and receive data

Now that we are ready, let's load the csv. ** Variable = open (path of file you want to read, "r") ** You can read the file data with.

# -*- coding: utf-8 -*-
# !/usr/bin/env python


f = open("addresslist.csv", "r")
print(f)
f.close()

Open the file with open and close the last opened file. I am printing the data I received before closing it. This time I put the csv file in the same folder, so It is possible to receive only by the file name. (If you want to read a file in another folder, enter the path.)

When I run the file

1.JPG

You can see that you have successfully received addresslist.csv.

I want to change the file name to be read each time

Well, it deviates a little from the original flow, but so far in the program The file name to be read was fixed as addresslist.csv. However, in some cases you may want to read a different file.

We will show you how to get and use the file name entered at the command prompt when executing the file.

** sys module ** Import.

# -*- coding: utf-8 -*-
# !/usr/bin/env python
import sys
args = sys.argv
print(args)

** sys.argv ** is entered as the python file name when executing the file from the command prompt, isn't it? When you enter it, the contents of the text entered after the python file name and separated by a space will be received as a list.

When you execute the file containing the above code, it will be as follows. 4.JPG

python allcode2.py "test" If you execute, you can see that ** ['allcode2.py','test'] ** is returned. python allcode2.py "test" "DjangoGirls" "elv" If you run You can see that ** ['allcode2.py','test','DjangoGirls','elv'] ** is returned. The list always starts with the name of the Python file you are running.

Use this feature to get the name of the file and open it.

# -*- coding: utf-8 -*-
# !/usr/bin/env python

import sys
import csv

args = sys.argv

filename = args[1]

f = open(filename, 'r')

for i in csv.DictReader(f):
    print(i)

I put the list in sys.argv in a variable called args. As explained above, the first element of the list will be the name of the executed file, so get the second element of the list. (For the list, click here](http://qiita.com/sandream/items/e7a215a27c080cf8ac03)) Get the name of the csv file you want to read with args [1] in the variable ** filename **.

Let's handle exceptions

Now, when the user executes the file, if he forgets to enter the file name, the file cannot be opened and the program will fail. Therefore, if you forget to enter the file name, or if you specify a file name that does not exist, quit the program and set it to display an easy-to-understand error. try: ** What you want to do ** except: ** Processing when the program tried under try cannot be executed as desired **

This time, I will introduce two error handling. (Someday ... I'd like to write various articles about error handling separately.)

The first is processing when the user forgets to enter the file name. If you specify ** filename = args [1] ** and the user does not enter a filename, an error will occur because [1] in the list does not exist.

Therefore, if an error occurs, I want to terminate the program and notify the user that the file name is not included, so do as follows.

try:
    filename = args[1]
except:
    sys.exit("No file Name!")

If you do this, if you execute it without specifying the CSV file name to read 8.JPG In this way, the characters specified in () of sys.exit () are displayed.

The next time the user enters a file name, if the file does not exist, the variable filename will contain the file name, When opening with ** f = open (filename,'r') **, an error will occur because the file does not exist.

Therefore, it is described as follows.

try:
    filename = args[1]
    f = open(filename, 'r')
except IOError as e:
    sys.exit("Unable to open file: {}".format(e))

** IOE error ** occurs when input or output fails. For example, an error that occurs when the disk is full or the input file does not exist. ** IOError as e ** means that when an exception occurs, the object containing that information is treated as an "e" and the processing inside it (in this case, only the information is displayed) is executed. This time, I wanted to display an error message, so I set it. If you don't want to use the error value, you don't need to write it.

If you specify an appropriate file name and execute it, 9.JPG It will be. After Unable to open file: is the IOError (the part marked e) error message.

In this program, when the above two errors occur, the program will be terminated even if exception handling is not performed. For example, in writing many programs, I want you to be able to execute this process, but if you can't, that's fine. At times try: ** What you want to do ** except: 　pass By doing so, you can use it to do nothing if an error occurs. I think it's good to remember.

Also, if you receive an error message, make a list and write it out to the text at the end, you can check what went wrong after the program is finished, and you can recover it, which is convenient. ..

Use csv.DictReader ()

This time, key the received data on the first line. The second and subsequent lines create a dictionary that is the value for each key.

So we use the csv module. First, import the module with ** import csv **.

# -*- coding: utf-8 -*-
# !/usr/bin/env python
import csv

f = open("addresslist.csv", "r")

for i in csv.DictReader(f):
    print(i)

f.close()

(For the sake of explanation, the exception handling introduced above is omitted once.)

csv.DictReader () uses the first line of the received csv data as a key The data from the next row will generate an object that will be the value of that key.

I want to extract the generated objects line by line, so use the for statement and I'm printing. If you execute the above, you will get the following results.

2.JPG

By the way, if you want to list the loaded CSV Use ** csv.reader () ** to get a list.

# -*- coding: utf-8 -*-
# !/usr/bin/env python
import csv

f = open("addresslist.csv", "r")

for i in csv.reader(f):
    print(i)

f.close()

Execution result 3.JPG

Generate in the data format you want to export

This time, the read CSV (addresslist.csv) is converted to the following format. Export as a separate CSV file (allcodelist.csv).

Address and phone number are appropriate information

addresslist.csv 6.JPG

allcodelist.csv 5.JPG

Create a list in the format to export.

Prepare an empty list (newcsv) and add information to the empty list (newcsv).

newcsv = []
top_list = [
        'name',
        'zip code',
        'address',
        'phone',
        'mobile_pyone'
                ]
newcsv.append(top_list)

First, I put the line of the item in a variable called top_list and add it to the list newcsv used for export. To add a list, write ** the list name you want to add.append (object to add) **.

Now that the items are complete, we will add more information. Generate the information received by csv.DictReader () in the required format and add it to the list.

There are various ways to generate a character string, but this time we will introduce ** join **. ** Specify the characters to be used for connecting. join (pass the information you want to connect in a list) **

What does that mean ... 7.JPG

You can see that the character string of the list specified after join is connected with-specified before join.

There are other ways to concatenate character strings, such as using format and +, which may be more common, but it is [Janken Poi in Python for beginners (answers and explanations)](http: // qiita) Please refer to the concatenation of the character strings of .com / sandream / items / 01374069f447b7748eba). This time, I did not introduce it on the above page, so I used join.

This time, we will connect the information of each item, so we will do as follows.

for i in csv.DictReader(f):
    line = []
    line.append(" ".join((i['lastname'], i['firstname'])))
    line.append(":".join(("Postal code", i['zip code'])))
    line.append("".join((i['address1'], i['address2'], i['address3'])))
    line.append("：".join(("phone number", i['phone'])))
    line.append("：".join(("Cellphone number", i['mobilephone'])))
    newcsv.append(line)

Since I want to create a list for one line once in the for minutes, I will create an empty list in a variable called line and put the information for each cell in it. When I'm done, I've added it to the export list (newcsv).

Export the generated data as CSV

First, set the expression format. ** csv.writer (file object, delimiter = delimiter, line terminator = line feed code) ** Is used.

For the file object, specify ** open (CSV file name to export, export method **). As for the export method, ** w ** will be overwritten and ** a ** will be added.

** delimiter ** is a specification of what to divide. The default is a comma. CSV is separated by commas, so it is not specified this time.

** line terminator = line feed code) ** is a line feed code. This time, specify ** / n **.

writer = csv.writer(open("allcodelist.csv", "w"), lineterminator='\n')

After setting the expression format, export it.

writer.writerows(newcsv)

** writerows (object to write) ** This time, the data you want to write is put in a variable called newcsv, so Specify newcsv in () of writerows ().

** writerows ** writes all the objects specified in () with the file name specified in ** open (file name to write, export method **).

Below is the entire program.

# -*- coding: utf-8 -*-
# !/usr/bin/env python
import sys
import csv

args = sys.argv
#Receive the file name entered at the command prompt
try:
    filename = args[1]
except:
    sys.exit("No file Name!")

#Receive the corresponding CSV using the received file name
try:
    filename = args[1]
    f = open(filename, 'r')
except IOError as e:
    sys.exit("Unable to open file: {}".format(e))

#Generate CSV item line to export
newcsv = []
top_list = [
        'name',
        'zip code',
        'address',
        'phone',
        'mobile_pyone'
                ]
newcsv.append(top_list)
for i in csv.DictReader(f):
    line = []
    line.append(" ".join((i['lastname'], i['firstname'])))
    line.append(":".join(("Postal code", i['zip code'])))
    line.append("".join((i['address1'], i['address2'], i['address3'])))
    line.append("：".join(("phone number", i['phone'])))
    line.append("：".join(("Cellphone number", i['mobilephone'])))
    newcsv.append(line)

#Export the generated data
writer = csv.writer(open("allcodelist.csv", "w"), lineterminator='\n')
writer.writerows(newcsv)

f.close()

It's been a long time, but I hope you find it helpful. Next time, based on this content, create a class and then create a class that inherits the created class! I would like to proceed to. (The article is currently under construction. It will be linked when completed.)

Recommended Posts

Python basics basic course CSV processing (functions and classes, part 1 CSV is read and written)
Python basic course (13 classes)
Python Basic Course (1 What is Python)
[Introduction to Data Scientists] Basics of Python ♬ Functions and classes
Python Basic Course (14 Modules and Packages)
Read Python csv and export to txt
Read CSV file with Python and convert it to DataFrame as it is
Read JSON with Python and output as CSV
#Python basics (functions)
Python basics: functions
[Python] Read Japanese csv with pandas without garbled characters (and extract columns written in Japanese)
Full-width and half-width processing of CSV data in Python
Installing Python 3 on Mac and checking basic operation Part 1
Python Basic Course (7 Dictionary)
Python basic course (9 iterations)
Python Basic Course (11 exceptions)
Python basic course (6 sets)
Python Basic Course (Introduction)
Python basic memorandum part 2
Python basic memo --Part 2
# 4 [python] Basics of functions
Read Python csv file
Python basic memo --Part 1
Python basic course (8 branches)
Python Basic Course (3 Python Execution)
[Python] Read the csv file and display the figure with matplotlib