[PYTHON] 2 ways to read all csv files in a folder


We have summarized how to read csv files (txt etc. are also possible) in the folder in order. I wrote it for those who want to know how to automatically read in order instead of writing the instructions to read files one by one when reading all the large number of csv files in an arbitrary folder. It is assumed that you know how to read individual csv files.

table of contents

  1. Pattern to read the names of all files
  2. Pattern to paste the path of all files into the csv file

1. Pattern to read the names of all files

First, import it.

import os 
import pandas as pd

Next, copy the address of the folder where the csv files you want to read are stored. From here, let's take a quick look.

#Paste the copied address.
csv_file=os.listdir('C:/Users/Satoru Mizu/Documents/Qiita')
>['First day.csv', 'the 2nd day.csv']
>First day.csv

#First, let's read the first file
file1=pd.read_csv('C:/Users/Satoru Mizu/Documents/Qiita/'+str(csv_file[0]),engine='python')

With the above operation, the first file among a large number of csv files can be read. When I check the contents,

>Column 1 Column 2 Column 3 Column 4
>0 A-kun B-kun C-kun D-kun
>1   1    2    3    4
>2   9   10   11   12

I was able to read it properly.

By using for minutes, you can write code that reads and processes csv files in order. A simple example is shown below.


Calculate the total score of Mr. A

for i in range(len(csv_file)): #Turn the for statement as many times as there are csv files.
  file =pd.read_csv('C:/Users/Satoru Mizu/Documents/Qiita/'+str(csv_file[i]),engine='python')
  A_sum += file.iloc[1][0] + file.iloc[2][0]

# 2. Pattern to paste the path of all files into csv

 First, select all the csv files in the folder where the csv files are stored.
 In that state, hold down "Shift" and right-click.
 I think a "copy of the path" will appear, so copy it.
 After that, create a new csv file (txt is also acceptable) in the folder where the program is running and paste it.
 In my case, it looks like this.

 C: \ Users \ Mizugoro \ Documents \ Qiita \ Day 1 .csv
 C: \ Users \ Mizugoro \ Documents \ Qiita \ Day 2 .csv

 Here, replace "\" with "/" (Ctrl + F). Then it looks like this

 C: / Users / Satoru Mizu / Documents / Qiita / Day 1 .csv
 C: / Users / Satoru Mizu / Documents / Qiita / Day 2.csv

csv_adress=pd.read_csv('File with pasted path.csv',engine='python')

>adress    C:/Users/Satoru Mizu/Documents/Qiita/First day.csv
>Name: 0, dtype: object
 Therefore, it cannot be used as a path when opening a csv file.

> C:/Users/Satoru Mizu/Documents/Qiita/First day.csv
 Then, it will be only the path.

>Column 1 Column 2 Column 3 Column 4
>0 A-kun B-kun C-kun D-kun
>1   1    2    3    4
>2   9   10   11   12
 I opened it properly ~

 Thank you for your hard work! !!

Recommended Posts

2 ways to read all csv files in a folder
Batch convert all xlsx files in the folder to CSV files
How to read CSV files in Pandas
How to combine all CSVs in a folder into one CSV
Convert UTF-8 CSV files to read in Excel
Create a command to delete all temporary files generated in a specific folder
Various ways to read the last line of a csv file in Python
[R] [Python] Memo to read multiple csv files in multiple zip files
How to read a CSV file with Python 2/3
[Python] Get the files in a folder with Python
Read CSV files uploaded to Flask without saving
How to get the files in the [Python] folder
How to read a file in a different directory
Various ways to extract columns in a NumPy array
How to reference static files in a Django project
How to read csv containing only integers in Python
[Python] Rename all image files in a specific folder by shooting date and time
Process the files in the folder in order with a shell script
Command to list all files in order of file name
How to display a specified column of files in Linux (awk)
I made a library to easily read config files with Python
Create a CSV reader in Flask
How to get all the possible values in a regular expression
6 ways to string objects in Python
Read files in parallel with Python
5 Ways to Create a Python Chatbot
Get a list of files in a folder with python without a path
I made a plugin to generate Markdown table from csv in Vim
Use shutil to delete all folders with a small number of files
I want to convert a table converted to PDF in Python back to CSV
[Python] How to read a csv file (read_csv method of pandas module)
I made a script in python to convert .md files to Scrapbox format
Generate all files with a specific extension
How to create a git clone folder
I want to print in a comprehension
How to convert csv to tsv in CLI
AHC task (3) Read CSV as a dictionary
Automatically update CSV files to AWS DynamoDB
How to get a stacktrace in python
Read and write csv files with numpy
Read Python csv and export to txt
Read and write JSON files in Python
Various ways to create a dictionary (memories)
[V11 ~] A memorandum to put in Misskey
Control files to build in Golang project
Books on data science to read in 2020
A note to load open data in CSV format into Cloudant on Bluemix
A confusing story with two ways to implement XGBoost in Python + overall notes
How to drop Google Docs in one folder in a .txt file with python
I want to see a list of WebDAV files in the Requests module
How to read a serial number file in a loop, process it, and graph it
How to write files that you should be careful about in all languages
How to get a list of files in the same directory with python
[Python] I want to collect specific files scattered in each folder in one place
Various ways to execute .py files on Windows
3 ways to parse time strings in python [Note]
Convert FBX files to ASCII <-> BINARY in Python
Add a command to mark similar files together
Try to calculate a statistical problem in Python
How to clear tuples in a list (Python)
How to embed a variable in a python string