[Python] Scan the inside of the folder including subfolders → Export the file list to CSV
This is my first post. I recently started studying Python, and although I still have half-baked knowledge, I would like to post it as a memorandum. I would be grateful if you could feel free to comment on any advice.
Thing you want to do
I'm using a departmental shared folder in a corporate environment, but the capacity is small and it fills up quickly. I want to visualize what files are occupying space in this folder. As a general rule, using free software is NG according to company rules, so I made a tool myself after studying.
What I studied
1. 1. Directory scanning with walk module
You can easily scan the folder with os.walk. (In the case of VBA, it was necessary to perform recursive processing, but it can be easily written by using walk)
How to use
for curDir, dirs, files in os.walk(target_folder):
For the folders described in target_folder_p above, the files are stored in files, the subfolders are stored in dirs, and the folders in which each is stored are stored in curDir.
After that, further for nesting in that for, files and dirs are expanded, and processing is performed.
Example of use:
python
import os
target_folder = r"C:\Users\aaa\Desktop\Photo folder"
for curDir, dirs, files in os.walk(target_folder):
for a_file in files:
print(f"{a_file}Is a folder:{curDir}It is stored in.")
result
Mandarin orange.jpg is a folder: C:\Users\aaa\Desktop\It is stored in the photo folder.
Apple.jpg is a folder: C:\Users\aaa\Desktop\It is stored in the photo folder.
banana.jpg is a folder: C:\Users\aaa\Desktop\It is stored in the photo folder.
dog.jpg is a folder: C:\Users\aaa\Desktop\Photo folder\Stored in animals.
Cat.jpg is a folder: C:\Users\aaa\Desktop\Photo folder\Stored in animals.
2. Output to csv file
For output to csv, open the file with the open function, convert it to csv with csv.writer, and write it with writerow or writerows.
Example of writing
python
import csv
with open (file name to output (full path),Read mode, newline="") as fo:
CSV_file = csv.writer(fo)
CSV_file.writerow(["A1", "B1", "C1", "D1"])
CSV_file.writerow([["A2", "B2", "C2", "D2"],["A3", "B3", "C3", "D3"]])
CSV_file.writerows([["A4", "B4", "C4", "D4"],["A5", "B5", "C5", "D5"]])
result:
write row writes one line
write rows write multiple lines at once
** Read mode (second argument of open function) **
- "w": Write-only (if there is an existing file, the contents will be erased and written)
- "a": Additional export (if there is an existing file, the data will be written at the end)
- "x": New export only (if there is an existing file, nothing is written)
- "r": Read-only
- "r +": literacy
I'm still studying the optional newline. However, if this is omitted, the data will be skipped by one line when it is output.
Deliverables
File list export.py
import os
import pathlib
import csv
from tkinter import filedialog
#Specify the folder to read
F_path = os.path.dirname(__file__)
target_folder = filedialog.askdirectory(initialdir=F_path)
target_folder_path = pathlib.Path(target_folder)
filedata = []
#Operate in the folder
for curDir, dirs, files in os.walk(target_folder_path):
for a_File in files:
file_size = os.path.getsize(os.path.join(curDir, a_File))
file_ext = os.path.splitext(a_File)
filedata.append([len(filedata)+1, a_File,file_ext[1], file_size,os.path.join(curDir, a_File)])
#Display reading result
print("No.File name Extension File size (Byte) Full path")
for i in range(len(filedata)):
print(f"{filedata[i][0]} {filedata[i][1]} {filedata[i][2]} {filedata[i][3]} {filedata[i][4]}")
#Confirmation of file output destination
print("The file list is output as CSV to the top level of the read folder.\n Enter the output file name (no extension required)")
print("(If you do not want to output, leave blank and press ENTER)")
csv_name = input("Output file name:")
if csv_name == "":
print("Did not output the file.")
else:
#Output the read result to a file
output_csv_path = os.path.join(target_folder_path, csv_name + ".csv")
with open(output_csv_path, "w", newline="") as fo:
CSV_file = csv.writer(fo)
CSV_file.writerow(["No.", "file name","extension", "File size (Byte)","full path"])
CSV_file.writerows(filedata)
print(f"The following file was output.=> {output_csv_path}")
result
No.File name Extension File size (Byte) Full path
1 Mushroom Takeko War.jpg .jpg 118437 C:\Users\aaa\Desktop\Photo folder\Mushroom Takeko War.jpg
2 Takenoko no Sato.jpg .jpg 93003 C:\Users\aaa\Desktop\Photo folder\Takenoko no Sato.jpg
3 cat 1.jpg .jpg 31115 C:\Users\aaa\Desktop\Photo folder\animal\Cat 1.jpg
4 cat 2.jpg .jpg 21160 C:\Users\aaa\Desktop\Photo folder\animal\Cat 2.jpg
The file list is output as CSV to the top level of the read folder.
Enter the output file name (no extension required)
(If you do not want to output, leave blank and press ENTER)
Output file name: output
The following file was output.=> C:\Users\aaa\Desktop\Photo folder\output.csv
Other memorandums / memos
- You can specify the folder to read in tkinter.filedialog.askdirectory by yourself. The location of this file is referenced by the optional initialdir.
- The read folder is converted to Path type with pathlib.Path, but it seems that it can be output without conversion. I forgot why I converted it to Path type when I created it.
- All the read data is stacked in a list (array) and output to csv at once. Since it was handled differently from the VBA array, I was worried about how to stack it. The difference between append and extend. Also, I want to organize.
- External libraries such as NamPy are likely to have a heavy file size when converted to an exe, so do not use them as much as possible for the time being.
- Get the file size with os.path.getsize (full path file name)
- Get the extension with os.path.splitext (filename, not the full path?). The return value is a tuple, the file name without the extension is stored in the first element, and the extension is stored in the second element. If there are multiple periods, the later period will be applied. (Example: test.png.jpg → can be divided into test.png and .jpg)
Recommended Posts