[Python] Read a csv file with a large data size using a generator


import csv


def csv_reader(file, header=False):
    with open(file, "r") as f:
        reader = csv.reader(f)

        if header:
            next(reader)

        yield from reader


def main():
    g = csv_reader("xxx.csv", header=False)
    print(next(g)) # =>First line

if __name__ == '__main__':
    main()




yield from reader

Is


for row in reader:
    yield row

But it can be implemented.

Recommended Posts

[Python] Read a csv file with a large data size using a generator
Read CSV file with python (Download & parse CSV file)
Read a character data file with numpy
Read Python csv file
Read data with python / netCDF> nc.variables [] / Check data size
[Python] Split a large Flask file using Blueprint
Read Python csv data with Pandas ⇒ Graph with Matplotlib
Read table data in PDF file with Python
Read line by line from a file with Python
Read and format a csv file mixed with comma tabs with Python pandas
Read csv with python pandas
Collectively register data in Firestore using csv file in Python
Process csv data with python (count processing using pandas)
Download csv file with python
Read json data with python
Consolidate a large number of CSV files in folders with python (data without header)
[Python] Read the csv file and display the figure with matplotlib
Output to csv file with Python
Read a file in Python with a relative path from the program
Creating a simple PowerPoint file with Python
Add a Python data source with Redash
Create a large text file with shellscript
[Automation] Read a Word document with Python
I made a configuration file with Python
[Automation] Read mail (msg file) with Python
Read a Python # .txt file for a super beginner in Python with a working .py
[Python] How to read a csv file (read_csv method of pandas module)
Read CSV file with Python and convert it to DataFrame as it is
Python / numpy> Read the data file with the item name line> Use genfromtxt ()
[python] Read data
Create a GIF file using Pillow in Python
Read a file containing garbled lines in Python
Read JSON with Python and output as CSV
Create a PDF file with a random page size
A story stuck with handling Python binary data
Folium: Visualize data on a map with Python
Draw a graph with matplotlib from a csv file
Write CSV data to AWS-S3 with AWS-Lambda + Python
Run a Python file from html using Django
Create a Photoshop format file (.psd) with python
Create a MIDI file in Python using pretty_midi
I want to write to a file with Python
Open a file dialog with a python GUI (tkinter.filedialog)
Extract data from a web page with Python
Various ways to read the last line of a csv file in Python
Summary of how to read numerical data with python [CSV, NetCDF, Fortran binary]
Convert csv, tsv data to matrix with python --using MovieLens as an example
Python --Read data from a numeric data file and find the multiple regression line.
Replace the named entity in the read text file with a label (using GiNZA)
Draw a line / scatter plot on the CSV file (2 columns) with python matplotlib
Data analysis with python 2
Create a data collection bot in Python using Selenium
Get a large amount of Starbucks Twitter data with python and try data analysis Part 1
Try creating a compressed file using Python and zlib
A4 size with python-pptx
Data analysis using Python 0
Get financial data with python (then a little tinkering)
I tried to touch the CSV file with Python
Every time I try to read a csv file using pandas, I get a numpy error.
[Python] Create a ValueObject with a complete constructor using dataclasses
Read CSV file: pandas