[PYTHON] Create a pandas Dataframe from a string.

Trigger

Note that I was having trouble converting a string to a pandas Dataframe. The trigger was a scene where I wanted to handle a file of about 300GB with pandas. It seems impossible with read_csv () directly, so it was good to cut it finely with open () or readline (), but by the way, it was the first time I made a Dataframe from a character string. When I looked it up, I couldn't find anything good.

solution

I tried using numpy's reshape ().

pd.DataFrame(np.array(data.split()).reshape(-1,7))

The image looks like this.

screenshot.png

Fortunately, the number of columns was fixed. If the number of lines is not fixed, it seems better to put -1 in the argument of reshape (). The number of columns is also the same. Please let me know if there is another good way.

Recommended Posts

Create a pandas Dataframe from a string.
Create a dataframe from excel using pandas
How to create a function object from a string
[Python pandas] Create an empty DataFrame from an existing DataFrame
[python] Create table from pandas DataFrame to postgres
Create a datetime object from a string in Python (Python 3.3)
Django: Import a class from a string
Python-dynamically call a function from a string
Create a random string in Python
Create an instance of a predefined class from a string in Python
DataFrame of pandas From creating a DataFrame from two lists to writing a file
How to create a clone from Github
# 5 [python3] Extract characters from a character string
Create a standings from JFL match results
[Pandas] Expand the character string to DataFrame
[Pandas_flavor] Add a method of Pandas DataFrame
Create a deb file from a python package
Generate a class from a string in Python
How to create a repository from media
Download Pandas DataFrame as a CSV file
[Python / Tkinter] Search for Pandas DataFrame → Create a simple search form to display
Create a game UI from scratch with pygame2!
Create a New Todoist Task from Python Script
Create a record table from JFL match results
Create a phylogenetic tree from Biopyton using ClustalW2
Create a protein sequence mutation library in pandas
Create a score ranking from JFL match results
Extract lines containing a specific "string" in Pandas
Create a decision tree from 0 with Python (1. Overview)
Convert pandas dataframe elements to regular string type
Create a Word Cloud from an academic program
Create a Django schedule
Create a Python module
Create a Bootable LV
Create a Python environment
Python script to create a JSON file from a CSV file
How to create a kubernetes pod from python code
Python application: Pandas # 3: Dataframe
Create a machine learning environment from scratch with Winsows 10
Create a slack bot
Create a decision tree from 0 with Python and understand it (3. Data analysis library Pandas edition)
Create a new csv with pandas based on the local csv
Convert from Pandas DataFrame to System.Data.DataTable using Python for .NET
Create a data frame from the acquired boat race text data
I made a tool to create a word cloud from wikipedia
Put the lists together in pandas to make a DataFrame
A handy function to add a column anywhere in a Pandas DataFrame
Python: Create a dictionary from a list of keys and values
Python> Read from a multi-line string instead of a file> io.StringIO ()
Create a GCE instance from a GCR Docker image using terraform
Create a message corresponding to localization with python translation string
Create a correlation diagram from the conversation history of twitter
[Python] Random data extraction / combination from DataFrame using random and pandas
Create a Wox plugin (Python)
Create a local pypi repository
Create a function in Python
Create a dictionary in Python
[Python] Use a string sequence
Create a (simple) REST server
Create a homepage with django
Is this string a decimal?