[PYTHON] A memorandum of trouble when formatting data

This article is a memorandum of the processing that was performed when the scraped data type was strange.

Check the data type

 df ['column name'] .dtype #DataFrame
 np.dtype () #numpy array

Data type change

 Convert to df ['column name']. astype (int) #int
 Convert to df ['column name']. dtype (float) #float
 df ['column name'] .dtype (str) # Convert to string

Confirmation of missing values

df.isnull()

Delete missing values

df.dropna()

Delete pandas line

 df.drop (number of lines)

Add column to pandas

 df ['new column name'] = [a, b, c, d, ...]

Recommended Posts

A memorandum of trouble when formatting data
A memorandum of kernel compilation
A small memorandum of openpyxl
A memorandum of method often used when analyzing data with pandas (for beginners)
A memorandum of using eigen3
A collection of methods used when aggregating data with pandas
A memorandum when using beautiful soup
[Python] A memorandum of beautiful soup4
A memorandum of files under conf.d
A memorandum of closure survey contents
A memorandum of speed of arbitrary degree diagonalization
A memorandum of understanding about django's QueryDict
Memorandum of Understanding when migrating with GORM
A memorandum of python string deletion process
Memorandum of introduction of EXODUS, a data model of the finite element method (FEM)
Memorandum of sed
A memorandum of studying and implementing deep learning
Python variadic memorandum when inheriting a defined class
A memorandum of extraction by python bs4 request
[Linux command] A memorandum of frequently used commands
A well-prepared record of data analysis in Python
[Data science memorandum] Handling of missing values ​​[python]
Be careful when differentiating the eigenvectors of a matrix
Generate a vertical image of a novel from text data
A memorandum when writing experimental code ~ Logging in python
Analysis of measurement data ①-Memorandum of understanding for scipy fitting-
A memorandum when an error occurs with pip install
A memorandum regarding Wifi connection when installing Arch Linux
A memorandum when making a surveillance camera with Raspberry Pi
Numerical summary of data
Memorandum of fastText (editing)
memorandum of vi command
A memorandum about matplotlib
A memorandum about Nan.
elasticsearch_dsl Memorandum of Understanding
Preprocessing of prefecture data
Selection of measurement data
[Django] A memorandum when you want to communicate asynchronously [Python3]
Recommendation of Jupyter Notebook, a coding environment for data scientists
A memorandum of stumbling on my personal HEROKU & Python (Flask)
A network diagram was created with the data of COVID-19.
[GCP] A memorandum when running a Python program on Cloud Functions
[Introduction to AWS] A memorandum of building a web server on AWS
A story about clustering time series data of foreign exchange
[Data science memorandum] Confirmation of the contents of DataFrame type [python]
Manage the overlap when drawing scatter plots with a large amount of data (Matplotlib, Pandas, Datashader)