[PYTHON] How to extract non-missing value nan data with pandas

How to extract data that is not missing value nan with pandas

I was just thinking about how to extract the list of member stores of Go To EAT in Chiba prefecture, so I summarized it with reference to the article.

import pandas as pd
import io

data = """
name,Number of times,start,End
Ponsuke,1,9:00,18:00
Ponsuke,2,18:00,
Ponsuke,3,9:00,13:00
Ponsuke,4,,
Ponsuke,5,9:00,
Ponsuke,6,18:00,
Ponsuke,7,12:00,
Ponsuke,8,12:00,
Ponsuke,9,,18:00
Ponsuke,10,,
"""

df = pd.read_csv(io.StringIO(data))

df
name Number of times start End
0 Ponsuke 1 9:00 18:00
1 Ponsuke 2 18:00 nan
2 Ponsuke 3 9:00 13:00
3 Ponsuke 4 nan nan
4 Ponsuke 5 9:00 nan
5 Ponsuke 6 18:00 nan
6 Ponsuke 7 12:00 nan
7 Ponsuke 8 12:00 nan
8 Ponsuke 9 nan 18:00
9 Ponsuke 10 nan nan

Extract data where one column is not NaN

df[(df.loc[:, "start"].notnull() == True)]

Extract data that is not NaN in both columns

df[(df.loc[:, ["start", "End"]].notnull() == (True, True)).all(axis=1)]

Extract data if either of the two columns is not NaN

df[(df.loc[:, ["start", "End"]].notnull() == (True, True)).any(axis=1)]

Extract data of "1 column is not NaN" & "1 column is NaN"

df[(df.loc[:, ["start", "End"]].notnull() == (False, True)).all(axis=1)]

Recommended Posts

How to extract non-missing value nan data with pandas
How to extract non-missing value nan data with pandas
How to convert horizontally held data to vertically held data with pandas
How to extract null values and non-null values with pandas
Extract the maximum value with pandas.
How to deal with imbalanced data
How to deal with imbalanced data
How to Data Augmentation with PyTorch
How to extract features of time series data with PySpark Basics
Try converting to tidy data with pandas
How to read problem data with paiza
How to create sample CSV data with hypothesis
Try to aggregate doujin music data with pandas
[Python] How to read excel file with pandas
How to scrape horse racing data with BeautifulSoup
How to use Pandas 2
Convert 202003 to 2020-03 with pandas
Data visualization with pandas
Data manipulation with Pandas!
Shuffle data with pandas
How to replace with Pandas DataFrame, which is useful for data analysis (easy)
How to use xgboost: Multi-class classification with iris data
How to scrape image data from flickr with python
How to scrape horse racing data using pandas read_html
How to access with cache when reading_json in pandas
How to get more than 1000 data with SQLAlchemy + MySQLdb
How to output CSV of multi-line header with pandas
How to convert JSON file to CSV file with Python Pandas
[Python] How to deal with pandas read_html read error
Extract the maximum value with pandas and change that value
How to update with SQLAlchemy?
How to use Pandas Rolling
How to cast with Theano
How to Alter with SQLAlchemy?
How to separate strings with','
How to RDP with Fedora31
How to handle data frames
Data processing tips with Pandas
How to Delete with SQLAlchemy?
How to extract any appointment in Google Calendar with Python
Ingenuity to handle data with Pandas in a memory-saving manner
Try to extract Azure SQL Server data table with pyodbc
How to extract other than a specific index with Numpy
How to get an overview of your data in Pandas
How to read an Excel file (.xlsx) with Pandas [Python]
How to create dataframes and mess with elements in pandas
Data science companion in python, how to specify elements in pandas
[Introduction to Python] How to get data with the listdir function
Try to extract the features of the sensor data with CNN
How to cancel RT with tweepy
[Python] How to FFT mp3 data
Python: How to use async with
How to read e-Stat subregion data
How to write soberly in pandas
[Python] How to use Pandas Series
How to use virtualenv with PowerShell
How to install python-pip with ubuntu20.04LTS
Versatile data plotting with pandas + matplotlib
How to install pandas on EC2 (How to deal with MemoryError and PermissionError)
How to get started with Scrapy
How to get started with Python