[PYTHON] I wrote the basic operation of Pandas with Jupyter Lab (Part 1)

This article is an article that I actually coded the basic operation of Pandas described in Kame (@usdatascientist)'s blog (https://datawokagaku.com/python_for_ds_summary/) using Jupyter Lab.

Summary of basic operations of Pandas

10th

import pandas as pd
import numpy as np

Series

data = {'name':'John', 'sex':'male', 'age': 22}
john_s = pd.Series(data)
print(john_s)

name    John
sex     male
age       22
dtype: object

array = np.array([10,20,30])
pd.Series(array)

0    10
1    20
2    30
dtype: int64

array = np.array([10,20,30])
labels = ['a','b','c']
pd.Series(array, labels)

a    10
b    20
c    30
dtype: int64

11th

How to make a DataFrame

Make from ndarray

data = {'name':'John', 'sex':'male', 'age': 22}
john_s = pd.Series(data)
print(john_s)
print(john_s['age'])

name    John
sex     male
age       22
dtype: object
22

ndarray = np.random.randint(5, size=(5,4))
pd.DataFrame(data=ndarray)

columns = ['a','b','c','d']
index = np.arange(0,50,10)
pd.DataFrame(data=ndarray, index=index, columns=columns)

Make from dictionary

data1 = {
    'name':'John',
    'sex':'male',
    'age':22
}
data2 = {
    'name':'Zack',
    'sex':'male',
    'age':30
}
data3 ={
    'name':'Emily',
    'sex':'female',
    'age':32
}
pd.DataFrame([data1, data2, data3])

df = pd.read_csv('train.csv')
df.head()

12th

Display the first 5 lines with .head ()

df.head()

Check statistics with .describe ()

df.describe()

type(df.describe()) #type is DataFrame

pandas.core.frame.DataFrame

Show list of columns in .columns

df.columns

Index(['PassengerId', 'Survived', 'Pclass', 'Name', 'Sex', 'Age', 'SibSp',
       'Parch', 'Ticket', 'Fare', 'Cabin', 'Embarked'],
      dtype='object')

type(df.columns) #type is index

pandas.core.indexes.base.Index

df.index #There is also an index.

RangeIndex(start=0, stop=891, step=1)

Get the Series with a specific column embraced with the bracket [].

df['Age'].head()

0    22.0
1    38.0
2    26.0
3    35.0
4    35.0
Name: Age, dtype: float64

type(df['Age'])

pandas.core.series.Series

Put a list of columns in the bracket [] and extract multiple columns at once

df[['Age','Parch','Fare']].head()

Get a specific row in Series with .iloc [int]

df.iloc[888] #index location

PassengerId                                         889
Survived                                              0
Pclass                                                3
Name           Johnston, Miss. Catherine Helen "Carrie"
Sex                                              female
Age                                                 NaN
SibSp                                                 1
Parch                                                 2
Ticket                                       W./C. 6607
Fare                                              23.45
Cabin                                               NaN
Embarked                                              S
Name: 888, dtype: object

df.iloc[888]['Age']

nan

np.isnan(df.iloc[888]['Age'])

True

np.random.seed(1)
ndarray = np.random.randint(10, size=(5,5))
columns = [0,1,2,3,4]
index = ['a','b','c','d','e']
df_1 = pd.DataFrame(data=ndarray, index=index, columns=columns)
df_1

df_1[0] 

a    5
b    0
c    2
d    2
e    1
Name: 0, dtype: int64

df_1.loc['c'] #When the line is not an int['str']To.

0    2
1    4
2    5
3    2
4    4
Name: c, dtype: int64

Drop certain rows and columns with Slicing

Drop index = 0 (0th column)

df.drop(0) .head()

Drop the'Age'column

df.drop('Age', axis=1) .head()

When dropping multiple columns, pass a list as an argument .drop ([]). Drop does not change the original df

df.drop(['Age','PassengerId'], axis=1) .head()

df.head()#Drop does not change the original df

There are two ways to overwrite df. Setting place = True will change the original DataFrame

df = pd.read_csv('train.csv')
df.drop(['Age', 'Cabin'], axis=1, inplace=True) 

df .head()

df = pd.read_csv('train.csv')
df = df.drop(['Age', 'Cabin'], axis=1)

id(df)

140285150057616

Get multiple lines with slicing

df.iloc[5:10]

13th

Filter the DataFrame by specific conditions

df = pd.read_csv('train.csv')
df = df['Survived'] == 1#Filter survivors
df.head()

0    False
1     True
2     True
3     True
4    False
Name: Survived, dtype: bool

filter = df['Survived'] ==1 #Put it in a variable called filter
df = df[filter]
df.head()

df = df[df['Survived'] ==1] #This is more common
df.head()

df[df['Survived'] ==1].describe() #Describe only survivor data

df.describe() #raw data

df[df['Age'] >= 60].describe() #'Age'>=60 only

df[(df['Age']>=60) & (df['Sex']=='female')] #Data for women over 60 years old only

df[(df['Pclass']==1) | (df['Age']<10)] #Data for 1st class or under 10 years old only
df.head()

If ~ (squiggle) is added, it can be filtered by NOT operation.

data =[{'Name':'John', 'Survived':True},
      {'Name':'Emily', 'Survived':False},
      {'Name':'Ben', 'Survived':True}]
df = pd.DataFrame(data)
df

It is often used when filtering by a column whose value is boolean.

df[df['Survived']==True] 

Since the Survived column is already Boolean, you don't need == True. Since df ['Survived'] is already a Boolean Series, you can filter it as it is as shown on the left.

df[df['Survived']] 

If you want to narrow down to Survived == False, you can do the following without having to do df [df ['Survived'== False]

df[~df['Survived']] 

Change index

Reallocate index with .reset_index ()

df = pd.read_csv('train.csv')
df = df[df['Sex']=='male']
df.head() #index is disjointed

Align indexes

As with .drop (), the original df is not overwritten, so if you want to update df, reassign it with inplace = True or df = df.reset_index ().

df.reset_index() .head()

Use .set_index () to index a specific column

Set index to ‘Name’

As with .reset_index (), you can overwrite the original df with inplace = True.

df.set_index('Name').head()