Rewrite field order nodes in SPSS Modeler with Python.

Field order nodes change the order of columns in SPSS Modeler. Let's rewrite this with Python pandas.

0. raw data

This is done using the following time-series sensor data.

■COND2n.csv Time: Uptime Power: Power Temperature: Temperature Pressure: Pressure Uptime: Uptime Status: Status code Outcome: error code

image.png

1m. Field order change Modeler version

Column order of "COND2n.csv" Time,Power,Temperature,Pressure,Uptime,Status,Outcome From the order of Time,Uptime,Power,Temperature,Pressure,Outcome,Status Change to.

image.png

Select Time and Uptime in the field order and set them to the top two. Next, set Outcome and Status to the two from the bottom.

The point is that there is an item "----- [Other fields] ------", and the other fields are automatically placed while maintaining the column order in the base table.

image.png

It is often used when you want to handle the same type of data such as Power, Temperature, Pressure as a set. This time there are only 3 columns, but when there are a lot of columns, it is very convenient because you do not have to specify 1 column or 1 column for all columns. Also, if a column for the sensor is added, it does not need to be modified.

1p. Field order change pandas version

There are several possible ways. The first simple way is to give a list of all columns after the change.

#Swap columns 1: List all columns
df1_1=df1[['Time','Uptime','Power','Temperature','Pressure','Outcome','Status']]
df1_1

image.png

However, this method can be tricky if you have a lot of columns, and it's less readable. Consider how to treat some columns as a set like Modeler.

The second method is to divide the dataframe into three parts. The point is that df1.loc [:,'Power':'Pressure'] creates dataframes for 3 columns of'Power','Temperature', and'Pressure'.

#Swap columns 2: Make 3 DFs and connect them
df1_2=pd.concat([df1[['Time','Uptime']],
                 df1.loc[:,'Power':'Pressure'],
                 df1[['Outcome','Status']]], axis=1, join='inner')
df1_2

However, this method creates a lot of dataframes in the middle, and I think that it is quite inefficient when the number of data is large.

The third method is After creating a list of column names with columns.tolist () collist [collist.index ('Power'): collist.index ('Pressure') +1] makes a list of column names of "'Power','Temperature','Pressure'".

#Swap columns 3: Create a list object of columns
collist=df1.columns.tolist()

df1_3=df1[['Time', 'Uptime']+
          collist[collist.index('Power'):collist.index('Pressure')+1]+
          ['Outcome','Status']]
df1_3

This method is lighter than the second method. However, it is a program that is not very readable.

The fourth method is a combination of the second and third methods. df1.loc [0: 0,'Power':'Pressure'] creates a dataframe of'Power', only one record of'Temperature','Pressure', and columns.tolist () only lists the columns. I'm taking it out.

#Swap columns 4: Create a column list object from the cut out DF
collist= df1.loc[0:0,'Power':'Pressure'].columns.tolist()

df1_3=df1[['Time', 'Uptime']+
          collist+
          ['Outcome','Status']]
df1_3

It's not very readable yet, but I think it's lighter than the second method and easier to read than the third method.

2. Sample

The sample is placed below.

stream https://github.com/hkwd/200611Modeler2Python/raw/master/fieldreorder/fieldreorder.str notebook https://github.com/hkwd/200611Modeler2Python/blob/master/fieldreorder/fieldreorder.ipynb

data https://raw.githubusercontent.com/hkwd/200611Modeler2Python/master/data/COND2n.csv

■ Test environment Modeler 18.2.2 Windows 10 64bit Python 3.7.9 pandas 1.0.5

3. Reference information

Field order node https://www.ibm.com/support/knowledgecenter/ja/SS3RA7_18.2.2/modeler_mainhelp_client_ddita/clementine/reorder_overview.html

Recommended Posts

Rewrite field order nodes in SPSS Modeler with Python.
Rewrite SPSS Modeler filter nodes in Python
Using Python with SPSS Modeler extension nodes ① Setup and visualization
Rewrite the record addition node of SPSS Modeler with Python.
Change node settings in supernodes with SPSS Modeler Python scripts
Rewrite the sampling node of SPSS Modeler with Python (2): Layered sampling, cluster sampling
Rewrite SPSS Modeler reconfigure node in Python. Aggregation by purchased product category
Natural order in python
Rewrite the sampling node of SPSS Modeler with Python ①: First N cases, random sampling
Rewrite duplicate record nodes in SPSS Modeler in Python. ① Identify the item you purchased first. (2) Identification of the top-selling item in the product category
Scraping with selenium in Python
Working with LibreOffice in Python
Scraping with chromedriver in python
Debugging with pdb in Python
Working with sounds in Python
Scraping with Selenium in Python
Scraping with Tor in Python
Tweet with image in Python
Combined with permutations in Python
Implement extension field in Python
Number recognition in images with Python
Testing with random numbers in Python
How to pass arguments to a Python script in SPSS Modeler Batch
Working with LibreOffice in Python: import
Scraping with Selenium in Python (Basic)
CSS parsing with cssutils in Python
Numer0n with items made in Python
Open UTF-8 with BOM in Python
Use rospy with virtualenv in Python3
Use Python in pyenv with NeoVim
Heatmap with Dendrogram in Python + matplotlib
Read files in parallel with Python
Password generation in texto with python
Use OpenCV with Python 3 in Window
Until dealing with python in Atom
Get started with Python in Blender
Working with DICOM images in Python
Write documentation in Sphinx with Python Livereload
Get additional data in LDAP with python
Spiral book in Python! Python with a spiral book! (Chapter 14 ~)
Try logging in to qiita with Python
Stress Test with Locust written in Python
Python3> in keyword> True with partial match?
Exclusive control with lock file in Python
Device monitoring with On-box Python in IOS-XE
Try working with binary data in Python
Draw Nozomi Sasaki in Excel with python
Display Python 3 in the browser with MAMP
Page cache in Python + Flask with Flask-Caching
Post Test 3 (Working with PosgreSQL in Python)
How to work with BigQuery in Python
Playing card class in Python (with comparison)
Dealing with "years and months" in Python
Replace non-ASCII with regular expressions in Python
Connect with mysql.connector with ssh tunnel in Python 3.7
One liner webServer (with CGI) in python
Get Started with TopCoder in Python (2020 Edition)
Easy image processing in Python with Pillow
To work with timestamp stations in Python
Call APIGateWay with APIKey in python requests
Read text in images with python OCR