Introduction to Structural Equation Modeling (SEM), Covariance Structure Analysis with Python

Introduction

I had to use Python to perform Structural Equation Modeling (SEM). However, since there were almost no Japanese articles to serve as a reference, I decided to write a summary article myself.

What is structural equation modeling?

--Structural equation modeling is also known as covariance structure analysis. --There are people who call it SEM (Esu, I, Emu) or SEM (Sem).

History of structural equation modeling

――It is said that it was born in the form of being implemented in a computer from the 1960s to the 1970s.

Algorithm type

  1. Simultaneous equation regression method
  2. Iterative maximum likelihood algorithm
  3. Iterative canonical phase matching algorithm

There are various algorithms such as. When I start digging deep here, it seems that I will not come back anymore. Therefore, I will omit it this time.

Attractiveness of structural equation modeling

――Personally, I think the biggest attraction is that you can execute the factor model and the regression model at the same time. ――In addition, you can set up a ** hypothetical model ** saying, "First of all, this is the model," and then program the model as it is and execute it. ――In other words, it is a ** flexible analysis method ** that allows you to immediately put your thoughts into shape. ――So, you can ** minimize the discussion, repeat SEM repeatedly, and refine the model **.

Observed variables and latent variables

In order to perform structural equation modeling, you need to know the idea of ​​observed variables and latent variables.

Observed variables

――As the name suggests, it is ** "observable variable" **. ――Specifically, it refers to the data that can be actually collected in university research, experiments, and business. ――Examples that often appear in structural equation modeling textbooks are subjects such as "humanities academic ability" and "national language, mathematics ...". --Observed variables correspond to "national language" and "mathematics" that can be observed as scores.

Latent variable

--The variables ** hidden behind the observable variables **. ――In the factor model, this is called ** factor (or common factor) **. ――The matching app can collect (≒ observe) data such as "annual income", "occupation", "house", and "hobby", but you cannot observe the "whether or not it is popular (degree of popularity)" behind it. ――Probably many people somehow * recognize that there are "people who are popular (people with a high degree of popularity)" **. I know that the index of popularity is built into the ** model of matching. However, it is difficult to obtain it as data. ** These variables are called latent variables.

Practical application

――That's why it is used in academic fields such as sociology and psychology where ** variables (latent variables) that cannot be easily obtained as data are required **. ――In fact, there are many things that cannot be acquired as data even in business **, so it can be applied to customer understanding and business structure understanding.

Path diagram

Once you know the difference between an observed variable and a latent variable, you also need to understand the path diagram. This is because this path diagram is a blueprint for a so-called hypothetical model (probably related to this).

pd.png

How to draw a path diagram

  1. Observed variables are ** rectangles **
  2. Latent variables are ** circle / ellipse **
  3. Causality is ** unidirectional arrow (many practices / straight lines) **
  4. Covariance (correlation) is ** bidirectional arrow (many dotted lines / curves) **

First, let's write the variables. I think that I will illustrate from the data that can be acquired, so I will write from the observed variables. Consider what invisible variables these observed variables are derived from. Then draw a unidirectional arrow for the derived variable. In the above path diagram, y1, y2, y3, y4 are considered to be the observed variables derived from the dem60 behind them, so the arrows extend from the latent variables to the observed variables. Also, due to the complexity of the figure, we are not plotting the covariance arrows at this time.

Why is it demopy?

With this basic knowledge, anyone can play structural equation modeling. Actually, before playing with semipy, why ** SEM ** in Python at this time in the first place?

--Python, which has a rich library of machine learning models, had the weakness of not being able to execute SEM. --The Institute of Statistical Mathematical Models includes ** R **, which is a strong programming language for statistical analysis. ――So, if you want to execute SEM using R or SPSS and really want to do it in Python, the mainstream method is to write it in full scratch or to connect it to Python by force using ** PypeR **. ――However, ** semipy has become version 2.0, which makes it more practical. ** ** ――From now on, as the number of users of this library increases, the number of methods that can be used will increase.

So, let's execute structural equation modeling with one Python using semopy.

Perform structural equation modeling using semopy

I'd really like to prepare an original dataset and play with it, but I don't have time, so I'll use the sample semopy: Structural Equation Modeling in Python to summarize the procedure.

0. Library installation

Installation with pip command

pip install semopy

Install with git command

git clone https://gitlab.com/georgy.m/semopy

1. Import semopy

--When setting the abbreviation as as sem, rewrite the part written as semopy as sem.

#Import semopy
import semopy as sem
#Also import the Model that instantiates the hypothetical model
from semopy import Model

--I was able to run it correctly without running from semopy import Model

2. Prepare the dataset

This time we will use the sample dataset, so get the sample data with the code below.

#Import sample data from the semopy library
from semopy.examples import political_democracy

#Get sample data and assign it to data
data = political_democracy.get_data()

data

スクリーンショット 2020-12-13 15.08.12.png

--If you see a result like this, it's OK.

Data type conversion

--semopy cannot be executed correctly unless the data type is float64 --If the data types are different, it is better to perform type conversion. --Sample data does not require type conversion, so proceed as it is

#Data type confirmation
data.dtypes

#Execution result
y1    float64
y2    float64
y3    float64
y4    float64
y5    float64
y6    float64
y7    float64
y8    float64
x1    float64
x2    float64
x3    float64
dtype: object

3. Set the model

Check the settings of the hypothetical model

--In the sample data, the hypothetical model has already been described. --Get the hypothetical model settings with get_mode () --Assign to the variable desc and try print ()

desc = political_democracy.get_model()
print(desc)

#Execution result
# measurement model
ind60 =~ x1 + x2 + x3
dem60 =~ y1 + y2 + y3 + y4
dem65 =~ y5 + y6 + y7 + y8
# regressions
dem60 ~ ind60
dem65 ~ ind60 + dem60
# residual correlations
y1 ~~ y5
y2 ~~ y4 + y6
y3 ~~ y7
y4 ~~ y8
y6 ~~ y8

--All of these are described in str type (character string) --Almost the same as the description method in R (the number of ~ in the regression model is different, so some correction is required)

Set up a hypothetical model

#Assign the hypothetical model to the variable desc
desc = '''
    # measurement model
    ind60 =~ x1 + x2 + x3
    dem60 =~ y1 + y2 + y3 + y4
    dem65 =~ y5 + y6 + y7 + y8
    # regressions
    dem60 ~ ind60
    dem65 ~ ind60 + dem60
    # residual correlations
    y1 ~~ y5
    y2 ~~ y4 + y6
    y3 ~~ y7
    y4 ~~ y8
    y6 ~~ y8
    '''

--Writing in str type is a simple story that you can enclose it in ''' when assigning it to a variable.

Description method

――In order to learn how to write, you need to have a deep understanding of datasets. --Political_democracy is Political Democracy: Industrialization And Political Democracy Dataset in lavaan: Latent Variable Analysis.

Overview of the dataset to use

|Physical name|Logical name | Japanese translation|Mold|Number of defects| |:--|:--|:--|:--| |y1|1960 Expert Evaluation of Freedom of the Press|float64|0| |y2|Freedom of political opposition in 1960|float64|0| |y3|Fairness of the 1960 elections|float64|0| |y4|Effectiveness of the legislature elected in 1960|float64|0| |y5|1965 Expert Evaluation of Freedom of the Press|float64|0| |y6|Freedom of political opposition in 1965|float64|0| |y7|Fairness of the 1965 election|float64|0| |y8|Effectiveness of the legislature elected in 1965|float64|0| |x1|Gross National Product (GNP) per capita in 1960|float64|0| |x2|Inanimate energy consumption per capita in 1960|float64|0| |x3|Percentage of labor force in industry in 1960|float64|0|

Structural equations | Latent variables = ~ Observed variables

――The latent variable of industry (Industry1960, ind60) in 1960 leads to the observed variables of GNP, energy consumption and the ratio of labor force. ――The latent variable of democracy in 1960 (Democracy1960, demo60) leads to the observation variables of expert evaluation of the press, freedom to oppose politics, election fairness, and government effectiveness. --The latent variable of democracy (Democracy1965, demo65) in 1966 leads to the observation variables of expert evaluation of the press, freedom to oppose politics, election fairness, and government effectiveness.

    # measurement model
    ind60 =~ x1 + x2 + x3
    dem60 =~ y1 + y2 + y3 + y4
    dem65 =~ y5 + y6 + y7 + y8

Measurement equation | Variable (result) ~ Variable (explanation)

--Industry in 1960 (Industry1960, ind60) and democracy in 1960 (Democracy1960, demo60) are at the same time, so there may be a causal relationship. --The 1965 democracy (Democracy1965, demo65) can be explained by the historical data of the 1960 industry (Industry1960, ind60) and the 1960 democracy (Democracy1960, demo60).

    # regressions
    dem60 ~ ind60
    dem65 ~ ind60 + dem60

Residual correlation | Variables ~~ Variables

--The 1960 press freedom (y1) will correlate with the 1965 press freedom (y5). --The freedom to oppose politics in 1960 (y2) can be explained by the fairness of elections at the same time (y4) and the freedom to oppose politics in 1965 (y6). etc…

# residual correlations
y1 ~~ y5
y2 ~~ y4 + y6
y3 ~~ y7
y4 ~~ y8
y6 ~~ y8

That's why I made a hypothetical model.

4. Perform learning

semopy has various modules as below

from see import see
see(semopy)

#Execution result
 ismodule             help()               .Model()
 .ModelEffects()      .ModelMeans()        .calc_stats()
 .constraints         .create_regularization()                  .efa
 .estimate_means()    .examples            .gather_statistics()
 .inspector           .means               .model               .model_base
 .model_effects       .model_means         .multigroup          .name
 .optimizer           .parser              .plot                .polycorr
 .regularization      .semplot()           .solver
 .startingvalues      .stats               .utils

However, running SEM is easy because it's almost the same process as building a machine learning model.

model () | Prepare the model to use

--Substitute a hypothetical model (desc in this case) as an argument to Model () and assign it to mod (abbreviation of model).

#Prepare a learning device
mod = Model(desc)

fit () | Perform learning

--The data used for learning (or analysis) is used as an argument (data in this case), and the result is assigned to res (abbreviation of result).

#Substitute the learning result for res
res = mod.fit(data)

#Display learning results
print(res)

#Expected execution result
Name of objective: MLW
Optimization method: SLSQP
Optimization successful.
Optimization terminated successfully.
Objective value: 0.508
Number of iterations: 52
Params: 2.180 1.819 1.257 1.058 1.265 1.186 1.280 1.266 1.482 0.572 0.838 0.624 1.893 1.320 2.156 7.385 0.793 5.067 0.347 3.148 1.357 4.954 3.430 3.951 0.172 0.120 0.082 2.352 3.256 0.467 0.448

I was able to execute it correctly

5. Evaluate the learning results

insepect () | Check parameter estimates

#Display a list of learning result parameters
inspect = mod.inspect()
print(inspect)

#Expected execution result
     lval  op   rval  Estimate   Std. Err   z-value      p-value
0   dem60   ~  ind60  1.482379   0.399024   3.71502   0.00020319
1   dem65   ~  ind60  0.571912   0.221383   2.58336   0.00978421
2   dem65   ~  dem60  0.837574  0.0984456   8.50799            0
3      x1   ~  ind60  1.000000          -         -            -
4      x2   ~  ind60  2.180494   0.138565   15.7363            0
5      x3   ~  ind60  1.818546   0.151993   11.9646            0
6      y1   ~  dem60  1.000000          -         -            -
7      y2   ~  dem60  1.256819   0.182687   6.87965  6.00009e-12
8      y3   ~  dem60  1.058174   0.151521    6.9837  2.87503e-12
9      y4   ~  dem60  1.265186   0.145151   8.71634            0
10     y5   ~  dem65  1.000000          -         -            -
11     y6   ~  dem65  1.185743   0.168908   7.02003  2.21823e-12
12     y7   ~  dem65  1.279717   0.159996   7.99841  1.33227e-15
13     y8   ~  dem65  1.266084   0.158238   8.00114  1.33227e-15
14  dem60  ~~  dem60  3.950849   0.920451    4.2923  1.76835e-05
15  dem65  ~~  dem65  0.172210   0.214861  0.801494     0.422846
16  ind60  ~~  ind60  0.448321  0.0866766   5.17234  2.31175e-07
17     y1  ~~     y5  0.624423   0.358435   1.74208    0.0814939
18     y1  ~~     y1  1.892743    0.44456   4.25756  2.06666e-05
19     y2  ~~     y4  1.319589    0.70268   1.87794    0.0603898
20     y2  ~~     y6  2.156164   0.734155   2.93693   0.00331475
21     y2  ~~     y2  7.385292    1.37567    5.3685  7.93938e-08
22     y3  ~~     y7  0.793329   0.607642   1.30558     0.191694
23     y3  ~~     y3  5.066628   0.951722   5.32365  1.01708e-07
24     y4  ~~     y8  0.347222   0.442234  0.785154     0.432363
25     y4  ~~     y4  3.147911   0.738841    4.2606  2.03874e-05
26     y6  ~~     y8  1.357037     0.5685   2.38705    0.0169843
27     y6  ~~     y6  4.954364   0.914284   5.41884   5.9986e-08
28     y7  ~~     y7  3.430032   0.712732   4.81251  1.49045e-06
29     x2  ~~     x2  0.119894  0.0697474   1.71897    0.0856192
30     x1  ~~     x1  0.081573  0.0194949   4.18432  2.86025e-05
31     y5  ~~     y5  2.351910   0.480369   4.89604  9.77851e-07
32     y8  ~~     y8  3.256389    0.69504   4.68518  2.79711e-06
33     x3  ~~     x3  0.466732  0.0901676   5.17628  2.26359e-07

calc_stats () | Display goodness of fit

#Show goodness of fit
stats = semopy.calc_stats(mod)
print(stats)

#Expected execution result
DoF  DoF Baseline       chi2  chi2 p-value  chi2 Baseline       CFI  \
Value   35            55  38.125446      0.329171     730.654577  0.995374   
           GFI      AGFI      NFI       TLI     RMSEA        AIC         BIC  \
Value  0.94782  0.918003  0.94782  0.992731  0.034738  60.983321  132.825453   
         LogLik  
Value  0.508339  

#It's hard to see, so transpose
print(stats.T)
                    Value
DoF             35.000000
DoF Baseline    55.000000
chi2            38.125446
chi2 p-value     0.329171
chi2 Baseline  730.654577
CFI              0.995374
GFI              0.947820
AGFI             0.918003
NFI              0.947820
TLI              0.992731
RMSEA            0.034738
AIC             60.983321
BIC            132.825453
LogLik           0.508339

--Since it is sample data, interpretation is omitted.

6. Output the path diagram

--For simple bar charts and scatter charts, Matplotlib is fine, but path charts are graphs consisting of nodes and edges. --Therefore, import Graphviz and output the path diagram.

Install Graphviz

Example of pip command

$ pip install graphviz

semplot () | Output path diagram

--semplot specifies the model to be used in the first argument (mod this time), and specifies the file name to be output in the second argument.

g = semopy.semplot(m, "sample.png ")

g

dot_no_bunsan.png

Also show covariance

--If you take plot_covs = True as an argument, the covariance arrow will also be plotted as a dotted line.

g = semopy.semplot(m, "sample.png ", plot_covs=True)

dot.png

Change the layout

--If you take engine =" circleo " as an argument, the illustration method will change to a circle type (circle type). --The default is a hierarchical type of engine =" dot "

g = semopy.semplot(mod, "sample.png ", plot_covs=True, engine="circo")

circo.png

Layout type

-Summary of how to draw a graph in Graphviz and dot language --Qiita

layout description
circo Circular graph
dot Hierarchical graph.Suitable for directed graphs.Default layout engine
fdp spring(Spring)Model graph.Suitable for undirected graphs.
neato spring(Spring)Model graph.Suitable for undirected graphs.
osage Array type graph.
sfdp Multi-scale version of fdp.Suitable for large undirected graphs.
twopi Radial graph.Nodes are arranged concentrically.

Impressions

――I thought this would be the first one, and when I played with semopy, there was already an article Structural Equation Modeling (SEM) in Python ~ Trying out the tutorial of semopy ~ ―― Qiita, which I had practiced earlier. There was a person ――The above article was also very helpful. ――For the path diagram, the point that the error variable is not output has not been solved yet, and the drawing function is stronger in R before. --The above may have a solution if you look at the semipy documentation. --Let's try structural equation modeling (SEM) in Python! I hope it will be helpful for those who say.

end

References

Recommended Posts

Introduction to Structural Equation Modeling (SEM), Covariance Structure Analysis with Python
Reading Note: An Introduction to Data Analysis with Python
20200329_Introduction to Data Analysis with Python Second Edition Personal Summary
Introduction to image analysis opencv python
Introduction to Data Analysis with Python P32-P43 [ch02 3.US Baby Names 1880-2010]
Introduction to Data Analysis with Python P17-P26 [ch02 1.usa.gov data from bit.ly]
Introduction to Bayesian Statistical Modeling with python ~ Trying Linear Regression with MCMC ~
From the introduction of JUMAN ++ to morphological analysis of Japanese with Python
Introduction to Python Image Inflating Image inflating with ImageDataGenerator
[Introduction to Python] Let's use foreach with Python
[Python] Introduction to CNN with Pytorch MNIST
"Introduction to data analysis by Bayesian statistical modeling starting with R and Stan" implemented in Python
I tried fMRI data analysis with python (Introduction to brain information decoding)
[Python] Easy introduction to machine learning with python (SVM)
Introduction to Artificial Intelligence with Python 1 "Genetic Algorithm-Theory-"
Markov Chain Chatbot with Python + Janome (1) Introduction to Janome
Markov Chain Chatbot with Python + Janome (2) Introduction to Markov Chain
Introduction to Artificial Intelligence with Python 2 "Genetic Algorithm-Practice-"
Introduction to Tornado (1): Python web framework started with Tornado
An introduction to statistical modeling for data analysis
Introduction to formation flight with Tello edu (Python)
[Introduction to minimize] Data analysis with SEIR model ♬
Introduction to Python with Atom (on the way)
Introduction to Generalized Linear Models (GLM) with Python
[Introduction to Udemy Python3 + Application] 9. First, print with print
I want to output a path diagram of distributed covariance structure analysis (SEM) by linking Python and R.
Data analysis with python 2
[Introduction to Python3 Day 8] Chapter 4 Py Skin: Code Structure (4.1-4.13)
[Introduction to Python] How to iterate with the range function?
[Chapter 5] Introduction to Python with 100 knocks of language processing
Voice analysis with python
An introduction to Python distributed parallel processing with Ray
Introduction to Mathematics Starting with Python Study Memo Vol.1
Introduction to Python language
Introduction to OpenCV (python)-(2)
[Chapter 3] Introduction to Python with 100 knocks of language processing
[Chapter 2] Introduction to Python with 100 knocks of language processing
Voice analysis with python
Data analysis with Python
[Technical book] Introduction to data analysis using Python -1 Chapter Introduction-
[Chapter 4] Introduction to Python with 100 knocks of language processing
An introduction to statistical modeling for data analysis (Midorimoto) reading notes (in Python and Stan)
Introduction to her made with Python ~ Tinder automation project ~ Episode 6
Introduction to her made with Python ~ Tinder automation project ~ Episode 5
Introduction to Python for VBA users-Calling Python from Excel with xlwings-
[Raspi4; Introduction to Sound] Stable recording of sound input with python ♪
Introduction to Statistical Modeling for Data Analysis GLM Model Selection
[Introduction to Python] How to get data with the listdir function
[Introduction to Udemy Python3 + Application] 51. Be careful with default arguments
Connect to BigQuery with Python
[Python] Morphological analysis with MeCab
Introduction to Python Django (2) Win
[Co-occurrence analysis] Easy co-occurrence analysis with Python! [Python]
Connect to Wikipedia with Python
Post to slack with Python 3
Sentiment analysis with Python (word2vec)
Introduction to RDB with sqlalchemy Ⅰ
Introduction to serial communication [Python]
Planar skeleton analysis with Python
Japanese morphological analysis with Python
Switch python to 2.7 with alternatives