[PYTHON] An error occurs when trying to import scikit-learn after connecting to Oracle with SQLAlchemy

phenomenon

If you import sklearn after actually connecting the DB from the ʻEngine instance of SQLAlchemy as shown below, ʻImport Error will occur.

`demo.Case of py / ImportError`


import sqlalchemy
import cx_Oracle

dsnStr = cx_Oracle.makedsn('localhost','1521','pdb-name')
connect_str = 'oracle://user:pass@' + dsnStr.replace('SID', 'SERVICE_NAME')
engine = sqlalchemy.create_engine(connect_str, max_identifier_length=128)

conn = engine.connect()

import sklearn # <-Here, "Import Error:DLL load failed "occurs

Full stack trace

Traceback (most recent call last):
  File "demo.py", line 12, in <module>
    import sklearn
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\sklearn\__init__.py", line 80, in <module>    
    from .base import clone
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\sklearn\base.py", line 21, in <module>
    from .utils import _IS_32BIT
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\sklearn\utils\__init__.py", line 23, in <module>
    from .class_weight import compute_class_weight, compute_sample_weight
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\sklearn\utils\class_weight.py", line 7, in <module>
    from .validation import _deprecate_positional_args
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\sklearn\utils\validation.py", line 25, in <module>
    from .fixes import _object_dtype_isnan, parse_version
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\sklearn\utils\fixes.py", line 18, in <module> 
    import scipy.stats
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\stats\__init__.py", line 388, in <module>
    from .stats import *
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\stats\stats.py", line 180, in <module>  
    from . import distributions
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\stats\distributions.py", line 8, in <module>
    from ._distn_infrastructure import (entropy, rv_discrete, rv_continuous,
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\stats\_distn_infrastructure.py", line 23, in <module>
    from scipy import optimize
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\optimize\__init__.py", line 387, in <module>
    from .optimize import *
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\optimize\optimize.py", line 36, in <module>
    from ._numdiff import approx_derivative
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\optimize\_numdiff.py", line 6, in <module>
    from scipy.sparse.linalg import LinearOperator
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\sparse\linalg\__init__.py", line 114, in <module>
    from .eigen import *
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\sparse\linalg\eigen\__init__.py", line 9, in <module>
    from .arpack import *
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\sparse\linalg\eigen\arpack\__init__.py", line 20, in <module>
    from .arpack import *
  File "C:\Users\kurukuruz\anaconda3\envs\sqla-demo\lib\site-packages\scipy\sparse\linalg\eigen\arpack\arpack.py", line 43, in <module>
    from . import _arpack
ImportError: DLL load failed while importing _arpack:The specified procedure cannot be found.

Cases actually encountered

I don't think you will write ʻimport in the middle of processing as described above in the .py` file, but you may encounter an error in the following cases.

--When importing additionally after working with Jupyter Notebook to some extent --When deserializing an object with joblib.load (...) etc: arrow_left: This is what I encountered

Countermeasures

As symptomatic treatment, it may be declared in advance in the code.

`Example of importing in advance`


import sqlalchemy
import cx_Oracle
import joblib
import sklearn # <-Not used explicitly in this file, but declared

dsnStr = cx_Oracle.makedsn('localhost','1521','pdb-name')
connect_str = 'oracle://user:pass@' + dsnStr.replace('SID', 'SERVICE_NAME')
engine = sqlalchemy.create_engine(connect_str, max_identifier_length=128)

conn = engine.connect()

joblib.load('path/to/file.dump') # <-Implicitly used sklearn here