Conclusion

unsolved

background

In the Dockerfile

From jupyter/jupyter/pyspark-notebook:~~~~~~~~~

Loading Pyspark as.

environment

Python 3.7.6
pyspark 2.4.5

Corresponding part

from pyspark.sql import SparkSession

error contents

/usr/local/spark/python/pyspark/__init__.py in <module>
     49 
     50 from pyspark.conf import SparkConf
---> 51 from pyspark.context import SparkContext
     52 from pyspark.rdd import RDD, RDDBarrier
     53 from pyspark.files import SparkFiles

/usr/local/spark/python/pyspark/context.py in <module>
     27 from tempfile import NamedTemporaryFile
     28 
---> 29 from py4j.protocol import Py4JError
     30 
     31 from pyspark import accumulators

ModuleNotFoundError: No module named 'py4j'

How to respond

This time on the jupyter notebook as it will cause problems

!pip install py4j

Corresponded with The following error will appear, so I would like to add it as soon as I understand how to deal with it without an error.

ERROR: pyspark 2.4.5 has requirement py4j==0.10.7, but you'll have py4j 0.10.9.1 which is incompatible.

reference

Recommended Posts

ModuleNotFoundError: No module named'py4j'

ModuleNotFoundError: No module named'flask_sqlalchemy'

[Python] ModuleNotFoundError: No module named'urlparse'

Python error: ModuleNotFoundError: No module named'flask'

ModuleNotFoundError in poetry: No module named'setuptools'

Today's python error: ModuleNotFoundError: No module named

Microsoft Cognitive Toolkit: ModuleNotFoundError: No module named'cntk._cntk_py'

ModuleNotFoundError: No module named'***' when running Flask

Today's python error: ModuleNotFoundError: No module named'requests'

ModuleNotFoundError in Python: No module named story

Today's python error: ModuleNotFoundError: No module named'bs4'

ModuleNotFoundError: No module named'_bz2' error in pyenv Python

Import cv2 ModuleNotFoundError: No module named'cv2' in python3

ModuleNotFoundError: No module What to do if you get'tensorflow.contrib'

Resolve No module named tkinter

How to resolve ModuleNotFoundError: No module named XXX in Jupyter Notebook

'lambda_function': No module named When'selenium' appears

Why ModuleNotFoundError: No module named'dataset.mnist' appears in "Deep Learning from scratch".

What to do when ModuleNotFoundError: No module named'XXX' occurs in Python

[Question] No module named'Selenium' in PyCharm

Error when installing opencv-python on jetson nano (ModuleNotFoundError: No module named'skbuild')

When lambda-uploader says No module named xxxx.

Suddenly with Python PyInstaller No module named pyinstaller

Django / Apache / mod_wsgi: No module named importlib

Import Error in Python3: No module named'xxxxx'

No module named'_ctypes' occurs when installing pipenv

Attempting to install opencv-python3 on jetson nano but error (ModuleNotFoundError: No module named'skbuild')

[PYTHON] ModuleNotFoundError: No module named'py4j'