I want to start a lot of processes from python

It ’s the process that starts os.system(cmd) But subprocess.check_output(cmd) But that's fine.

If you throw a lot of jobs to another server with ssh, Write failed: Broken pipe And fork: Resource temporarily unavailable And shell request failed on channel 0 I was in trouble because I got an error like that.

You can increase the upper limit of the number of processes with ulimit, Increase the time to wait for a server response in .ssh / config, During various trials and errors.

I will update it when it is resolved.

http://d.hatena.ne.jp/usata3/20061110/p2 http://blog.ybbo.net/2013/06/19/mac%E3%81%AEssh%E3%81%A7%E3%80%8Cwrite-failed-broken-pipe%E3%80%8D%E3%81%8C%E5%87%BA%E3%81%9F%E5%A0%B4%E5%90%88%E3%81%AE%E5%AF%BE%E5%87%A6%E6%B3%95/

Recommended Posts

I want to start a lot of processes from python
NikuGan ~ I want to see a lot of delicious meat! !!
I want to send a message from Python to LINE Bot
I want to use jar from python
I want to build a Python environment
I want to email from Gmail using Python.
I want to use ceres solver from python
[Python] I want to manage 7DaysToDie from Discord! 2/3
I want to make C ++ code from Python code!
I want to write to a file with Python
[Python memo] I want to get a 2-digit hexadecimal number from a decimal number
I want to color a part of an Excel string in Python
Python: I want to measure the processing time of a function neatly
I want to embed a variable in a Python string
I want to easily implement a timeout in python
I want to detect images of cats from Instagram
I want to iterate a Python generator many times
I want to generate a UUID quickly (memorandum) ~ Python ~
I want to write in Python! (2) Let's write a test
I want to randomly sample a file in Python
I want to work with a robot in python.
I want to install a package of Php Redis
[Python3] I want to generate harassment names from Japanese!
[Python] I want to make a nested list a tuple
Python + selenium to GW a lot of e-mail addresses
I want to send a business start email automatically
I want to run a quantum computer with Python
I want to get / execute variables / functions / classes of external files from Python
I want to specify another version of Python with pyvenv
[Python] I made a system to introduce "recipes I really want" from the recipe site!
I want to start a jupyter environment with one command
[Python] I want to get a common set between numpy
I want to tell people who want to import from a higher directory with Python direnv
I want to collect a lot of images, so I tried using "google image download"
I want to install a package from requirements.txt with poetry
I want to know the features of Python and pip
I want to debug with Python
I want to make input () a nice complement in python
[Python] I want to make a 3D scatter plot of the epicenter with Cartopy + Matplotlib!
I want to use a wildcard that I want to shell with Python remove
I tried to make a regular expression of "amount" using Python
I tried to make a regular expression of "time" using Python
I want to output the beginning of the next month with Python
I tried to create a list of prime numbers with python
I made a lot of files for RDP connection with Python
I tried to make a regular expression of "date" using Python
I want to do a full text search with elasticsearch + python
I want to use both key and value of Python iterator
[Introduction] I want to make a Mastodon Bot with Python! 【Beginners】
I want to make a parameter list from CloudFormation code (yaml)
I did a lot of research on how Python is executed
I want to print in a comprehension
Send a message from Python to Slack
Connect a lot of Python or and and
I want to analyze logs with Python
I want to play with aws with python
I want to extract an arbitrary URL from the character string of the html source with python
I want to clear up the question of the "__init__" method and the "self" argument of a Python class.
A story I was addicted to when inserting from Python to a PostgreSQL table
(Python Selenium) I want to check the settings of the download destination of WebDriver
I want to convert a table converted to PDF in Python back to CSV