U lu f1@sdZddlZddlZddlZddlZddlZddlZddlZddlm Z ddl m Z m Z m Z ddlmZddlmZddlmZddlZddlmZmZmZejrdd lmZejZed d d Zdd d dZdadeeeeedddZeed ddZ Gddde!Z"dS)zUtilities for working with multiple processes, including both forking the server into multiple processes and managing subprocesses. N)hexlify)Future"future_set_result_unless_cancelled%future_set_exception_unless_cancelled)ioloop) PipeIOStream)gen_log)OptionalAnyCallable)Listreturnc Cs`tdkr dSz tWStk r*YnXz tdWSttfk rPYnXtddS)z1Returns the number of processors on this machine.NSC_NPROCESSORS_CONFz1Could not detect number of processors; assuming 1) multiprocessing cpu_countNotImplementedErrorossysconfAttributeError ValueErrorrerrorrr3/tmp/pip-unpacked-wheel-bmg6zs32/tornado/process.pyr1s   rcCsjdtjkrdSddl}ztttdd}Wn,tk rZttdt A}YnX| |dS)Nrandomri) sysmodulesrintrrurandomrtimegetpidseed)rr#rrr_reseed_randomAs r$) num_processes max_restartsrc sPtjdkrtd|dkrd}tdks*t|dks:|dkr@t}td|itt tdfdd }t |D]}||}|dk rp|Sqpd}rBt \}}|krq |}t |rtd ||t |n8t |dkrtd ||t |ntd ||q|d 7}||kr,td||}|dk r|SqtddS)aqStarts multiple worker processes. If ``num_processes`` is None or <= 0, we detect the number of cores available on this machine and fork that number of child processes. If ``num_processes`` is given and > 0, we fork that specific number of sub-processes. Since we use processes and not threads, there is no shared memory between any server code. Note that multiple processes are not compatible with the autoreload module (or the ``autoreload=True`` option to `tornado.web.Application` which defaults to True when ``debug=True``). When using multiple processes, no IOLoops can be created or referenced until after the call to ``fork_processes``. In each child process, ``fork_processes`` returns its *task id*, a number between 0 and ``num_processes``. Processes that exit abnormally (due to a signal or non-zero exit status) are restarted with the same id (up to ``max_restarts`` times). In the parent process, ``fork_processes`` calls ``sys.exit(0)`` after all child processes have exited normally. max_restarts defaults to 100. Availability: Unix win32zfork not available on windowsNdrzStarting %d processes)ircs.t}|dkrt|a|S||<dSdSNr)rforkr$_task_id)r)pidchildrenrr start_childsz#fork_processes..start_childz1child %d (pid %d) killed by signal %d, restartingz3child %d (pid %d) exited with status %d, restartingz!child %d (pid %d) exited normallyrz"Too many child restarts, giving up)rplatform Exceptionr,AssertionErrorrrinforr rangerwaitpop WIFSIGNALEDwarningWTERMSIG WEXITSTATUS RuntimeErrorexit) r%r&r0r)idZ num_restartsr-statusZnew_idrr.rfork_processesSsV         r@cCstS)zpReturns the current task id, if any. Returns None if this process was not created by `fork_processes`. )r,rrrrtask_idsrAc@seZdZdZeZdZiZeeddddZ e e gdfdddd Z de d d d dZeddddZeddddZeddddZee ddddZe ddddZdS) Subprocessa Wraps ``subprocess.Popen`` with IOStream support. The constructor is the same as ``subprocess.Popen`` with the following additions: * ``stdin``, ``stdout``, and ``stderr`` may have the value ``tornado.process.Subprocess.STREAM``, which will make the corresponding attribute of the resulting Subprocess a `.PipeIOStream`. If this option is used, the caller is responsible for closing the streams when done with them. The ``Subprocess.STREAM`` option and the ``set_exit_callback`` and ``wait_for_exit`` methods do not work on Windows. There is therefore no reason to use this class instead of ``subprocess.Popen`` on that platform. .. versionchanged:: 5.0 The ``io_loop`` argument (deprecated since version 4.1) has been removed. FN)argskwargsrc Osztj|_g}g}|dtjkrZt\}}||d<| ||f| |t ||_ |dtjkrt\}}||d<| ||f| |t ||_ |dtjkrt\} } | |d<| | | f| | t | |_ztj|||_Wn$|D]} t| qYnX|D]} t| q"|jj|_dD]&} t|| sBt|| t|j| qBd|_d|_dS)Nstdinstdoutstderr)rErFrG)rZIOLoopcurrentio_loopgetrBSTREAMrpipeextendappendrrErFrG subprocessPopenproccloser-hasattrsetattrgetattr_exit_callback returncode) selfrCrDZpipe_fdsto_closeZin_rZin_wZout_rZout_wZerr_rZerr_wfdattrrrr__init__sF            zSubprocess.__init__)callbackrcCs*||_t|tj|j<t|jdS)aRuns ``callback`` when this process exits. The callback takes one argument, the return code of the process. This method uses a ``SIGCHLD`` handler, which is a global setting and may conflict if you have other libraries trying to handle the same signal. If you are using more than one ``IOLoop`` it may be necessary to call `Subprocess.initialize` first to designate one ``IOLoop`` to run the signal handlers. In many cases a close callback on the stdout or stderr streams can be used as an alternative to an exit callback if the signal handler is causing a problem. Availability: Unix N)rVrB initialize_waitingr-_try_cleanup_process)rXr]rrrset_exit_callbacks zSubprocess.set_exit_callbackTz Future[int]) raise_errorrcs*ttddfdd }||S)a2Returns a `.Future` which resolves when the process exits. Usage:: ret = yield proc.wait_for_exit() This is a coroutine-friendly alternative to `set_exit_callback` (and a replacement for the blocking `subprocess.Popen.wait`). By default, raises `subprocess.CalledProcessError` if the process has a non-zero exit status. Use ``wait_for_exit(raise_error=False)`` to suppress this behavior and return the exit status without raising. .. versionadded:: 4.2 Availability: Unix N)retrcs,|dkrrtt|dn t|dS)Nrunknown)rCalledProcessErrorr)rcfuturerbrrr](s  z*Subprocess.wait_for_exit..callback)rrra)rXrbr]rrfr wait_for_exits zSubprocess.wait_for_exitr cCs,|jr dSt}|tj|jd|_dS)aInitializes the ``SIGCHLD`` handler. The signal handler is run on an `.IOLoop` to avoid locking issues. Note that the `.IOLoop` used for signal handling need not be the same one used by individual Subprocess objects (as long as the ``IOLoops`` are each running in separate threads). .. versionchanged:: 5.0 The ``io_loop`` argument (deprecated since version 4.1) has been removed. Availability: Unix NT) _initializedasyncioget_event_loopZadd_signal_handlersignalSIGCHLD_cleanupclsZlooprrrr^4s zSubprocess.initializecCs(|js dSt}|tjd|_dS)z Removes the ``SIGCHLD`` handler.NF)rirjrkZremove_signal_handlerrlrmrorrr uninitializeIs  zSubprocess.uninitializecCs"t|jD]}||qdS)N)listr_keysr`)rpr-rrrrnRszSubprocess._cleanup)r-rcCsfzt|tj\}}Wntk r,YdSX|dkr:dS||ksFt|j|}|j|j |dSr*) rwaitpidWNOHANGChildProcessErrorr3r_r7rIZ add_callback_set_returncode)rpr-Zret_pidr?Zsubprocrrrr`Ws  zSubprocess._try_cleanup_process)r?rcCsptjdkrd|_n4t|r,t| |_nt|s:tt||_|j|j _|j rl|j }d|_ ||jdS)Nr') rr1rWrr8r: WIFEXITEDr3r;rQrV)rXr?r]rrrrwcs    zSubprocess._set_returncode)T)__name__ __module__ __qualname____doc__objectrKrir_r r\r rraboolrh classmethodr^rqrnr`rwrrrrrBs '  rB)N)#r}rjrrrlrOrr!binasciirZtornado.concurrentrrrZtornadorZtornado.iostreamrZ tornado.logrtypingr r r TYPE_CHECKINGr rerrr$r,r@rAr~rBrrrrs8      `