Subprocess

Python >= 3.5: Returncode, errors and standard output

If one uses Python >= 3.5 there is a new run command which also returns the standard output.

>>> import subprocess
>>> result = subprocess.run(["ls", "-l"], stdout=subprocess.PIPE)
>>> result.stdout.decode('utf-8')
`total 0\n----------- root  root  0 Jan 00:01 filename\n'

Returncode (Python < 3.5)

from subprocess import call
retcode = call(["matlab", "-nodesktop", "-no"])

Standard output (Python 2.7 to 3.4)

>>> output = subprocess.check_output(["ls", "-l"])
>>> output.stdout.decode('utf-8')

Mind that there is no stdout=subprocess.PIPE, but one only can get the standard output and not the error output or return code, like with run.

Thanks to this stackoverflow post and of cause the Official docu.

Capturing standard output of other's libraries

If you do not write the subprocess-calls yourself, but use a library that calls shell commands and you do not want to patch it, there is also a solution for you.

This solution involves the cStringIO library which is TODO not available for PyPy and dramatically changed from Python 2 to 3, since the string handling completely changed, so this solution is only valid for python 2.

import sys                                                                                                    

from cStringIO import StringIO                                                                                

class capture(list):                                                                                          
    def __enter__(self):                                                                                      
        self._stdout = sys.stdout                                                                             
        sys.stdout = self._stringio = StringIO()                                                              
        return self                                                                                           
    def __exit__(self, *args):                                                                                
        self.extend(self._stringio.getvalue().splitlines())                     
        del self._stringio                                                                                    
        sys.stdout = self._stdout                                                                             

import contextlib                                                                                             
# Older/alternative version using contextlib
# @contextlib.contextmanager                                                                                  
# def capture():                                                                                              
#     import sys                                                                                              
#     from cStringIO import StringIO                                                                          
#     print("capture")                                                                                        
#     oldout, olderr = sys.stdout, sys.stderr                                                                 
#     print("pre try")                                                                                        
#     try:                                                                                                    
#         out = [StringIO(), StringIO()]                                                                      
#         print("pre out in")                                                                                 
#         sys.stdout, sys.stderr = out                                                                        
#         print("pre yield")                                                                                  
#         yield out                                                                                           
#         print("post yield")                                                                                 
#     finally:                                                                                                
#         sys.stdout, sys.stderr = oudout, olderr                                                             
#         out[0] = out[0].getvalue()                                                                          
#         out[1] = out[1].getvalue()                                                                          
#         print("end finally")

If you then want to get the stdout of a library call you have got a context manager. In the following example I get the stdout produced by the pyepics library and if there is there is output an error occurred and I need to run the request again.

def getpv(pv, rerun=0):
    _LIMIT = 10
    with capture() as out:
        data = epics.caget(pv)
    if out and rerun < _LIMIT:
        print("waiting 1 s..., out={}".format(out))
        time.sleep(1)
        return getpv(pv)
    return data

The important part is the with capture() as out. See also my other blog post on that topic.