Pipettor Library

Function Interface

run(cmds, stdin=None, stdout=None, stderr=<class 'pipettor.devices.DataReader'>, logger=None, logLevel=None)

Construct and run a process pipeline.

Parameters:

• cmds – A list (or tuple) of arguments for a single process, or a list of such lists for a pipeline. Arguments are converted to strings.

• stdin – Input to the first process. Can be None (inherit), filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataWriter object.

• stdout – Output from the last process. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object.

• stderr – stderr for the pipeline. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object. It may also be the class pipettor.DataReader itself. See discussion below.

• logger – Name of the logger or a Logger instance. If None, the default pipettor logger is used.

• logLevel – Log level to use instead of the default.

runout(cmds, stdin=None, stderr=<class 'pipettor.devices.DataReader'>, binary=False, logger=None, logLevel=None, buffering=-1, encoding=None, errors=None, newline=None)

Construct and run a process pipeline, returning the output.

Parameters:

• cmds – A list (or tuple) of arguments for a single process, or a list of such lists for a pipeline. Arguments are converted to strings.

• stdin – Input to the first process. Can be None (inherit), filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataWriter object.

• stdout – Output from the last process. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object.

• stderr – stderr for the pipeline. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object. It may also be the class pipettor.DataReader itself. See discussion below.

• logger – Name of the logger or a Logger instance. If None, the default pipettor logger is used.

• logLevel – Log level to use instead of the default.

• buffering – controls buffering. See open() for more details.

• encoding – name of the encoding used to decode or encode the file See open() for more details.

• errors – how encoding errors are handled. See open() for more details.

• newline – controls how universal newlines works See open() for more details.

Raises:

pipettor.ProcessException – if the a process in pipeline exits with a non-zero status

Use str.splitlines() to split the result into lines.

They specification of stderr controls how errors are reported. If a pipettor.DataReader class is provided for stderr, a DataReader object will be created from it for each process to collect that processes stderr output. If the pipeline fails, the contents of stderr from the first process that failed will be included in the pipettor.ProcessException object. If an instance of pipettor.DataReader is provided, stderr from all processes is combined and that is returned in the failure. It is recommended that a DataReader object be created with errors="backslashreplace" to prevent invalid UTF-8 characters from generating confusing errors.

runlex(cmds, stdin=None, stdout=None, stderr=<class 'pipettor.devices.DataReader'>, logger=None, logLevel=None)

Call pipettor.run(), first splitting commands specified as strings are split into arguments using shlex.split.

See run() for details.

runlexout(cmds, stdin=None, stderr=<class 'pipettor.devices.DataReader'>, logger=None, logLevel=None, buffering=-1, encoding=None, errors=None)

Call pipettor.runout(), first splitting commands specified as strings are split into arguments using shlex.split.

See runout() for details.

Pipeline Classes

class Pipeline(cmds, *, stdin=None, stdout=None, stderr=<class 'pipettor.devices.DataReader'>, logger=None, logLevel=None)

Create and run a process pipeline. Create an instances setups a project to run. Once constructed, the pipeline is started with the start(), poll(), or wait() functions.

Parameters:

• cmds – A list (or tuple) of arguments for a single process, or a list of such lists for a pipeline. Arguments are converted to strings.

• stdin – Input to the first process. Can be None (inherit), filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataWriter object.

• stdout – Output from the last process. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object.

• stderr – stderr for the pipeline. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object. It may also be the class pipettor.DataReader itself. See discussion below.

• logger – Name of the logger or a Logger instance. If None, the default pipettor logger is used.

• logLevel – Log level to use instead of the default.

failed()

check if any process failed, call after poll() or wait()

property finished

determined if been detected as finished (waited on)

kill(sig=Signals.SIGTERM)

send a signal to all of the processes in the pipeline

poll()

Check if all of the processes have completed. Return True if it has, False if it hasn’t.

property running

determined if this process has been running

shutdown()

Close down the pipeline prematurely. If the pipeline is running, it’s killed. This does not report errors from child process and differs from wait in the fact that it doesn’t start the pipeline if it has not been started, just frees up open pipes. Primary intended for error recovery

start()

start processes

wait()

Wait for all of the process to complete. Generate an exception if any exits non-zero or signals. Starts process if not already running.

class Popen(cmds, mode='r', *, stdin=None, stdout=None, stderr=None, logger=None, logLevel=None, buffering=-1, encoding=None, errors=None, newline=None)

File-like object of processes to read from or write to a Pipeline. Only one of stdin or stdout maybe specified, with the other end being read or written via the Popen instance.

• Read pipeline (‘r’): stdin → cmd[0] → … → cmd[n] → Popen

• Write pipeline (‘w’): Popen → cmd[0] → … → cmd[n] → stdout

Parameters:

• mode – specifies the mode in which the file is opened. It defaults to ‘r’. See open() for more details.

• cmds – A list (or tuple) of arguments for a single process, or a list of such lists for a pipeline. Arguments are converted to strings.

• stdin – Input to the first process. Can be None (inherit), filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataWriter object.

• stdout – Output from the last process. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object.

• stderr – stderr for the pipeline. Can be None (inherit), a filename, file-like object, file descriptor, a pipettor.File object, or a pipettor.DataReader object. It may also be the class pipettor.DataReader itself. See discussion below.

• logger – Name of the logger or a Logger instance. If None, the default pipettor logger is used.

• logLevel – Log level to use instead of the default.

• buffering – controls buffering. See open() for more details.

• encoding – name of the encoding used to decode or encode the file See open() for more details.

• errors – how encoding errors are handled. See open() for more details.

• newline – controls how universal newlines works See open() for more details.

Raises:

pipettor.ProcessException – if the a process in pipeline exits with a non-zero status

They specification of stderr controls how errors are reported. If a pipettor.DataReader class is provided for stderr, a DataReader object will be created from it for each process to collect that processes stderr output. If the pipeline fails, the contents of stderr from the first process that failed will be included in the pipettor.ProcessException object. If an instance of pipettor.DataReader is provided, stderr from all processes is combined and that is returned in the failure. It is recommended that a DataReader object be created with errors="backslashreplace" to prevent invalid UTF-8 characters from generating confusing errors.

close()

wait for process to complete, with an error if it exited non-zero

property closed

bool. True if the file has been closed.

Type:

closed

failed()

check if any process failed, call after poll() or wait()

fileno()

get the integer OS-dependent file handle

property finished

determined if been detected as finished (waited on)

flush()

Flush the internal I/O buffer.

isatty()

Return a bool indicating whether this is an ‘interactive’ stream.

kill(sig=Signals.SIGTERM)

send a signal to all of the processes in the pipeline

poll()

poll is not allowed for Pipeline objects

read(size=-1)

readable()

Return a bool indicating whether object was opened for reading.

readline(size=-1)

readlines(size=-1)

property running

determined if this process has been running

seek(pos, whence=0)

Changing stream position not supported

seekable()

Not seekable

shutdown()

Close down the pipeline prematurely. If the pipeline is running, it’s killed. This does not report errors from child process and differs from wait in the fact that it doesn’t start the pipeline if it has not been started, just frees up open pipes. Primary intended for error recovery

start()

start processes

tell()

Return an int indicating the current stream position.

truncate(pos=None)

Truncate file unsupported

wait()

wait to for processes to complete, generate an exception if one exits no-zero

writable()

Return a bool indicating whether object was opened for writing.

write(str)

Write string str to file.

writelines(lines)

Write a list of lines to the stream.

Line separators are not added, so it is usual for each of the lines provided to have a line separator at the end.

Process I/O Classes

class DataReader(*, binary=False, buffering=-1, encoding=None, errors=None, newline=None)

Object to asynchronously read data from process into memory via a pipe. A thread is use to prevent deadlock when both reading and writing to a child pipeline.

Specifying binary access results in data of type bytes, otherwise str type is returned. The buffering, encoding, and errors arguments are as used in the open() function.

A reader maybe read from multiple process.

close()

close pipes and terminate thread

property data

return buffered data as a string or bytes

class DataWriter(data, *, buffering=-1, encoding=None, errors=None, newline=None)

Object to asynchronously write data to process from memory via a pipe. A thread is use to prevent deadlock when both reading and writing to a child pipeline. Text or binary output is determined by the type of data.

The buffering, encoding, and errors arguments are as used in the open() function.

close()

close pipes and terminate thread

class File(path, mode='r')

A file path for input or output, used for specifying stdio associated with files. Mode is one of r, w, or a

close()

close file if open

Logging Control

LOGGER_NAME = 'pipettor'

Name of default logger for pipettor

setDefaultLogging(logger, level)

Set both default logger and level. Either can be None to leave as default

setDefaultLogger(logger)

Set the default pipettor logger used in logging command and errors. If None, removes default logging. The logger can be the name of a logger or the logger Default logger is pipettor

getDefaultLogger()

return the current value of the pipettor default logger

setDefaultLogLevel(level)

Set the default pipettor log level to use in logging command and errors. Standard value is logging.DEBUG

getDefaultLogLevel()

Get the default pipettor log level to use in logging command and errors.

Exceptions

class PipettorException

Base class for Pipettor exceptions.

class ProcessException(procDesc, returncode=None, stderr=None)

Exception associated with running a process. A None returncode indicates a exec failure.