pyworkflow.protocol.executor module

This module have the classes for execution of protocol steps. The basic one will run steps, one by one, after completion. There is one based on threads to execute steps in parallel using different threads and the last one with MPI processes.

class pyworkflow.protocol.executor.MPIStepExecutor(hostConfig, nMPI, comm, **kwargs)[source]

Bases: pyworkflow.protocol.executor.ThreadStepExecutor

Run steps in parallel using threads. But call runJob through MPI workers.

runJob(log, programName, params, numberOfMpi=1, numberOfThreads=1, env=None, cwd=None)[source]

This function is a wrapper around runJob, providing the host configuration.

runSteps(steps, stepStartedCallback, stepFinishedCallback, checkStepsCallback, stepsCheckSecs=3)[source]

Create threads and synchronize the steps execution. n: the number of threads. stepsCheckSecs:

rate of how many seconds between stepsCheckCallback calls
class pyworkflow.protocol.executor.QueueStepExecutor(hostConfig, submitDict, nThreads, **kwargs)[source]

Bases: pyworkflow.protocol.executor.ThreadStepExecutor

runJob(log, programName, params, numberOfMpi=1, numberOfThreads=1, env=None, cwd=None)[source]

This function is a wrapper around runJob, providing the host configuration.

class pyworkflow.protocol.executor.StepExecutor(hostConfig, **kwargs)[source]

Bases: object

Run a list of Protocol steps.

getGpuList()[source]

Return the GPU list assigned to current thread.

runJob(log, programName, params, numberOfMpi=1, numberOfThreads=1, env=None, cwd=None)[source]

This function is a wrapper around runJob, providing the host configuration.

runSteps(steps, stepStartedCallback, stepFinishedCallback, stepsCheckCallback, stepsCheckSecs=3)[source]
class pyworkflow.protocol.executor.StepThread(thId, step, lock)[source]

Bases: threading.Thread

Thread to run Steps in parallel.

run()[source]

Method representing the thread’s activity.

You may override this method in a subclass. The standard run() method invokes the callable object passed to the object’s constructor as the target argument, if any, with sequential and keyword arguments taken from the args and kwargs arguments, respectively.

class pyworkflow.protocol.executor.ThreadStepExecutor(hostConfig, nThreads, **kwargs)[source]

Bases: pyworkflow.protocol.executor.StepExecutor

Run steps in parallel using threads.

getGpuList()[source]

Return the GPU list assigned to current thread or empty list if not using GPUs.

runSteps(steps, stepStartedCallback, stepFinishedCallback, stepsCheckCallback, stepsCheckSecs=3)[source]

Create threads and synchronize the steps execution. n: the number of threads. stepsCheckSecs:

rate of how many seconds between stepsCheckCallback calls