pyworkflow.project.project module

exception pyworkflow.project.project.MissingProjectDbException

Bases: Exception

exception pyworkflow.project.project.ModificationNotAllowedException

Bases: Exception

class pyworkflow.project.project.Project(domain, path)

Bases: object

This class will handle all information related with a Project

Create a new Project instance. :param domain: The application domain from where to get objects and

protocols.

Parameters

path – Path where the project will be created/loaded

checkIsAlive(protocol)

Check if a protocol is alive based on its jobid or pid

checkJobId(protocol)

Check if a running protocol is still alive or not. The check will only be done for protocols that have been sent to a queue system.

checkPid(protocol)

Check if a running protocol is still alive or not. The check will only be done for protocols that have not been sent to a queue system.

static cleanProjectName(projectName)

Cleans a project name to avoid common errors Use it whenever you want to get the final project name pyworkflow will end up. Spaces will be replaced by _

closeMapper()

configureLogging()

continueProtocol(protocol)

This function should be called to mark a protocol that have an interactive step waiting for approval that can continue

copyProtocol(protocol)

Make a copy of the protocol, Return a new instance with copied values.

create(runsView=1, readOnly=False, hostsConf=None, protocolsConf=None, comment=None)

Prepare all required paths and files to create a new project.

Parameters

• runsView – default view to associate the project with

• readOnly – If True, project will be loaded as read only.

• hostsConf – Path to the host.conf to be used when executing protocols

• protocolsConf – Not used.

createMapper(sqliteFn)

Create a new SqliteMapper object and pass as classes dict all globals and update with data and protocols from em.

createSettings(runsView=1, readOnly=False)

deleteProtocol(*protocols)

deleteProtocolOutput(protocol, output)

Delete a given object from the project. Usually to clean up some outputs.

doesProtocolExists(protId)

exportProtocols(protocols, filename)

Create a text json file with the info to import the workflow into another project. This method is very similar to copyProtocol

Parameters

• protocols – a list of protocols to export.

• filename – the filename where to write the workflow.

fixLinks(searchDir)

getComment()

Returns the project comment. Stored as CreationTime comment.

getCreationTime()

Return the time when the project was created.

getDbLastModificationDate()

Return the last modification date of the database

classmethod getDbName()

Return the name of the database file of projects.

getDbPath()

Return the path to the sqlite db.

getDomain()

getElapsedTime()

Returns the time elapsed from the creation to the last execution time.

getGraphFromRuns(runs)

This function will build a dependencies graph from a set of given runs.

Parameters

runs – The input runs to build the graph

Returns

The graph taking into account run dependencies

getHostConfig(hostName)

getHostNames()

Return the list of host name in the project.

getLeftTime()

getLocalConfigHosts()

Return the local file where the project will try to read the hosts configuration.

getLogPath(*paths)

getName()

getObjId()

Return the unique id assigned to this project.

getObject(objId)

Retrieve an object from the db given its id.

getPath(*paths)

Return path from the project root

getProjectLog()

static getProtWorkingDir(protocol)

Return the protocol working directory

getProtocol(protId, fromRuns=False)

Returns the protocol with the id=protId or raises an Exception

Parameters

• protId – integer with an existing protocol identifier

• fromRuns – If true, it tries to get it from the runs list (memory) avoiding querying the db.

getProtocolCompatibleOutputs(protocol, classes, condition)

Getting the outputs compatible with an object type. The outputs of the child protocols are excluded.

getProtocolFromRuns(protId)

Returns the protocol with the id=protId from the runs list (memory) or None

getProtocolView()

Returns de view selected in the tree when it was persisted

getProtocolsByClass(className)

getProtocolsDict(protocols=None, namesOnly=False)

Creates a dict with the information of the given protocols.

Parameters

• protocols – list of protocols or None to include all.

• namesOnly – the output list will contain only the protocol names.

getProtocolsJson(protocols=None, namesOnly=False)

Wraps getProtocolsDict to get a json string

Parameters

• protocols – list of protocols or None to include all.

• namesOnly – the output list will contain only the protocol names.

getRelatedObjects(relation, obj, direction=0, refresh=False)

Get all objects related to obj by a give relation.

Parameters

• relation – the relation name to search for.

• obj – object from which the relation will be search, actually not only this, but all other objects connected to this one by the pwobj.RELATION_TRANSFORM.

• direction – Not used

• refresh – If True, cached objects will be refreshed

getRuns(iterate=False, refresh=True, checkPids=False)

Return the existing protocol runs in the project.

getRunsGraph(refresh=False, checkPids=False)

Build a graph taking into account the dependencies between different runs, ie. which outputs serves as inputs of other protocols.

getSettings()

getSettingsCreationTime()

getShortName()

getSourceChilds(obj)

Return all the objects have used obj as a source.

getSourceGraph(refresh=False)

Get the graph from the SOURCE relation.

getSourceParents(obj)

Return all the objects that are SOURCE of this object.

getTmpPath(*paths)

getTransformGraph(refresh=False)

Get the graph from the TRANSFORM relation.

isInReadOnlyFolder()

isLink()

Returns if the project path is a link to another folder.

isReadOnly()

iterSubclasses(classesName, objectFilter=None)

Retrieve all objects from the project that are instances

of any of the classes in classesName list.

Params:

classesName: String with commas separated values of classes name. objectFilter: a filter function to discard some of the retrieved objects.

launchProtocol(protocol, wait=False, scheduled=False, force=False)

In this function the action of launching a protocol will be initiated. Actions done here are:

1. Store the protocol and assign name and working dir

2. Create the working dir and also the protocol independent db

3. Call the launch method in protocol.job to handle submission:

   mpi, thread, queue.

If the protocol has some prerequisites (other protocols that needs to be finished first), it will be scheduled.

Parameters

• protocol – Protocol instance to launch

• wait – Optional. If true, this method will wait until execution is finished. Used in tests.

• scheduled – Optional. If true, run.db and paths already exist and are preserved.

• force – Optional. If true, launch is forced, regardless latter dependent executions. Used when restarting many protocols a once.

launchWorkflow(workflowProtocolList, mode=0)

This function can launch a workflow from a selected protocol in two modes depending on the ‘mode’ value (RESTART, CONTINUE) Actions done here are:

1. Check if the workflow has active protocols.

2. Fix the workflow if is not properly configured

3. Restart or Continue a workflow starting from the protocol depending

   on the ‘mode’ value

load(dbPath=None, hostsConf=None, protocolsConf=None, chdir=True, loadAllConfig=True)

Load project data, configuration and settings.

Parameters

• dbPath – the path to the project database. If None, use the project.sqlite in the project folder.

• hostsConf – where to read the host configuration. If None, check if exists in .config/hosts.conf or read from ~/.config/scipion/hosts.conf

• protocolsConf – Not used

• chdir – If True, os.cwd will be set to project’s path.

• loadAllConfig – If True, settings from settings.sqlite will also be loaded

loadProtocols(filename=None, jsonStr=None)

Load protocols generated in the same format as self.exportProtocols.

Parameters

• filename – the path of the file where to read the workflow.

• jsonStr –

Note: either filename or jsonStr should be not None.

needRefresh()

True if any run is active and its timestamp is older than its corresponding runs.db NOTE: If an external script changes the DB this will fail. It uses only in memory objects.

newProtocol(protocolClass, **kwargs)

Create a new protocol from a given class.

openedAsReadOnly()

resetProtocol(protocol)

Stop a running protocol

resetWorkFlow(workflowProtocolList)

This function can reset a workflow from a selected protocol :param initialProtocol: selected protocol

saveProtocol(protocol)

saveSettings()

scheduleProtocol(protocol, prerequisites=[], initialSleepTime=0)

Schedule a new protocol that will run when the input data is available and the prerequisites are finished.

Parameters

• protocol – the protocol that will be scheduled.

• prerequisites – a list with protocols ids that the scheduled protocol will wait for.

• initialSleepTime – number of seconds to wait before checking input’s availability

setComment(newComment)

Sets the project comment

setDbPath(dbPath)

Set the project db path. This function is used when running a protocol where a project is loaded but using the protocol own sqlite file.

setReadOnly(value)

stopProtocol(protocol)

Stop a running protocol

stopWorkFlow(activeProtList)

This function can stop a workflow from a selected protocol :param initialProtocol: selected protocol