pyworkflow.project.project module

exception pyworkflow.project.project.MissingProjectDbException[source]

Bases: Exception

exception pyworkflow.project.project.ModificationNotAllowedException[source]

Bases: Exception

class pyworkflow.project.project.Project(domain, path)[source]

Bases: object

This class will handle all information related with a Project

Create a new Project instance. :param domain: The application domain from where to get objects and

protocols.

Parameters

path – Path where the project will be created/loaded

checkPid(protocol)[source]

Check if a running protocol is still alive or not. The check will only be done for protocols that have not been sent to a queue system.

static cleanProjectName(projectName)[source]

Cleans a project name to avoid common errors Use it whenever you want to get the final project name pyworkflow will endup. Spaces will be replaced by _

closeMapper()[source]
continueProtocol(protocol)[source]

This function should be called to mark a protocol that have an interactive step waiting for approval that can continue

copyProtocol(protocol)[source]

Make a copy of the protocol, Return a new instance with copied values.

create(runsView=1, readOnly=False, hostsConf=None, protocolsConf=None)[source]

Prepare all required paths and files to create a new project.

Parameters
  • runsView – default view to associate the project with

  • readOnly – If True, project will be loaded as read only.

  • hostsConf – Path to the host.conf to be used when executing protocols

  • protocolsConf – Not used.

createMapper(sqliteFn)[source]

Create a new SqliteMapper object and pass as classes dict all globals and update with data and protocols from em.

createSettings(runsView=1, readOnly=False)[source]
deleteProtocol(*protocols)[source]
deleteProtocolOutput(protocol, output)[source]

Delete a given object from the project. Usually to clean up some outputs.

doesProtocolExists(protId)[source]
exportProtocols(protocols, filename)[source]

Create a text json file with the info to import the workflow into another project. This method is very similar to copyProtocol

Parameters
  • protocols – a list of protocols to export.

  • filename – the filename where to write the workflow.

getCreationTime()[source]

Return the time when the project was created.

getDbLastModificationDate()[source]

Return the last modification date of the database

classmethod getDbName()[source]

Return the name of the database file of projects.

getDbPath()[source]

Return the path to the sqlite db.

getDomain()[source]
getElapsedTime()[source]

Returns the time elapsed from the creation to the last execution time.

getGraphFromRuns(runs)[source]

This function will build a dependencies graph from a set of given runs.

Parameters

runs – The input runs to build the graph

Returns

The graph taking into account run dependencies

getHostConfig(hostName)[source]
getHostNames()[source]

Return the list of host name in the project.

getLeftTime()[source]
getLocalConfigHosts()[source]

Return the local file where the project will try to read the hosts configuration.

getLogPath(*paths)[source]
getName()[source]
getObjId()[source]

Return the unique id assigned to this project.

getObject(objId)[source]

Retrieve an object from the db given its id.

getPath(*paths)[source]

Return path from the project root

static getProtWorkingDir(protocol)[source]

Return the protocol working directory

getProtocol(protId)[source]
getProtocolView()[source]

Returns de view selected in the tree when it was persisted

getProtocolsByClass(className)[source]
getProtocolsDict(protocols=None, namesOnly=False)[source]

Creates a dict with the information of the given protocols.

Parameters
  • protocols – list of protocols or None to include all.

  • namesOnly – the output list will contain only the protocol names.

getProtocolsJson(protocols=None, namesOnly=False)[source]

Wraps getProtocolsDict to get a json string

Parameters
  • protocols – list of protocols or None to include all.

  • namesOnly – the output list will contain only the protocol names.

getRelatedObjects(relation, obj, direction=0, refresh=False)[source]

Get all objects related to obj by a give relation.

Parameters
  • relation – the relation name to search for.

  • obj – object from which the relation will be search, actually not only this, but all other objects connected to this one by the pwobj.RELATION_TRANSFORM.

  • direction – Not used

  • refresh – If True, cached objects will be refreshed

getRuns(iterate=False, refresh=True, checkPids=False)[source]

Return the existing protocol runs in the project.

getRunsGraph(refresh=False, checkPids=False)[source]

Build a graph taking into account the dependencies between different runs, ie. which outputs serves as inputs of other protocols.

getSettings()[source]
getSettingsCreationTime()[source]
getShortName()[source]
getSourceChilds(obj)[source]

Return all the objects have used obj as a source.

getSourceGraph(refresh=False)[source]

Get the graph from the SOURCE relation.

getSourceParents(obj)[source]

Return all the objects that are SOURCE of this object.

getTmpPath(*paths)[source]
getTransformGraph(refresh=False)[source]

Get the graph from the TRANSFORM relation.

isInReadOnlyFolder()[source]

Returns if the project path is a link to another folder.

isReadOnly()[source]
iterSubclasses(classesName, objectFilter=None)[source]
Retrieve all objects from the project that are instances

of any of the classes in classesName list.

Params:

classesName: String with commas separated values of classes name. objectFilter: a filter function to discard some of the retrieved objects.

launchProtocol(protocol, wait=False, scheduled=False, force=False)[source]

In this function the action of launching a protocol will be initiated. Actions done here are:

  1. Store the protocol and assign name and working dir

  2. Create the working dir and also the protocol independent db

  3. Call the launch method in protocol.job to handle submission:

    mpi, thread, queue, and also take care if the execution is remotely.

If the protocol has some prerequisites (other protocols that needs to be finished first), it will be scheduled.

Parameters
  • protocol – Protocol instance to launch

  • wait – Optional. If true, this method will wait until execution is finished. Used in tests.

  • scheduled – Optional. If true, run.db and paths already exist and are preserved.

  • force – Optional. If true, launch is forced, regardless latter dependent executions. Used when restarting many protocols a once.

launchWorkflow(workflowProtocolList, mode=0)[source]

This function can launch a workflow from a selected protocol in two modes depending on the ‘mode’ value (RESTART, CONTINUE) Actions done here are:

  1. Check if the workflow has active protocols.

  2. Fix the workflow if is not properly configured

  3. Restart or Continue a workflow starting from the protocol depending

    on the ‘mode’ value

load(dbPath=None, hostsConf=None, protocolsConf=None, chdir=True, loadAllConfig=True)[source]

Load project data, configuration and settings.

Parameters
  • dbPath – the path to the project database. If None, use the project.sqlite in the project folder.

  • hostsConf – where to read the host configuration. If None, check if exists in .config/hosts.conf or read from ~/.config/scipion/hosts.conf

  • protocolsConf – Not used

  • chdir – If True, os.cwd will be set to project’s path.

  • loadAllConfig – If True, settings from settings.sqlite will also be loaded

loadProtocols(filename=None, jsonStr=None)[source]

Load protocols generated in the same format as self.exportProtocols.

Parameters
  • filename – the path of the file where to read the workflow.

  • jsonStr – Not used.

Note: either filename or jsonStr should be not None.

needRefresh()[source]

True if any run is active and its timestamp is older than its corresponding runs.db NOTE: If an external script changes the DB this will fail. It uses only in memory objects.

newProtocol(protocolClass, **kwargs)[source]

Create a new protocol from a given class.

openedAsReadOnly()[source]
resetProtocol(protocol)[source]

Stop a running protocol

resetWorkFlow(workflowProtocolList)[source]

This function can reset a workflow from a selected protocol :param initialProtocol: selected protocol

saveProtocol(protocol)[source]
saveSettings()[source]
scheduleProtocol(protocol, prerequisites=[], initialSleepTime=0)[source]

Schedule a new protocol that will run when the input data is available and the prerequisites are finished.

Parameters
  • protocol – the protocol that will be scheduled.

  • prerequisites – a list with protocols ids that the scheduled protocol will wait for.

  • initialSleepTime – number of seconds to wait before checking input’s availability

setDbPath(dbPath)[source]

Set the project db path. This function is used when running a protocol where a project is loaded but using the protocol own sqlite file.

setReadOnly(value)[source]
stopProtocol(protocol)[source]

Stop a running protocol

stopWorkFlow(activeProtList)[source]

This function can stop a workflow from a selected protocol :param initialProtocol: selected protocol