scipion.install.funcs module

class scipion.install.funcs.Command(env, cmd, targets=None, **kwargs)[source]

Bases: object

execute()[source]

class scipion.install.funcs.Environment(**kwargs)[source]

Bases: object

addLibrary(name, **kwargs)[source]

Add library <name> to the construction process.

Checks that the needed programs are in PATH, needed libraries can be found, downloads the given url, untars the resulting tar file, configures the library with the given flags, compiles it (in the given buildDir) and installs it.

If default=False, the library will not be built.

Returns the final targets, the ones that Make will create.

addPackage(name, **kwargs)[source]

Download a package tgz, untar it and create a link in software/em. Params in kwargs:

Parameters

tar – the package tar file, by default the name + .tgz. Pass None or VOID_TGZ if there is no tar file.
commands – a list with actions to be executed to install the package
buildDir – Optional folder where build/extraction happens. If not passed will be inferred from tgz’s name
neededProgs – Optional, list of programs needed. E.g: make, cmake,…
version – Optional, version of the package.
libChecks – Optional, a list of the libraries needed. E.g: libjpeg62, gsl (GSL - GNU Scientific Library)

addPipModule(name, version='', pipCmd=None, target=None, default=True, deps=[])[source]

Add a new module to our built Python. Params in kwargs:

Parameters

name – pip module name
version – module version - must be specified to prevent undesired updates.
default – Optional. True if the module has to be installed right after the installation/update of the plugin.

:returns target containing the pip module definition

addTarget(name, *commands, **kwargs)[source]

addTargetAlias(name, alias)[source]: Add an alias to an existing target. This function will be used for installing the last version of each package.

execute()[source]

static getBin(name)[source]

static getBinFolder(*paths)[source]

static getEm(name)[source]

static getEmFolder()[source]

static getIncludeFolder()[source]

getLib(name)[source]

static getLibFolder(*paths)[source]

getLibSuffix()[source]

static getLogFolder(*path)[source]

getPackage(name)[source]

getPackages()[source]: Return all plugin packages

getProcessors()[source]

static getPython()[source]

static getPythonPackagesFolder()[source]

static getSoftware(*paths)[source]

getTarget(name)[source]

getTargetList()[source]

getTargets()[source]

static getTmpFolder()[source]

hasPackage(name)[source]: Returns true if it has the package

hasTarget(name)[source]

printHelp()[source]

setDefault(default)[source]: Set default values of all packages to the passed parameter

updateCudaEnviron(package)[source]: Update the environment adding CUDA_LIB and/or CUDA_BIN to support packages that uses CUDA. package: package that needs CUDA to compile.

class scipion.install.funcs.InstallHelper(packageName: str, packageHome: Optional[str] = None, packageVersion: str = '1.0')[source]

Bases: object

### This class is intended to be used to ease the plugin installation process.

#### Usage: InstallHelper class needs to be instanciated before it can be used. After that, commands can be chained together to run them in the defined order. The last command always needs to be addPackage().

#### Example: installer = InstallHelper() # Instanciating class

installer.getCloneCommand(‘test-package’, ‘/home/user/myCustomPath’, ‘github.com/myRepo’) # Cloning GitHub repository

installer.getCondaenvCommand(‘test-package’) # Creating conda enviroment

installer.addPackage(env, ‘test-package’) # Install package

#### It can also be done in a single line: installer.getCloneCommand(‘test-package’, ‘/home/user/myCustomPath’, ‘github.com/myRepo’).getCondaenvCommand(‘test-package’).addPackage(env, ‘test-package’)

#### If you want to check the command strings you are producing, use the function getCommandList() instead of addPackage() and assign it to a variable so you can print it.

### Constructor for the InstallHelper class.

#### Parameters: packageName (str): Name of the package. packageHome (str): Optional. Path to the package. It can be absolute or relative to current directory. packageVersion (str): Optional. Package version.

DEFAULT_VERSION = '1.0'

addCommand(command: str, targetName: str, workDir: str = '')[source]

### This function adds the given command with target to the command list. ### The target file needs to be located inside packageHome’s directory so Scipion can detect it.

#### Parameters: command (str): Command to be added. targetName (str): Name of the target file to be produced after commands are completed successfully. workDir (str): Optional. Directory where the command will be executed from.

#### Usage: installer.addCommand(‘python3 myScript.py’, ‘MYSCRIPT_COMPLETED’, workDir=’/home/user/Documents/otherDirectory’)

#### This function call will generate the following commands: cd /home/user/Documents/otherDirectory && python3 myScript.py && touch /home/user/scipion/software/em/test-package-1.0/MYSCRIPT_COMPLETED

addCommands(commandList: List[str], binaryName: Optional[str] = None, workDir: str = '', targetNames: List[str] = [])[source]

### This function adds the given commands with targets to the command list.

#### Parameters: commandList (list[str]): List containing the commands to add. binaryName (str): Optional. Name of the binary. Default is package name. workDir (str): Optional. Directory where the commands will be executed from. targetNames (list[str]): Optional. List containing the name of the target files for this commands.

#### Usage: installer.addCommands([‘python3 myScript.py’, ‘ls’], binaryName=’myBinary’, workDir=’/home/user/Documents/otherDirectory’,

targetNames=[‘MYSCRIPT_COMPLETED’, ‘DIRECTORY_LISTED’])

#### This function call will generate the following commands: cd /home/user/Documents/otherDirectory && python3 myScript.py && touch /home/user/scipion/software/em/test-package-1.0/MYSCRIPT_COMPLETED

cd /home/user/Documents/otherDirectory && ls && touch /home/user/scipion/software/em/test-package-1.0/DIRECTORY_LISTED

addCondaPackages(packages: List[str], binaryName: Optional[str] = None, binaryVersion: Optional[str] = None, channel: Optional[str] = None, targetName: Optional[str] = None)[source]

### This function returns the command used for installing extra packages in a conda enviroment.

#### Parameters: binaryName (str): Name of the binary. Default is package name. packages (list[str]): List of conda packages to install. binaryVersion (str): Optional. Binary’s version. Default is package version. channel (str): Optional. Channel to download the package from. targetName (str): Optional. Name of the target file for this command.

#### Usage: installer.addCondaPackages(packages=[‘pytorch==1.1.0’, ‘cudatoolkit=10.0’], binaryName=’myBinary’,

binaryVersion=‘1.5’, channel=’conda-forge’, targetName=’CONDA_PACKAGES_INSTALLED’)

#### This function call will generate the following command: eval “$(/home/user/miniconda/bin/conda shell.bash hook)”&& conda activate myBinary-1.5 && conda install -y pytorch==1.1.0 cudatoolkit=10.0 -c conda-forge && touch CONDA_PACKAGES_INSTALLED #### The path in the first command (eval …) might vary, depending on the value of CONDA_ACTIVATION_CMD in your scipion.conf file.

addPackage(env, dependencies: List[str] = [], default: bool = True, **kwargs)[source]

### This function adds the given package to scipion installation with some provided parameters.

#### Parameters: env: Scipion enviroment. dependencies (list[str]): Optional. List of dependencies the package has. default (bool): Optional. Defines if this package version is automatically installed with the plugin. **kwargs: Optional. Other possible keyword parameters that will be directly passed to env.addPackage. Intended for cases where multiple versions of the same package coexist in the same plugin.

#### Usage: installer.addPackage(env, dependencies=[‘wget’, ‘conda’], default=True)

getCloneCommand(url: str, binaryFolderName: str = '', targeName: Optional[str] = None)[source]

### This function creates the neccessary command to clone a repository from Github.

#### Parameters: url (str): URL to the git repository. binaryFolderName (str): Optional. Name of the binary directory. targetName (str): Optional. Name of the target file for this command.

#### Usage: installer.getCloneCommand(‘https://github.com/myRepo.git’, binaryFolderName=’myCustomBinary’, targeName=’BINARY_CLONED’)

#### This function call will generate the following command: cd /home/user/scipion/software/em/test-package-1.0 && git clone https://github.com/myRepo.git myCustomBinary && touch BINARY_CLONED

getCommandList() → List[Tuple[str, str]][source]

### This function returns the list of commands with targets for debugging purposes.

#### Returns: (list[tuple[str, str]]): Command list with target files.

#### Usage: commandList = installer.getCommandList()

getCondaEnvCommand(binaryName: Optional[str] = None, binaryPath: Optional[str] = None, binaryVersion: Optional[str] = None, pythonVersion: Optional[str] = None, requirementsFile: bool = True, requirementFileName: str = 'requirements.txt', requirementList: List[str] = [], extraCommands: List[str] = [], targetName: Optional[str] = None)[source]

### This function creates the command string for creating a Conda enviroment and installing required dependencies for a given binary inside a package.

#### Parameters: binaryName (str): Optional. Name of the binary. Default is package name. binaryPath (str): Optional. Path to the binary. It can be absolute or relative to current directory. binaryVersion (str): Optional. Binary’s version. Default is package version. pythonVersion (str): Optional. Python version needed for the package. requirementsFile (bool): Optional. Defines if a Python requirements file exists. requirementFileName (bool): Optional. Name of the Python requirements file. requirementList (list[str]): Optional. List of Python packages to be installed. Can be used together with requirements file, but packages cannot be repeated. extraCommands (list[str]): Optional. List of extra conda-related commands to execute within the conda enviroment. targetName (str): Optional. Name of the target file for this command.

#### Usage: installer.getCondaEnvCommand(binaryName=’myBinary’, binaryPath=’/home/user/scipion/software/em/test-package-1.0/myBinary’, binaryVersion=‘1.5’, pythonVersion=‘3.11’,

requirementsFile=True, requirementFileName=’requirements.txt’, requirementList=[‘torch==1.2.0’, ‘numpy’], extraCommands=[‘conda info –envs’], targetName=’CONDA_ENV_CREATED’)

#### This function call will generate the following command: eval “$(/home/user/miniconda/bin/conda shell.bash hook)”&& conda create -y -n myBinary-1.5 python=3.11 && conda activate myBinary-1.5 && cd /home/user/scipion/software/em/test-package-1.0/myBinary && conda install pip -y && $CONDA_PREFIX/bin/pip install -r requirements.txt && $CONDA_PREFIX/bin/pip install torch==1.2.0 numpyconda info –envs && cd /home/user/scipion/software/em/test-package-1.0 && touch CONDA_ENV_CREATED #### The path in the first command (eval …) might vary, depending on the value of CONDA_ACTIVATION_CMD in your scipion.conf file.

getExtraFile(url: str, targetName: str, location: str = '.', workDir: str = '', fileName: Optional[str] = None)[source]

### This function creates the command to download with wget the file in the given link into the given path. ### The downloaded file will overwrite a local one if they have the same name. ### This is done to overwrite potential corrupt files whose download was not fully completed.

#### Parameters: url (str): URL of the resource to download. targetName (str): Name of the target file for this command. location (str): Optional. Location where the file will be downloaded. It can be absolute or relative to current directory. workDir (str): Optional. Directory where the file will be downloaded from. fileName (str): Optional. Name of the file after the download. Use intended for cases when expected name differs from url name.

#### Usage: installer.getExtraFile(‘https://site.com/myfile.tar’, ‘FILE_DOWNLOADED’, location=’/home/user/scipion/software/em/test-package-1.0/subdirectory’, workDir=’/home/user’, fileName=’test.tar’)

#### This function call will generate the following command: cd /home/user && mkdir -p /home/user/scipion/software/em/test-package-1.0/subdirectory && wget -O /home/user/scipion/software/em/test-package-1.0/subdirectory/test.tar https://site.com/myfile.tar && touch /home/user/scipion/software/em/test-package-1.0/FILE_DOWNLOADED

getExtraFiles(fileList: List[Dict[str, str]], binaryName: Optional[str] = None, workDir: str = '', targetNames: Optional[List[str]] = None)[source]

### This function creates the command to download with wget the file in the given link into the given path. ### The downloaded file will overwrite a local one if they have the same name. ### This is done to overwrite potential corrupt files whose download was not fully completed.

#### Parameters: fileList (list[dict[str, str, str]]): List containing files to be downloaded. Example: [{‘url’: url1, ‘path’: path1, ‘name’: ‘test.tar’}, {‘url’: url2, ‘path’: path2, ‘name’: ‘test2.tar’}] binaryName (str): Optional. Name of the binary. Each file is a list contaning url and location to download it. Paths can be an empty string for default location. workDir (str): Optional. Directory where the files will be downloaded from. targetNames (list[str]): Optional. List containing the name of the target files for this commands.

#### Usage: installer.getExtraFiles(

[
{‘url’: ‘https://site.com/myfile.tar’, ‘path’: ‘/home/user/scipion/software/em/test-package-1.0/subdirectory1’, ‘name’: ‘test.tar’}, {‘url’: ‘https://site.com/myfile.tar2’, ‘path’: ‘/home/user/scipion/software/em/test-package-1.0/subdirectory2’, ‘name’: ‘test2.tar2’}

], binaryName=’myBinary’, workDir=’/home/user’, targetNames=[‘DOWNLOADED_FILE_1’, ‘DOWNLOADED_FILE_2’])

#### This function call will generate the following commands: cd /home/user && mkdir -p /home/user/scipion/software/em/test-package-1.0/subdirectory1 && wget -O /home/user/scipion/software/em/test-package-1.0/subdirectory1/test.tar https://site.com/myfile.tar && touch /home/user/scipion/software/em/test-package-1.0/DOWNLOADED_FILE_1

cd /home/user && mkdir -p /home/user/scipion/software/em/test-package-1.0/subdirectory2 && wget -O /home/user/scipion/software/em/test-package-1.0/subdirectory2/test2.tar2 https://site.com/myfile.tar2 && touch /home/user/scipion/software/em/test-package-1.0/DOWNLOADED_FILE_2

getFileDict(url: str, path: str = '.', fileName: Optional[str] = None) → Dict[str, str][source]: This function generates the dictionary for a downloadable file.

class scipion.install.funcs.Link(packageLink, packageFolder)[source]

Bases: object

createPackageLink(packageLink, packageFolder)[source]: Create a link to packageFolder in packageLink, validate that packageFolder exists and if packageLink exists it is a link. This function is supposed to be executed in software/em folder.

class scipion.install.funcs.Target(env, name, *commands, **kwargs)[source]