sitetools - WesternX’s Python Setup¶

The dev-install Command¶

A dev-install command exists to assist in the installation of tools for local development. It will first assert that a virtualenv_ exists in your home, clone the tool, and finally install it into your virtualenv.

The first time you run dev-install (or dev) on a platform, you will be prompted to create your venv:

$ dev true
Could not find existing development virtualenv.
A virtualenv can be created in these locations:
    1) ~/dev/venv-osx/bin/python
    2) ~/key_tools/venv-osx/bin/python
Which one do you want to create? (1): 1
New python executable in /home/mboers/dev/venv-osx/bin/python
Installing setuptools, pip...done.

Now you can install tools:

$ dev-install --list
< big list of tools and their repos >

$ dev-install sgfs
< snip >
Successfully installed sgfs
Cleaning up...

The dev-status Command¶

A dev-status command exists to help you figure out how your local tools relate to those in production:

$ dev-status
    WARNING: You are behind by 6 commits.
==> sgsession
    Working directory is dirty.
==> sgfs
    Up to date.

Here we can see that my copy of key_base is behind by 6 commits, I have uncommitted work on sgsession, and my sgfs is up to date.

We can have dev-status bring everything up to date for us:

$ dev-status -nu
==> key_base
Updating 05eb284..fc9c4f8
Fast-forward
    Up to date.
==> sgsession
    Working directory is dirty.
==> sgfs
    Up to date.

The dev Command Wrapper¶

A dev command exists that will run any other command from within an automatically constructed development environment. Effectively, any Python imports will use your local packages, and any executables will be sourced from your local paths. Any packages or executables not found locally will fall back on the production tools.

It does this by searching for development tools in KS_DEV_SITES, and appends those that exist to KS_SITES. It also looks across the standard PATH for directories that fall within the production environment, and prepends the development versions of those paths.

Lets look at some examples. I can bring up a Python shell in the development environment:

$ dev python
Python 2.6.6 (r266:84292, Nov  1 2010, 12:40:26)
[GCC 4.2.1 (Apple Inc. build 5646)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ks
>>> ks
<module 'ks' from '/home/mboers/dev/key_base/python/ks.py'>

Notice how the ks package was located in my home directory. I can also launch the development version of the toolbox:

$ dev which toolbox
/home/mboers/dev/key_base/2d/bin/toolbox
$ dev toolbox

To drop into a development shell:

$ dev bash
$ which toolbox
/home/mboers/dev/key_base/2d/bin/toolbox

You can also use other developer’s environments!

$ dev -u mreid which toolbox
/home/mreid/dev/key_base/2d/bin/toolbox

See dev --help for all of the options.

Environment Variables¶

KS_SITES¶

A colon-delimited list of sites to add as pseudo site-packages (see python_setup).

If the “site” is a directory, it will be processed as if it were a site-packages directory.

If the “site” is a file named python, it will search for the corresponding site-packages directory.

If the current environment (equivalent to sys.executable) is found in this list then it will be used as a centering point for the other sites listed. Anything before the current environment will be prepended to sys.path, and anything after the current environment will be appended.

KS_DEV_SITES¶

A colon-delimited list of sites to look for when using the dev command. Any ~ found may refer to any requested user’s home.

If unset, defaults to ~/dev:~/dev/venv/bin/python.

API Reference¶

class sitetools.sites.SysPathInserter(index=None)¶

Class to insert a series of paths into sys.path incrementally.

add(path)¶

Add the given path to the decided place in sys.path

sitetools.sites.add_site_dir(dir_name, before=None, _path=None)¶

Add a pseudo site-packages directory to sys.path.

Parameters:	dir_name (str) – The directory to add. before (str) – A directory on sys.path that new paths should be

inserted before.

Looks for .pth files at the top-level and __site__.pth files within top-level directories.

sitetools.sites.add_site_list(dir_list)¶

Add a list of pseudo site-packages to sys.path.

This centers the list on sys.path around the current environment. I.e. if this environment is in the list, then directories before it in the list will be prepended to sys.path, and directories after it will be appended to sys.path.

Actual Variables¶

KS_ENVIRON_DIFF¶

A set of variables to update (or delete) from Python’s os.environ at startup. This is used to force variables that are nessesary for startup to not propigate into the next executable.

Warning

Do not use this directly, as the format is subject to change without notice. Instead, use sitecustomize.environ.freeze().

API Reference¶

sitetools.environ.freeze(environ, names, label=None)¶

Flag the given names to reset to their current value in the next Python.

Parameters:	environ (dict) – The environment that will be passed to the next Python. names – A list of variable names that should be reset to their current value (as in environ) when the next sub-Python starts. label (str) – A name for this environment freeze; the default of None will be unfrozen at startup.

This is useful to reset environment variables that are set by wrapper scripts that are nessesary to bootstrap the process, but we do not want to carry into any subprocess. E.g. LD_LIBRARY_PATH.

This may be called multiple times for the same label as it updates any existing freezes.

Usage:

import os
from subprocess import call

from sitecustomize.environ import freeze

env = dict(os.environ)
env['DEMO'] = 'one'
freeze(env, ['DEMO'])
env['DEMO'] = 'two'

call(['python', '-c', 'import os; print os.environ["DEMO"]'], env=env)
# Prints: one

sitetools.environ.unfreeze(label, pop=False, environ=None)¶

Reset the environment to its state before it was frozen by freeze().

Parameters:	label (str) – The name for the frozen environment. pop (bool) – Destroy the freeze after use; only allow unfreeze once. environ (dict) – The environment to work on; defaults to os.environ.
Returns:	A context manager to re-freeze the environment on exit.

Usage:

unfreeze('nuke')

# or

with unfreeze('maya'):
    # do something

Debugging via Logging¶

It is highly recommended that you use DEBUG level logging instead of print statements, since they will not show up to the end users unless it is requested, and then key locations which need debugging output will already have it without having to re-determine where the trouble parts are. It is recommended to use the following pattern at the top of your files:

import logging
log = logging.getLogger(__name__)

# Do stuff.

log.debug('Something crazy is happening...')

You can then get those debug logs dumped to your terminal by using the dev wrapper in verbose mode:

$ dev -v python -m my.awesome.module
2013-04-07 13:48:08,416 DEBUG my.awesome.module: Something crazy is happening...

Environment Variables¶

KS_PYTHON_LOG_FILE¶

A format string for determining where to save logging logs. Defaults (in the WesternX environment) to:

/Volumes/VFX/logs/{date}/{login}@{ip}/{time}.{pid}.log

Keys available include: date, time, login, ip, and pid.

KS_LOG_LEVELS¶

A space-or-comma-delimited list of logger names and minimum record levels. E.g.:

$ export KS_LOG_LEVELS=:WARNING,mayatools:DEBUG

would set the general logging threshold to logging.WARNING, but anything within mayatools to logging.DEBUG.

In an emergency this can effectively disable the logging system by setting:

$ export KS_LOG_LEVELS=:100

which is too high for any (built-in) log levels.