Techniques for sharing Python scripts

Following are some techniques you can use when sharing your Python scripts with others.

Finding data relative to the script location

Your script may need to use what programmers call hard-wired paths—paths to existing data that is not passed as an argument. For example, you may need to set the symbology property of your output parameter to an existing layer file or clip your data to a known dataset.

If you are sharing your tool with others, you need to make sure that your script can find the data it needs. A good way to do this is to use the ToolShare folder structure and place your project data in the ToolData folder and your scripts in the Scripts folder. If you follow this pattern, then you can always find your data relative to the location of your script.

When your script is run, the path to the script can be found using the following:

scriptPath = sys.path[0]

Once this location is known, you can find your project data relative to this location. The following code snippet demonstrates this:

import arcpy
import os
import sys

# Get the pathname to this script
#
scriptPath = sys.path[0]
arcpy.AddMessage("Script folder: " + scriptPath)

# Get the pathname to the ToolShare folder
#
toolSharePath = os.path.dirname(scriptPath)
arcpy.AddMessage("ToolShare folder: " + toolSharePath)

# Now construct pathname to the ToolData folder
#
toolDataPath = os.path.join(toolSharePath, "ToolData")
arcpy.AddMessage("ToolData folder: " + toolDataPath)

# Create the pathname to the parks feature class found in the ToolData folder
#
parkPath = os.path.join(toolDataPath, "Project.gdb/Parks")
arcpy.AddMessage("Parks feature class: " + parkPath)

Note:

If you embed your script code, sys.path[0] will return the location of the toolbox (which is where the code is located).

Finding a scratch workspace

If you create scratch data within your script, you need a scratch workspace where you can create and subsequently delete your scratch data.

Here are some things to keep in mind about finding a scratch workspace:

• You need write permission to the workspace.
• If the scratch workspace geoprocessing environment is set, use it—you have to assume it has been intentionally set by the user to a writeable location. In the case of a geoprocessing service running on an ArcGIS server, the scratch workspace will be automatically set by ArcGIS Server to a specific writeable folder.
• Be cautious of using the current workspace environment as a scratch workspace—it may be a remote database, and you never want to use a remote database as a scratch location, since the overhead of communicating with the remote database defeats the purpose of quickly creating, writing, and deleting scratch data.
• If the scratch workspace environment has not been set, you need to find a writeable location. If this is the case, there a several options:
  • If you're using the ToolShare folder structure, then you can use scratch.gdb in the Scratch folder. If you are practicing defensive programming, you should check that scratch.gdb exists—the user may have deleted it.
  • If one of your script parameters is an output dataset, such as a feature class, you can safely assume you have write permission to the location of the output dataset, and you can use that as a scratch workspace. However, you need to check if the location is a feature dataset or a remote database. (As mentioned above, you never want to use a remote database as a scratch location, because the overhead of communicating with the remote database defeats the purpose of quickly creating, writing, and deleting scratch data.)
  • When all else fails, you can always use the system temp directory.
• Feature classes within shapefile and personal geodatabase workspaces have a two gigabyte size limit. While two gigabytes is a lot of data, it is still a limit. A file geodatabase workspace has a default limit of one terabyte (1024 gigabytes) for a dataset. If you think that your scratch datasets could exceed two gigabytes, use a file geodatabase as the scratch workspace.
• Use the CreateScratchName function to create a unique dataset name in your scratch workspace.

The following code example shows a very defensive implementation of finding a scratch workspace. This code

• Doesn't use the current workspace as a scratch workspace, but can be easily modified to do so
• Assumes you're using the ToolShare folder structure
• Attempts to find and use a file geodatabase and uses a shapefile workspace as a last resort

import arcpy
from arcpy import env
import sys
import os

def getScratchWorkspace(outDataset):
  # outDataSet is assumed to be the full pathname to a dataset. Typically,
  #  this would be a tool's output parameter value.
  #
  # Get the scratch workspace environment. If it's set, just return it.
  #
  scratchWS = env.scratchWorkspace
  if scratchWS:
    return scratchWS
 
  # Let's go fishing...
  #
  # If you're using the ToolShare folder structure, look for scratch.gdb in
  #  the Scratch folder.
  #
  scriptPath      = sys.path[0]
  toolSharePath   = os.path.dirname(scriptPath)
  scratchWS       = os.path.join(toolSharePath, "Scratch/scratch.gdb")
  if not arcpy.Exists(scratchWS):
    scratchWS = ""

  # No scratch workspace environment and no scratch.gdb in the ToolShare folder
  #
  if not scratchWS:
    # Get the workspace of the output dataset (if any passed in)
    #  by going up one level
    #
    if outDataset:
      scratchWS = os.path.dirname(str(outDataset))

      # If this isn't a workspace, go up another level and
      #  test again. 
      #
      desc = arcpy.Describe(scratchWS)
      if desc.dataType.upper() <> "WORKSPACE":
        scratchWS = os.path.dirname(scratchWS)
        desc = arcpy.Describe(scratchWS)
        if desc.dataType.upper() <> "WORKSPACE":
          scratchWS = ""

  # If we have a workspace, make sure it's not a remote (SDE) database.
  #  If it is remote, set workspace to the system temp directory.
  #
  # If we don't have a workspace, just set it to the system temp directory.
  #
  usingTemp = False
  if scratchWS:
      desc = arcpy.Describe(scratchWS)
      if desc.workspaceType.upper() == "REMOTEDATABASE":
          scratchWS = arcpy.GetSystemEnvironment("TEMP")
          usingTemp = True
  else:
      scratchWS = arcpy.GetSystemEnvironment("TEMP")
      usingTemp = True

  # If we're using the system temp directory (a shapefile workspace), look 
  #  for a scratch file geodatabase.  If it exists, use it.  If it doesn't, 
  #  create it.
  #
  if usingTemp:
    scratchWS = os.path.join(scratchWS, "scratch.gdb")
    if arcpy.Exists(scratchWS):
      return scratchWS
    else:
      arcpy.CreateFileGDB_management(arcpy.GetSystemEnvironment("TEMP"),
                                     "scratch.gdb") 

  return scratchWS

# Main demonstration routine
#  One optional input parameter, a feature class. 
#
aDatasetpath = arcpy.GetParameterAsText(0)
scratch = getScratchWorkspace(aDatasetpath)
arcpy.AddMessage("Scratch workspace: " + scratch)

# Create a scratch feature class in the scratch workspace
#
scrname = arcpy.CreateScratchName("temp", "","featureclass", scratch)

arcpy.AddMessage("Scratch feature class is: " + scrname)
arcpy.CreateFeatureclass_management(scratch, os.path.basename(scrname), "point")
arcpy.AddMessage(arcpy.GetMessages())

Sharing Python modules

Like any modern programming language, Python allows you to call routines found in other Python scripts. As you develop more and more Python code, you'll probably want to develop Python routines to share among scripts. The purpose of this section is to briefly show you how you can share routines and give you enough information that you can effectively research and implement sharing of routines, starting with the official Python Web site (http://www.python.org).

Here are the contents of a script, helloworld.py:

def dosomething():
    print "Hello world"
def somethingelse():
    print "Goodbye world"

Here are the contents of a script, main.py:

import sys, os, helloworld
helloworld.dosomething()
helloworld.somethingelse()

The script main.py imports the helloworld module (a module name is equal to the script name minus the .py extension), along with the sys and os module. Note that the .py extension on helloworld is not needed (and, in fact, not allowed).

The script helloworld.py implements two routines (using the def statement) called dosomething, which prints the ubiquitous "Hello world", and somethingelse, which prints the less ubiquitous "Goodbye world". When main.py executes, it will call these two routines, which will print "Hello world" and "Goodbye world".

The two modules shown above lack a number of things, such as importing arcpy, fetching geoprocessing environments, error handling, and the like. These topics are all covered in A quick tour of Python.

Where Python looks for modules

When main.py is executed, the import directive causes Python to look for a file named helloworld.py in its list of system directory paths. The first place Python looks is the current directory—the same directory as the script containing the import directive (in this case, main.py). You can display a list of these paths in the interactive window of PythonWin with:

import sys
sys.path

You cannot enter a path in the import directive such as

import E:\SharedScripts\helloworld

Rather, you must alter the list of directories where Python looks for modules. This list of directories is contained in the Windows environment setting called PYTHONPATH, which is installed by ArcGIS. To alter this setting, do the following:

1. Under the Windows Start menu, click Settings > Control Panel.
2. Locate and open the System file.
3. Click the Advanced tab, then click Environment Variables.
4. Under System Variables, scroll to the PYTHONPATH variable and click to select it.
   • If the PYTHONPATH variable does not exist, click New. In the Variable Name: text box, input PYTHONPATH.
   • In the Variable value: text box, input <ArcGIS install directory>\bin;<ArcGIS install directory\arcpy (for example, C:\Program Files\ArcGIS\Desktop10.0\bin;C:\Program Files\ArcGIS\Desktop10.0\arcpy).
   • Click OK.
5. If the PYTHONPATH variable exists, click Edit.

   The contents of PYTHONPATH should have <ArcGIS install directory>\bin;<ArcGIS install directory\arcpy as the first entry. This is where the arcgisscripting module resides. You can append more paths to directories containing your Python modules. The paths are separated by semicolons, and there must not be any spaces around the semicolon.

You can also append a path within your code with

sys.path.append("e:\sharedmodules")

Sharing Python modules

If you are delivering script tools to other users, and the scripts are importing other modules, you have two choices:

1. All scripts reside in the same directory.
2. You instruct users to install the additional modules somewhere on their system and have them modify the PYTHONPATH variable.

Using sys.path.append() is not recommended, since it requires that the Python code be altered by users.

Paths and the escape character

Programming languages that have their roots in UNIX and the C programming language, like Python, treat the backslash (\) as the escape character. For example, \n is used to insert a carriage return when writing text output, and \t is used to insert a tab character. If a path in your script uses backslashes as the separator, Python will scan it and substitute a carriage return when it encounters a \n and a tab for a \t. (There are other escape character sequences besides \n and \t.)

The easiest way to guard against this is to convert paths into Python raw strings using the r directive, as shown below. This instructs Python to ignore backslashes.

thePath = r"E:\data\teluride\newdata.gdb\slopes"

Learn more about setting paths to data

Checking for licenses

If your script is using extensions or relies on tools that are unavailable at the ArcView or ArcEditor product level, you need to first check for licenses and product levels.

Learn more about license checking in scripts