The method I use to load the geoprocessor, while verbose, ensures that if anyone loads the geoprocessor before they use any of my module functions, any of their settings remain intact. This is achieved as follows:

1. Check if the arcgisscripting module was loaded.
   1. If it was not loaded, obviously the geoprocessor is not instantiated. Import, instantiate and return.
2. If the arcgisscripting module was loaded, check the variables the user created in the main program.
   1. If a variable is found to be of type ‘geoprocessor object‘, then return that object.
3. If no variable is found of type ‘geoprocessor object‘, create it and return it.

The code can be found below. Please review and use as you wish. If you have any comments or questions about it, please leave a comment.

"""
Loads the geoprocessor safely, ensuring that it is not loaded
twice (to avoid overwriting parameters).

Usage: load()

Returns: a geoprocessor object.

Note: Geoprocessor Version returned is 9.3

"""
import sys

def load(version=None):
    # Check if the arcgisscripting module was loaded
    if "arcgisscripting" not in sys.modules:
        # The arcgisscripting module was never loaded Try to load it.
        print "arcgisscripting not loaded"
        print "Trying to load arcgisscripting"
        try:
            import arcgisscripting
        except ImportError, errorDetail:
            sys.exit("Error loading arcgisscriping. Is ArcGIS installed?\n" + errorDetail)
        # Try to create a geoprocessor instantiation
        print "Trying to create the geoprocessor instance"
        try:
            gp = arcgisscripting.create(9.3)
        except:
            sys.exit("Error creating geoprocessor object. Exiting\n" + str(sys.exc_info()))
        # Return the geoprocessor instance
        print "Done!"
        return gp
    else:
        # arcgisscripting was loaded. Is the geoprocessor instantiated?
        print "arcgisscripting was already loaded. Searching for geoprocessor..."
        # Get the global variables
        globalVars = vars(sys.modules["__main__"])
        for item, value in globalVars.iteritems():
            # Loop through global variables, ignore internals
            if not item.startswith("__"):
                # Check the type of the variable
                if type(globalVars[item]).__name__ == 'geoprocessing object':
                    # If the variable is the geoprocessor, return it
                    print "Found geoprocessor instance as", item
                    return globalVars[item]
        # If we are here, the module is loaded but no gp instantiated
        # Try to create a geoprocessor instantiation
        try:
            print "arcgisscripting loaded but not instantiated. Instantiating now..."
            gp = sys.modules["arcgisscripting"].create(9.3)
        except:
            sys.exit("Error creating geoprocessor object. Exiting\n" + str(sys.exc_info()))
        # Return the geoprocessor instance
        return gp
 

Related posts:

1. Understanding the Geoprocessor Programming Model part 1

1. A free geoprocessor: While this is unlikely to happen in the foreseeable future, there are elements of the geoprocessor that can be made freely available, either as a separate Python module or through a license check for parts of the objects. Specifically, I am referring to some of the List* methods (ListRasters(), ListFeatureClasses(), etc.) that do not require deep access to data (this would include ProductInfo(), validators, and hopefully even the Describe object). Why would ESRI want to do this? To ensure that limited functionality is available to everyone independent of cost. Why restrict people from seeing the power of the geoprocessor? Let them join the fun, and perhaps become customers too.
2. Path traversal options: The List*() methods come in the list again, with a slight change. Since many users use directory structures to store data that are deep (subfolder within a subfolder within a folder), it would be nice if the geoprocessor could read through them all and return refernces to them based on the current folder. Before one jumps in to say that the Python os module can help, please remember that most users are not aware of os.walk, and trying to deal with ERI GRIDS (which are a folder themselves) presents a challenge even for experienced Python programmers. Why would ESRI want to do this? To make life easier for the majority of users out there with data in directory trees.
3. Expose numpy in the geoprocessor: The geoprocessor as is right now (9.3.1) uses the excellent numpy module to perform matrix algebra (think of raster manipulation). Yet, when one wishes to run numpy commands, one needs to manually read raster files with GDAL, import them as numpy arrays (default), perform operations, and translate back to raster. ESRI must have modules for dealing with this, and we want them. Why would ESRI want to do this? Right now, raster manipulation through Python is done outside the geoprocessor. Most people turn to open source tools to manipulate data, which leads to less and less users relying on ESRI for this. Why pay when free software will do it? The capability is there, and we need to access it too.
4. A faster starting geoprocessor: Right now, on my dual-core, 4GB RAM machine, creating the geoprocessing object takes a noticeably long time (in the orders of a few seconds). Why is that the case today? If the geoprocessor is written as C module, the bottleneck is somewhere else. Regardless, this needs to be address, especially for using an interactive shell or multiple parallel processes involving the geoprocessor. Why would ESRI want to do this? Simply because a faster startup time is time-saving and psychologically makes people feel better about the tools at hand.
5. Multi-core aware geoprocessor: Right now, ESRI operates on one core per command (on the desktop). Similarly, the geoprocessor is a one-core operation. Yet many operations can run in parallel, and indeed, many people on the ESRI forums try endlessly to achieve multi-core operations. This is not easy of course, and not many people know how to do it. Why would ESRI want to do this? Increase in speed for one, but most importantly, to introduce a way to achieve this for everyone that wants it. Right now, there are as many solutions to this as there are people needing it. One simple and good way to achieve this would allow for unification of effort, and some good documentation be written.
6. Exposure to the desktop application when run through it: If you remember the good old VBA days (which will not be included on any version after 10, beware), you could manipulate the application via scripting. You could add data to a map, reference the 3rd item on the table of contents, etc. We want a return to this regime, but this time with Python. Why would ESRI want to do this? Something will need to replace VBA, and what better language than the chosen one for geoprocessing scripting? Also, access to such functionality opens up development to a larger audience, which can help bolster the industry by itself.
7. Make every method return a Python list for multiple data: The cursor methods in ArcGIS right now return an enumeration object. While this is understandable (ESRI doesn’t want people manipulating the order of items or inserting items in the middle of the list), it is not to the best interest of the geoprocessing model to return both enumeration objects and Python lists. At the very least, the enumeration object should have more Pythonic methods, like append() to add an element in the end or pop() to remove the last item. Why would ESRI want to do this? Consistency. There was an attempt to become more Pythonic in 9.3, so why not keep up with the theme and be more consistent at the same time?

Do you have any suggestions to include in this list? Please leave a comment. Or if you think I am off my head and don’t know what I am talking about.

Related posts:

1. Understanding the Geoprocessor Programming Model part 1

The Describe Realm

As mentioned earlier, the geoprocessor gives us the ability to access or create data to perform various operations. These operation vary in type and content, but more often than not are performed on data we are not intimately familiar with. This can become a problem quite quickly, as we need to know the names of fields in the datasets, as well as other characteristics (the type of data we are dealing with, spatial references, geographical extent and other). The solution to this lack of knowledge comes through the describe object which provides users with methods to identify all unknown information beforehand. In the most basic form of usage, one uses the describe object on a dataset, accessing the bottom left box in the diagram, the Describe Object Properties. The Python code to achieve this follows:

import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

dataset = r"c:\data\testData.shp"
# The location of the data. This is usually
#   not hardcoded but discovered with List methods

dsc = gp.Describe(dataset)
# Create a describe object for the dataset

print dsc.DataType
# Print the data type of our dataset

print dsc.CatalogPath
# Print the path to the dataset

With the above simple example, we managed to read the Data Type and Catalog Path of a dataset. While this seems simple, it can become a very powerful tool, when on considers the possible return values (listed below).

As you can see, the return values as listed above correspond to different objects in our diagram (the boxes in the first image above). This means that it would be trivial for us to move between the first access we have into the Describe realm (through the Describe object) and move to any other box of our choice when we are not familiar with what possible data we have.

Retrieving Information about data: Feature Classes

FeatureClass Object Properties

The most commonly used types of data in GIS now seems to be feature classes, either standalone (in the form of shapefiles) or in a geodatabase. As such, we will use the example of a feature class to retrieve information about out data. This is of course not the only type of data one can use, so please refer to the image on the top for more information.

As you can see from the diagram on the left, feature classes have specific information associated with them, including what feature type they are of, their shape type, and the shape field name we used in part two of this series. We will dive right into it by trying to identify what shape type our feature class has, and what type of feature it is.

import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

dataset = r"c:\data\testData.shp"
# The location of the data. This is usually
#   not hardcoded but discovered with List methods

dsc = gp.Describe(dataset)
# Create a describe object for the dataset
# As we know the dataset is of type FeatureClass,
#   all the properties of the FeatureClass object
#   are available to us now.

print dsc.ShapeType
# Print what type of shape we have
#   (point, polygon, polyline, etc)

print dsc.FeatureType
# Print what type of features are stored
#   (simple, annotation, dimension, etc)

With the above code we can identify what type of shape we have and the type of feature stored. For possible values to these properties, please check this help file.

Table Properties

In the FeatureClass Properties object image above, on the bottom, you can see that there are also Table Properties and Dataset Properties listed. This means that when you have a feature class, you automatically gain access to those properties as well (as a feature class has them embedded). Therefore, we can use this to our advantage to further retrieve information about our feature class, like the name of all fields within the feature class. The example below shows you how to achieve this.

import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

data = r"c:\data\testData.shp"
# The location of the data. This is usually
#   not hardcoded but discovered with List methods

dsc = gp.Describe(data)
# Create a describe object for the data
# As we know the dataset is of type FeatureClass,
#   all the properties of the FeatureClass object
#   and inherited objects are available to us now.

dataFlds = dsc.Fields
# Retrieve a list of fields in the table of the
#   feature class as a Python list. Note the fields
#   are of Field Object type.

for fld in dataFields:
    print fld.Name

Dataset Properties

What he have done on the above example is run the ListFields() method mentioned in an older article (Using the Geoprocessor’s List Methods). Similarly, one can follow the Dataset Properties object (shown on your right) to access information like the extent of the data, or the spatial reference system. Don’t forget, both of those return an object, which has a corresponding box in the geoprocessor programming model diagram, so you need to treat them as such.

In this installment, I have shown you how one can use the Describe object the geoprocessor makes available to us to retrieve information about our data. While I did not provide complete examples of every single possible option available, hopefully by reading this post you are able to navigate your options better to achieve your goals.

As always, comments are encouraged and I am willing to answer any questions you may have.

Related posts:

1. Accessing Geometries using the Geoprocessor (Updated)

In this post I will finish covering the bottom left part of the model that deals with direct access to features and their geometry. In order to do that, we will discuss cursors, features and geometries.

Spatial Data Primer

Before we begin discussing deep access to data, we need to define what spatial data are. In essence, there are two types of spatial data, vector and raster data. An easy way to understand the difference is to imagine raster data as images, while vector data are defined by points, lines and polygons. What is special about spatial data is their dimension of space, or location. Raster data offer extensive coverage of an area, in which our spatial data offer a regular grid of cells that provide data. Space in this case is depended on the location of the cell in the raster file (and therefore implicit location). Vector files on the other hand contain explicit information about the location of things (providing coordinates). Data are then associated with the locations of features. We will be dealing with vector data mostly, so a further investigation is warranted.

In order to represent the world in spatial data, we choose a type of vector data we want to use (for example, choosing points to represent trees, lines to represent road networks, etc). Each of the features in the world get assigned two types of data: spatial characteristics (for a point, an x,y pairs, for a line, a series of points, etc.) and non-spatial characteristics (think of them as attributes). A file that contains all the trees in a neighborhood then includes a list of all the trees, with each tree having a location attribute, and a set of attributes that describe the tree (like type of tree, or age of tree, etc). Therefore, spatial data are defined as follows:

• A file dealing with a specific type of feature in the world.
  • The file has a list of features that all satisfy the requirement of belonging in that type of feature.
    • Each feature contains a spatial footprint for where it is located.
    • Along with location, each feature contains zero or more characteristics that are aspatial but refer to the feature itself.

Similarly, when one tries to access the geometry or the characteristics of a specific feature in our data, one has to first access the file we need, within the file, identify the right feature we need, and then eventually, within the feature find the geometry and/or the characteristics associated with it.

Accessing Data Using the Geoprocessor

From the Geoprocessor to Geometry

The image above shows the path one needs to travel within the Geoprocessor to access a Geometry. The cyan box on the left represents the commands available to the geoprocessor object, as discusses in the previous post. The commands that include the word Cursor have an arrow to their right, that show they can access rows, which is actually a representation of the spatial data file in Python. From there on, one can access a single row (the third box from the left), which can eventually lead to the Geometry object on the left. So in the geoprocessing world, in order to access a geometry (or spatial characterics of a feature), we need to do the following:

1. Create the geoprocessor object.
2. Call the appropriate cursor command to receive a rows object, which is a Python representation of the file we need to access.
3. Within the rows object, we call the Next() or NewRow() command to receive an individual feature, represented as a single row.
4. The row object allows us to directly access characteristics of the file, including the Geometry object. Please note that the characteristics are actually variables now, and not methods.

A question people often ask is how one knows which path to follow when they need to access a specific object (or box in the diagram). If you are not intimately familiar with the model, there is one method to follow: Identify the object you want, and then follow the arrows in reverse to find the methods that can get you there. Once you have those steps reverse them and you have a workflow. Remember, you will always need to end up in the cyan, geoprocessor object, as this is your only starting point.

From the Geoprocessor to Spatial File Access

There are three methods to access your spatial file, as shown in the diagram, explained below:

• InsertCursor: The insert cursor allows you to access a spatial file with the sole purpose of adding new features (or rows). Use this cursor to add more features to a file.
• SearchCursor: The search cursor allows you to access a spatial file with the sole purpose of reading features (rows). Use this cursor to read the characteristics of each feature.
• UpdateCursor: The update cursor allows you to access a spatial file in order to update values in each feature (row), or completely remove it. Use this cursor to update characteristics (including geometry) of a feature, or to remove the feature itself.

The required parameter for each of these cursor is the file you want to access, called InputValue in the diagram. Here is an example of how to use each cursor:

import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

fcWrite = gp.InsertCursor(r"c:\data\testWrite.shp")
# Create an insert cursor that points to the testWrite.shp file.
# The returned value fcWrite is actually a rows object.
# You are now able to add new features to the file.

fcRead = gp.SearchCursor(r"c:\data\testRead.shp")
# Create a search cursor that points to the testRead.shp file.
# The returned value fcRead is actually a rows object.
# You are now able to read the information about features from the file.

fcUpdate = gp.UpdateCursor(r"c:\data\testUpdate.shp")
# Create an update cursor that points to the testUpadate.shp file.
# The returned value fcUpdate is actually a rows object.
# You are now able to update values in features, of completely remove them from the file.

If you follow the above examples, you manage to escape from the geoprocessor object (the cyan box) and move to the rows object, the first yellow box on the left. Note that the rows object has different methods available depending on how you access it.

From the Rows Object to a Single Row

From the file to a single row

Now that you have access to the rows object, you can continue accessing the file by individual feature (or row). Depending on the Cursor method you used, you will have the following options available to you:

• SearchCursor Rows Object: This provides you with read-only access to the elements/features of your file. As such, there are two methods available: Next() and Reset(). As you may infer from their name, Next() will provide to you the next feature as a row object, while the Reset() method simply returns the pointed to the specific row in the beginning of the file.
• InsertCursor Rows Object: As the InsertCursor() method provides the ability to add new features in the file, the methods available are NewRow(), InsertRow() and Reset(). This creates some confusion, but it is easy to distinguish what happens when you understand the internals. NewRow() creates a new and empty row object in memory, which allows you to create your new feature. This row object remains in memory until one calls the InsertRow() function, which will save the row object to your file. The Reset() method functions as with the SearchCursor Rows object explained above.
• UpdateCursor Rows Object: As the UpdateCursor() provides you the most functionality, it comes with more methods: Next(), Reset(), UpdateRow() and DeleteRow(). The Next() and Reset() methods function the same way as with the SearchCursor Rows object. UpdateRow() is similar to the InsertRow() method of the InsertCursor Rows object. Given a row object, it will update the feature the cursor is set on. Similarly, the DeleteRow() method will delete the current row object the cursor points to.

Examples on how to use the commands follow:

Example 1: Writing a new feature

# Example 1: Create a new Row
import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

fcWrite = gp.InsertCursor(r"c:\data\testWrite.shp")
# Create an insert cursor that points to the testWrite.shp file.
# The returned value fcWrite is actually a rows object.

fcWrite.Reset()
# Reset the cursor to the rows object

fcRow = fcWrite.NewRow()
# Create a new row object in memory, saved in the fcRow variable

# Manipulate the row here.

fcWrite.InsertRow(fcRow)
# Save the row object back to our file

Example 2: Reading a feature

 # Example 2: Read the n feature of a file
 import arcgisscripting
 # Import the module provided by ESRI

 gp = arcgisscripting.create(9.3)
 # Create the geoprocessor object

 fcRead = gp.InsertCursor(r"c:\data\testRead.shp")
 # Create an insert cursor that points to the testRead.shp file.
 # The returned value fcRead is actually a rows object.

 fcRead.Reset()
 # Reset the cursor to the rows object

 fcRow = fcRead.Next()
 # Read the first row of the file and save it to the fcRow variable

 # Read the information for the feature here

# If you need to find a specific feature, you need to continue your search
# Use a while loop to achieve this
while fcRow:
    # Test whether this feature is the one you want
    # When you found the feature, you can use break to end the loop
    # If the feature is not the right one, keep trying
    fcRow = fcRead.Next()
 

Example 3: Updating and Deleting Rows

# Example 3: Updating and Deleting Rows
import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

fcUpdate = gp.UpdateCursor(r"c:\data\testUpdate.shp")
# Create an update cursor that points to the testUpadate.shp file.
# The returned value fcUpdate is actually a rows object.

fcUpdate.Reset()
# Reset the cursor to the rows object

fcRow = fcUpdate.Next()
# Receive the first row

# Loop through the rows until you find one to delete and one to update
while fcRow:
    # If this is the row that needs deleting, then
        fcUpdate.DeleteRow(fcRow)
        # Call the DeleteRow() method with the current row as a parameter
    # If this is the row that needs updating, then
        # Perform the update to the row
        fcUpdate.UpdateRow(fcRow)
        # Call the UpdateRow() method with the current row as a parameter

Manipulating Individual Features and Geometries

If you followed the above examples, then you are aware on how one can access an individual feature from a file. The question then remains on how one manipulates that row. Below are some explanations on how one can perform this.

Individual Features (row) and their Geometry

As with the Rows object, there are different versions of the Row object depending on who calls it. The two possibilities are:

• Row object derived from a SearchCursor: This object provides the GetValue() method that allows one to read values associated with a field. Along with the method, there is a collection of read-only variables that represent the fields/attributes the row represents.
• Row object derived from an InsertCursor or UpdateCursor: This object provides what the read-only version provides, with the variables representing fields/attributes being write-enabled as well. The SetValue() method is also provided to allow one to set new values to fields.

The Field Object

Before we continue on with the idea of geometries, we need to show how one can access the field names. Often we know the names of fields/attributes because we are familiar with the dataset or have access to the metadata. Another way to get familiar with the names of the fields though exists, and is through the ListFields() method available to the geoprocessor object. This command will return a Python list that includes all the attributes available to you, as Field objects. What this object provides to you is a set of variables that are read-only and make available to you information like the name, alias, domain, editable ability and more information about the field. Please have a look at the diagram on the right for a full listing of information included in a Field object.

# Example of finding out the fields available in a file
import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

fieldsList = gp.ListFields(r"c:\data\test.shp")
# Call the ListFields() method on the test.shp file
# The returned list is saved in fieldsList

for field in fieldsList:
# Loop through all the field
    print field.Name
# Print the name of the field

This is one way to also identify the field that includes the geometry. Unfortunately, depending on the type of file you have (or database for that matter), the name may change. There is a trick to find the actual name of the field that stores geometry, but it will be covered in another part. For now, here is how one can do it:

# Example of finding out the fields available in a file
import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

fcDsc = gp.Describe(r"c:\data\test.shp")
# Get the descriptor of the file
geometryField = fcDsc.ShapeFieldName
# Get the geometry(shape) field name from the description

As you now know what the name of the geometry field is, you can access is as mentioned before with the GetValue() method. Similarly, you can save a geometry using the SetValue() method. It is important to remember that geometries in the ESRI world are Geometry objects of multiple parts which are held in Array objects holding one or more Point objects. This means when you use the GetValue() method, you first receive a Geometry object. Within that object, you will call the GetPart() method to receive an Array object, which in turn holds a number of Point objects, which are called through the Next() method. An example follows to help you understand this.

# Example of accessing a Geometry of a feature / print geometry type
import arcgisscripting
# Import the module provided by ESRI

gp = arcgisscripting.create(9.3)
# Create the geoprocessor object

fcDsc = gp.Describe(r"c:\data\test.shp")
# Get the descriptor of the file
geometryField = fcDsc.ShapeFieldName
# Get the geometry(shape) field name from the description

srcCursor = gp.SearchCursor(r"c:\data\test.shp")
# Create a search cursor to the file

row = srcCursor.Next()
# Grab the first row

while row:
# Loop through the features/rows
rowGeom = row.Getvalue(geometryField)
# Grab the geometry of the row

print rowGeom.Type
# Print the type of the geometry

row = srcCursor.Next()
# Grab the next row

With this installment, you should have the knowledge to navigate the Geoprocessor Programming Model on the bottom, where accessor methods for files reside. Also, you should be able to access a Geometry object for every vector dataset you have available. In the next installment we will further navigate the programming model.

Note: Thanks to Eric for catching a Python error.

Related posts:

1. Accessing Geometries using the Geoprocessor (Updated)

A question often asked when people venture into the wonderful world of Python Geoprocessing with ESRI is how one can read the programming model they make available on their website. As it may not be easily interpreted when one begins programming, I will do my best to unpack it and explain how one can use it more effectively. All images presented here are extracts from the actual model presented by ESRI on their website.

Dual Usages for the Geoprocessor

The geoprocessor within Python can be used for two different purposes: to access tools provided by ESRI to manipulate data (the tools one sees in the ArcToolbox), and to programatically manipulate data using the underlying structures provided by ESRI. To further explain this process, a little understanding of how software works is needed. A software package usually involves multiple components. At the heart of software are basic operations and data models, the core of your software. This core deals with things like accessing and saving data, manipulating numbers and the like. This functionality is usually bundled together to create the tools the software produces. In the case of ArcMap, a tool like buffer could use components about reading and writing data, coordinate transformation routines, and probably some sort of functionality that allows manipulation of spatial data. To bring it back to the geoprocessor, and why it is so much more powerful than using the software by itself, what you get access to is not only the tools ESRI has created for you by compiling multiple elements, but you also get access to a lot of those elements yourself that allows you to write your own buffer tool for example.

Understanding what the Geoprocessor is

When one talks about the geoprocessor, one is referring to a Python object built by ESRI, that allows one to call many different functions to perform operations. As in most Python code, the definitions of the object reside in a module that is provided to you. As with any other module, in order to access it through Python you need to import it. This is done by simply typing import arcgisscripting in your Python code.

import arcgisscripting
# Imports the module provided by ESRI

In this case, the module provides a single command accessible to you called create. This exposed function allows one to instantiate the geoprocessor object. Simply calling the create() command is enough to return to you a geoprocessor object. You can also pass parameters to it, but the only one at the moment is the number 9.3, to create a geoprocessor object that returns values based on the 9.3 iteration of the programming model.

gp = arcgisscripting.create(9.3)
# Instantiates the geoprocessor and assigns the name gp to it

Beginning our Travel in the Geoprocessor Programming Model

Geoprocessor Object

The box on the right represents your geoprocessor. While it seems that these are the commands available to the arcgisscripting module, that is incorrect. Therefore, in the Geoprocessor Programming Model, one always enters through the cyan box, as seen on your right. There are few interesting things to note here:

1. There are a lot of names appearing on the list.
2. Each name has a symbol ahead of it.
3. Most name have a parenthesis to their right.
4. A few of those names also have more information on the very right.

Here is how one needs to interpret this part of the diagram. The symbols on the left define what the name represents.

Geoprocessor Variables

If there is a box connected to a dash, or two boxes, then the name represents a variable. What this means is that the name represents a value. There are a few options here:

1. Box on the left of the dash: This represents a built-in variable that is read-only. You as a user have no power to change these value, but can only read it. An example is the variable called MessageCount. The value represents how many messages the geoprocessor object has thus far.
2. Box on both sides of the dash: This represents a variable you can also write too. This is a value that the software can use, and you are allowed to overwrite it with your own value. An example is the OverwriteOutput variable, which allows you to tell the geoprocessor whether it has permission to overwrite existing files.

Example usages for geoprocessor variables is below (click to expand):

import arcgisscripting
# Import the module provided by ESRI
gp = arcgisscripting.create(9.3)
# Instantiate the geoprocessor object

gp.OverwriteOutput = True
# Set the OverwriteOutput variable to True

print gp.MessageCount
# Print the number of messages on the object

Geoprocessor methods

Boxes are not the only symbols that appear on the dashes. There is also an arrow, which represents a function or procedure you are allowed to call. What this means is that by using that name, one is telling the geoprocessor to perform an operation. Some operations also allow you to define variables to be used on the operations. For example, RemoveToolBox is an operation that will remove a Toolbox from the geoprocessor, as long as you can provide its name. There are two types of operations available to you, and to distinguish them one needs to see what appears to the right of the name. here are the possibilities:

1. Methods that do not return anything: A lot of functions do not return anything to the user, and you can see this by the lack of any more information to the right of the name of the function. They perform an operation the user asks for, but do not return something the user can work with, other than a message of their success or a preset value. ResetEnvironments() is an example of such a method.
2. Methods that return objects one can manipulate: Some of the functions in the list return a new object that the user can manipulate. These objects are often Python objects (like a Python List) that contain other ESRI objects, or sometimes an ESRI defined object directly that allows us to further manipulate what the software does. An example is the InsertCursor() method, that will return an object back called a rows object. The ListFields() method returns a Python List object populated with ESRI defined Field objects.

The methods that return an object or a collection of objects also have an arrow on their right linking them with the object they return. For example, all the cursor methods connect to the row object to allow the user to see what that object provides. Similarly, the list methods oftentimes have an arrow to connect them to other objects, like the field object. The special case is the method called CreateObject(), which allows you to create a large set of objects from scratch, like spatial reference, points, valute table, etc.

Example usages of methods follow:

 import arcgisscripting
 # Import the module provided by ESRI
 gp = arcgisscripting.create(9.3)
 # Instantiate the geoprocessor object

 gp.ResetEnvironments()
 # Reset all environments on the geoprocessor object

 tools = gp.ListTools()
 # Returns a Python list of all the tools available, save in the tools variable
for tool in tools:
print tool
# Loop through each tool and print it
 

In the net part of this series I will investigate the objects connected with cursors (meaning the rows object and all connected object, like the row object (singular) and geometry).

Geoprocessor Access to Tools

The geoprocessor object also allows users to call on tools that ESRI provides. Those tools are identical to the tools found in the ArcToolbox. A list of them, with a description, is available from ESRI here (note, it is a large PDF file). Note that the Python scripting syntax is not included in that file, but the help file contains a Python example for each tool. You will need to search for it yourself unfortunately, but the thing one needs to remember is that the tool name remains the same, and all parameters to be passed to it must be within parenthesis.

I hope you enjoyed this article and be sure to visit for the remaining parts.

Related posts:

1. Loading the Geoprocessor Safely

1. The method of invoking the geoprocessor is very similar for both, with the exception of an optional parameters when creating the object.
2. Geoprocessing methods do not return Python built-in data types but rather ESRI iterable objects.
3. Methods that used to return True/False values now return the Python True or False object rather than a string or 0/1.

The following table illustrates the differences:

gp = arcgisscripting.create()

gp = arcgisscripting.create(9.3)

# Create a list of available feature classes
fcs = gp.ListFeatureClasses()
# Iterate through said feature classes
fcs.Reset()
fc = fcs.Next()
while fc:
    # Do work here
    fc = fcs.Next()

# Create a list of available feature classes
fcs = gp.ListFeatureClasses()
# Iterate through said feature classes
for fc in fcs:
    # Do work here

# Identify the fields of a feature class
dsc = gp.describe(fc)
# Get the fields from the description
flds = dsc.Fields
# Iterate through the fields
flds.Reset()
field = flds.Next()
while field:
    # Do work here
    field = flds.Next()

# Identify the files of a feature class
dsc = gp.describe(fc)
# Get the fields from the description
flds = dsc.Fields
# Iterate through the fields
for field in flds:
    # Do work here

# Tell the geoprocessor to overwrite output by default.
gp.OverWriteOutput = 1

# Tell the geoprocessor to overwrite output by default
gp.OverWriteOutput = True

There is not list of commands that need the changes to the best of my knowledge. But a quick look through the Geoprocessor Programming Model for ArcGIS 9.2 can show which methods return an enumeration unit. In brief, ListDatasets, ListFeatureClasses, ListRasters, ListTables, ListWorkspaces, ListEnvironments, ListToolboxes, ListTools and all other methods that return an Array Object. For a complete listing of the changes, please read the ESRI document for Geoprocessor changes in 9.3.

Related posts:

1. Geoprocessing Iteration with Python

Geoprocessing in Python provides multiple methods that can help one iterate. Depending on the dataset types you have, you can access list of data easily and effortlessly. The following code will show you how this is done in version 9.3, although earlier versions are as easy to use as well.

In order to iterate a process, we need a list of files we will use, and then a series of commands to be run on those files. There are multiple ways to achieve this through the operating system (os and sys modules), but since we do have that pesky little problem of shapefiles having multiple files associated with them, or geodatabases being a single representation for the OS, we need a different solution. And here lies the power of the geoprocessor. It allows one to list data the way ESRI can recognize them: workspaces, (feature) datasets, feature classes, tables and rasters. So let’s begin.

Oftentimes, analysts store data in folders. If you are conducting a study in 10 different regions, it is often the case you save the data files for each region in a different folder. A folder structure like the following is common:

• \myStudy\
• \myStudy\region1\
• \myStudy\region2\
• \myStudy\region3\
• …

We therefore need a method that allows us to go into the master folder (\myStudy\ in this case) and read the files in each folder. To achive this, we will read the names of folders first. The command ListWorkspaces(widcard, type) can help us do that, by accepting a wildcard as a first argument (any combination of * can help narrow things down) and the type of workspace we want (from Access, Coverage, FileGDB Folder, SDE or ALL). An example follows:

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
masterFolder = r"path\to\folder" # Define the master folder
gp.workspace = masterFolder # Define the location to run the commands on
workspaces = gp.ListWorkspaces("*", "Folder") # List all FOLDERS in master workspace
for workspace in workspaces: # Iterate through the workspaces one at a time
print workspace # Print the workspace name
# Do you work here

At this point, you have a Python list object that holds the names of all the subfolders that you may want to use. The next step is to identify a list of all data within that folder. To achieve that, we use one of the following commands, depending on the type of data we have: ListDatasets(wildcard, type), ListFeatureClasses(wildcard, type), ListTables(wildcard, type) or ListRasters(wildcard, type). The wildcard is defined as above, but the type is different. Type actually refers to the type of listing. For datasets, the type is feature,TIN, raster, CAD or ALL. For feature classes the types are Annotation, arc, dimension, label, line, node, point, polygon, region, route or tic. Table types can be DBF, INFO or ALL. Similarly, raster types can be BMP, GIF, IMG, JP2, JPG, PNG, TIFF, GRID or ALL. We will contnue our example by searching for feature classes (shapefiles) in the folders we identified above.

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
masterFolder = r"path\to\folder" # Define the master folder
gp.workspace = masterFolder # Define the location to run the commands on
workspaces = gp.ListWorkspaces("*", "Folder") # List all FOLDERS in master workspace
for workspace in workspaces: # Iterate through the workspaces one at a time
gp.workspace = workspace # Change to the new workspace
fcs = gp.ListFeatureClasses() # No parameter defaults to all
for fc in fcs: # Iterate through each file in the folder now
print fc # print the feature class
# Do individual file work here

As you can see we managed to begin with a master folder and continue to analyze individual files within all subfolders of the initial master folder. This is all done in a few lines of code, minimize complexity and confussion for us. In less that 6 lines of Python code you can begin processing each file in your computer quickly and easily.

While listing of other types of files has not been shown, it is quite easy to do by using the appropriate command (as mentioned above). Dealing with databases is similar, as you can imagine and use the database as your master folder. Do not forget that “subfolders” in the database are called datasets, so you will need to use ListDatasets() instead of ListWorkspaces().

I hope this will be useful to some of you out there.

Related posts:

1. Using the geoprocessor’s List* Methods

ESRI apparently recognizes this issue, and has produced a Technical Article for it (#22668) that discusses the issue when people use enterprise geodatabases. I was running up the same problem though myself when dealing with a personal geodatabase (Jet Engine). Here is what I was trying to do (in Python):

1. Read the results of the OD Matrix (origin destination matrix)
2. When one record is read, do the following:
   1. Extract information about source ID and destination ID
   2. Using MakeFeatureLayer with a query, built two layers (one for origins, one for destinations)
   3. Run a network Route solution based on the two files
   4. Store the resulting route into a feature class collecting them all

Now, while this may sound simple (and also makes people wonder why ESRI does not give the option of presenting the actual route with the OD Matrix command), there are problems. What I noticed was that the results of MakeFeatureLayer were inconsistent through the loop. Always, when reaching a specific OD Matrix record, the command returned an empty feature layer. This was happening 48 times over my 393 routes, so it was bothering me.

Searching through the excellent ESRI Support site,  I found a Forum Posting describing similar problems, which also referenced Technical Article #22668 (link above). The technical article though mentions that the problem is only exhibited with enterprise level geodatabases (in essense, the memory the command has to store these is limited, so it saves things to the database, which the command isn’t aware of). I thought I should give it a try, and to my amazement, it worked wonders. I used the value they recommended to change and I had only 8 unsolvable routes (6 times less than before!). Increasing the value slightly yielded only 6 unsolvable routes, so this seems promising.

The details on what one is to do are mentioned in the technical article, and recreated here for your convenience. Please remember to backup before these changes, and don’t forget that messing with the registry can mess with your operating system.

1. Use regedit and open the HKEY_CURRENT_USER registry entry.
2. Navigate to the Software key
3. Expand the ESRI Key
4. Open the Geodatabase Key (if it does not exist, simply right-click on ESRI and create a new one)
5. Expand the Settings Key (or create it if not present)
6. Add a DWORD value called SelectionThreshold if not already present.
7. Enter a new value, in decimal base, of 5000 (ESRI recommendation, I used 7000 myself)
8. Close regedit and you are done.

I hope someone else finds this useful out there.

Related posts:

1. Geoprocessing Iteration with Python

Geometry: A geometry in this case is the geographical representation of an object as a point, or line, or polygon.
Feature Class: Many people get confused by the term. In essence, if you are in the shapefile world, this would be your shapefile. If you are using geodatabases, a feature class is a collection of objects. In both cases, it is a collection of features.
Feature: A feature is the digital representation of an object in the real world using a vector-based data type. For example, a tree can be a point, a road can be a line, countries would be polygons. Each tree would be a feature, as would each roads, etc. Features have both a spatial representation as well as an attribute representation (think of the object in the world and its properties).
Cursor: A cursor is an object that provides us access to a feature class. In other worlds, it is our way to communicate with a feature class through our programming language (Python). There are three types of cursors: Search (only reads feature classes), Insert (only adds features) and Update (can update or delete a feature).

Accessing geometries then, from the above definitions, would require us to (1) create a cursor to a feature class, (2) find the feature we want to read the geometry of, and (3) read the actual geometry. In reverse order, you may want to think of it this way: a geometry belongs to a specific feature, which belongs to a feature class, which is accessible through a cursor. Similarly, our Python code would do something similar, as shown below (only reads a geometry object):

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
featureClassToRead = r"path\to\file" # Define the feature class to read
searchCursor = gp.SearchCursor(featureClassToRead) # Create a SEARCH cursor
feature = searchCursor.Next() # Grab the first feature (row) from the feature class
featureGeometry = feature.shape # Grab the feature's geometry object 

After importing (line 1) and initializing (line 2) the geoprocessor, we create a search cursor (line 4) that pointed to our feature class. Within the cursor, we requested the first feature (line 5) and proceeded retrieving the feature’s geometry (line 6).

All other cursors will use a very similar approach. The difference will be line 4, in which we run the method that creates a cursor. If there is a need for an Insert cursor, you would run the InsertCursor command, or the UpdateCursor if you need the update cursor. With each cursor, you grab the first feature (row) and then manipulate it by accessing its individual variable (one of which is the geometry, called Shape).

Another important element is how one accesses the geometry. While there is no problem accessing the geometry by requesting the .shape variable present in each feature, this changes depending on your use of a shapefile or a geodatabase. To solve this problem, you can use the feature class description, and retrieve its .ShapeFieldName parameter. This is shown below:

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
featureClass = "path\to\file" # Define the feature class to describe
description = gp.Describe(featureClass) # Call the Describe function
shapeFieldName = description.ShapeFieldName # Grab the parameter and save it
searchCursor = gp.SearchCursor(featureClass) # Create a new search cursor
feature = searchCursor.Next() # Grab the first feature
featureGeometry = feature.GetValue(shapeFieldName) # Grab the feature's geometry

You would  now receive the geometry object, as with the sample code above, independent of the type of file you are reading.

Of course, we rarely need to read only the very first feature of a feature class. An easy method to read each feature is using a while loop, as follows:

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
featureClass = "path\to\file" # Define the feature class to describe
description = gp.Describe(featureClass) # Call the Describe function
shapeFieldName = description.ShapeFieldName # Grab the parameter and save it
searchCursor = gp.SearchCursor(featureClass) # Create a new search cursor
feature = searchCursor.Next() # Grab the first feature
while feature: # Create a loop that runs while we have features
# Work with the feature here
feature = searchCursor.Next() # Grab the next feature in feature classs

This take advantage of the fact that the geoprocessor will return a None object when we reach the end of the feature class, which would let the loop know we are done.

Related posts:

1. Understanding the Geoprocessor Programming Model part 2