michalisavraam.org blog

There are multiple problems analysts face when they have to deal with processing multiple data files. There is the issue of identifying similarities and commonalities in files, and then of course how to automate the processing so they don’t have to run a program multiple times with the same parameters of separate files. In the world of ESRI’s GIS analysis, this can be performed quite easily with the help of Geoprocessing, either in Python or the Model Builder. Below is sample code that allows the iteration over a number of datasets.

Geoprocessing in Python provides multiple methods that can help one iterate. Depending on the dataset types you have, you can access list of data easily and effortlessly. The following code will show you how this is done in version 9.3, although earlier versions are as easy to use as well.

In order to iterate a process, we need a list of files we will use, and then a series of commands to be run on those files. There are multiple ways to achieve this through the operating system (os and sys modules), but since we do have that pesky little problem of shapefiles having multiple files associated with them, or geodatabases being a single representation for the OS, we need a different solution. And here lies the power of the geoprocessor. It allows one to list data the way ESRI can recognize them: workspaces, (feature) datasets, feature classes, tables and rasters. So let’s begin.

Oftentimes, analysts store data in folders. If you are conducting a study in 10 different regions, it is often the case you save the data files for each region in a different folder. A folder structure like the following is common:

• \myStudy\
• \myStudy\region1\
• \myStudy\region2\
• \myStudy\region3\
• …

We therefore need a method that allows us to go into the master folder (\myStudy\ in this case) and read the files in each folder. To achive this, we will read the names of folders first. The command ListWorkspaces(widcard, type) can help us do that, by accepting a wildcard as a first argument (any combination of * can help narrow things down) and the type of workspace we want (from Access, Coverage, FileGDB Folder, SDE or ALL). An example follows:

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
masterFolder = r"path\to\folder" # Define the master folder
gp.workspace = masterFolder # Define the location to run the commands on
workspaces = gp.ListWorkspaces("*", "Folder") # List all FOLDERS in master workspace
for workspace in workspaces: # Iterate through the workspaces one at a time
print workspace # Print the workspace name
# Do you work here

At this point, you have a Python list object that holds the names of all the subfolders that you may want to use. The next step is to identify a list of all data within that folder. To achieve that, we use one of the following commands, depending on the type of data we have: ListDatasets(wildcard, type), ListFeatureClasses(wildcard, type), ListTables(wildcard, type) or ListRasters(wildcard, type). The wildcard is defined as above, but the type is different. Type actually refers to the type of listing. For datasets, the type is feature,TIN, raster, CAD or ALL. For feature classes the types are Annotation, arc, dimension, label, line, node, point, polygon, region, route or tic. Table types can be DBF, INFO or ALL. Similarly, raster types can be BMP, GIF, IMG, JP2, JPG, PNG, TIFF, GRID or ALL. We will contnue our example by searching for feature classes (shapefiles) in the folders we identified above.

import arcgisscripting # Import the geoprocessor capabilities
gp = arcgisscripting.create(9.3) # Create the geoprocessor object
masterFolder = r"path\to\folder" # Define the master folder
gp.workspace = masterFolder # Define the location to run the commands on
workspaces = gp.ListWorkspaces("*", "Folder") # List all FOLDERS in master workspace
for workspace in workspaces: # Iterate through the workspaces one at a time
gp.workspace = workspace # Change to the new workspace
fcs = gp.ListFeatureClasses() # No parameter defaults to all
for fc in fcs: # Iterate through each file in the folder now
print fc # print the feature class
# Do individual file work here

As you can see we managed to begin with a master folder and continue to analyze individual files within all subfolders of the initial master folder. This is all done in a few lines of code, minimize complexity and confussion for us. In less that 6 lines of Python code you can begin processing each file in your computer quickly and easily.

While listing of other types of files has not been shown, it is quite easy to do by using the appropriate command (as mentioned above). Dealing with databases is similar, as you can imagine and use the database as your master folder. Do not forget that “subfolders” in the database are called datasets, so you will need to use ListDatasets() instead of ListWorkspaces().

I hope this will be useful to some of you out there.

(No Ratings Yet)
 Loading ...

Related posts:

1. Using the geoprocessor’s List* Methods
2. Python Geoprocessing in ArcGIS 9.2 vs. 9.3
3. Understanding the Geoprocessor Programming Model part 3
4. Understanding the Geoprocessor Programming Model part 2
5. Accessing Geometries using the Geoprocessor (Updated)