The new ESRI Support website

The ESRI family of websites have been seeing a lot of new updates as of late. The screenshot on the right is of the support site that ESRI offers, which hadn’t changed in quite a while. Notice the changes, including the lack of links to the web help (which has proven so helpful in the past to show people help files), and no direct link to the forums prominent in the page (it is located on the left-hand side navigation bar). Instead, people are directed to the new ESRI Resources site, which appears to be the new organization of resources for ESRI products. It is nice to see ESRI trying to separate the official support requests from the support resources, even though I am not sure which design will be more useful to the majority of users.

The new ArcGIS Resource Center

The new resources page, for those that did not notice it, is no longer on the esri.com domain, but rather moves to the arcgis.com domain we discussed in the previous post. In the meantime, arcgis.com is still under construction, which the “Coming soon” phrase over an oblique image of the earth. If anyone has any more insights to this, please do share them on the comments section below.

Related posts:

1. ArcGIS.com now online

ArcGIS.com Online Screenshot from 03/23/10

ArcGIS.com is now online, with a new look and design, and everyone’s favorite web phrase: “Coming Soon…“. A little bit of searching around the internet reveals some information from Vector1Media regarding a cloud-version of ArcGIS in collaboration with Amazon, providing on-the-go GIS functionality.

This is a dramatic change from the previous functionality of the ArcGIS.com domain, which redirected to the esri.com section about ArcGIS. The Internet Archive has a number of older snapshots available here.

Note that the site is currently only available to ESRI employees, that may actually have to request access too. Curiously, the http://maps.arcgis.com/ domain mentioned in this ESRI Blog entry redirects to the same page, even though the screenshots seem to differ. This of course will be different from ArcGIS Online, which is available now in Beta.

Related posts:

1. ESRI Updating Website ahead of new release

The ArcToolbox

The ArcToolbox of ArcGIS 9.3

The ArcToolbox is a collection of tools provided by ESRI that achieve a number of geoprocessing operations. They are logically organized in themes based on the functionality, and are available to use without any customization. These operations can be very powerful and achieve great results. As you can see from the image on the right, there is a number of tools available to you to use. Depending on the extensions you have installed, you will see some different results, but the Geocoding Tools we mentioned above are available with any kind of license. To use any of these geoprocessing tools, you simply double-click on its name, and a window will pop up asking you for the required inputs and outputs. Pay attention to all the options available to you, and make use of the help button to clarify what the operation does, and how it is achieved.

The Model Builder

The Model Builder Window

The Model Builder is a wonderful program written by ESRI that allows you to graphically define geoprocessing operations. Using a simple tool, a user can drag and drop operations and data on a canvas, and connect them with directional arrows (signifying direction of data). On the example on the right, we follow the Address Geocoding operation. There are three data inputs for the operation, including an address locator, address fields to be used, and finally the input table with the addresses. There is one more data element in the picture, the output feature, on the far right. Note that all data are represented by ovals. The operation, which is in the middle, receives data from all the arrows pointing to it, and returns data to the arrow pointing outwards. Notice that the operation is in a rectangular box rather than an oval, to distinguish it.

The Command Line

The Command Line Window

The command line window is one more way to achieve geoprocessing when one is using ArcGIS. The window simply allows one to type commands, an hit enter to execute them. Some users are more familiar with this interface, and like the speed of which you can type operations. Typing a few letters in the top box will give you a list of all commands starting with those letters. Pressing space will bring the tooltip shown in the top of the image on your right. As you can see, it tells you what the tool expects to run. In this case, it expects an input table (in_table), an address locator (address_locator), the address fields (in_address_fields) and where to save the output (output_feature). Optionally, you can set the out_relation_type (notice the curly braces this time).

Python Scripting

Python Scripting Geocoding

The perhaps most complex way to perform geocoding is done through Python scripting. The example on the right actually presents geoprocessing code. What happens when one uses Python Scripting to create geoprocessing operations is similar with the command line option talked about earlier, with some slight changes. Python scripting gives you the option to interact with the data deeper. What this means is that the user can now access an individual point by latitude and longitude. And since all vector spatial data are built from points, this means you have deep access to any spatial data using Python (to see more about Accessing Geometries using the Geoprocessor click here). Also, as Python is a general use scripting language, you can find reusable code by other users (in the form of modules) to achieve things like reading Excel files (click here for an overview).

And that is what geoprocessing is. Operations that work on spatial data. In the ArcGIS realm, that means either using ArcToolbox, the Model Builder, the Command Line interface or creating your own Python Scripts to achieve your desired results.

Related posts:

1. Understanding the Geoprocessor Programming Model part 1

1. A free geoprocessor: While this is unlikely to happen in the foreseeable future, there are elements of the geoprocessor that can be made freely available, either as a separate Python module or through a license check for parts of the objects. Specifically, I am referring to some of the List* methods (ListRasters(), ListFeatureClasses(), etc.) that do not require deep access to data (this would include ProductInfo(), validators, and hopefully even the Describe object). Why would ESRI want to do this? To ensure that limited functionality is available to everyone independent of cost. Why restrict people from seeing the power of the geoprocessor? Let them join the fun, and perhaps become customers too.
2. Path traversal options: The List*() methods come in the list again, with a slight change. Since many users use directory structures to store data that are deep (subfolder within a subfolder within a folder), it would be nice if the geoprocessor could read through them all and return refernces to them based on the current folder. Before one jumps in to say that the Python os module can help, please remember that most users are not aware of os.walk, and trying to deal with ERI GRIDS (which are a folder themselves) presents a challenge even for experienced Python programmers. Why would ESRI want to do this? To make life easier for the majority of users out there with data in directory trees.
3. Expose numpy in the geoprocessor: The geoprocessor as is right now (9.3.1) uses the excellent numpy module to perform matrix algebra (think of raster manipulation). Yet, when one wishes to run numpy commands, one needs to manually read raster files with GDAL, import them as numpy arrays (default), perform operations, and translate back to raster. ESRI must have modules for dealing with this, and we want them. Why would ESRI want to do this? Right now, raster manipulation through Python is done outside the geoprocessor. Most people turn to open source tools to manipulate data, which leads to less and less users relying on ESRI for this. Why pay when free software will do it? The capability is there, and we need to access it too.
4. A faster starting geoprocessor: Right now, on my dual-core, 4GB RAM machine, creating the geoprocessing object takes a noticeably long time (in the orders of a few seconds). Why is that the case today? If the geoprocessor is written as C module, the bottleneck is somewhere else. Regardless, this needs to be address, especially for using an interactive shell or multiple parallel processes involving the geoprocessor. Why would ESRI want to do this? Simply because a faster startup time is time-saving and psychologically makes people feel better about the tools at hand.
5. Multi-core aware geoprocessor: Right now, ESRI operates on one core per command (on the desktop). Similarly, the geoprocessor is a one-core operation. Yet many operations can run in parallel, and indeed, many people on the ESRI forums try endlessly to achieve multi-core operations. This is not easy of course, and not many people know how to do it. Why would ESRI want to do this? Increase in speed for one, but most importantly, to introduce a way to achieve this for everyone that wants it. Right now, there are as many solutions to this as there are people needing it. One simple and good way to achieve this would allow for unification of effort, and some good documentation be written.
6. Exposure to the desktop application when run through it: If you remember the good old VBA days (which will not be included on any version after 10, beware), you could manipulate the application via scripting. You could add data to a map, reference the 3rd item on the table of contents, etc. We want a return to this regime, but this time with Python. Why would ESRI want to do this? Something will need to replace VBA, and what better language than the chosen one for geoprocessing scripting? Also, access to such functionality opens up development to a larger audience, which can help bolster the industry by itself.
7. Make every method return a Python list for multiple data: The cursor methods in ArcGIS right now return an enumeration object. While this is understandable (ESRI doesn’t want people manipulating the order of items or inserting items in the middle of the list), it is not to the best interest of the geoprocessing model to return both enumeration objects and Python lists. At the very least, the enumeration object should have more Pythonic methods, like append() to add an element in the end or pop() to remove the last item. Why would ESRI want to do this? Consistency. There was an attempt to become more Pythonic in 9.3, so why not keep up with the theme and be more consistent at the same time?

Do you have any suggestions to include in this list? Please leave a comment. Or if you think I am off my head and don’t know what I am talking about.

Related posts:

1. Understanding the Geoprocessor Programming Model part 1

Related posts:

1. Python Geoprocessing in ArcGIS 9.2 vs. 9.3

1. The method of invoking the geoprocessor is very similar for both, with the exception of an optional parameters when creating the object.
2. Geoprocessing methods do not return Python built-in data types but rather ESRI iterable objects.
3. Methods that used to return True/False values now return the Python True or False object rather than a string or 0/1.

The following table illustrates the differences:

gp = arcgisscripting.create()

gp = arcgisscripting.create(9.3)

# Create a list of available feature classes
fcs = gp.ListFeatureClasses()
# Iterate through said feature classes
fcs.Reset()
fc = fcs.Next()
while fc:
    # Do work here
    fc = fcs.Next()

# Create a list of available feature classes
fcs = gp.ListFeatureClasses()
# Iterate through said feature classes
for fc in fcs:
    # Do work here

# Identify the fields of a feature class
dsc = gp.describe(fc)
# Get the fields from the description
flds = dsc.Fields
# Iterate through the fields
flds.Reset()
field = flds.Next()
while field:
    # Do work here
    field = flds.Next()

# Identify the files of a feature class
dsc = gp.describe(fc)
# Get the fields from the description
flds = dsc.Fields
# Iterate through the fields
for field in flds:
    # Do work here

# Tell the geoprocessor to overwrite output by default.
gp.OverWriteOutput = 1

# Tell the geoprocessor to overwrite output by default
gp.OverWriteOutput = True

There is not list of commands that need the changes to the best of my knowledge. But a quick look through the Geoprocessor Programming Model for ArcGIS 9.2 can show which methods return an enumeration unit. In brief, ListDatasets, ListFeatureClasses, ListRasters, ListTables, ListWorkspaces, ListEnvironments, ListToolboxes, ListTools and all other methods that return an Array Object. For a complete listing of the changes, please read the ESRI document for Geoprocessor changes in 9.3.

Related posts:

1. Geoprocessing Iteration with Python