Geography 293: Geospatial Analysis with Python

Chapter 8 covers manipulating spatial and tabular data. You may want to briefly refamiliarize yourself with “for” and “while” loops before reading this chapter. It would also be wise to familiarize yourself with Structured Query Language (SQL).

Important details to remember:

– Once a script creates an exclusive lock, the lock will continue to exist until the script releases the lock.

Terms

cursor – is a database technology term for accessing a set of records in a table. It works the way list functions work.

search cursor – retrieves rows

insert cursor – inserts rows

update cursor – updates and deletes rows

rows – records in a table

with statements – Good to use when you have two operations that you want to run as a pair

data locks – prevent multiple processes from changing the same table at the same time in two different applications. (includes shared lock and exclusive locks)

shared lock – is applied when a table or dataset is accessed

exclusive lock – are applied when changes are being made to a table or dataset

parsing – splitting the fully qualified name for a dataset into its components

Table 8.1 (pg 248) describes the cursor methods of each class

Chapter 7 reviews the types of errors you can come across while scripting and the most common procedures used to solve them.

Python IDEs have built-in error checking processes that will alert you of any known errors in the script. If you read the error carefully, you may be able to fix the error yourself, without seeking further help. You should keep indentation and spacing consistent in your scripts or you may produce an error. Syntax errors may be caught as they appear in Spyder and PyCharm. Spyder will display a small red x which opens a pop-up window for syntax errors that appear as you type. Spyder can catch other types of errors such as inconsistent indentation. In PyCharm you can produce a general report to see what types of errors are in your script by clicking Code (at the top of the menu) > Inspect code and then selecting the current script.

There are multiple ways to debug your code. The main ways include carefully reading error messages, adding print messages to your script, selectively commenting out code, or using a python debugger. Debugging will not tell you why a script failed, but it will tell you the line in which it failed. (See pg 224-234 for reference of these techniques.)

You can help handle exceptions by using a “try-except” statement (see pg 237). You can use these to produce text that can help you understand what went wrong. An example for a tuple is seen below:

except (ZeroDivisionError, ValueError):
print (“Your entries were not valid.”)

Using a Python debugger (pg 229):
1. Check for syntax errors then save the script.
2. Set breakpoints, allowing you to pause running the script at specific lines.
3. Run the script with a debugger.
4. Walk through the script while looking out for values of important variables.

Syntax errors – errors involved with spelling, punctuation, indentation, or other conventions of the Python language. If you click Run > Check Module and a syntax error appears, a red bar will appear on the line of which the error resides.

Excepts – Events that have no real error but are a result of the data and the script run. An example of this would be to write a script to produce a count of the feature classes in a feature class list. If there are zero feature classes in the list then the code may run perfectly fine but produce no output. This is not truly an error as the code has run correctly.

commenting out code – Placing a double hashtag (##) before lines of code helps cancel out that line in order to isolate parts of the script. This helps find errors in the script.

breakpoints – The code will run until it reaches a breakpoint. They are good for debugging scripts. To set a breakpoint, right-click a line of code and select Set Breakpoint. This will highlight the line yellow.

This chapter took a lot longer than the others but was probably the most engaging as you will finally be able to understand the concepts behind Python scripting. This chapter focuses on the fundamentals of Python scripting so that you will be able to make your own scripts. It would be wise to really pay attention to the chapter and to use it as a reference when creating your own code. Below is only some of the information provided by the chapter.

Python uses several different data types in its language. Data types determine what type of values can be held by an object. Some are the most common are strings, numbers, Booleans, and lists. Strings are usually text but can include numbers and other types of characters as well. Numbers can appear as integers (whole numbers) or floats (numbers with a decimal point). Boolean values are either noted as True or False and are useful for establishing or evaluating conditions. Lists are just as they sound and can contain a series of numbers, text, or other values.

You want to be careful when naming objects in Python as there are a number of ways of producing an error with a name (see pg 91-92). The chapter encourages consistency with how you name objects. One of the prefered methods is utilizing snake_case where each word is uncapitalized and instead of using spaces, underscores are used between each word to increase legibility. Python has several built-in functions and an error will result if you try to name an object with the same name as a function.

Furthermore, when writing scripts it may be wise to includes comments to provide context for the script. You can add comments by writing a line preceded by a number sign (#). Any text that comes after the number sign will not be considered code. An example of this could be (# Create output feature class) with the lines of code to create an output feature class found below.

Table References:

Table 4.1. Common operators for integers and floats in Python (pg. 89)

(Examples: * stands for multiplication and ** stands for exponent)

Table 4.2 Built-in functions in Python (pg. 102)

(Example: asbs(x) with “x” being some number will return the absolute value of that number)

Table 4.3 Comparison operators in Python

(Examples: == means equal to with != means not equal to)

Chapter 3 gives the rundown on how geoprocessing works in ArcGIS Pro with details on tools, models, and scripting. To complete some form of geoprocessing you must input some form of data (a feature class, raster, or table) into a geoprocessing tool (often with additional parameters) to produce an output.

Just like in ArcGIS Mapper, ArcGIS Pro contains toolboxes with tools inside them. You can locate geoprocessing tools in a variety of ways but you must first open the geoprocessing pane to do this. To do this navigate to Analysis Tab and click Tools. You can now search for tools by name and add frequently used tools to a list of favorites. Recently used tools will also show up in the favorites. When searching for tools the description of what the tool does will also appear. By right-clicking a tool and selecting “Batch” you can run the tool to produce multiple outputs with similar parameters.

Information about other types of tools is mentioned in the chapter. Most tools are built into the program but you can also utilize script tools made from Python code and model tools created using ModelBuilder. You can make your own model processes by opening ModelBuilder and creating your own models. The easiest way to open ModelBuilder is to navigate to the Analysis tab and click on the ModelBuilder button. This opens a window into which you can input data and tools and create connections between them. ModelBuilder is similar to Python coding but is more limited. It is still good to become familiar with ModelBuilder as it essentially allows you to create scripts visually rather than with text.

        Chapter 6 took around an hour to read. It mostly handles how to confirm that data exists, creating lists to see where data is, and creating lists for specific data (ex. Isolating shapefiles). You can also use python scripting to describe the characteristics of a feature class, shapefile, raster, or table. An example of this would be to determine if a shapefile is a polyline through a Describe ( ) function. Using a script to produce a list can help later on when using batch processing. 

Key terms:

System paths – the paths recognized by the Windows operating system

Catalog paths – the paths that only ArcGIS Pro recognizes

List comprehension – used to create lists from existing files in a variety of ways

Wildcard – All functions have a wildcard parameter (*), which defines a name filter. A wildcard is a symbol that represents one or more characters. 

 

What key similarities and differences are there between Describe ( ) and da.Describe ( ) functions?

For the Describe ( ) function, the properties of the object are dynamic. For the da.Describe ( ) function, the keys of the dictionary are dynamic. 

 

This chapter was alright compared to the others but there are definitely a few concepts I do not quite understand yet.

ArcPy is built into ArcGIS Pro if you are using IDLE. Other IDE’s like Spyder requires extra steps to access ArcPY. 

Key Terms:

Class- are used to create objects which have properties and methods

Environment- Hidden parameters that influence how a tool is run

Hard-coded- the parameters are not set as variables but use the values assigned to them directly. 

Namespace- a system to make sure all the names are unique and can be used without any conflicts

nontool function- A function in Python performs a specific task. ArcPy provides several functions that are not tools, otherwise known as nontool functions. Nearly all  geoprocessing tools in ArcGIS Pro or functions in ArcPy, but not all ArcPy functions are geoprocessing tools in ArcGIS Pro.

Package- a collection of modules, functions, and classes that adds functionality to Python

Property- a workspace is an example of a property that an object has

toolbox alias- tools may share the same tool name so they require a specific tool alias when applied in ArcPy (ex. Clip_management)

tool label- a tool name with no spaces (ex. AddField [as in ArcPy])

tool name- a tool label that contains spaces (ex. Add Field [in Data Management toolbox])

factory code (of a coordinate system)/well-known ID (WKID)- A series of numbers used to represent a name. WKIDs are often used for map projections because their names can be written out in different ways but not always be recognized by Python.

Workspace- provides a default location for the files you will be working with, such as inputs and outputs of geoprocessing tools.

 

Review Questions:

Explain the relationship between the geoprocessing tools in ArcGIS Pro and the functions in ArcPy?

ArcPy must be imported first in order to use its geoprocessing tools. ArcPy makes Python scripting easier and more powerful. 

What exactly is the difference between required and optional parameters of geoprocessing tools, and how does this impact writing code to use these tools in a script?

Required and optional parameters are just as they sound. Required parameters are required for the script to run properly while optional parameters are not. There are a variety of ways to write the optional parameters in your script without them actually doing anything. This can be done by writing “”, “#”, or None in its place. This will leave the optional parameters at their default values.

Give an example of the use of variables for parameters instead of hard-coding values to make your code more versatile?

You could write as follows:

import arcpy

arcpy.env.workspace = “C:/Data”

infc = “streams.shp”

clipfc = “study.shp”

outfc = “results.sph”

arcpy.Clip_analysis(infc, clipfc, outfc)

If you are writing scripts or tools to be shared with others it is wise to use variables instead of hard-coding. Variables also make it easier to reuse part of your code.

Explain some of the uses of the “Result” object: 

ArcPy returns the output of a tool as a Result object. The result object can consist of a string, number, or a Boolean value. (See page 163)

Why are classes used as input parameters for geoprocessing tools?

ArcPy classes are often used as shortcuts for tool parameters that would otherwise have a more complicated equivalent. They are often used to avoid using lengthy strings.

Explain how ArcGIS Pro is licensed and how this impacts handling licensing when writing scripts.

Licensing effects which geoprocessing tools can be used by the user.

 

 

This chapter is not too vigorous, but I will definitely have to refer back to it if I want to download Spyder or PyCharm down the line.

Chapter 2 focuses on the various Python editors and what makes each one unique. A Python editor, otherwise known as integrated development environments (IDEs) is used to write, test, and fix code. Python comes with a built-in editor known as IDLE, but Spyder and PyCharm are also recommended IDEs that must be installed separately. This chapter explains how to open and make a shortcut for IDLE as well as directions for installing Spyder and PyCharm, which require some extra steps. The editors may look different but the code works for all of them in the same way. In other words, you do not need to change how you write your code when using another editor. Both Spyder and Python are free and are cross-platform, so they will work on different computer systems. In summary, Spyder is easier to use, but PyCharm has more features within the user interface. (PyCharm also has an optional dark theme.)