Geography 293: Geospatial Analysis with Python

 Chapter 3: 

• The chapter gets started with geoprocessing and how to use it in parallel with the Python Window. 
• Getting the Geoprocessing pane to open and understanding how it works was easy for me as I’ve used some of the tools for GIS 191
• I came across three different types of tools: 
  • Built-in tools: Built with compiled programming languages like C++ For example The Clip Tool 
  • Script Tools: Built using scripting languages like Python. For example Mean Center. 
  • Model Tools: Built using models like Model Builder. For example My Model  
• The dialog box is also something that I’m well acquainted with due to GIS 191
• I also used a map from one of the GIS 191 databases to try the “Clip” and “Intersect” tools
• I also came across the Environment Settings in the Analysis Tab, which can be specified for better efficient usage of the tools. 
• These environment settings can be coded, or used through individual tools as well \
• I also came across Batch Processing, which is running the same tools multiple times with different parameters. Dynamic naming of the tool’s output datasets must receive special consideration so that a unique name is used for every iteration of the batch tool. Batch tool outputs are dynamically named using the %Name% variable in output parameters.t
• Batch mode can also take in multiple raster variables and determine multiple statistics in one go 
• Then I used the Model Builder. I had used this before for GIS 191. Unfortunately, the dataset I was using did not have variables of different kinds, which posed a problem in connecting the shapes, I believe, But I did not fret about it as I had done this before. 
• For scripting, I noticed that some of the ways, the code for Clip(Analysis Tool) worked had been changed. 
• I just had to mention the input layers when I used Clip and the output name, and it gave back the perfect result when I ran the code
• I also saw the Python Code for “Split By Attribute” in the Analysis Toolbox, which was really cool. 

Chapter 4: 

• The 4th chapter is about learning how Python works and its syntax 
• This chapter was a bit easy for me because I’ve done Python before and only had to review some stuff to get started 
• I also came across Unicode (which helps you print out special characters that might not be in English) which I hadn’t before. 

Chapter 1: 

• The chapter mentions Python as a “scripting language” more than a programming language. Although I’ve seen this word around a lot in the CS Jargon, I haven’t really understood what it meant. The book mentions a clear distinction between the two. 
• The chapter also has a structure mentioned, as in a brief of what each chapter in the book entails, and that gives me a clear idea of what I’m going to be doing this semester. 
• It also mentions the advantages of using Python instead of just relying on the tools offered by ArcGIS Pro. 
• Was a very laid back introduction to  using Python in ArcGIS Pro, and gave me a gist of what I was getting myself into. 

Chapter 2: 

• I had trouble as soon as I started. The Python Command Line doesn’t work or show the same things as the textbook. I tried to get the Command Line(Prompt) and IDLE started but wasn’t able to. I got the following error, every time I tried to run: 
• Traceback (most recent call last):
•   File “<pyshell#4>”, line 1, in <module>
•     print(numpy._file_)
• AttributeError: ‘module’ object has no attribute ‘_file_’
• >>> 

 

• I think downloading PyCharm might work? Not sure 
• I got through it! The Python Window in ArcGIS Pro worked in the end and I did not need to download PyCharm as well. 
• The Python Window can be accessed by: 
  • (Assuming you have a map opened) → View Tab → Python Pane. (It should show up at the bottom) 
• The second chapter was an introduction to different types of Python IDEs and environments that can be used to go through with the book. Choosing the Python Window would be better in my opinion since there would be no need for extra downloads. 

 

I got through the first chapter in 20 minutes, however it took some time for the second one. But, I think now that I have figured this out, there should not be much trouble.

So excited to be in this course!! I decided against posting a picture of mine, since it’s already there somewhere on the 191 site, here’s another gorgeous picture:

Python Scripting for ArcGIS Pro

The book is helpful to start to get an understanding of Python, but it is difficult to interpret sometimes as it is prone to using jargon. It does define the jargon, or scripting terms, and explain how the terms relate to each other, but it does get confusing sometimes if you are unfamiliar with a lot of these concepts. The weakest part of the book is that it reads like a walkthrough, but there is no actual data package that comes with it so you are not able to perform all of the scripts it explains. Although the book works as a learning tool, I believe it works better as a reference guide.

Individual Project: ERSI Learning Courses

Even though I was not able to make as much progress as I wanted with the ESRI learning course, I do think the courses make it easier to learn Python much more than the book as you are able to actively learn through working with datasets and writing scripts. Additionally, you are immediately able to see the results of your scripts which makes you feel much more accomplished. There are many learning courses that you should be able to access and many of them are introductory level courses. The directions for the learning courses that I worked through were pretty thorough, so the biggest learning curve comes from navigating PyCharm than writing the code itself.

The work that was completed was certainly worthwhile; however, I hit an error that caused more errors and delays which prohibited me from completing the second learning course I was on.  The main error occurred after I added a python interpreter to PyCharm as instructed in the “Python Scripting for Geoprocessing” course. After the interpreter was added and I ran the completed code and an error code appeared in the console. It reads as follows: C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\numpy\__init__.py:143: UserWarning: mkl-service package failed to import, therefore Intel(R) MKL initialization ensuring its correct out-of-the-box operation under condition when Gnu OpenMP had already been loaded by Python process is not assured. Please install mkl-service package, see http://github.com/IntelPython/mkl-service from . import _distributor_init

I tried following the link but that lead to many dead ends. I also discovered that PyCharm stopped recognizing arcpy as a module around this time. This may have been from the version of Python I was using but after I tried changing it, the error still appeared.

I then tried following the file path listed in the error code: “C:\Program Files\ArcGIS\Pro\bin\Python\envs\arcgispro-py3\lib\site-packages\numby\__init__.py:143.” I could not find “__init__.py:143” in the numpy folder.

I went to the settings and clicked on the Python interpreter and saw that the packages are actually listed. I found the mkl-service package in the list so I wasn’t sure why it wasn’t working. I clicked the plus sign to add a package, searched for the mkl-service package, and clicked install. However, an error occurred claiming that I did not “have the write permissions to the target environment.

I worked with someone from OWU tech support and we discovered that I did have administration access and we tried a couple of alternative methods but determined that the error may stem from a problem with the actual PyCharm software itself. Hopefully, you don’t encounter any of these errors as they are a pain to deal with.

Overall, I am proud of the work I was able to complete. I definitely think the learning courses are worth exploring and should maybe be the primary focus of the course, rather than reading the book. I think I retained knowledge better from the courses than from the book itself, but this could be a reflection of my learning style. The book is still a worthy investment. Taking down notes will slow your progress, but they definitely will help you retain more information.

Mack Wade

 

Get Started with Python in ArcGIS Pro

In this course I learned how to write code to determine the number of features for all the feature classes in a workspace. This course also introduced some of the basics of Python syntax as I wrote code into the Python window in ArcGIS Pro.  

In order to begin this project I had to download data from ESRI’s website. I really enjoyed this course and chose to do it because of the downloadable data that I could use as I followed along, unlike the book which did not provide downloadable datasets for it’s walkthroughs. 

The course provided a few shapefiles to work out of, but for this exercise I used the Ambulances.shp. 

First, I ran a tool using Python. In Arc there are many ways and routes to complete one task. This task was to find the count of the items in the shapefile. You can do this in the attribute table or by using the Get Count tool in the Toolbox. You can also find the count by running a script in the Python window. 

To do this you take go to the toolbox history > right click the Get Count tool > Send to Python Window. After sending the window, I manually entered “Ambulances” since that’s the shapefile I want to be working out of. The script looked like this:

arcpy.management.GetCount(“ambulances”)

After completing this task, I learned that when completing a script in the Python window, the history is updated to include the task. 

The next task that I practiced was inputing an assignment statement. 

y = 73 

Then, 

x = 37

Then,

x * y 

Then, after pressing enter, the Python window answered with 2701

Next, I did a similar thing but with Temperature. 

temp_c = 17

temp_f = temp_c * 9 / 5 + 32

print(temp_f)

The Python window responded with 62.6

The next assignment was learning how to use the Python window as a tool to help you when writing code. You can do this by hovering over the prompt and reading the information that the window provides. 

There is also a system of code completion. For example, if you want the “Get Count” tool, you can typer “Get” in the Python Window and suggestions will pop up to help you finish the prompt easier. 

I really enjoyed this tutorial. I thought that it was very easy to follow and was helpful. I see myself using more of ESRI’s resources in the future if or when needed when writing scripts. I think that if any student was interested in this course in the future these tutorials would be a better resource than the book we used.

I had some screen shots from this assignment but they did not get added into the blog format. If you would like me to send those over to you. let me know.

Background — 

This independent study was more so learning R rather than spending more time in project-based learning as I did with python. R is a very powerful data-driven language and is great for using large data sets. R is vastly used for geospatial analysis as well. I created a county population map of the United States using census data from built-in libraries whose primary use is for geospatial analysis. The code looks very simple, but it was not. I had significantly more problems with R than I did with Python. The syntax used in R is an absolute cluster and I despise it. 

 

Steps 

1. Import libraries usmap and ggplot2. 
2. Add some data to plot the map, data = countypop draws the county lines and values creates data with the built-in census data via a CSV. 
3. Create a scale and create breaks 
4. Change the scale fill to one that’s not a constant color for easier viewing
5. Cry because you can’t create an interactive map with tmap 

 

The Code — 

library(usmap)

library(ggplot2)

plot_usmap(data = countypop, values = “pop_2015”, color = “grey”) + 

  scale_fill_viridis_c(name = “County Population 2015”, label = scales::comma, option = ‘D’, trans = “log2”,  breaks = trans_breaks(“log2”, function(x) 2^x))+

theme(legend.position = “right”)

 

 

Problems — 

Like I stated before, I had tons of issues with this project. I programmed the map at least 15 different ways and it never looked like how I wanted it to. I think R is a little over my head when it comes to the math-based stuff like the scale. The breaks were never how I wanted them but I chose one that seemed the most reasonable.

 Another issue that I had was this map is a static map, it cannot move and you cannot click on counties to see their population. I tried for at least three weeks to create an interactive map through the library T map. It’s literally very simple, you just join the counties and the county population CSV to create the interactive map. The issue that I had was joining the two tables by their geoID. It never seemed to work and I literally even created my own spreadsheet with the counties’ populations, names, states, and geoID. Didn’t work. This was the most frustrating part of the project that I dealt with. 

Again, I took more of a book approach to this course so it definitely felt like I could’ve figured it out by making other things within R. I can follow the code and see what someone is doing, but trying to do it on your own starting out is very difficult. Also, R is just way worse and not as user-friendly as Python. I feel that I could definitely become better at R if I just spent more time with it, and also brushed up on my math skills a little more. 

 

Background –    For this independent study, I wanted to solve a problem that isn’t necessarily tedious but just annoying. For me, that was creating true color images through ArcPro. Working with Dr. Rowley has made me quite familiar with creating true colors, mostly creating several of them over a melt season for the Sermeq Kujalleq Ablation Region. By creating this script, it just makes everything a little easier so you don’t have to manually import which .TIFF files you want. I’ve had some experience with python before starting this course so it definitely was a little easier than I expected, and the syntax of python is just leaps and bounds ahead of that stinky language R. 

 

Introduction – (what each step does)

1. Create a geodatabase or .gdb — doing this creates a server like storage house your tiff files to make calculations and pull the data back down. 
2. Create Mosaic Dataset — This creates a dataset to house your images, another important step before using the composite band’s tool. 
3. Add rasters to Mosaic — Add your images to the dataset 
4. Composite Bands Tool — From the mosaic dataset, import them to the composite band’s tool, which creates the Truecolor image raster for you. 

 

 

 

What Did I learn? —

I learned a lot from this project, firstly that ERSI software is insanely tedious when it comes to using their programs. I needed to jump through several hoops just to create a geodatabase. The geodatabase is created through ArcServer – everything must be run through the server and you need to have the software installed to even create one. 

I learned to problem solve way better on my own, knowing exactly what to search to find solutions to my problem was a huge success. I genuinely didn’t have any problems with this project, nothing that was too halting that I felt like I needed to stop the project or require outside help. I dedicated about 8-10 hours a week to this project and it was mostly spent debugging and finding solutions to my code. 

One issue that came across was at the very beginning of the project. To use the Arcpy library, you cannot have the newest version of python installed. The newest version is 3.10, but arcpy is currently only compatible with 3.7 and below. I felt like this project was a great introduction to the GIS developing world, and I am very glad that I took it. Overall, I am very confident in my work for this course. I believe that this script has significant use for GIS creators and researchers.