Frawley: R Geospatial Wrap Up

Background — 

This independent study was more so learning R rather than spending more time in project-based learning as I did with python. R is a very powerful data-driven language and is great for using large data sets. R is vastly used for geospatial analysis as well. I created a county population map of the United States using census data from built-in libraries whose primary use is for geospatial analysis. The code looks very simple, but it was not. I had significantly more problems with R than I did with Python. The syntax used in R is an absolute cluster and I despise it. 

 

Steps 

  1. Import libraries usmap and ggplot2. 
  2. Add some data to plot the map, data = countypop draws the county lines and values creates data with the built-in census data via a CSV. 
  3. Create a scale and create breaks 
  4. Change the scale fill to one that’s not a constant color for easier viewing
  5. Cry because you can’t create an interactive map with tmap 

 

The Code — 

library(usmap)

library(ggplot2)

plot_usmap(data = countypop, values = “pop_2015”, color = “grey”)

  scale_fill_viridis_c(name = “County Population 2015”, label = scales::comma, option = ‘D’, trans = “log2”,  breaks = trans_breaks(“log2”, function(x) 2^x))+

theme(legend.position = “right”)

 

 

Problems — 

Like I stated before, I had tons of issues with this project. I programmed the map at least 15 different ways and it never looked like how I wanted it to. I think R is a little over my head when it comes to the math-based stuff like the scale. The breaks were never how I wanted them but I chose one that seemed the most reasonable.

 Another issue that I had was this map is a static map, it cannot move and you cannot click on counties to see their population. I tried for at least three weeks to create an interactive map through the library T map. It’s literally very simple, you just join the counties and the county population CSV to create the interactive map. The issue that I had was joining the two tables by their geoID. It never seemed to work and I literally even created my own spreadsheet with the counties’ populations, names, states, and geoID. Didn’t work. This was the most frustrating part of the project that I dealt with. 

Again, I took more of a book approach to this course so it definitely felt like I could’ve figured it out by making other things within R. I can follow the code and see what someone is doing, but trying to do it on your own starting out is very difficult. Also, R is just way worse and not as user-friendly as Python. I feel that I could definitely become better at R if I just spent more time with it, and also brushed up on my math skills a little more. 

 

Frawley: Python Wrap Up

Background –    For this independent study, I wanted to solve a problem that isn’t necessarily tedious but just annoying. For me, that was creating true color images through ArcPro. Working with Dr. Rowley has made me quite familiar with creating true colors, mostly creating several of them over a melt season for the Sermeq Kujalleq Ablation Region. By creating this script, it just makes everything a little easier so you don’t have to manually import which .TIFF files you want. I’ve had some experience with python before starting this course so it definitely was a little easier than I expected, and the syntax of python is just leaps and bounds ahead of that stinky language R. 

 

Introduction – (what each step does)

  1. Create a geodatabase or .gdb — doing this creates a server like storage house your tiff files to make calculations and pull the data back down. 
  2. Create Mosaic Dataset — This creates a dataset to house your images, another important step before using the composite band’s tool. 
  3. Add rasters to Mosaic — Add your images to the dataset 
  4. Composite Bands Tool — From the mosaic dataset, import them to the composite band’s tool, which creates the Truecolor image raster for you. 

 

 

 

What Did I learn?

I learned a lot from this project, firstly that ERSI software is insanely tedious when it comes to using their programs. I needed to jump through several hoops just to create a geodatabase. The geodatabase is created through ArcServer – everything must be run through the server and you need to have the software installed to even create one. 

I learned to problem solve way better on my own, knowing exactly what to search to find solutions to my problem was a huge success. I genuinely didn’t have any problems with this project, nothing that was too halting that I felt like I needed to stop the project or require outside help. I dedicated about 8-10 hours a week to this project and it was mostly spent debugging and finding solutions to my code. 

One issue that came across was at the very beginning of the project. To use the Arcpy library, you cannot have the newest version of python installed. The newest version is 3.10, but arcpy is currently only compatible with 3.7 and below. I felt like this project was a great introduction to the GIS developing world, and I am very glad that I took it. Overall, I am very confident in my work for this course. I believe that this script has significant use for GIS creators and researchers. 

Frawley: Chapter 4 Notes

Chapter 4 Notes: This was an amazing review for me, essentially a complete introduction to python and its syntax and how to use it. This was genuinely the best python tutorial that I have read just because of how concise and to the point it was. I’ve read about two different intermediate-level python books and they didn’t compare to the examples and just the great layout the author put forth. Anyone could read through this chapter a few times and receive a pretty intermediate-level understanding of python. Genuinely you can mentally program in your head once learning the syntax. For me, this has helped me visualize projects easier and how I would accomplish them. I spent at least two hours reading this chapter several times to absorb the massive amount of information thrown at you. 

It helped me ponder on a semester project. I’m possibly thinking about attempting to automate a mapping sequence and run data extraction by teaching the program how to locate and measure each feature. On what, I am not exactly sure yet. 

 

What are the main data types in python? Which data types are sequences? Which data types are mutable, which are immutable? 

Strings, tuples, lists, dictionaries, boolean expressions, integers, float. Strings, tuples, and numbers are immutable. Lists and dictionaries are mutable, meaning they can be modified. 

 

Describe the difference between true division and floor division. 

Floor division divides with no remainder, while true division divides into a whole number with a leftover remainder. 

 

Explain what the dynamic assignment of a variable means. 

Dynamic assignment of variables occurs after you assign a value to it. Specifically, y = 6 is an integer, y =  10.33 is a float, and y = “Mark” is a string. The variable you assigned the value to becomes that exact data type you applied to it. 

 

Give examples of variable names and why they are good. 

Counter, field5, my_file. Underscores are great for creating a unique variable name. All lowercase or including a number helps differentiate the variables from the rest of the code to avoid confusion. 

 

What’s the difference between an expression and a statement in python, give examples.

An expression is a value with no variable assignment. For example, 3 * 10 and the console will calculate it and return 30. A statement is a function that assigns a value to a variable. For example, x = 40 * 3. The x itself is not a value, it is just assigned one, and to display the result you need to incorporate the print() function to display the value. Print(x) and it would return 120. 

 

Explain the use of quotation marks in python

Quotation marks assign a collection of characters as a string, which is designated to a variable. 

 

What are Unicode strings? 

Strings are essentially plain text, and Unicode allows that plain text to be converted into other languages, and with Unicode you can use different letters from other languages by using their designated ASCII code. 

 

Name five built in functions in python and provide an example of each one’s use. 

Print(x) : Prints a result from a function. Print (name) – Mark

pow(x,y) : First number is multiplied by a ‘power’  of the later value pow(2,3) – 8 

string (x) : Can convert an integer into a string to be able to display the integer in a normal line 

of string characters. str(10) = “10”

abs(x) : Returns absolute value of a number. abs(-8) – 8

float(x) : converts a string or number into a float. float(“8.0”) – 10.0 

 

What is dot notation? 

Shows that method belongs to the object. What comes after the dot belongs to what came before it. 

 

Describe the use of forward and reverse indexing sequences. Provide an example of slicing strings and lists. 

Forward indexing is used to extract or display certain characters in a string, list, dictionary, or tuple, by a specific number correlated to the character. For example, indexing starts at 0, so indexing a string can retrieve a certain number or letter. Mystring = “Mark is the best” 

Mystring[0] would return “M”. Slicing strings and lists can be used to take a chunk of letters or values from the string or list. This is done by using the index numbers and a colon to separate the first and last values you want to return from the string or list. 

 

Name three methods of string objects in python, provide an example of each one’s use. 

.join(x) – Joins values in a list into one string. 

Leela_list = [ ‘she’ , ‘needs’ , ‘dog’ , ‘food’] 

Leelastring = “”

leela_list.join(leelastring) = ‘she needs dog food’ 

 

.split(x) – Can turn a string into a list by splitting each word into substrings

bestdog = “Leela is the best dog.”

Bestdog.split(“ ”)  = [‘Leela’ , ‘is’ , ‘the’ , ‘best’, ‘dog’] 

 

.replace(x) – replace all occurrences of a specific substring with another substring

 

myhouse = “609 logan”

myhouse.replace(“609”, “Toronto”)

 

What are some key similarities between lists and tuples? 

They can both hold multiple values, except tuples are immutable. 

 

Describe the use of boolean values to evaluate a condition. 

Boolean values are true/false values that can differentiate if a value is there or not, essentially a yes or no. 

X = 3 

X == 8

Returns false 

Describe the key steps to create a dictionary and add new items to the dictionary. 

The first step to creating a dictionary is to assign a variable name to it and set it equal to empty braces. For example,

 teams = {}

To add to a dictionary, you include brackets to the variable with a pair of quotation marks inside and set it equal to the complete string. 

teams[“CHI”] = “Chicago Blackhawks” 

And to add to the dictionary you continue to use the same syntax. 

 

Why should backslashes be avoided when writing paths in python? 

Because a single backslash is used to denote a new line of text in strings. 

 

Describe how branching is implemented in python. 

Branching is implemented with if statements, telling the compiler that it should take one path or another depending on the returned value. 

 

What is a compound statement in python? 

A compound statement is a block of code expected to run together as a block, which is denoted by indentation. 

 

What are the two main looping structures in python? Describe each. 

While loops check for the value of a variable and require an exit statement. 

For loops repeat a block of code for each element in that sequence 

 

What approaches can be used to split long lines of code into multiple lines and improve readability? 

Enter a backslash in a defined string within quotation marks. 

Implied line continuation (), [], {}. 

Hanging indentation 

 

Why should you add comments to your scripts? 

To help separate parts of the code into blocks

Designate certain code with comments to know that this block is performing a particular function within the script.

 

Frawley: Chapter 3 Notes

Chapter 3 Notes:

Types of tools, batch processing. Model tools vs script tools, how to automate workflows. Dynamic naming for batch processing %Name%. Took me about 40 minutes to read thoroughly. 

Chapter review questions: 

 

Describe some of the general elements of the geoprocessing framework in ArcGIS Pro? 

Collection of tools organized into toolboxes, python window for executing python, geoprocessing history, Model Builder.

 

What are the three types of tools in ArcGIS Pro? 

Geoprocessing, analysis tools, geocoding, cartography. 

 

Explain the difference between system tools and custom tools. 

System tools are premade executable tools that you can drag and drop your files into to complete a task/output. Custom tools are obtained by the user through their own creation or collection of third-party tools. 

 

Provide examples of commonly used environments and how they impact running geoprocessing tools. 

Environments are where the workspace is located essentially, and environments can be modified to use specific values for the output of the geoprocessing tools. 

 

What are the strengths and limitations of batch processing in ArcGIS Pro? 

Batch processing uses the same tool multiple times using different parameters. Different iterations use different parameters to finish their processing. For tools with multiple input parameters, you can only choose one to use as a batch parameter. 

 

Explain the use of dynamic naming in batch processing. 

Creates a different output for each individual parameter. %Name% uses the input variable name and combines it with the remaining part of the naming scheme. %Name%_final will turn all inputs into separate outputs with their respective names. 

 

What are some of the differences and similarities between model tools and script tools in ArcGIS Pro? 

Model tools help you execute multiple tools at the same time, while script tools help you create your own detailed tool by connecting applications, better programming logic, and can be run by themselves on a stand-alone script reader. 

 

Why would you automate a specific workflow instead of a model? 

It’s more reliable to automate one workflow and observe it, rather than automating a model and trying to decipher the results after all functions are completed running. 

 

What are ways to run python code in the context of automating workflows in ArcGIS Pro? 

Scheduling python scripts to run at an exact time, by using the task scheduler.

 

Frawley: Chapter 2 Notes

Chapter 2 Notes:

The autocomplete feature allows you to not need to memorize all function/statement diction and their uses.  Explains different IDEs and their uses. How to use the python window in Arc. Explains which tools are used with the help of python for specific geoprocessing tasks. 

Quick and easy to read. Probably only took around 25 minutes. The chapter review questions are the following: 

 

What does IDE stand for? 

Integrated Development Environment.

 

What does it mean that python is an interpreted language? 

Interpreted language means that it doesn’t need to compile into binary before the script runs. 

 

Which python editor comes installed with every version of python? 

Command prompt, or IDLE (Integrated DeveLopment Environment)

 

What are some of the recommended python editors to work with ArcGIS Pro 

PyCharm, Spyder, Visual Studio

 

What are some of the differences and similarities between python editors? 

Essentially they’re similar in every aspect except Spyder is more well supported with geospatial programming and visualization. 

 

What are the general steps to install and configure each of the python editors? 

Download the python language from their website, decide which IDE you want to work with and go to their website and download the version for your operating system. Once you download the IDE of choice, follow the setup instructions and begin writing your code. I also recommend giving a designated folder to keep each python project organized and by itself. 

 

What is the default environment when working with python and ArcGIS Pro 

IDLE 

 

What is syntax highlighting? 

Syntax highlighting assigns different colors to statements, functions, text, etc. 

 

What is a file extension for a python script file? 

.py 

 

How do you open the Python window in ArcGIS Pro? 

Click the toolbox at the top of ArcGIS Pro and click open the python window. 

 

What are the two sections of the python window, and what purpose does each one serve? 

The top is the transcript, and the bottom window is called the prompt. The transcript is blank until you start scripting in the prompt window. When you run the code in the prompt, it displays the script on the above transcript window. 

 

What are some typical examples of autocompletion prompts in the python window, and how do they assist in writing proper code? 

They reduce the amount of code you need to memorize since the list of autocomplete prompts provides the name of different functions and descriptions on how to use them. This results in fewer syntax errors and is faster than typing the full name of the function. 

 

Which features of the python window illustrate that it is fully integrated with the geoprocessing framework of ArcGIS Pro? 

Most of the geoprocessing framework and data extraction/management tools in ArcGIS Pro are written in Python 3, so you are able to manipulate default python tools within the software using the Python window. Within the python geoprocessing tools, you can manipulate the script to incorporate multiple layers or rasters at once, instead of individually. 

 

Frawley: Chapter 1 Notes

Chapter 1 Notes:

Python is an interpreted language that is easy to understand and has a simple syntax. Python 3 is the latest version of python that is used with ArcGIS Pro. 

This chapter was a relatively easy read and clear to understand. This chapter just provided the basics of what Python is and shows some example scripts that you will eventually understand. This chapter took me about 20 minutes to read, so I’m not entirely sure about the questions included because I read all of the chapters before completing each review question set. I completed the end of chapter questions and they are as follows:

 

What are some of the key features of Python that make it suitable as a scripting language to work with ArcGIS Pro? 

Python is easy to learn, free and open-source, multi-OS platform use, and it is object-oriented, meaning that it can interact with multiple interacting objects within the script. 

 

What does it mean that python is an interpreted language? 

Interpreted language means that its code doesn’t need to be compiled into binary when you run it. 

 

What are some of the differences between scripting and programming? 

Scripting essentially takes components that are already made and combines them together to create something, while programming is more about creating something from the ground up on your own with unique code. 

 

Which version of python is used with ArcGIS pro? 

Python 3 is the version used with ArcGIS Pro. Python 3 was introduced as the newest version of the language in 2008. 

 

What are some of the reasons to invest time and effort into learning python? 

 Python can automate workflows and tasks to reduce the amount of time and effort into collecting and sorting data. Python is one of the most commonly used programming languages within the geospatial community.