Difference between revisions of "Geocoding and Reverse Geocoding Addresses in QGIS"

From CUOSGwiki
Jump to navigationJump to search
 
(42 intermediate revisions by the same user not shown)
Line 13: Line 13:
   
 
==Data and Software==
  
==Data and Software==
  +
QGIS offers several plugins on how to translate a list of addresses to coordinates on a map, or vice versa. We will be using a plugin called MMQGIS to simulate the following theoretical problem: '''we want to know locate the most popular Mexican restaurants around Ottawa so we can choose the best place to open our new restaurant'''. The data used for this tutorial was gathered by compiling the first 20 results of a Google maps search of local Mexican restaurants. Their addresses were collected and formatted in an Excel file, available HERE.
−
We will be using QGIS to geocode a theoretical problem
  
  +
  +
QGIS 3.22 download available at: https://www.qgis.org/en/site/forusers/download.html
  +
  +
===Data Format===
  +
*Note: Only relevant if you are using your own data.
  +
Before you get started with geocoding using QGIS, make sure that your addresses are stored in a CSV/Excel file that is formatted for UTF-8. The addresses should follow this format: '''Name (optional), Street Address, City, State/Province, Country'''
  +
  +
[[File:data6.PNG]]
  +
  +
===Installing MMQGIS===
  +
MMQGIS is a set of Python plugins for manipulating vector map layers in Quantum GIS: CSV input/output/join, geocoding, geometry conversion, buffering, hub analysis, simplification, column modification, and simple animation. MMQGIS provides an alternative to the Processing toolbox, with verbose progress reporting, an intuitive user interface, direct shapefile/CSV-file access, and some additional capabilities missing from other plugin sets (QGIS Plugins Repository).
  +
MMQGIS is a solid choice for geocoding because it allows for a large number of addressed to be geocoded at the same time, and also supports multiple web services for geocoding (Google Maps, OpenStreetMap, US Census Bureau, ESRI Server).
  +
  +
  +
*To install MMQGIS, first select Plugins -> Manage and Install Plugins.
  +
[[File:data1.PNG]]
  +
  +
*Next, begin searching for "MMQGIS" in the search bar, it should be the first one to pop up. Select Install Plugin.
  +
[[File:data2.PNG]]
  +
  +
==Geocoding the Addresses==
  +
  +
Now that we have our data set, and the plugin installed, we are ready to start geocoding.
  +
  +
* Under the MMQGIS tab, pan down to Geocode -> Geocode CSV with Web Service.
  +
[[File:data3.PNG]]
  +
  +
* We should see the following pop-up:
  +
[[File:data4.PNG|frame]]
  +
  +
In the Input CSV File field, we will use our selectee data set of restaurants. Make sure the Address, City, State, and Country fields align correctly with their respective column in the Excel sheet. The Web Service we will use is OpenStreetMap / Nominatim, make sure that this field is correct. In the Output File Field, choose an appropriate name for the resulting shapefile to be stored as, and save it in your working folder. The Not Found Output List field is a list of every address that could not be geocoded. Save it in your working folder. Once all of your field are correctly filled, hit Apply.
  +
  +
  +
  +
===Google Maps Web Service Option===
  +
  +
Another web service option supported by MMQGIS is Google Map's web service. The Google Maps Platform web services are a collection of HTTP interfaces to Google services providing geographic data for user maps applications. Google Maps Platform web services are an interface for requesting Maps API data from external services and using the data within your Maps applications. These products are secured from unauthorized use by restricting API calls to those that provide proper authentication credentials. These credentials are in the form of an API key - a unique alphanumeric string that associates your Google billing account with your project, and with the specific API or SDK. The Maps APIs web services use HTTP(S) requests to specific URLs, passing URL parameters and/or JSON-format POST data as arguments to the services. Generally, these services return data in the HTTP(S) request as either JSON or XML for parsing and/or processing by your application. (Google Maps Platform)
  +
  +
Follow this link to generate an API key: https://console.cloud.google.com/project/_/google/maps-apis/credentials
  +
  +
This tutorial will not cover how to geocode using Google's API key's, but it is as simple as selecting Google instead of OpenStreetMap, and inserting your API key in the required input box.
  +
  +
* Now that we have applied our geocoding via MMQGIS, we should see our new feature layer containing our new geocoded point locations. It should be noted that the larger the dataset, or the more addresses used, will sometimes take much longer to geocode.
  +
[[File:points1.PNG]]
  +
  +
* And if we open the Attribute Table, we see our geocoded addresses, along with some added descriptive attributes, including latitude and longitude.
  +
[[File:points2.PNG]]
  +
  +
* If we check our output CVS file, it will show us the addresses that were unable to be geocoded, for whatever reason.
  +
[[File:points3.PNG]]
  +
  +
===Attribute Table===
  +
You'll notice that even though we have our latitude and longitude, their formatting in the attribute table is not ideal, as they are both located in the same column. To adjust the columns to show Lat and Long separately, open the attribute table and select the Field Calculator.
  +
  +
  +
[[File:table1.PNG]]
  +
  +
* We will create a new field, and call it X, for latitude. Change the “Output field type” to “Decimal number (real)”. And the “Precision”, 5. In the Expression field, type '''$X'''. Hit OK. Repeat for longitude, but call the field Y.
  +
[[File:table2.PNG]]
  +
  +
* After creating our new fields, we should be able to see our changes in the attribute table.
  +
[[File:table3.PNG]]
  +
  +
===Reverse Geocoding===
  +
Reverse geocoding is simply the process by which geographic coordinates are converted into a physical address.
  +
Since we have extrapolated our latitude and longitude coordinates from our addresses in the step above, we can use these coordinates to check and make sure our data point locations line up correctly.
  +
  +
* Open MMQGIS, and select Reverse Geocode.
  +
[[File:reverse1.PNG]]
  +
  +
* Input our restaurants CVS file as the layer input. Choose OpenStreetMap again for the web service.
  +
[[File:reverse2.PNG]]
  +
  +
* We should the new "Reverse" feature layer aligned with our old point locations. To make sure they are correct, we can use the Identify tool to display the attributes.
  +
[[File:reverse3.PNG]]
  +
[[File:reverse4.PNG]]
  +
  +
==Conclusion==
  +
  +
Geocoding is a very simple, effective way to collect latitude and longitude coordinates which can be further used for GIS analysis, cartography, decision making workflow, transaction mash-up, or injected into larger business processes. Or, geocoding can be used in services such as routing and local search. The MMQGIS plugin within QGIS is one of the easiest and most user-friendly geocoding services available for GIS programs. The ease of use and access, combined with the freely available software makes QGIS an easy choice to execute address geocoding.
  +
  +
==References==
  +
  +
* ArcMap. What is geocoding?-ArcMap | Documentation. (n.d.). Retrieved December 20, 2021, from https://desktop.arcgis.com/en/arcmap/latest/manage-data/geocoding/what-is-geocoding.html
  +
  +
* Goldberg, D. W. (n.d.). From Text to Geographic Coordinates: The Current State of Geocoding. Retrieved December 20, 2021, from https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.117.471
  +
  +
* McKie, D. (n.d.). Geocoding using QGIS . Geocoding Using QGIS. Retrieved December 20, 2021, from http://www.davidmckie.com/Geocoding%20Using%20QGIS_New.pdf
  +
  +
* University of Western Ontario. (n.d.). Process for Geocoding in QGIS Using Google and OpenStreetMap Web Map Services. Geocoding Using QGIS. Retrieved December 20, 2021, from https://www.lib.uwo.ca/madgic/projects/gis/Geocoding%20Using%20QGIS.pdf
  +
  +
* What is geocoding? definition and faqs. OmniSci. (n.d.). Retrieved December 20, 2021, from https://www.omnisci.com/technical-glossary/geocoding

Latest revision as of 10:25, 21 December 2021

Introduction

Geocoding is the computational process by which a text-based, physical address is converted into geographic coordinates, which can be used for a variety of mapping applications. Geocoding requires a reference dataset and a geocoding algorithm, each of which is composed of sub-operations and sub-components that work together to transform physical, input data into numerical, spatial data. Reverse geocoding is the process by which geographic coordinates are converted into a physical address.

Why Geocode?

Geocoding allows us to quickly find various types of locations, including historical buildings, houses, restaurants, and stores. From simple data analysis to business and customer management and distribution techniques, there is a wide range of applications geocoding can be used for, as we can identify and analyze patterns within the data. With geocoded addresses, you can spatially display the address locations and begin to recognize patterns within the information.

Geocoding software is implemented in a variety of industries, including but not limited to:

  • Health: assess patient access to healthcare facilities; study epidemiological patterns of a disease
  • Finance: determine lending activity in the community, including demographics data to assist in fair lending efforts
  • Public Safety: direct emergency response via locally developed street files and E911 points
  • Military: Military Grid Reference System (MGRS) is the geocoordinate standard used by NATO militaries
  • Commercial: directly factor geography into the business analytics process; monitor shipping patterns and customer sales

Data and Software

QGIS offers several plugins on how to translate a list of addresses to coordinates on a map, or vice versa. We will be using a plugin called MMQGIS to simulate the following theoretical problem: we want to know locate the most popular Mexican restaurants around Ottawa so we can choose the best place to open our new restaurant. The data used for this tutorial was gathered by compiling the first 20 results of a Google maps search of local Mexican restaurants. Their addresses were collected and formatted in an Excel file, available HERE.

QGIS 3.22 download available at: https://www.qgis.org/en/site/forusers/download.html

Data Format

  • Note: Only relevant if you are using your own data.

Before you get started with geocoding using QGIS, make sure that your addresses are stored in a CSV/Excel file that is formatted for UTF-8. The addresses should follow this format: Name (optional), Street Address, City, State/Province, Country

Data6.PNG

Installing MMQGIS

MMQGIS is a set of Python plugins for manipulating vector map layers in Quantum GIS: CSV input/output/join, geocoding, geometry conversion, buffering, hub analysis, simplification, column modification, and simple animation. MMQGIS provides an alternative to the Processing toolbox, with verbose progress reporting, an intuitive user interface, direct shapefile/CSV-file access, and some additional capabilities missing from other plugin sets (QGIS Plugins Repository). MMQGIS is a solid choice for geocoding because it allows for a large number of addressed to be geocoded at the same time, and also supports multiple web services for geocoding (Google Maps, OpenStreetMap, US Census Bureau, ESRI Server).


  • To install MMQGIS, first select Plugins -> Manage and Install Plugins.

Data1.PNG

  • Next, begin searching for "MMQGIS" in the search bar, it should be the first one to pop up. Select Install Plugin.

Data2.PNG

Geocoding the Addresses

Now that we have our data set, and the plugin installed, we are ready to start geocoding.

  • Under the MMQGIS tab, pan down to Geocode -> Geocode CSV with Web Service.
Data3.PNG
  • We should see the following pop-up:
Data4.PNG

In the Input CSV File field, we will use our selectee data set of restaurants. Make sure the Address, City, State, and Country fields align correctly with their respective column in the Excel sheet. The Web Service we will use is OpenStreetMap / Nominatim, make sure that this field is correct. In the Output File Field, choose an appropriate name for the resulting shapefile to be stored as, and save it in your working folder. The Not Found Output List field is a list of every address that could not be geocoded. Save it in your working folder. Once all of your field are correctly filled, hit Apply.


Google Maps Web Service Option

Another web service option supported by MMQGIS is Google Map's web service. The Google Maps Platform web services are a collection of HTTP interfaces to Google services providing geographic data for user maps applications. Google Maps Platform web services are an interface for requesting Maps API data from external services and using the data within your Maps applications. These products are secured from unauthorized use by restricting API calls to those that provide proper authentication credentials. These credentials are in the form of an API key - a unique alphanumeric string that associates your Google billing account with your project, and with the specific API or SDK. The Maps APIs web services use HTTP(S) requests to specific URLs, passing URL parameters and/or JSON-format POST data as arguments to the services. Generally, these services return data in the HTTP(S) request as either JSON or XML for parsing and/or processing by your application. (Google Maps Platform)

Follow this link to generate an API key: https://console.cloud.google.com/project/_/google/maps-apis/credentials

This tutorial will not cover how to geocode using Google's API key's, but it is as simple as selecting Google instead of OpenStreetMap, and inserting your API key in the required input box.

  • Now that we have applied our geocoding via MMQGIS, we should see our new feature layer containing our new geocoded point locations. It should be noted that the larger the dataset, or the more addresses used, will sometimes take much longer to geocode.

Points1.PNG

  • And if we open the Attribute Table, we see our geocoded addresses, along with some added descriptive attributes, including latitude and longitude.

Points2.PNG

  • If we check our output CVS file, it will show us the addresses that were unable to be geocoded, for whatever reason.

Points3.PNG

Attribute Table

You'll notice that even though we have our latitude and longitude, their formatting in the attribute table is not ideal, as they are both located in the same column. To adjust the columns to show Lat and Long separately, open the attribute table and select the Field Calculator.


Table1.PNG

  • We will create a new field, and call it X, for latitude. Change the “Output field type” to “Decimal number (real)”. And the “Precision”, 5. In the Expression field, type $X. Hit OK. Repeat for longitude, but call the field Y.

Table2.PNG

  • After creating our new fields, we should be able to see our changes in the attribute table.

Table3.PNG

Reverse Geocoding

Reverse geocoding is simply the process by which geographic coordinates are converted into a physical address. Since we have extrapolated our latitude and longitude coordinates from our addresses in the step above, we can use these coordinates to check and make sure our data point locations line up correctly.

  • Open MMQGIS, and select Reverse Geocode.

Reverse1.PNG

  • Input our restaurants CVS file as the layer input. Choose OpenStreetMap again for the web service.

Reverse2.PNG

  • We should the new "Reverse" feature layer aligned with our old point locations. To make sure they are correct, we can use the Identify tool to display the attributes.

Reverse3.PNG Reverse4.PNG

Conclusion

Geocoding is a very simple, effective way to collect latitude and longitude coordinates which can be further used for GIS analysis, cartography, decision making workflow, transaction mash-up, or injected into larger business processes. Or, geocoding can be used in services such as routing and local search. The MMQGIS plugin within QGIS is one of the easiest and most user-friendly geocoding services available for GIS programs. The ease of use and access, combined with the freely available software makes QGIS an easy choice to execute address geocoding.

References

Retrieved from "https://dges.carleton.ca/CUOSGwiki/index.php?title=Geocoding_and_Reverse_Geocoding_Addresses_in_QGIS&oldid=18070"