Difference between revisions of "Exploring Clustering In QGIS"

From CUOSGwiki
Jump to navigationJump to search
 
(42 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Introduction ==
== Introduction ==
The purpose of this project is to explore the capabilities of open source software such as QGIS. QGIS is one of the many open source softwares we have used in GEOM 4008. In this tutorial which I am writing in 2021, I will be going into vector analysis tools in QGIS which are used to create clusters. I will go step by step going through the Density Based and K-means methods of clustering and I will give the results of my findings.
The purpose of this project is to explore the capabilities of open source software such as QGIS, which is one of many open source softwares used in GEOM 4008. This tutorial will explore vector analysis tools used to create clusters and distinct boundaries. An in-depth approach will breakdown each step for new QGIS users, please feel free to skip any steps you are already familiar with.

The use of clusters is a practical tool in GIS, it can help group vector data points into separate clusters or groups which the amount is fully configurable by the user. This can be helpful if you have a vast area and you want to divide a task into multiple areas and divide and conquer to accomplish a certain goal. The [https://pro.arcgis.com/en/pro-app/2.8/tool-reference/spatial-statistics/an-overview-of-the-mapping-clusters-toolset.htm option] is given in ArcPro and while it may have a bigger collection of clustering options, QGIS still has clustering capabilities which are DBSCAN(density based) and K-Means. We will be using the newest version of [https://qgis.org/en/site/forusers/download.html QGIS] at the time of tutorial, version 3.22. The most important part for the start of the project is downloading the latest version, getting the proper data and proper projections which I will go step by step in detailing below. The scope of my project will be going as a City of Ottawa official that wants to investigate incidents(vehicle, bicycle, etc...) and he feels the city is too big and the goal of this tutorial will be on how we can designate multiple smaller areas for him to work with.
Cluster analysis is a practical GIS technique which can group vector points into corresponding clusters based on distance, density, or set number of points, and is fully configurable by the user. This can be helpful if you have a large study extent and you want to geographically divide a variables into multiple areas, such as detecting regions of high painted turtle sightings or identifying crime hotspots. [https://pro.arcgis.com/en/pro-app/2.8/tool-reference/spatial-statistics/an-overview-of-the-mapping-clusters-toolset.htm ArcGIS Pro] has a larger collection of clustering options, however QGIS still has clustering capabilities which are DBSCAN and K-Means.

Here we will explore a local problem: A City of Ottawa employee on the Traffic Services Division team wants to investigate vehicle collision trends by locating accident clusters to recommend updated patrol divisions and officer coverage based on municipality demand.


==Resources==
==Resources==
Line 7: Line 10:
===Required Software===
===Required Software===


To run this analysis, it is best to have the latest version of QGIS. I will detail the steps into getting the newest version.
To run this analysis, it is best to have the latest version of QGIS.


*If you have an old version, I would start by deleting that one before proceeding to further steps as having two versions of QGIS is not necessary and takes space on the hard drive.
*If you have an old version, it is recommended to delete this before proceeding to further steps as having two versions of QGIS is not necessary and takes space on the hard drive.


*Click on this [https://qgis.org/en/site/forusers/download.html link] and you will see many options on operating systems you can download QGIS on
*Click on this [https://qgis.org/en/site/forusers/download.html link] and you will see many options on operating systems you can download QGIS on.


*Select either the fastest (4.2.2) or the more stable (3.44.14) download depending on your preference and it should start downloading
*Select either the fastest (4.2.2) or the more stable (3.44.14) download depending on your preference and it should start downloading.


*Once it's done downloading add it to your hard drive(if not already done automatically) and you should be ready to use the open source software
*Once it's done downloading, add it to your hard drive (if not already done automatically) and you are ready!


=== Required Datasets ===
=== Required Datasets ===


The data that we will be using will be in two forms. Referencing data so we can visualize where we are and data we will use for the clustering.
The data that we will be using will be in two forms: Referencing data so we can visualize where we are and vector data to use for clustering.


Referencing
Referencing
Line 44: Line 47:


===Projection===
===Projection===
Projection is an important step for DBSCAN clustering, you may encounter problems if you do not do it this way. NAD83, specifically UTM zone 18, is an appropriate projection for this project as we are looking at a municipal problem, Ottawa traffic collisions. Projecting to a projection such as this one will be important considering we are dealing with distances when using the DBSCAN clustering.
Projection is an important step for DBSCAN clustering, you may encounter problems if you do not do it this way. NAD83, specifically UTM zone 18, is an appropriate projection for this project as we are looking at a municipal problem, Ottawa traffic collisions. Projection is important as DBSCAN clustering relies on map distances.

Important: All vector data should be projected that way.
Tip: All vector data should be projected that way.

To first orient yourself, we'll add Google Satellite basemap.
To first orient yourself, we'll add Google Satellite basemap.
*Select '''Plugins''', then '''Manage and Install Plugins...''', search "Basemaps", select '''Basemaps''' and '''Install'''. [[File:BasemapPlugin.png|none|600px|link=MediaWiki]]
*Select '''Plugins''', then '''Manage and Install Plugins...''', search "Basemaps", select '''Basemaps''' and '''Install'''. [[File:BasemapPlugin.png|none|600px|link=MediaWiki]]
Line 59: Line 64:
*Choose a location in your project folder and name it. Ex: Traffic_Collisions2019_Reprojected.
*Choose a location in your project folder and name it. Ex: Traffic_Collisions2019_Reprojected.
*On '''CRS''', click the world icon, search for "EPSG:2959" or "NAD83(CRSR) / UTM zone 18N", select it and press '''OK'''. [[File:Reprojecting_Layer.png|none|500px|link=MediaWiki]]
*On '''CRS''', click the world icon, search for "EPSG:2959" or "NAD83(CRSR) / UTM zone 18N", select it and press '''OK'''. [[File:Reprojecting_Layer.png|none|500px|link=MediaWiki]]




===Symbology===
===Symbology===


Symbology is a great visual method to understand and create distinction of map layers. It is recommended to try out various symbol combinations for all layers. This is exceptionally important at the end when analyzing our results and displaying them for an audience.
We use symbology as a method of better reference. I would play around as you wish in the properties of the Wards, Collisions and Roads layer. To make sure there isn't too much colour similarities. I recommend to also do this post clustering as you will be classifying separate clusters with different colours. The next step will help us classify roads better.
*Right click on the layer, go to '''Properties''', select '''Symbology''', and click '''OK''' once satisfied. Make sure to click '''Apply''' each time you change the symbology. [[File:SymbologyTab.png|none|600px|link=MediaWiki]]
*Right click on the layer, go to '''Properties''', select '''Symbology''', and click '''OK''' once satisfied. Make sure to click '''Apply''' each time you change the symbology. [[File:SymbologyTab.png|none|600px|link=MediaWiki]]


Line 79: Line 82:


*Repeat this for other roads by clicking '''+''' icon to add a new rule, type 'Other Roads' in '''Label''', and select '''Else''' as it's the other condition. In the '''Layers''' pane it should look something like this: [[File:RoadSeperation.png|none|300px|link=MediaWiki]]
*Repeat this for other roads by clicking '''+''' icon to add a new rule, type 'Other Roads' in '''Label''', and select '''Else''' as it's the other condition. In the '''Layers''' pane it should look something like this: [[File:RoadSeperation.png|none|300px|link=MediaWiki]]




== Analysis ==
== Analysis ==
Line 91: Line 92:
===DBSCAN clustering===
===DBSCAN clustering===
Density based clustering in QGIS will help us determine and locate clusters of accidents based on the point densities within our collisions dataset. This requires a minimum cluster size (# of points) and a maximum distance between these points. A scenario where this would be practical is if City of Ottawa wanted to know how many patrollers need to be allocated for this specific location.
Density based clustering in QGIS will help us determine and locate clusters of accidents based on the point densities within our collisions dataset. This requires a minimum cluster size (# of points) and a maximum distance between these points. A scenario where this would be practical is if City of Ottawa wanted to know how many patrollers need to be allocated for this specific location.



====Running the tool====
====Running the tool====
Line 98: Line 98:
**Tips: For min size, it is best to use a value greater than the dimension you are working in. For example, this is 2D so I would like to set my value at 3 or higher. Max distance depends fully on the input data and logical sense associated with it. It is recommended to attempt many combinations of each parameter to see what yields the best result for your objective.
**Tips: For min size, it is best to use a value greater than the dimension you are working in. For example, this is 2D so I would like to set my value at 3 or higher. Max distance depends fully on the input data and logical sense associated with it. It is recommended to attempt many combinations of each parameter to see what yields the best result for your objective.


*Based on this scenario, the parameter combination which captured both rural and urban clusters were a min size of 4 and distance of 500m.
Based on this scenario, the parameter combination which captured both rural and urban clusters were a min size of 4 and distance of 500m.
[[File:DBSCAN_rural_4min500m.png|none|500px|link=MediaWiki]]
[[File:DBSCAN_rural_4min500m.png|none|500px|link=MediaWiki]]


Line 116: Line 116:
*Select one of the points in the mega cluster using the selection icon to find its cluster ID. [[File:FindingLargeClusterID.png|none|500px|link=MediaWiki]]
*Select one of the points in the mega cluster using the selection icon to find its cluster ID. [[File:FindingLargeClusterID.png|none|500px|link=MediaWiki]]
*In the cluster layer attribute table, select '''Select by Expression''' and enter "CLUSTER_ID IS 'X'", then click '''Select Features''', and export this selected feature layer. [[File:SelectingLargeCluster.png|none|500px|link=MediaWiki]]
*In the cluster layer attribute table, select '''Select by Expression''' and enter "CLUSTER_ID IS 'X'", then click '''Select Features''', and export this selected feature layer. [[File:SelectingLargeCluster.png|none|500px|link=MediaWiki]]
*Repeat the same DBSCAN steps for urban clusters, however change the min cluster to a much higher value to account for frequent collision occurrence in urban centres.
*Repeat the same DBSCAN steps for urban clusters, however change the min cluster to a much higher value to account for frequent collision occurrence in urban centers. For this scenario, a value of 5x higher (min = 20) was chosen.


Below are the new results. It is a better representation than just running the dataset on a one-fits-all parameter, however the core of Ottawa still shows weak context: [[File:Final_DBSCAN_result.png|none|800px|link=MediaWiki]]
Below are the new results. It is a better representation than just running the dataset on a one-fits-all parameter, however the core of Ottawa still shows weak context: [[File:Final_DBSCAN_result.png|none|800px|link=MediaWiki]]


===K-means clustering===
===K-means clustering===
[[File:kmeans.png|150px|thumb|right|Figure 8: K-Means classification settings]] [[File:kmeansex.png|150px|thumb|right|Figure 9: K-Means example]]
K-means clustering will create clusters based on the amount given so you tell QGIS how many you want and they will give you that amount. Useful if you already have a set of officials and you want to split them.


K-means clustering creates clusters based on the number of clusters specified by the user. This is useful if you already have a set number of patrollers or divisions and want to geographically split their patrolling areas. Currently, Ottawa has 3 patrol operation [https://www.ottawapolice.ca/en/who-we-are/about-us/ divisions] which would show a practical, simplistic use of K-means. Assuming there is a surplus in available resources and we can show ideal, complex scenarios, the tutorial separates Ottawa into 20 divisions.
The steps are described below:


*Select '''K-means clustering''' tool, add the reprojected collisions layer into '''Input layer''', change '''Number of clusters''' to 20, provide a name and source location under '''Clusters''', and press '''Run'''. [[File:Kmeans_20_tool.png|none|500px|link=MediaWiki]]
*Select "K-means clustering"


*To visualize the output, repeat the same steps as done in for the DBSCAN clustering '''Visualizing results'''.
*You will get a window(as pictured in Figure 8) and select that same projected collisions vector layer from the other form of clustering


Below are the results. K-means clustering excels at dividing urban centers, unlike DBSCAN, and overall is better at geographically spreading operation divisions equally: [[File:Kmeans20_map_result.png|none|800px|link=MediaWiki]]
*You will then select a number of clusters that you want to separate the data in(as in the pictured example on that same Figure)

*Click OK

Figure 9 shows the results

To visualize the clustering, repeat the same steps to differentiate the colours as done in for the DBSCAN clustering. The result should be similar in terms of the coloured points we would get as a result despite the clusters being different in high likelihood.


===Concave Hull===
===Concave Hull===
[[File:selectbyex.png|150px|thumb|right|Figure 10: Select by expression window]]
We have now got our clusters that we desire but we want the officials to get a region assigned. How can we get those regions traced out more clearly. We have create some vector layers which will be polygons that will contain the points in each cluster. You will do those steps for both DBSCAN and K-means methods as they will be identical.


We now have our clusters and want to assign officials to a region with fixed boundaries. Concave Hull creates a vector polygon by tightly enclosing a boundary around the outer points within a cluster. By inputting either DBSCAN or K-means clustering output, we can make these polygons. Since both processes are identical, we will use K-means as it resulted in a preferred output.
To add those polygons we need to first select them by an expression:


With both K-means generating 20 clusters and DBSCAN with over 50, a streamlined method is more efficient than manually filtering shared cluster IDs to make individual polygons. Thus, we can apply batch processing to automate the Concave Hull tool.
*Right click on the Clusters


====Cluster splitting====
*Click on "Open Attribute Table"
Here we can split the individual cluster layer into multiple layers corresponding to its ID.
*Under '''Processing Toolbox''' search for and select '''Split Vector Layer''', and add the K-means layer to '''Input layer'''
*Using the drop down tab for '''Unique ID field''', select the attribute field for cluster IDs
*In '''Output file type''' select '''shp.''', then name and select the file destination. [[File:Split_IDs.png|none|500px|link=MediaWiki]]
*Find the chosen directory and its shapefiles under '''Browser''' tab, then drag and drop them into the map pane. [[File:FindingSplitLayers.png|none|200px|link=MediaWiki]]


====Running the tool====
*Click on "select features using an expression" which is in the icons over the attribute table
*After opening '''Concave Hull (by layer)''' tool, select '''Run as Batch Process'''. [[File:CH_Batch_Button.png|none|200px|link=MediaWiki]]

*Under '''Input layer''' double click '''Autofill...''' and choose '''Select Files...''' to add all 20 shp. layers. [[File:Select_Batch_Layer.png|none|200px|link=MediaWiki]]
*Then you type "CLUSTER_ID"(it show start showing up once you start typing it) and "=0" which is the first cluster(as pictured on Figure 10)
*Under '''Threshold''' insert "1" into the first row, then click '''Autofill...''' and '''Fill down'''. Do the same for '''Allow Holes''' and '''Multipart geometry into singleparts''' using "No".

**A threshold of 1 produces smoother boundaries with less extreme vertices, to capture a larger extent of the region.
*Click the "Select Features" button at the bottom of the window
*Under '''Concave Hull''' select the path and input a name.
[[File:concavehull.png|150px|thumb|right|Figure 11: Concave hull settings example]] [[File:concavehullres.png|150px|thumb|right|Figure 12: Concave hull outcome]]
*Lastly, make sure to check '''Load layers on completion''' to see the output in the map pane and press '''Run'''. [[File:CH_BatchProcess2.png|none|900px|link=MediaWiki]]

Keep that window open and go back to your processing toolbox:

*Search "Concave hull" or go to Vector geometry and select "Concave hull(alpha shapes)"

You will see a window like on Figure 11

*Choose your Cluster that you worked on then check the "Selected Features Only" box

*Give the threshold 0.8 and uncheck the "Allow Holes" box.


====Interpreting Results====
*Click OK


Below is the result of Concave Hull polygons overlayed by points for the specific cluster. This output provides us with a baseline on how to identify regions where patrolling operations should be designated.
You will then go back to the select by expression window and then change the CLUSTER_ID value to the next cluster and repeat for every cluster.


Further analysis can be made to account for the cluster size of a specific region. For example, urban clusters (e.g. Centretown) have higher frequency of collisions occurring, thus assigning more patrol officers to these municipalities can account for this demand. Whereas rural locations such as North Gower can be under less surveillance.
This should get you a polygon with a region for each official to go towards.


[[File:CH_Results_blackpoints_map.png|none|900px|link=MediaWiki]]
The outcome shown on Figure 12


== Conclusion ==
== Conclusion ==


The clustering methods in QGIS are very useful and relatively easy to use as shown in this tutorial. When using vector point data and there is a need for something to be divided, I think clustering does a good job at determining areas in which we can divide some tasks and for that it makes it an effective tool to use. I will say that the DBSCAN clustering method is incredibly particular and has given me some trouble to understand how it works. However, the solution to that issue which I have described is very straightforward and it is important to really follow the steps to use the tool. Doesn't matter if it's trying to investigate collisions or divide areas of garbageman using h
These clustering methods in QGIS are invaluable and relatively easy to use as shown in this tutorial. If there is a need for something to be divided, clustering performs well with vector data, making it an effective tool to use. Whether you are investigating collision trends or assigning door-to-door sales representative using house locations, these QGIS tools have incredible potential in terms of urban planning or any purposes where discrete divisions are required. While it may not be as advanced and intricate as other tools, such as ArcGIS Pro, QGIS is a great alternative to complete these tasks for its open source and easily accessible nature.
ouse locations, that tool has incredible potential in terms of urban planning or any purposes where we would need to divide and conquer. It might not be as pronounced as the tool we have for ArcGIS but it is importantly a good open source alternative that will do the job when other more expensive software is not widely available.


== Future Work ==
== Future Work ==


There definitely is a lot of future work in terms of the potential of something like this. You can add more components and maybe include some other GIS tools such as network analysis to calculate the most cost affective ways to investigate those areas(less gas consumption is better and could save city money). Use interpolation to maybe predict possible incidences of accidents. Future work more from a QGIS perspective is to include more methods of clustering as well. There is a lot that can be done around vector points that contain location data but that would be more suited for a separate tutorial not to overwhelm a user.
There definitely is a lot of future work in terms of the potential of something like this. You can add more components and maybe include some other GIS tools such as network analysis to calculate the most cost affective ways to investigate those areas (less gas consumption is better and could save city money). Use interpolation to maybe predict possible incidences of accidents. Future work more from a QGIS perspective is to include more methods of clustering as well. There is a lot that can be done around vector points that contain location data but that would be more suited for a separate tutorial not to overwhelm a user.


== References ==
== References ==

Latest revision as of 19:57, 6 October 2026

Introduction

The purpose of this project is to explore the capabilities of open source software such as QGIS, which is one of many open source softwares used in GEOM 4008. This tutorial will explore vector analysis tools used to create clusters and distinct boundaries. An in-depth approach will breakdown each step for new QGIS users, please feel free to skip any steps you are already familiar with.

Cluster analysis is a practical GIS technique which can group vector points into corresponding clusters based on distance, density, or set number of points, and is fully configurable by the user. This can be helpful if you have a large study extent and you want to geographically divide a variables into multiple areas, such as detecting regions of high painted turtle sightings or identifying crime hotspots. ArcGIS Pro has a larger collection of clustering options, however QGIS still has clustering capabilities which are DBSCAN and K-Means.

Here we will explore a local problem: A City of Ottawa employee on the Traffic Services Division team wants to investigate vehicle collision trends by locating accident clusters to recommend updated patrol divisions and officer coverage based on municipality demand.

Resources

Required Software

To run this analysis, it is best to have the latest version of QGIS.

  • If you have an old version, it is recommended to delete this before proceeding to further steps as having two versions of QGIS is not necessary and takes space on the hard drive.
  • Click on this link and you will see many options on operating systems you can download QGIS on.
  • Select either the fastest (4.2.2) or the more stable (3.44.14) download depending on your preference and it should start downloading.
  • Once it's done downloading, add it to your hard drive (if not already done automatically) and you are ready!

Required Datasets

The data that we will be using will be in two forms: Referencing data so we can visualize where we are and vector data to use for clustering.

Referencing

  • The Roads dataset will be used to visualize the roads, this is important to add as those incidents and vector data points are on roads so it can be a helpful tool to illustrate the incidents in a smaller scale.
  • The Wards dataset is a wards dataset which will be used to see where those incidents are with an Ottawa city border background. All data points will be located within that dataset.

Clustering

  • The Traffic Collisions by Location in 2019 dataset is the data set with all the traffic collisions and their frequencies. Their location information will help us separate them into clusters. Once you have added this dataset, I recommend a change as name as it will automatically save as a complicated alphanumeric sequence.

Environment Setup

Add vector data

  • All datasets can be downloaded by clicking on the cloud icon and selecting Shapefile. Organize these files into a project folder and then extract them.
    Download Data.png


  • Go on QGIS and go to Layer menu, Add Layer and select Add Vector Layer
    QGIS AddLayer.png


  • Under Source, click on the 3 dots, find and select each of your project's downloaded datasets (shp.), and Add.
    Upload Files.png


Projection

Projection is an important step for DBSCAN clustering, you may encounter problems if you do not do it this way. NAD83, specifically UTM zone 18, is an appropriate projection for this project as we are looking at a municipal problem, Ottawa traffic collisions. Projection is important as DBSCAN clustering relies on map distances.

Tip: All vector data should be projected that way.

To first orient yourself, we'll add Google Satellite basemap.

  • Select Plugins, then Manage and Install Plugins..., search "Basemaps", select Basemaps and Install.
    BasemapPlugin.png


  • Reselect Plugins, Basemap Management, Load Basemaps, then choose whichever you feel is best at representing your data. Google Satellite is a common and versatile option. Select your choosing and then Load.
    GoogleImagery.png


Now it's time to set the projection, to do so we will save those features with a new CRS:

  • Right click the Traffic Collisions 2019 layer, select Export then Save Feature As.
    ExportingLayers.png


  • Choose a location in your project folder and name it. Ex: Traffic_Collisions2019_Reprojected.
  • On CRS, click the world icon, search for "EPSG:2959" or "NAD83(CRSR) / UTM zone 18N", select it and press OK.
    Reprojecting Layer.png

Symbology

Symbology is a great visual method to understand and create distinction of map layers. It is recommended to try out various symbol combinations for all layers. This is exceptionally important at the end when analyzing our results and displaying them for an audience.

  • Right click on the layer, go to Properties, select Symbology, and click OK once satisfied. Make sure to click Apply each time you change the symbology.
    SymbologyTab.png


Rule based classification

To visually separate the highways from other roads, this can be done through symbology as well.

  • By going to the Symbology tab again, hover over the top drop down menu and select Rule-Based. Double click the row, type 'Highway' in Label, and then click the expression icon.
    Rulebased .png


  • Under the expression tab, enter "SUBCLASS" = 'Highway' by either manually typing it or under Field and Values, double click on SUBCLASS, double click =, type 'Highway', and then select OK.
    RuleBased Highways.png


  • Repeat this for other roads by clicking + icon to add a new rule, type 'Other Roads' in Label, and select Else as it's the other condition. In the Layers pane it should look something like this:
    RoadSeperation.png

Analysis

Once you are done setting up the environment, we can now start with the main purpose of the tutorial which is exploring two classification tools, DBSCAN and K-means, coupled with a boundary tool, Concave hull. Once this is done we will have a good picture on how we can proceed in assigning areas to investigate collisions in Ottawa.

  • To find the necessary tools, under the Processing tab select Toolbox, search or scroll to find both clustering methods within Vector analysis or Concave hull (by layer) within Vector geometry.
    FindingClusteringTools.png


DBSCAN clustering

Density based clustering in QGIS will help us determine and locate clusters of accidents based on the point densities within our collisions dataset. This requires a minimum cluster size (# of points) and a maximum distance between these points. A scenario where this would be practical is if City of Ottawa wanted to know how many patrollers need to be allocated for this specific location.

Running the tool

  • Select DBSCAN clustering, add projected collisions layer into Input layer, select a desired value for Minimum cluster size and Maximum distance between cluster points - we are working in meters so it is best to stay in this. Then label and specify the path and press Run.
    • Tips: For min size, it is best to use a value greater than the dimension you are working in. For example, this is 2D so I would like to set my value at 3 or higher. Max distance depends fully on the input data and logical sense associated with it. It is recommended to attempt many combinations of each parameter to see what yields the best result for your objective.

Based on this scenario, the parameter combination which captured both rural and urban clusters were a min size of 4 and distance of 500m.

DBSCAN rural 4min500m.png

Visualizing results

You will now have a new vector layer called "Clusters" with a table entry named "CLUSTER_ID". To visually differentiate these clusters, use unique colours.

  • Right click Clusters layer, go to "Symbology", select Categorized in the drop down menu of classification.
  • In Value, select "CLUSTER_ID", click on Classify to get a preview, scroll down and uncheck all other values to remove the noise (points that didn't fall into a cluster), and click Apply.
    DBSCAN cluster symbology.png

This is how it should appear:

DBSCAN rural output.png

Interpreting results

If the min cluster size was larger than 4, it would fail to capture rural areas. However, the output also poorly represents Ottawa's urban clusters in the downtown area by creating one mega cluster. The mega cluster contained over half of the dataset recorded collisions, thus providing no useful context. To account for this, re-running this tool on the mega cluster with same max density distance but large min clusters can help determine urban clusters.

  • Select one of the points in the mega cluster using the selection icon to find its cluster ID.
    FindingLargeClusterID.png
  • In the cluster layer attribute table, select Select by Expression and enter "CLUSTER_ID IS 'X'", then click Select Features, and export this selected feature layer.
    SelectingLargeCluster.png
  • Repeat the same DBSCAN steps for urban clusters, however change the min cluster to a much higher value to account for frequent collision occurrence in urban centers. For this scenario, a value of 5x higher (min = 20) was chosen.

Below are the new results. It is a better representation than just running the dataset on a one-fits-all parameter, however the core of Ottawa still shows weak context:

Final DBSCAN result.png

K-means clustering

K-means clustering creates clusters based on the number of clusters specified by the user. This is useful if you already have a set number of patrollers or divisions and want to geographically split their patrolling areas. Currently, Ottawa has 3 patrol operation divisions which would show a practical, simplistic use of K-means. Assuming there is a surplus in available resources and we can show ideal, complex scenarios, the tutorial separates Ottawa into 20 divisions.

  • Select K-means clustering tool, add the reprojected collisions layer into Input layer, change Number of clusters to 20, provide a name and source location under Clusters, and press Run.
    Kmeans 20 tool.png
  • To visualize the output, repeat the same steps as done in for the DBSCAN clustering Visualizing results.

Below are the results. K-means clustering excels at dividing urban centers, unlike DBSCAN, and overall is better at geographically spreading operation divisions equally:

Kmeans20 map result.png

Concave Hull

We now have our clusters and want to assign officials to a region with fixed boundaries. Concave Hull creates a vector polygon by tightly enclosing a boundary around the outer points within a cluster. By inputting either DBSCAN or K-means clustering output, we can make these polygons. Since both processes are identical, we will use K-means as it resulted in a preferred output.

With both K-means generating 20 clusters and DBSCAN with over 50, a streamlined method is more efficient than manually filtering shared cluster IDs to make individual polygons. Thus, we can apply batch processing to automate the Concave Hull tool.

Cluster splitting

Here we can split the individual cluster layer into multiple layers corresponding to its ID.

  • Under Processing Toolbox search for and select Split Vector Layer, and add the K-means layer to Input layer
  • Using the drop down tab for Unique ID field, select the attribute field for cluster IDs
  • In Output file type select shp., then name and select the file destination.
    Split IDs.png
  • Find the chosen directory and its shapefiles under Browser tab, then drag and drop them into the map pane.
    FindingSplitLayers.png

Running the tool

  • After opening Concave Hull (by layer) tool, select Run as Batch Process.
    CH Batch Button.png
  • Under Input layer double click Autofill... and choose Select Files... to add all 20 shp. layers.
    Select Batch Layer.png
  • Under Threshold insert "1" into the first row, then click Autofill... and Fill down. Do the same for Allow Holes and Multipart geometry into singleparts using "No".
    • A threshold of 1 produces smoother boundaries with less extreme vertices, to capture a larger extent of the region.
  • Under Concave Hull select the path and input a name.
  • Lastly, make sure to check Load layers on completion to see the output in the map pane and press Run.
    CH BatchProcess2.png

Interpreting Results

Below is the result of Concave Hull polygons overlayed by points for the specific cluster. This output provides us with a baseline on how to identify regions where patrolling operations should be designated.

Further analysis can be made to account for the cluster size of a specific region. For example, urban clusters (e.g. Centretown) have higher frequency of collisions occurring, thus assigning more patrol officers to these municipalities can account for this demand. Whereas rural locations such as North Gower can be under less surveillance.

CH Results blackpoints map.png

Conclusion

These clustering methods in QGIS are invaluable and relatively easy to use as shown in this tutorial. If there is a need for something to be divided, clustering performs well with vector data, making it an effective tool to use. Whether you are investigating collision trends or assigning door-to-door sales representative using house locations, these QGIS tools have incredible potential in terms of urban planning or any purposes where discrete divisions are required. While it may not be as advanced and intricate as other tools, such as ArcGIS Pro, QGIS is a great alternative to complete these tasks for its open source and easily accessible nature.

Future Work

There definitely is a lot of future work in terms of the potential of something like this. You can add more components and maybe include some other GIS tools such as network analysis to calculate the most cost affective ways to investigate those areas (less gas consumption is better and could save city money). Use interpolation to maybe predict possible incidences of accidents. Future work more from a QGIS perspective is to include more methods of clustering as well. There is a lot that can be done around vector points that contain location data but that would be more suited for a separate tutorial not to overwhelm a user.

References


Very good support video k-means clustering: