Calculating historical agricultural land differences using QGIS - Revision history

Smitch: fixing a few typos and grammar errors

2020-10-20T13:37:46Z

fixing a few typos and grammar errors

Opowell at 23:55, 7 October 2020

2020-10-07T23:55:06Z

Opowell at 23:52, 7 October 2020

2020-10-07T23:52:52Z

Opowell: /* Step 4: Dissolve Polygons */

2020-10-07T23:02:40Z

Step 4: Dissolve Polygons

Opowell at 22:38, 7 October 2020

2020-10-07T22:38:39Z

Smitch: /* Introduction */

2020-01-05T15:00:00Z

Introduction

Smitch: /* Data */

2020-01-05T00:44:26Z

Data

Smitch: /* Introduction */ Adapt text to better reflect aims of this course, and minor wording fixes

2020-01-05T00:43:11Z

Introduction: Adapt text to better reflect aims of this course, and minor wording fixes

JamesGriffin3: /* Final Output */

2019-12-21T16:56:59Z

Final Output

JamesGriffin3: /* References */

2019-12-21T16:28:09Z

References

@@ Line 3: / Line 3: @@
 One issue when working with large data sets is the massive resources required to render the files. This is not so bad when you have a computer powerful enough to handle them, or you absolutely need to have data that detailed, but there are problems in other situations. I found myself in such a situation about a year ago. I was attempting to graphically represent agricultural land attrition in Southern Ontario, from 1983 – 2017.
-The data I had downloaded was incredibly detailed, which is perfect if you wanted to grab a subset and analyze it on a micro level, but not so great if you want to analyze at a macro level. While I was able to complete my analysis, it was frustratingly time consuming. Every map movement caused a render lag, and any use of analysis tools resulted in a long wait.
+The data I had downloaded were incredibly detailed, which is perfect if you wanted to grab a subset and analyze it on a micro level, but not so great if you want to analyze at a macro level. While I was able to complete my analysis, it was frustratingly time consuming. Every map movement caused a render lag, and any use of analysis tools resulted in a long wait.
 This leads me to now. While I was not able to think up a way to do it back then, I figured now might be the time to try. This lead me to a feature in QGIS called ‘dissolve’. While this is a general tool in many GIS software packages, I decided to see if I could improve my past project, using QGIS.
@@ Line 26: / Line 26: @@
-In this image we can see the problem. The data is so dense that changes between the two cannot be seen. This data is really meant for querying, not so much for display.
+In this image we can see the problem. The data are so dense that changes between the two cannot be seen. This data is really meant for querying, not so much for display.
 [[File:problem.jpg|500px|thumb|center|Figure 1. Problem with data display]]
@@ Line 48: / Line 48: @@
 ===Step 3: Check Data Validity===
-This is a very important step. Unlike ArcGIS, if there is any errors or invalid data, QGIS will quit whatever process is running.
+This is a very important step. Unlike ArcGIS, if there are any errors or invalid data, QGIS will quit whatever process is running.
 Now that we have deleted all unnecessary fields, we need to be sure that there are no errors in our shapefile geometry. To do this, we need to use the Check Validity tool.
@@ Line 122: / Line 122: @@
 ==Conclusion==
-when you are working with large datasets that have lots of polygons you can use the dissolve tool to make the data more manageable however we need to be careful that we still keep the data accurate. If your computer is not powerful enough to do the calculations involved in this tool we also can use cloud computing to get this done in a more reasonable time. And in terms of the data we used in the example it turned out that using this method we were more easily able to tell that the amount of agricultural land decline by 66.8% from 1983 to 2017.
+When you are working with large datasets that have lots of polygons you can use the dissolve tool to make the data more manageable, however we need to be careful that we still keep the data accurate. If your computer is not powerful enough to do the calculations involved in this tool we also can use cloud computing to get this done in a more reasonable time. And in terms of the data we used in the example it turned out that using this method we were more easily able to tell that the amount of agricultural land declined by 66.8% from 1983 to 2017.
 == References ==

@@ Line 122: / Line 122: @@
 ==Conclusion==
-when you are working with large datasets that have lots of polygons you can use the dissolve tool to make the data more manageable however we need to be careful that we still keep the data accurate. If your computer is not powerful enough to do the calculations involved in this tool we also can use cloud computing to get this done in a more reasonable time.
+when you are working with large datasets that have lots of polygons you can use the dissolve tool to make the data more manageable however we need to be careful that we still keep the data accurate. If your computer is not powerful enough to do the calculations involved in this tool we also can use cloud computing to get this done in a more reasonable time. And in terms of the data we used in the example it turned out that using this method we were more easily able to tell that the amount of agricultural land decline by 66.8% from 1983 to 2017.
 == References ==

@@ Line 119: / Line 119: @@
 </td></tr></table>
+==Conclusion==
+when you are working with large datasets that have lots of polygons you can use the dissolve tool to make the data more manageable however we need to be careful that we still keep the data accurate. If your computer is not powerful enough to do the calculations involved in this tool we also can use cloud computing to get this done in a more reasonable time.
 == References ==

@@ Line 70: / Line 70: @@
 ===Step 4: Dissolve Polygons===
-The dissolve tool will allow us to merge all the individual polygons into one. This should make it much less resource intensive to render the data. It will also merge the area for each polygon into one attribute. This will make it easy to compare with the field calculator later on. Fair warning, this process for both files was somehow the longest, and most varied in terms of time. Each file will need anywhere from 15 minutes to more than five hours to process. This appears to be based on the number of attributes, as well as relative density of the data. With closer together 'dense' data being faster to process than more spread out 'less dense' data.
+The dissolve tool will allow us to merge all the individual polygons into one. This should make it much less resource intensive to render the data. It will also merge the area for each polygon into one attribute. This will make it easy to compare with the field calculator later on. Fair warning, this process for both files was somehow the longest, and most varied in terms of time. Each file will need anywhere from 15 minutes to more than five hours to process. This appears to be based on the number of attributes, as well as relative density of the data. With closer together 'dense' data being faster to process than more spread out 'less dense' data. think of this tool being like an addition tool for your polygons it will add all of the surface area into one so that when we move it around and manipulate it the computer doesn't have to do this with over 5000 polygons it just has to do this with one polygon which is much simpler. we can use this tool for this because we are using area as our variable some variables would need to be kept in their separate polygons, so make sure to check that your data is compatible before using this tool.
 [[File:dissolve_nav.jpg|500px|thumb|center|Figure 9. Navigate to the check validity tool]]
 Once completed, we can then finish the processing by dissolving the individual breaks, and creating a seamless surface.

@@ Line 3: / Line 3: @@
 One issue when working with large data sets is the massive resources required to render the files. This is not so bad when you have a computer powerful enough to handle them, or you absolutely need to have data that detailed, but there are problems in other situations. I found myself in such a situation about a year ago. I was attempting to graphically represent agricultural land attrition in Southern Ontario, from 1983 – 2017.
-The data I had downloaded were incredibly detailed, perfect for if you wanted to grab a subset and analyze it on a micro level, but not so great when you want to analyze at a macro level. While I was able to complete my analysis, it was frustratingly time consuming. Every map movement caused a render lag, and any use of analysis tools resulted in a long wait.
+The data I had downloaded was incredibly detailed, which is perfect if you wanted to grab a subset and analyze it on a micro level, but not so great if you want to analyze at a macro level. While I was able to complete my analysis, it was frustratingly time consuming. Every map movement caused a render lag, and any use of analysis tools resulted in a long wait.
-This leads me to now. While I was not able to think up a way to do it back then, I figured now might be the time to try. This lead me to a feature in QGIS called ‘dissolve’. While this is a general tool in many GIS software packages, the purpose of the assignment creating this tutorial was to explore new software, so I decided to see if I could improve my past project, using QGIS.
+This leads me to now. While I was not able to think up a way to do it back then, I figured now might be the time to try. This lead me to a feature in QGIS called ‘dissolve’. While this is a general tool in many GIS software packages, I decided to see if I could improve my past project, using QGIS.
+==Cloud Computing==
+Cloud computing is a fairly recent solution to having a large dataset that can take a long time to compute and also solve the issue of not having the correct resources in order to render the files. cloud computing can be useful if for some reason you can't use the dissolve tool mentioned above or if you do not have a particularly powerful computer, it can allow you to still do the work that you want to get done in a reasonable amount of time. there are several different ways to do this cloud computing and it depends on the type of software that you want to use however several large online companies such as amazon and google have services that if you're data is too large and you can not compute the layers you want to on the computer that you have then you can rent space on their servers to do this. this is a good way of getting your layers computed while still being able to do other work in the meantime.
 ==Data==

@@ Line 8: / Line 8: @@
 ==Data==
-Two files are needed for thhis project, the 1983 Agricultural Resource Inventory, and the most current Agricultural Resource Inventory. The source is constantly being updated, so I have provided a link to the homepage of each file, rather than a direct download link. Once on the page, select 'open', and the data will begin downloading.
+Two files are needed for this project: the 1983 Agricultural Resource Inventory, and the most current Agricultural Resource Inventory. The source data are constantly being updated, so I have provided a link to the homepage of each file, rather than a direct download link. Once on the page, select 'open', and the data will begin downloading.
 For the 1983 data, please visit: https://geohub.lio.gov.on.ca/datasets/agricultural-resource-inventory-1983

@@ Line 2: / Line 2: @@
 ==Introduction==
-One issue when working with large sized data sets is the massive resources required to render the files. This is not so bad when you have a computer powerful enough to handle it, or you absolutely need to have data that detailed, but it is when you do not. I found myself in that situation about a year ago. I was attempting to graphically represent agricultural land attrition in Southern Ontario, from 1983 – 2017.
+One issue when working with large sized data sets is the massive resources required to render the files. This is not so bad when you have a computer powerful enough to handle them, or you absolutely need to have data that detailed, but there are problems in other situations. I found myself in such a situation about a year ago. I was attempting to graphically represent agricultural land attrition in Southern Ontario, from 1983 – 2017.
-The data I had downloaded was incredibly detailed, perfect for if you wanted to grab a subset and analyze it on a micro level. Not so great when you are trying to analyze on a macro level. While I was able to complete my analysis, it was frustratingly time consuming. Every map movement caused a render lag, and any use of analysis tools resulted in a long wait.
+The data I had downloaded were incredibly detailed, perfect for if you wanted to grab a subset and analyze it on a micro level, but not so great when you want to analyze at a macro level. While I was able to complete my analysis, it was frustratingly time consuming. Every map movement caused a render lag, and any use of analysis tools resulted in a long wait.
-This leads me to now. While I was not able to think up a way to do it back then, I figured now might be the time to try. This lead me to a feature in QGIS called ‘dissolve’. While it is also available in ArcPro, that is not the purpose of this assignment. I decided to see if I recreate my project, using QGIS instead of Arc, and perhaps improve upon it a little.
+This leads me to now. While I was not able to think up a way to do it back then, I figured now might be the time to try. This lead me to a feature in QGIS called ‘dissolve’. While this is a general tool in many GIS software packages, the purpose of the assignment creating this tutorial was to explore new software, so I decided to see if I could improve my past project, using QGIS.
 ==Data==

@@ Line 110: / Line 110: @@
 [[File:final_output.jpg|500px|thumb|center|Figure 15. Final map output after all processing]]
+Using the field calculator (sort of) we are able to calculate the percent difference between 1983 and 2017. The difference is highlighted in red. 66.8% drop from 1983 to 2017.
+[[File:percent.jpg|500px|thumb|center|Figure 16. Percent difference between 1983 & 2017]]
 </td></tr></table>

← Older revision		Revision as of 16:28, 21 December 2019
Line 114:		Line 114:

	== References ==		== References ==

	<ref> <sup>1</sup> The European Space Agency,[https://sentinels.copernicus.eu/web/sentinel/missions/sentinel-2 "Sentinel-2"], Retrieved: 2019-11-05 </ref>