Difference between revisions of "Automating SAGA Workflows Using Command Line Scripting"

From CUOSGwiki
Jump to navigationJump to search
(Rewrote introduction to clarify tutorial goals and wording)
(Updated installation instructions to more clearly explain setting the path variable)
Line 8: Line 8:
   
 
Automating GIS tasks offers industry a means to efficiently repeat the same task. Doing so can reduce manual errors in processes as well as increase productivity. Many platforms offer this feature, but this tutorial will focus on the System for Automated Geoscientific Analyses (SAGA), created by the Department of Physical Geography at the University of Hamburg in Germany. It is one of the more popular free open-source software and offers both a command line option as well as Python integration for executing GIS tasks.
 
Automating GIS tasks offers industry a means to efficiently repeat the same task. Doing so can reduce manual errors in processes as well as increase productivity. Many platforms offer this feature, but this tutorial will focus on the System for Automated Geoscientific Analyses (SAGA), created by the Department of Physical Geography at the University of Hamburg in Germany. It is one of the more popular free open-source software and offers both a command line option as well as Python integration for executing GIS tasks.
 
=== Installation ===
 
 
To install the software, please use the links below to install SAGA or Python.
 
 
[[File:logo_saga.png|100px|link=http://www.saga-gis.org/en/index.html]] [[File:python.jpeg|150px|link=https://www.python.org/]]
 
 
   
 
=== Objectives ===
 
=== Objectives ===
Line 24: Line 17:
 
3. Provide implementation differences between Windows and Unix environments
 
3. Provide implementation differences between Windows and Unix environments
   
  +
== Getting Started ==
   
 
=== Installation ===
   
 
To install the software, please use the links below to install SAGA or Python as required.
== Methodology ==
 
   
 
[[File:logo_saga.png|100px|link=http://www.saga-gis.org/en/index.html]] [[File:python.jpeg|150px|link=https://www.python.org/]]
   
  +
To access the SAGA tools from a command line environment, you will use the saga_cmd tool (saga_cmd.exe on Windows). To make running this easier, you can add the path to the SAGA directory to your system path so that your operating system knows where to find saga_cmd. To do so, use the following commands depending on your operating system, replacing [SAGA_PATH] with the path to the directory where SAGA was installed. Otherwise, you'll have to enter the full path every time.
'''SAGA_CMD'''
 
   
  +
{| class="wikitable"
Exploring the environment of the command line application (saga_cmd)
 
  +
|+ Setting the PATH variable
  +
|-
  +
! Windows !! Mac / Unix
  +
|-
  +
| <code>set PATH=%PATH%;[SAGA_PATH]</code> || <code>export PATH=$PATH:[SAGA_PATH]</code>
  +
|}
   
The following are the basics for understanding the use of SAGA at the command line. Users can use the command line to quickly and efficiently run scripts to execute functions with specified parameters. This can be done either at a Windows command prompt, or a UNIX shell prompt. Although there are various UNIX shells available, probably the most popular one now, thanks to being the default shell for MacOS and most Linux distributions, is Bash. In the examples below, Bash syntax is used, and "[linux]" is used as a short form for any operating system that is running the bash shell.
 
   
1. First, you many need to make an alteration to your shell environment so that it knows where to find the saga program. The "path" is a variable telling the shell where and how to access files. If you type saga_cmd at the command prompt and get an error saying that the command is not found, you need to add the directory that has your saga executable program in it to the path. Find that directory and substitute the full path name to it where it says "PATHNAME", below. To modify the path variable the following command is used:
 
   
 
== Methodology ==
set PATH=%PATH%;C:\”PATHNAME” [windows]
 
   
OR
 
   
 
'''SAGA_CMD'''
export PATH=$PATH:/”PATHNAME” [linux]
 
   
 
Exploring the environment of the command line application (saga_cmd)
In both examples above, the path to the saga_cmd program should be appended to the pre-existing path. You can check this below.
 
   
 
The following are the basics for understanding the use of SAGA at the command line. Users can use the command line to quickly and efficiently run scripts to execute functions with specified parameters. This can be done either at a Windows command prompt, or a UNIX shell prompt. Although there are various UNIX shells available, probably the most popular one now, thanks to being the default shell for MacOS and most Linux distributions, is Bash. In the examples below, Bash syntax is used, and "[linux]" is used as a short form for any operating system that is running the bash shell.
To see where a variable is that already has a path set, users can “print” the contents of the variable by using the following:
 
 
echo %PATH% [windows]
 
 
OR
 
 
echo $PATH [linux]
 
   
Now we can start using the saga executable at the command/shell prompt.
 
   
 
To access help files the following script can be used:
 
To access help files the following script can be used:

Revision as of 09:04, 1 October 2020


Please note that this tutorial is not quite finished. Since there isn't a lot of material out there on this topic, it will hopefully prove useful to anyone researching this topic, and future contributions/edits are certainly welcome!


Introduction

Automating GIS tasks offers industry a means to efficiently repeat the same task. Doing so can reduce manual errors in processes as well as increase productivity. Many platforms offer this feature, but this tutorial will focus on the System for Automated Geoscientific Analyses (SAGA), created by the Department of Physical Geography at the University of Hamburg in Germany. It is one of the more popular free open-source software and offers both a command line option as well as Python integration for executing GIS tasks.

Objectives

1. Understand how to call SAGA tasks from the command line

2. Understand how to automate sequences of tasks using the command line and Python

3. Provide implementation differences between Windows and Unix environments

Getting Started

Installation

To install the software, please use the links below to install SAGA or Python as required.

Logo saga.png Python.jpeg

To access the SAGA tools from a command line environment, you will use the saga_cmd tool (saga_cmd.exe on Windows). To make running this easier, you can add the path to the SAGA directory to your system path so that your operating system knows where to find saga_cmd. To do so, use the following commands depending on your operating system, replacing [SAGA_PATH] with the path to the directory where SAGA was installed. Otherwise, you'll have to enter the full path every time.

Setting the PATH variable
Windows Mac / Unix
set PATH=%PATH%;[SAGA_PATH] export PATH=$PATH:[SAGA_PATH]


Methodology

SAGA_CMD

Exploring the environment of the command line application (saga_cmd)

The following are the basics for understanding the use of SAGA at the command line. Users can use the command line to quickly and efficiently run scripts to execute functions with specified parameters. This can be done either at a Windows command prompt, or a UNIX shell prompt. Although there are various UNIX shells available, probably the most popular one now, thanks to being the default shell for MacOS and most Linux distributions, is Bash. In the examples below, Bash syntax is used, and "[linux]" is used as a short form for any operating system that is running the bash shell.


To access help files the following script can be used:

$ saga_cmd [--help] [windows]

OR

saga_cmd [-h] [linux]


This pulls up a way to decode command scripts (see figure 1)

Fig1.jpg

Figure 1. help file outputs in SAGA.

By using various series of this language, the user is able to automatically pick from the library, module, options and produce a script.

To call on the library, a simple call can be used (“saga_cmd”). Simply calling on the command line automatically produces a list of the module libraries (see figure 2).


 Fig2.jpg

Figure 2. Command line results in SAGA for calling on the existing modules by using the command line.


To call on the library, which identifies functionality within a module, the user must input the name of the module after calling on “saga_cmd”. The result ends up with a list of the tools and functions of that specific library (see figure 3).

Fig3.jpg

Figure 3. SAGA output results where calling on the command line and the associated library (this is using saga_cmd ta_lighting to call on the lighting module in SAGA).


For example, a workflow for hillshade analysis (calculating the angle between the surface and the incoming light beams in radians) can be used by automating the process of taking ASCII grids or DEMS, converting them into SAGA grids and then creating an analytical hillshade for each tile. The following command would be used to pull up all parameters in the hillshade analysis (see figure 4). Parameters must be defined and then can be run within the command line.

$ saga_cmd ta_lighting "Analytical Hillshading"

Reminder: (“ta_lighting” refers to the module it belongs to and “Analytical Hillshading” refers to the function name in the library of the module).

Fig4.jpg

Figure 4. Variables to define in hillshade analysis.


Also, when left clicking “Analytical Hillshading” in the index, you can select “Create Script Command File” (see figure 5), where you can copy and paste command snippets to alter parameters (see figure 6).

Fig5.jpg

Figure 5. Left clicking in index allows option to “Create Script Command File” to enable editing process of parameters included in module.


Fig6.jpg

Figure 6. Script command created in process of figure 5.


When doing more basic tasks in SAGA, the command line is the most useful, as it requires the least amount of code associated with these processes. If the user is more interested in doing a more complicated analysis, or combine various tools and functions, this is where an outside program needs to be used. In this case, Python has been chosen to modify scripts and increase the functionality of SAGA.


Using Python

By using a similar workflow as the saga_cmd, the following can be used in Python to do the same tasks.

To call on module library:

mlb= saga_api.CSG_Module_Library

To choose a module from the library:

m= mlb.Get_Module(1)

  • The number 1 refers to the ID number of the module (see figure 1 for that list in saga_cmd). You can also add in the name of the module, m= mlb.Get_Module(‘Analytical Hillshade”)

To get and set parameters:

P =m.Get_Parameters()

P(‘FILE’).Set_Value(fASC)


To run module:

print m.Get_Name() + ‘ : ’ = p( ‘FILE’ ) .asString()

(Sorry, to be continued...)

For More Information...

http://sourceforge.net/p/saga-gis/wiki/Creating%20Python%20scripts/


http://sourceforge.net/projects/saga-gis/files/SAGA%20-%20Documentation/Tutorials/Command_Line_Scripting/