This tutorial teaches you how to create workflows that run two or more services in sequence. It assumes that you’ve familiarised yourself with the basic concepts of Steep and its core data models (workflows and service metadata), and that you’ve already submitted your first workflow.
Step 1: Add more service metadata
In this tutorial, we are going to create a workflow that downloads a
.zip file from an HTTP server and then extracts it. For this, we will use the
unzip programs. Similar to the first tutorial, we need to add service metadata for these services to the
First, add the following metadata for the
Similar to the
cp service from the first tutorial, this service has two parameters. However, in this case, the output parameter has been specified first and has a label
-O. Steep will respect the order of the parameters and the label when calling
wget later during workflow execution and generate a command line in the following form:
The output parameter also has a
fileSuffix. Specifying this will make sure Steep generates an output filename with the correct extension (in this case
.zip) later during workflow execution.
Next, we need to add the metadata for the
In this case, the output parameter has a label
-d and is a directory. Steep
is able to automatically create output directories during workflow execution.
Don’t forget to restart Steep after changing the service metadata configuration file.
Step 2: Create the workflow
Create a new file
download.yaml and paste the following workflow into it:
The workflow consists of two execute actions calling the
download and the
extract service. It downloads the source code of this website from GitHub and extracts it into Steep’s output directory.
Steep is able to automatically detect the dependency between the two actions based on their inputs and outputs.
The output of the
download action is stored in the variable
o1, which is in turn used as input for the
extract action. This tells Steep to execute the download action first, wait for it to finish, and then execute the extract action.
Step 3: Submit the workflow
Run the following command on your terminal to submit the new workflow to Steep:
Similar to the first tutorial, Steep will return a workflow ID.
Wait until the workflow has finished. If it was successful, you will find the extracted archive in Steep’s output directory:
Since we did not specify the
store flag for the
download action, the downloaded
.zip file will be stored in Steep’s temporary directory, which is
/tmp by default:
[WORKFLOW ID] with the ID returned by Steep.