Tutorials

Your first workflow

Learn how to run your first simple workflow. This tutorial assumes that you’ve already downloaded and extracted Steep and that it is running and listens to incoming requests on port 8080.

A Steep workflow consists of a set of processing services that are executed in a specific order (see How does Steep work?). Every service that should be executed has to be described with so-called service metadata.

In order to run a workflow, you therefore need at least two things:

The workflow itself as a YAML document or JSON file (we prefer YAML for its readability).
The service metadata for all services that are used in the workflow. This metadata has to be stored in the services configuration file. The default path to this file is conf/services/services.yaml in Steep’s application directory but can be configured.

Step 1: Add service metadata

Let’s assume we want to create a workflow that creates a copy of a file. For this, we can use the cp command, which is built-in into all UNIX distributions (under Windows, you can use the copy command instead). Open the file conf/services/services.yaml and add service metadata below.

The default distribution of Steep already contains metadata for this service, so you actually don’t need to do anything here.

conf/services/services.yaml

yaml

- id: cp
  name: Copy
  description: Create a copy of a file
  path: cp
  runtime: other
 
  parameters:
    - id: input_file
      name: Input file name
      description: The file to copy
      type: input
      cardinality: 1..1
      dataType: file
 
    - id: output_file
      name: Output file name
      description: The target to where the file will be copied
      type: output
      cardinality: 1..1
      dataType: file

This metadata describes a service with the ID cp. It has a human-readable name and description. The path attribute specifies the path to the service’s executable. Since the cp command is built-in, we can just use cp here (replace this with copy under Windows).

The service has two parameters, input_file and output_file, which represent the service’s input and output parameters, respectively. Both have a cardinality of 1..1, which means they are mandatory. They also have human-readable names and descriptions.

Steep uses this metadata later during workflow execution to generate a command line in the following form:

cp <input_file> <output_file>

Step 2: Restart Steep

Restart Steep if it is running, so it can pick up the updated configuration.

Step 3: Create the workflow

Now, let’s create a workflow that uses the new cp service. Create a file copy.yaml anywhere on your computer and add the following content:

copy.yaml

api: 4.7.0
actions:
  - type: execute
    service: cp
    inputs:
      - id: input_file
        value: /path/to/example.txt
    outputs:
      - id: output_file
        var: o1
        store: true

The workflow consists of one execute action calling our cp service. /path/to/example.txt is the path to the file that should be copied. Replace this with the absolute path to any file on your computer.

The name of the output file will be generated later during workflow execution by Steep. It will be stored in the variable o1.

Any output filename generated is relative to either Steep’s temporary directory (tmpPath) or output directory (outPath), which can be configured in the general configuration file. In our workflow, we set store to true to generate a filename in the output directory.

Step 4: Submit the workflow

It’s now time to submit the workflow to Steep. Run the following command on your terminal:

Terminal

shell

curl -X POST http://localhost:8080/workflows --data-binary @copy.yaml

Steep will return a submission object with runtime information about the workflow including a unique ID. You can either open the web-based user interface at http://localhost:8080 or monitor the workflow execution on the command line:

Terminal

shell

curl http://localhost:8080/workflows/[WORKFLOW ID]

Replace [WORKFLOW ID] with the unique ID returned by Steep after submitting the workflow.

When the workflow has finished, its status will change to SUCCESS. You will find the file copy in the output directory, which is /tmp/steep/out by default. Steep creates a subdirectory for each workflow based in the workflow’s ID:

Terminal

shell

ls /tmp/steep/out/[WORKFLOW ID]

How does Steep work?

Sequential workflows