Data models

Service metadata

Service metadata is used to describe the interface of a processing service so it can be executed by Steep.

Property	Type	Description
id (required)	string	A unique service identifier
name (required)	string	A human-readable name
description (required)	string	A human-readable description
path (required)	string	Relative path to the service executable in the service artefact (or a container image if `runtime` equals `docker` or `kubernetes`)
runtime (required)	string	The name of the runtime to execute the service (valid values are `other`, `docker`, `kubernetes`, or the name of a custom runtime)
parameters (required)	array	A list of service parameters
runtimeArgs (optional)	array	An optional list of arguments to pass to the runtime
requiredCapabilities (optional)	array	A set of strings specifying capabilities a host system must provide to be able to execute this service. See also setups.
retries (optional)	object	An optional retry policy specifying how often the execution of this service should be retried in case of an error. Can be overridden in an executable action.
maxInactivity (optional)	duration or object	An optional duration or timeout policy that defines how long the execution of this service can take without producing any output (i.e. without writing anything to the standard output and error streams) before it is automatically cancelled or aborted. Can be overridden in an executable action and can be combined with `maxRuntime` and `deadline` (see below). Note that a service cancelled due to inactivity is still subject to any configured retry policy, which means its execution may be retried even if one attempt timed out. If you want to cancel a long-running service immediately even if there is a retry policy configured, use a `deadline`.
maxRuntime (optional)	duration or object	An optional duration or timeout policy that defines how long the execution of this service can take before it is automatically cancelled or aborted, even if the service regularly writes to the standard output and error streams. Can be overridden in an executable action and can be combined with `maxInactivity` (see above) and `deadline` (see below). Note that a service cancelled due to a too long runtime is still subject to any configured retry policy, which means its execution may be retried even if one attempt timed out. If you want to cancel a long-running service immediately even if there is a retry policy configured, use a `deadline`.
deadline (optional)	duration or object	An optional duration or timeout policy that defines how long the execution of this service can take at all (including all retries and their associated delays) until it is cancelled or aborted. Can be overridden in an executable action and can be combined with `maxInactivity` and `maxRuntime` (see above).

Example

id: cp
name: cp
description: Copies files
path: cp
runtime: other
parameters:
  - id: no_overwrite
    name: No overwrite
    description: Do not overwrite existing file
    type: input
    cardinality: 1..1
    label: '-n'
    dataType: boolean
    default: false
  - id: input_file
    name: Input file name
    description: Input file name
    type: input
    cardinality: 1..1
    dataType: file
  - id: output_file
    name: Output file name
    description: Output file name
    type: output
    cardinality: 1..1
    dataType: file

Service parameters

This data model describes the parameters that can be passed to a processing service.

Property	Type	Description
id (required)	string	A unique parameter identifier
name (required)	string	A human-readable name
description (required)	string	A human-readable description
type (required)	string	The type of this parameter. Valid values: `input`, `output`
cardinality (required)	string	A string in the form `lower..upper` specifying how many times the parameter must appear at least (lower limit) and how many times it can appear at most (upper limit). The character `n` can be used for the upper limit to specify an arbitrary number. The lower limit must not be greater that the upper limit. Examples cardinalities are listed below.
dataType (optional)	string	The type of the parameter value. Steep treats parameters differently depending on the data type (see description below).
default (optional)	string	An optional default value for this parameter that will be used if the lower limit of `cardinality` is 1 but no parameter value is given in the workflow.
fileSuffix (optional)	string	An optional suffix that should be appended to the generated filename of an `output` parameter. This property is typically used for file extensions (including the dot), e.g. `".xml"` or `".json"`.
label (optional)	string	An optional string that will be used as a label for the parameter in the service call. Examples are `-i`, `--input`, `--resolution`, etc.

Example cardinalities:

"0..1" means the parameter is optional (it can appear 0 times or 1 time)
"1..1" means the parameter is mandatory (it must appear 1 time)
"1..n" means it must appear at least once or many times (no upper limit)

Data type:

Steep treats parameters differently depending on the type and dataType:

If type is "output" and dataType is "directory", Steep will create a new directory for the service’s output and recursively search it for result files after the service has been executed.
If type is "input" and dataType is "directory", Steep will find the common parent directory of the files from the parameter’s value and pass it to the service. For example, if the parameter’s value is an array with the elements ["/tmp/a.txt", "/tmp/b.txt", "/tmp/subdir/c.txt"], Steep will pass "/tmp/" to the service.
If type is "input", dataType is not "directory", but the parameter’s value is an array, Steep will duplicate the parameter as many times as there are items in the array (given that the cardinality has no upper limit).
If type is "input", dataType is "boolean", and the parameter has a label, Steep will pass the label to the service if the parameter’s value is true and ignore the parameter if the value is false.
Otherwise, this property can be an arbitrary string. New data types with special handling can be added through output adapter plugins.

Example

id: no_overwrite
name: No overwrite
description: Do not overwrite existing file
type: input
cardinality: 1..1
label: '-n'
dataType: boolean
default: false

Runtime arguments

Runtime arguments are similar to service parameters, except they are passed to the runtime that executes the service instead of the service itself.

Property	Type	Description
id (required)	string	A unique argument identifier
name (required)	string	A human-readable name
description (required)	string	A human-readable description
dataType (optional)	string	The type of the parameter value. Typically `"string"` or `"boolean"`. The same rules apply as for service parameters.
label (optional)	string	An optional string that will be used as a label for the parameter. Examples are `-v`, `--volume`, `--entrypoint`, etc.
value (optional)	string	An optional value for this parameter.

Example

id: volume
name: Volume mount
description: Mount data directory
label: '-v'
value: /data:/data

Runtimes

Steep provides a set of default runtimes to execute processing services: other, docker, and kubernetes. More runtimes can be added through plugins.

Other runtime

Name: other

The other runtime is the default runtime of Steep. The service will be executed like a normal executable program on the same host as the Steep agent.

The path attribute of the service metadata refers to the program to execute. It can either be an absolute path to an executable binary or shell script or a name of a built-in shell function.

Docker runtime

Name: docker

This runtime executes the service through Docker.

The service metadata attribute path specifies the Docker image to run. It can be an image name followed by an optional tag and/or digest (e.g. ubuntu or ubuntu:23.10).

The attribute runtimeArgs specifies parameters that should be forwarded to the docker run command. For example, the following runtime argument mounts a volume to the started container:

runtimeArgs:
  - id: myCustomVolumeMount
    name: Data volume
    description: Mount /data into the container
    label: -v
    value: /data:/data

Please refer to the documentation of the docker run command for a complete list of possible arguments.

Kubernetes runtime

Name: kubernetes

This runtime executes the service as a Kubernetes job.

The service metadata attribute path specifies the container image to run. It can be an image name followed by an optional tag and/or digest (e.g. ubuntu or ubuntu:23.10). See the Kubernetes documentation for more information about valid image names.

The runtimeArgs attribute can be used to override or amend the Kubernetes runtime configuration as specified in the steep.yaml file. Find a list of valid properties below.

Note that some of the properties are objects or arrays although the runtime arguments data model only allows strings as values. While reading the configuration file services.yaml, Steep automatically converts objects and arrays to strings and the Kubernetes runtime interprets those correctly later.

env

A list of environment variables that should be injected into the container. The list items should be Kubernetes environment variable objects. See the Kubernetes API reference for more information. The following example specifies an environment variable FOO with the value bar:

runtimeArgs:
  - id: env
    name: Environment variables
    description: Additional environment variables
    value:
      - name: FOO
        value: bar

The environment variables specified here will be added to the ones set in the Kubernetes runtime configuration through the steep.runtimes.kubernetes.env configuration item.

imagePullPolicy

The image pull policy for the started Kubernetes job. See the Kubernetes documentation or the imagePullPolicy parameter of the Container object in the Kubernetes API reference for more information.

runtimeArgs:
  - id: imagePullPolicy
    name: Image pull policy
    description: Image pull policy
    value: Always

This value overrides the one specified in the Kubernetes runtime configuration through the steep.runtimes.kubernetes.imagePullPolicy configuration item.

imagePullSecrets

A list of image pull secrets for the started job. See the Kubernetes documentation or the Kubernetes API reference for more information.

runtimeArgs:
  - id: imagePullSecrets
    name: Image pull secrets
    description: Image pull secrets
    value:
      - name: my-secret

The image pull secrets specified here will be added to the ones set in the Kubernetes runtime configuration through the steep.runtimes.kubernetes.imagePullSecrets configuration item.

volumes

A list of Kubernetes volumes. The Kubernetes runtime attaches these volumes to the started pod. For example, the following configuration defines a volume that refers to a host path:

runtimeArgs:
  volumes:
    - name: steep-tmp-path
      hostPath:
        path: /path/to/steep/tmp_path

You can also refer to persistent volume claims:

runtimeArgs:
  volumes:
    - name: steep-tmp-path
      persistentVolumeClaim:
        claimName: steep-tmp-path-volume-claim

See the Kubernetes API reference for more information about volumes.

The volumes specified here will be added to the ones set in the Kubernetes runtime configuration through the steep.runtimes.kubernetes.volumes configuration item.

volumeMounts

A list of Kubernetes volume mount objects. The Kubernetes runtime mounts volumes specified here into the started container. For example, the following configuration mounts a volume with the name steep-tmp-path to the path /tmp/steep/tmp inside the container:

volumeMounts:
  - name: steep-tmp-path
    mountPath: /tmp/steep/tmp

The volume steep-tmp-path must be defined through another runtime argument with the ID volumes or through the configuration item steep.runtimes.kubernetes.volumes.

See the Kubernetes API reference for more information about volume mount objects.

The volume mounts specified here will be added to the ones set in the Kubernetes runtime configuration through the steep.runtimes.kubernetes.volumeMounts configuration item.

Submissions

Macros