Data models

Full-text search

Steep contains a powerful full-text search engine that be used to find submissions and process chains in the backend database. The search engine can be accessed through the /search HTTP endpoint or through the web-based user interface.

Query language

The focus of Steep’s query language is on end users. It is lightweight and contains only a few operators. There are no logical combinators such as AND or OR. The main idea is that Steep tries to find as many objects matching the query as possible and then sorts them by relevance, so the best matches are on the top of the list. It is assumed that workflows have a higher relevance than process chains. If you are using the search through the web interface, you will typically find what you are looking for on one of the first pages. You rarely have to access any page after the second or third one. This is similar to how a web search engine works.

A search query consists of one or more of the following elements:

Term

A string that should appear somewhere in the document to find.

Put one or more terms into quotation marks (single or double) if you want to look for exact matches (including spaces). Within quoted terms, quotation marks can be escaped with the backslash character \.

Examples:

docker
"exact match"
"an \"exact match\" with quotation marks"

Date

A string in the form yyyy-MM-dd

The operators < (less than), <= (less than or equal to), > (greater than), and >= (greater than or equal to) can be used to find objects with a date that is before or after the given one.

Examples:

2024-08-15
<2024-08-15
>=2024-08-15

Date/time

A string in the form yyyy-MM-dd'T'HH:mm[:ss]. Seconds are optional.

The operators < (less than), <= (less than or equal to), > (greater than), and >= (greater than or equal to) can be used to find objects with a date/time pair that is before or after the given one.

Examples:

2024-08-15T08:26
2024-08-15T08:26:55
>2024-08-15T08:26
<=2024-08-15T08:26:55

Time range

A string representing a time range in the form yyyy-MM-dd['T'HH:mm[:ss]]..yyyy-MM-dd['T'HH:mm[:ss]]. Times as well as seconds within times are optional. The time range is inclusive, which means that objects with a date that matches the given start date as well as those with a date matching the given end date are included.

Examples:

2024-08-14..2024-08-15
2024-08-15T08:25..2024-08-15T08:26

Locator

A string starting with in: and denoting the attribute that should be compared with the give term(s), dates, date/time pairs, and time ranges. See Attributes below for a complete list of all possible attributes.

Example: in:name

Type

A string starting with is: and denoting the type of documents to search. Possible values are is:workflow and is:processchain.

Filter

A string that consists of an attribute name, followed by a colon and a term, date, date/time pair, or time range that should appear in this attribute. See Attributes below for a complete list of all possible attributes.

Examples:

name:Elvis
start:<=2024-08-15

Attributes

Possible values (including aliases) for attributes are:

id
name
error, errormessage
rc, cap, reqcap, capability, requiredcapability, rcs, caps, reqcaps, capabilities, requiredcapabilities
source
start, startTime
end, endTime

See the submission and process chain data models for more information about these attributes.

Search examples

filename highmemory: Search for objects that contain the terms filename or highmemory in any of their attributes. Since results are sorted by relevance, objects matching both terms will appear at the top of the list.
"exact match": Use quotation marks to search for literal (exact) strings.
filename is:workflow: Search for the term filename but only in workflows.
error:127 is:processchain: Search for process chains whose error message contains the term 127.
highmemory in:requiredcapabilities: Search for objects where the term highmemory appears in the list of required capabilities. Does not include objects where the term highmemory appears in any other attribute but the required capabilities.
rcs:highmemory: Search for objects whose required capabilities (alias rcs) contain the term highmemory.
filename status:error: Search for objects containing the term filename but only if their status equals error.
2024-08-15: Search for workflows or process chains that have started or finished on 15 August 2024.
<2024-08-15T08:26: Search for workflows or process chains that have started or finished before 08:26 on 15 August 2024.
>=2024-08-15T08:26:55: Search for workflows or process chains that have started or finished at or after 08:26:55 on 15 August 2024.
start:2024-08-14..2024-08-15: Search for workflows or process chains that have been started between 14 August 2024 and 15 August 2024 (inclusive).
filename in:source is:workflow rcs:highmemory: Search for workflows containing the term filename in their source and whose required capabilities contain the term highmemory.
exit code 127 in:error is:processchain: Search for process chains containing the terms exit, code, or 127 in their error message. Objects containing all terms in their error message will get a higher relevance and appear at the top of the list.

Search results

The HTTP endpoint /search returns a list of search result objects. Each such object refers to an item found and contains information about which properties of this item have matched with which search term and where in these properties the search term has been found.

The following list describes the properties of this object. For more information about each individual property, please refer to the description of the submission and process chain data models.

Property	Type	Description
id	string	The ID of the found item
type	string	The item’s type. Possible values are `workflow` and `processChain`.
requiredCapabilities	array	The capabilities required for this item to be executed.
status	string	The current status of the item
name (optional)	string	An optional human-readable name (only included if `type` is `workflow` and if the workflow has a name)
startTime (optional)	string	An ISO 8601 timestamp denoting the date and time when the item was started. May be `null` if the execution has not started yet.
endTime (optional)	string	An ISO 8601 timestamp denoting the date and time when the execution of the item has finished. May be `null` if the execution has not finished yet.
matches	array	A list of match objects denoting which properties have matched and what term was found at which location.

Example

id: az2ne7scskxqqzduorzq
type: workflow
requiredCapabilities:
  - sleep
  - docker
status: SUCCESS
name: Docker Sleep
startTime: '2022-06-02T05:43:10.397144Z'
endTime: '2022-06-02T05:43:42.534494Z'
matches:
  - locator: name
    fragment: Docker Sleep
    termMatches:
      - term: docker
        indices:
          - 0
  - locator: requiredCapabilities
    fragment: docker
    termMatches:
      - term: docker
        indices:
          - 0

Matches

A match object specifies a which property of an item has matched with which search term and where the match has occurred.

Property	Type	Description
locator	string	The property in which the match was found. Possible values are `errorMessage`, `id`, `name`, `requiredCapabilities`, `source`, `status`, `startTime`, `endTime`. See the description of the submission and process chain data models for more information about these properties.
fragment	string	A fragment of the property’s value (an ‘excerpt’ or a ‘preview’). If the property’s value is small enough, the fragment might even contain the whole value.
termMatches	array	A list of matches within `fragment` (see below)

termMatches is an array of objects with the following properties:

Property	Type	Description
term	string	The matched term from the search query
indices	array	The start positions (relative to `fragment`) at which the term was found

Example

locator: name
fragment: This is a docker-based service showcasing how docker ...
termMatches:
  - term: docker
    indices:
      - 10
      - 46

Setups

HTTP endpoints