Advanced tutorials

Bring your own service

In this tutorial, you’ll learn how to create a custom processing service and integrate it into Steep. We’ll use the image tiling service from the previous tutorial as an example. So far, we’ve relied on the pre-built Docker image, but what if you had to write the service from scratch or create your own Docker image?

Write a service from scratch

We use the tiling service as an example, because it’s relatively small and easy to understand. Also, it has been written in JavaScript, which makes it easy to create executable scripts.

In general, you can use any programming language to create processing services. Anything, that’s executable can be integrated into Steep.

Create a new directory, change into it, and initialize a new npm package:

Terminal

shell

mkdir tiling-service
cd tiling-service
npm init -y

Install the only dependency for the service:

Terminal

shell

npm i sharp

Then, create a new file tile.js in your directory and paste the following code into it:

tile.js

javascript

#!/usr/bin/env node
const fs = require("fs/promises")
const path = require("path")
const sharp = require("sharp")
 
if (process.argv.length < 5) {
  console.error("Usage: tile.js <input_image> <output_directory> <number_of_cols>")
  process.exit(1)
}
 
let input = process.argv[2]
let output_dir = process.argv[3]
let n = process.argv[4]
 
async function tile() {
  await fs.mkdir(output_dir, { recursive: true })
  let metadata = await sharp(input).metadata()
  let dx = metadata.width / n
  let dy = metadata.height / n
  for (let x = 0; x < n; x++) {
    for (let y = 0; y < n; y++) {
      await sharp(input)
        .extract({ left: dx * x, top: dy * y, width: dx, height: dy })
        .toFile(path.join(output_dir, `${x}_${y}.jpg`))
    }
  }
}
 
tile().catch(e => {
  console.error(e)
  process.exit(1)
})

The service accepts three command line arguments: the name of the image to tile, the output directory where to store the tiles, and the number of columns/rows. The main function tile creates the output directory, reads the input file, and then creates the tiles.

Note, in case of an error, the service fails with an exit code of 1. This is important so the workflow execution will be aborted by Steep and the error won’t be suppressed.

Also, the service follows all guidelines from the section on processing services.

Make the service executable

The script contains a Shebang at the beginning, which tells your system’s program loader to execute it through Node.js.

Make the service executable as follows:

Terminal

shell

chmod +x tile.js

Add service metadata

Finally, open the file conf/services/services.yaml in your Steep installation and add the following metadata to register the service:

conf/services/services.yaml

yaml

- id: tile
  name: Tiling
  description: Split an image into tiles
  path: tile.js   # replace this
  runtime: other
 
  parameters:
    - id: input_file
      name: Input image
      description: The input image to tile
      type: input
      cardinality: 1..1
      dataType: string
 
    - id: output_directory
      name: Output directory
      description: The directory where the tiles should be stored
      type: output
      cardinality: 1..1
      dataType: directory
 
    - id: num_tiles
      name: Number of columns/rows
      description: >-
        The number of columns and rows to split the image into. A value of
        5 means the image will be split into 5x5 tiles.
      type: input
      cardinality: 1..1
      dataType: string

Replace tile.js in the path attribute with the absolute path to tile.js on your system.

That’s it! 🎉 You’ve created your first service from scratch and integrated into Steep. It can now be called via its ID as seen in the previous tutorials.

Create a Docker image

In a production environment, you’ll probably want to convert your custom service into a Docker image. This makes it easier to deploy the service (including all its dependencies) to multiple machines in your environment.

Create a new Dockerfile in the directory of the tiling service you’ve created above and paste the following code into it:

Dockerfile

dockerfile

FROM node:18-slim
 
COPY . /tiling-service
WORKDIR /tiling-service
RUN npm ci
 
ENTRYPOINT ["node", "./tile.js"]

The Dockerfile is very short in this case. It just includes the base image for Node.js, copies the source code, and installs the dependencies.

We set the entrypoint to ["node", "./tile.js"], so Steep can later simply append the service arguments to the docker run command when it creates the container.

Build the Docker image

Run the following command to build the Docker image for the tiling service:

Terminal

shell

docker build -t tiling-service .

Modify the service metadata

Finally, modify the service metadata you’ve added above and change the path to the name of the built Docker image tiling-service. Also change runtime to docker.

conf/services/services.yaml

yaml

- id: tile
  name: Tiling
  description: Split an image into tiles
  path: tiling-service
  runtime: docker
 
  ...

If you want to deploy your image to multiple machines in your distributed environment, you’ll most likely want to push your image into a Docker registry. Please refer to the Docker documentation for more information.

Aerial image segmentation

Workflows