Pipeline examples

Pipelines provide a means of composing both Restspace services and services found anywhere on the internet, and as such are extremely powerful. To provide a sense of this and to show how to build pipelines to solve common use cases, here is a list of example pipelines. Bear in mind that while most are shown using urls from within the Restspace domain, they could be used (maybe with slight adjustment) with any suitable url on the internet.

Fetch files in a directory as a zip

GET /base-path/json/dir1

[
  "GET /$*/",
  "GET /$*/${files[].name} ~$<0",
  "zip"
]

This pipeline has 3 steps. The first gets the directory specified in the part of the request after the base path. This is returned as a JSON object specifying the contents of the directory. The second uses data expansion to get in parallel all the files in the directory. Each one is named after the final element of its path (~$<0). Then finally all the files read are combined into a zip file before returning this to the user.

Universal copy file

POST /base-path?from=https://some-domain.com/file0&to=/files/file1

[
  "GET $?(from)",
  "PUT $?(to)"
]

This simple pipeline will stream a url into another url. The first element GETs from the url given as the from query string argument and the second PUTs to the url given as the to query string argument.

Email a template and an attachment

POST /base-path/fred@jones.com
{
  "subject": "Here's an image",
  "message": "This very special image is for you"
}

[
  [
    "POST /templates/email.html ~html",
    "~$this",
    "GET /images/logo-blue.png ~logo-blue.png"
  ],
  "multipart",
  "/email/send/$<0"
]

This pipeline in parallel:

  • POSTs the submitted JSON into the template email.html, outputting the HTML email body, naming it 'html'
  • Passes through all the JSON sent
  • GETs a file

It then combines the results of all these as a multipart message which it sends to the email send function.

Email send takes the last url segment as the email address to which to send (this could be multiple comma separated addresses). It takes any files it receives as attachments, any field values as specifying email details such as in this case html, the html body, and subject, the email subject.

Fetch related data

GET /base-path/post-name

[
  "GET /json/post/$<0",
  [
    "~$this",
    "GET /json/user/${user} ~userDetail"
  ],
  "jsonObject"
]

This shows how a pipeline can manage fetching related records. Essentially the base record will have a url or part of a path giving you the reference to a related record. You fetch the base record as the first pipeline request. Then in parallel you fetch the related records using data substitution into the url spec, while the "~$this" item in the parallel pipeline will pass through the data of the message unchanged, naming it $this, a special name which tells the jsonObject join operator to merge all the fields of this data into the output JSON rather than adding it as a field value.

The output is the base record with an added field userDetail whose value is the full user record.