HashiCorp Nomad Task Dependencies

Nomad 0.11 introduces the lifecycle stanza for tasks which can be used to express task dependencies. This can be used to express task dependencies between tasks in a task group, and it can even be leveraged to express inter-job task dependencies with Consul.

In this blog post, we’ll go over the lifecycle stanza, task dependency patterns, and how to implement a sidecar task dependency in your own job.

»The lifecycle stanza

The lifecycle stanza enables users to control when a task is run within the lifecycle of a task group allocation.

»lifecycle Parameters

• hook - Specifies when a task should be launched within the lifecycle of a group. Hooks break up the allocation lifecycle into phases. If a task lifecycle specifies Prestart Hook, then those tasks will be started before the main tasks are started. Currently only Prestart Hooks are supported, but later 0.11 releases will add PostStart, PreStop and PostStop hooks.
• sidecar - Indicates if a task should run for the full duration of an allocation (sidecar = true), or whether the task should be ephemeral and run until completion (sidecar = false) before starting the next lifecycle phase in the allocation.

»Task Dependency Patterns

The combination of prestart hooks and the sidecar flag creates two task dependency patterns for prestart tasks: init tasks and sidecar tasks.

Init tasks are ephemeral prestart tasks that must run to completion before the main workload is started. They are commonly used to download assets or to create necessary tables for an extract-transform-load (ETL) job.

You can create an init task by adding a lifecycle stanza with hook set to prestart and sidecar to false as below.

      lifecycle {
        hook    = "prestart"
        sidecar = false
      }

Sidecar tasks are prestart tasks that are started before main workload starts and run for the lifetime of the main workload. Typical sidecars tasks are log forwarders, proxies, and for platform abstractions.

You can create a sidecar task by adding a lifecycle stanza with hook set to prestart and sidecar to true as below.

      lifecycle {
        hook    = "prestart"
        sidecar = true
      }

»Describing Sidecar Tasks in a Nomad Job

Now we will demonstrate how to configure two remote_syslog containers — one for stderr and one for stdout—to ship log events to Papertail for a sample Redis instance. In this example, the log-shippers are sidecar tasks and the Redis instance is the main task.

job "example" {
  datacenters = ["dc1"]
  group "cache" {
    task "remote_syslog_stdout" {
      driver = "docker"
      config {
        image = "octohost/remote_syslog"
        args = [
         # REPLACE placeholders with your Papertrail information.
         "-p", "«papertrail port»", "-d", "logs.papertrailapp.com", "/alloc/logs/redis.stdout.0"
        ]
     }
      lifecycle {
        sidecar = true
        hook = "prestart"
      }
    }
    task "remote_syslog_stderr" {
      driver = "docker"
      config {
        image = "octohost/remote_syslog"
        args = [
         # REPLACE placeholders with your Papertrail information.
         "-p", "«papertrail port»", "-d", "logs.papertrailapp.com", "/alloc/logs/redis.stderr.0"
        ]
     }
      lifecycle {
        sidecar = true
        hook = "prestart"
      }
    }
    task "redis" {
      driver = "docker"
      config {
        image = "redis:3.2"
        port_map {
          db = 6379
        }
      }
      resources {
        cpu    = 500
        memory = 256
        network {
          mbits = 10
          port  "db"  {}
        }
      }
    }
  }
}

Once you run the job, you can navigate to your Papertrail UI and view the logs.

»Getting Started

We're releasing Nomad's Task Dependencies feature in beta for Nomad 0.11 to get feedback from our practitioners. We’d like to hear what other task dependencies features you are interested in seeing. Feel free to try it out and give us feedback in the issue tracker. To see this feature in action, please register for the upcoming live demo session here. Learn more about Task dependencies at the HashiCorp Learn website.