Use Dagger with Tekton

Introduction

Tekton is an open source framework for CI/CD implementation, with built-in support for Kubernetes and other CI/CD tools. This guide explains how to run Dagger pipelines using Tekton.

Requirements

This guide assumes that:

You have a basic understanding of Kubernetes and Tekton.
You have a project with a Dagger pipeline in a Git repository. If not, follow the steps in Appendix A to create and populate a GitHub repository with a simple project and Dagger pipeline.

Step 1: Install Tekton

The first step is to install Tekton in the Kubernetes cluster and the Tekton CLI.

Follow the Tekton installation instructions and the Tekton CLI installation steps, adjusting them as needed to your own requirements. Once you've successfully installed Tekton and the Tekton CLI, continue to the next step.

Step 2: Install the `git-clone` Task

Next, install the git-clone Task from Tekton Hub. This Task adds repository cloning capabilities to your Tekton Pipeline, and is used by the Tekton Pipeline defined later in this guide.

tkn hub install task git-clone

Step 3: Create a Tekton Pipeline, Task and PipelineRun

Follow the steps below:

Define a new Tekton Pipeline as follows, in a file named git-pipeline.yaml.

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: dagger-pipeline
spec:
  description: |
    This pipeline clones a Git repository, then runs the Dagger pipeline.
  params:
  - name: repo-url
    type: string
    description: The Git repository clone URL
  - name: dagger-cloud-token
    type: string
    description: The Dagger Cloud token
  workspaces:
  - name: shared-data
    description: |
      This workspace contains the cloned repository files, so they can be read by the
      next task.
  tasks:
  - name: fetch-source
    taskRef:
      name: git-clone
    workspaces:
    - name: output
      workspace: shared-data
    params:
    - name: url
      value: $(params.repo-url)
  - name: dagger
    runAfter: ["fetch-source"]
    taskRef:
      name: dagger
    workspaces:
    - name: source
      workspace: shared-data
    params:
      - name: dagger-cloud-token
        value: $(params.dagger-cloud-token)

This Pipeline references two Tasks:

The git-clone Task, to check out the Git repository for the project into a Tekton Workspace;
A custom dagger Task, to run the Dagger pipeline for the project (defined below).

Define a new Tekton Task as follows, in a file named dagger-task.yaml.

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: dagger
spec:
  description: Run Dagger pipeline
  workspaces:
  - name: source
  params:
    - name: dagger-cloud-token
      type: string
      description: Dagger Cloud Token
  volumes:
    - name: dagger-socket
      emptyDir: {}
    - name: dagger-storage
      emptyDir: {}
  sidecars:
    - name: dagger-engine
      image: registry.dagger.io/engine:v0.9.1
      args:
        - "--oci-max-parallelism"
        - "num-cpu"
      securityContext:
        privileged: true
        capabilities:
          add:
            - ALL
      readinessProbe:
        exec:
          command: ["buildctl", "debug", "workers"]
      volumeMounts:
        - mountPath: /var/run/buildkit
          name: dagger-socket
        - mountPath: /var/lib/dagger
          name: dagger-storage
      env:
        - name: DAGGER_CLOUD_TOKEN
          value: $(params.dagger-cloud-token)
  steps:
  - name: read
    image: docker:dind
    workingDir: $(workspaces.source.path)
    script: |
      #!/usr/bin/env sh
      apk add nodejs npm curl
      curl https://dl.dagger.io/dagger/install.sh | BIN_DIR=/usr/local/bin sh
      npm i @dagger.io/dagger@latest
      dagger run node ./ci/index.mjs
    volumeMounts:
      - mountPath: /var/run/dagger
        name: dagger-socket
    env:
      - name: _EXPERIMENTAL_DAGGER_RUNNER_HOST
        value: unix:///var/run/dagger/buildkitd.sock
      - name: DAGGER_CLOUD_TOKEN
        value: $(params.dagger-cloud-token)

This Task installs the dependencies needed to execute the Dagger pipeline for the project (which was checked out in the previous Tekton Pipeline) and then executes the pipeline using the npm run ci command.

Define a new Tekton PipelineRun as follows, in a file named git-pipeline-run.yaml.

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  generateName: clone-read-run-
spec:
  pipelineRef:
    name: dagger-pipeline
  podTemplate:
    securityContext:
      fsGroup: 65532
  workspaces:
  - name: shared-data
    volumeClaimTemplate:
      spec:
        accessModes:
        - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi
  params:
  - name: repo-url
    value: YOUR_REPOSITORY_URL
  - name: dagger-cloud-token
    value: YOUR_DAGGER_CLOUD_TOKEN

This PipelineRun corresponds to the Tekton Pipeline created previously. It executes the Tekton Pipeline with a given set of input parameters: the Git repository URL and an optional Dagger Cloud token. Update the YOUR_REPOSITORY_URL placeholder with the correct repository URL and, if you have a Dagger Cloud token, replace the YOUR_DAGGER_CLOUD_TOKEN placeholder with the token or a Kubernetes secret holding the token.

A few important points to note:

In the Task, the Dagger Engine runs as a sidecar and shares a socket with the Task itself. The Task uses dind as its runtime in order to have Docker available.
Setting the DAGGER_CLOUD_TOKEN environment variable in the Task and the dagger-cloud-token parameter in the Pipeline and PipelineRun is only necessary if integrating with Dagger Cloud.

Step 4: Run the Pipeline

Apply the configuration:

kubectl apply -f dagger-task.yaml
kubectl apply -f git-pipeline-yaml

Run the Tekton Pipeline:
```
kubectl create -f git-pipeline-run.yaml
```
The output will look something like pipelinerun.tekton.dev/clone-read-run-mwvkm created To see the logs from the PipelineRun, obtain the PipelineRun name from the output and run tkn pipelinerun logs clone-read-run-<id> -f.

Once the PipelineRun has successfully completed, run it again. Dagger's caching should result in a significantly faster second execution.

Conclusion

This example demonstrated how to integrate Dagger into a Tekton Pipeline using a custom Task.

To learn more about Dagger, use the API Key Concepts page and the Go, Node.js and Python SDK References. For more information on Tekton, refer to the official documentation.

Appendix A: Create a GitHub repository with an example project

This guide assumes that you have a Git repository with a project and a Dagger pipeline. If you don't, follow the steps below to create a GitHub repository and add a simple project and Dagger pipeline to it.

Clone the Dagger starter application:

git clone git@github.com:dagger/hello-dagger.git

Create a new ci subdirectory:

cd hello-dagger
mkdir ci && cd ci

Add a simple Dagger pipeline as index.mjs:

index.mjs
import { connect } from "@dagger.io/dagger"

connect(
  async (client) => {
    // use a node:16-slim container
    // mount the source code directory on the host
    // at /src in the container
    const source = client
      .container()
      .from("node:16-slim")
      .withDirectory(
        "/src",
        client.host().directory(".", {
          exclude: ["node_modules/", "ci/", "build/", ".git/"],
        }),
      )

    // set the working directory in the container
    // install application dependencies
    const runner = source
      .withWorkdir("/src")
      .withMountedCache(
        "/src/node_modules",
        client.cacheVolume("node_module_cache"),
      )
      .withExec(["npm", "install"])

    // run application tests
    const test = runner.withExec(["npm", "test", "--", "--watchAll=false"])

    // build application
    // write the build output to the host
    await test
      .withExec(["npm", "run", "build"])
      .directory("./build")
      .export("./build")

    // use an nginx:alpine container
    // copy the build/ directory into the container filesystem
    // at the nginx server root
    // publish the resulting container to a registry
    const imageRef = await client
      .container()
      .from("nginx:1.23-alpine")
      .withDirectory(
        "/usr/share/nginx/html",
        client.host().directory("./build"),
      )
      .publish("ttl.sh/hello-dagger-" + Math.floor(Math.random() * 10000000))
    console.log(`Published image to: ${imageRef}`)
  },
  { LogOutput: process.stderr },
)

Commit the changes:

cd ..
git add .
git commit -m "Initial commit"

gh auth login

Create a repository in your GitHub account and push the changes to it:

gh repo create myapp --push --remote upstream --source . --public

Use Dagger with Tekton

Introduction​

Requirements​

Step 1: Install Tekton​

Step 2: Install the git-clone Task​

Step 3: Create a Tekton Pipeline, Task and PipelineRun​

Step 4: Run the Pipeline​

Conclusion​

Appendix A: Create a GitHub repository with an example project​