Skip to main content

Quickstart

Use caching​

One of Dagger's most powerful features is its ability to cache data across pipeline runs. This is typically most useful when dealing with package managers such as npm, maven, pip and similar. For these tools to cache properly, they need their own cache data (usually a directory) to be persisted between runs.

Dagger lets you define one or more such directories as cache volumes and persist their contents across runs. This enables you to reuse the contents of the cache volume(s) every time the pipeline runs, and thereby speed up pipeline operations.

note

Cache volumes are not meant to be shared outside of Dagger; they are for persisting specific parts of the internal state of your pipeline, for optimal use of your tool's native caching features.

You may have noticed that the example pipeline executes the npm install command to download the application's dependencies every time the pipeline runs. Since these dependencies are usually locked to specific versions in the application's manifest, re-downloading them on every pipeline run is inefficient and time-consuming.

This step is, therefore, a good candidate for a cache volume. Let's update the pipeline accordingly.

tip

The npm install command is appropriate for a React application, but other applications are likely to use different commands. Modify your Dagger pipeline accordingly.

package main

import (
	"context"
	"fmt"
	"math"
	"math/rand"
	"os"

	"dagger.io/dagger"
)

func main() {
	ctx := context.Background()

	// initialize Dagger client
	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
	if err != nil {
		panic(err)
	}
	defer client.Close()

	// create a cache volume
	nodeCache := client.CacheVolume("node")

	// use a node:16-slim container
	// mount the source code directory on the host
	// at /src in the container
	// mount the cache volume to persist dependencies
	source := client.Container().
		From("node:16-slim").
		WithDirectory("/src", client.Host().Directory(".", dagger.HostDirectoryOpts{
			Exclude: []string{"node_modules/", "ci/", "build/"},
		})).
		WithMountedCache("/src/node_modules", nodeCache)

		// set the working directory in the container
		// install application dependencies
	runner := source.WithWorkdir("/src").
		WithExec([]string{"npm", "install"})

		// run application tests
	test := runner.WithExec([]string{"npm", "test", "--", "--watchAll=false"})

	// first stage
	// build application
	buildDir := test.WithExec([]string{"npm", "run", "build"}).
		Directory("./build")

		// second stage
		// use an nginx:alpine container
		// copy the build/ directory from the first stage
		// publish the resulting container to a registry
	ref, err := client.Container().
		From("nginx:1.23-alpine").
		WithDirectory("/usr/share/nginx/html", buildDir).
		Publish(ctx, fmt.Sprintf("ttl.sh/hello-dagger-%.0f", math.Floor(rand.Float64()*10000000))) //#nosec
	if err != nil {
		panic(err)
	}

	fmt.Printf("Published image to: %s\n", ref)
}

This revised pipeline now uses a cache volume for the application dependencies.

  • It uses the client's CacheVolume() method to initialize a new cache volume.
  • It uses the Container.WithMountedCache() method to mount this cache volume at the node_modules/ mount point in the container.
  • It uses the Container.WithExec() method to define the npm install command. When executed, this command downloads and installs dependencies in the node_modules/ directory. Since this directory is defined as a cache volume, its contents will persist even after the pipeline terminates and can be reused on the next pipeline run.

Run the pipeline by executing the command below from the application directory:

dagger run go run ci/main.go

This revised pipeline produces the same result as before.

Run the pipeline a few times. Notice that on the first run, the application dependencies are downloaded as usual. However, since the dependencies are cached, subsequent pipeline runs will skip the download operation and be significantly faster (assuming that there are no other changes to the application code).

note

In addition to cache volumes, Dagger has a separate cache for pipeline operations. Changes in cache volumes or secrets do not invalidate the Dagger pipeline operations cache.