Copy Embedded Directories into a Container

Embedded directories in Go allow bundling static files with the compiled binary. This guide demonstrates copying embedded directories using Dagger, a Go-specific feature not available in Python or Node.js.

Dagger does not expose the option to copy entire directories as a single step (yet), whether it is between containers or from an embedded directory to a container. It is, however, doable by traversing the directory tree.

Assume that you have a Dagger CI tool containing the following code structure, which contains an example directory:

tree
.
├── go.mod
├── go.sum
├── main.go
└── example
    └── foo.go

The following example demonstrates how to copy an embedded directory:

package main

import (
	"context"
	"embed"
	"fmt"
	"io/fs"

	"dagger.io/dagger"
)

// create a copy of an embed directory
func copyEmbedDir(e fs.FS, dir *dagger.Directory) (*dagger.Directory, error) {
	err := fs.WalkDir(e, ".", func(path string, d fs.DirEntry, err error) error {
		if err != nil {
			return err
		}
		if d.IsDir() {
			return nil
		}

		content, err := fs.ReadFile(e, path)
		if err != nil {
			return err
		}

		dir = dir.WithNewFile(path, string(content))

		return nil
	})
	if err != nil {
		return nil, err
	}
	return dir, nil
}

//go:embed example
var e embed.FS

func main() {
	ctx := context.Background()

	// Init Dagger client
	client, err := dagger.Connect(ctx)
	if err != nil {
		panic(err)
	}
	defer client.Close()

	// Copy embed files to dir, a newly created directory.
	dir := client.Directory()
	dir, err = copyEmbedDir(e, dir)
	if err != nil {
		panic(err)
	}

	// Mount above directory ID and
	container := client.Container().From("alpine:3.16.2").WithDirectory("/embed", dir)

	// List files
	out, err := container.WithExec([]string{"ls", "-lR", "/embed/"}).Stdout(ctx)
	if err != nil {
		panic(err)
	}

	fmt.Printf("%s", out)
}

Attempt to run the code and print the content of the /embed directory:

➜  dagger run go run .
/embed/:
total 4
drwxr-xr-x    1 root     root          4096 Oct 31 16:49 example

/embed/example:
total 4
-rw-r--r--    1 root     root            50 Oct 31 16:49 foo.go

In this case, the function succeeds in copying the embedded example directory.

warning

You may encounter errors if your directory contains +1000 files, due to the concatenation of the queries.