Skip to main content

Deploy AWS Lambda Functions with Dagger

Introduction​

This tutorial teaches you how to create a local Dagger pipeline to update and deploy an existing AWS Lambda function using a ZIP archive.

Requirements​

This tutorial assumes that:

  • You have a basic understanding of the JavaScript programming language.
  • You have a basic understanding of the AWS Lambda service. If not, learn about AWS Lambda.
  • You have a Go, Node.js or Python development environment. If not, install Go, Python or Node.js.
  • You have Docker installed and running on the host system. If not, install Docker.
  • You have the Dagger CLI installed in your development environment. If not, install the Dagger CLI.
  • You have an AWS account with appropriate privileges to create and manage AWS Lambda resources. If not, register for an AWS account.
  • You have an existing AWS Lambda function with a publicly-accessible URL in Go, Node.js or Python, deployed as a ZIP archive. If not, follow the steps in Appendix A to create an example AWS Lambda function.

Step 1: Create a Dagger pipeline​

The first step is to create a Dagger pipeline to build a ZIP archive of the function and deploy it to AWS Lambda.

  1. In the function directory, install the Dagger SDK:

    go get dagger.io/dagger

  2. Create a new sub-directory named ci. Within the ci directory, create a file named main.go and add the following code to it.

    package main

    import (
    	"context"
    	"encoding/json"
    	"fmt"
    	"log"
    	"os"

    	"dagger.io/dagger"
    )

    func main() {

    	// check for required variables in host environment
    	vars := []string{"AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_DEFAULT_REGION"}
    	for _, v := range vars {
    		if os.Getenv(v) == "" {
    			log.Fatalf("Environment variable %s is not set", v)
    		}
    	}

    	// initialize Dagger client
    	ctx := context.Background()
    	client, err := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
    	if err != nil {
    		panic(err)
    	}
    	defer client.Close()

    	// set AWS credentials as client secrets
    	awsAccessKeyId := client.SetSecret("awsAccessKeyId", os.Getenv("AWS_ACCESS_KEY_ID"))
    	awsSecretAccessKey := client.SetSecret("awsSecretAccessKey", os.Getenv("AWS_SECRET_ACCESS_KEY"))

    	awsRegion := os.Getenv("AWS_DEFAULT_REGION")

    	// get reference to function directory
    	lambdaDir := client.Host().Directory(".", dagger.HostDirectoryOpts{
    		Exclude: []string{"ci"},
    	})

    	// use a node:18-alpine container
    	// mount the function directory
    	// at /src in the container
    	// install application dependencies
    	// create zip archive
    	build := client.Container().
    		From("golang:1.20-alpine").
    		WithExec([]string{"apk", "add", "zip"}).
    		WithDirectory("/src", lambdaDir).
    		WithWorkdir("/src").
    		WithEnvVariable("GOOS", "linux").
    		WithEnvVariable("GOARCH", "amd64").
    		WithEnvVariable("CGO_ENABLED", "0").
    		WithExec([]string{"go", "build", "-o", "lambda", "lambda.go"}).
    		WithExec([]string{"zip", "function.zip", "lambda"})

    	// use an AWS CLI container
    	// set AWS credentials and configuration
    	// as container environment variables
    	aws := client.Container().
    		From("amazon/aws-cli:2.11.22").
    		WithSecretVariable("AWS_ACCESS_KEY_ID", awsAccessKeyId).
    		WithSecretVariable("AWS_SECRET_ACCESS_KEY", awsSecretAccessKey).
    		WithEnvVariable("AWS_DEFAULT_REGION", awsRegion)

    	// add zip archive to AWS CLI container
    	// use CLI commands to deploy new zip archive
    	// and get function URL
    	// parse response and print URL
    	response, err := aws.
    		WithFile("/tmp/function.zip", build.File("/src/function.zip")).
    		WithExec([]string{"lambda", "update-function-code", "--function-name", "myFunction", "--zip-file", "fileb:///tmp/function.zip"}).
    		WithExec([]string{"lambda", "get-function-url-config", "--function-name", "myFunction"}).
    		Stdout(ctx)
    	if err != nil {
    		panic(err)
    	}

    	var data struct {
    		FunctionUrl string
    	}

    	err = json.Unmarshal([]byte(response), &data)
    	if err != nil {
    		panic(err)
    	}

    	fmt.Printf("Function updated at: %s\n", data.FunctionUrl)
    }

    This file performs the following operations:

    • It imports the Dagger SDK.
    • It checks for AWS credentials and configuration in the host environment.
    • It creates a Dagger client with Connect(). This client provides an interface for executing commands against the Dagger engine.
    • It uses the client's SetSecret() method to set the AWS credentials as secrets for the Dagger pipeline.
    • It uses the client's Host().Directory() method to obtain a reference to the current directory on the host, excluding the ci directory. This reference is stored in the source variable.
    • It uses the client's Container().From() method to initialize a new container image from a base node:18-alpine image. This method returns a Container representing an OCI-compatible container image.
    • It uses the previous Container object's WithDirectory() method to return the container image with the host directory written at the /src path, and the WithWorkdir() method to set the working directory in the container image.
    • It chains together a series of WithExec() method calls to install dependencies and build a ZIP deployment archive containing the function and all its dependencies.
    • It uses the client's Container().From() method to initialize a new aws-cli AWS CLI container image.
    • It uses the Container object's WithSecretVariable() and WithEnvVariable() methods to inject the AWS credentials (as secrets) and configuration into the container environment, so that they can be used by the AWS CLI.
    • It copies the ZIP archive containing the new AWS Lambda function code from the previous node:18-alpine container image into the aws-cli container image.
    • It uses WithExec() method calls to execute AWS CLI commands in the container image to upload and deploy the ZIP archive and get the function's public URL. If these operations complete successfully, it prints a success message with the URL to the console.

  3. Run the following command to update go.sum:

    go mod tidy
tip

Most Container object methods return a revised Container object representing the new state of the container. This makes it easy to chain methods together. Dagger evaluates pipelines "lazily", so the chained operations are only executed when required - in this case, when the container is published. Learn more about lazy evaluation in Dagger.

Step 2: Test the Dagger pipeline​

Configure the credentials and default region for the AWS CLI as environment variables on the local host by executing the commands below. Replace the KEY-ID and SECRET placeholders with the AWS access key and secret respectively, and the REGION placeholder with the default AWS region.

export AWS_ACCESS_KEY_ID=KEY-ID
export AWS_SECRET_ACCESS_KEY=SECRET
export AWS_DEFAULT_REGION=REGION

Once the AWS CLI environment variables are set, you're ready to test the Dagger pipeline. Do so by making a change to the function and then executing the pipeline to update and deploy the revised function on AWS Lambda.

If you are using the example application function in Appendix A, the following command modifies the function code to display a list of commits (instead of issues) from the Dagger GitHub repository:

sed -i -e 's|/dagger/issues|/dagger/commits|g' lambda.py

After modifying the function code, execute the Dagger pipeline:

dagger run go run ci/main.go

Dagger performs the operations defined in the pipeline script, logging each operation to the console. At the end of the process, the ZIP archive containing the revised function code is deployed to AWS Lambda and a message similar to the one below appears in the console output:

Function updated at: https://...

Browse to the public URL endpoint displayed in the output to verify the output of the revised AWS Lambda function.

Conclusion​

This tutorial walked you through the process of creating a local Dagger pipeline to update and deploy a function on AWS Lambda. It used the Dagger SDKs and explained key concepts, objects and methods available in the SDKs to construct a Dagger pipeline.

Use the API Key Concepts page and the Go, Node.js and Python SDK References to learn more about Dagger.

Appendix A: Create an example AWS Lambda function​

This tutorial assumes that you have an AWS Lambda function written in Go, Node.js or Python and configured with a publicly-accessible URL. If not, follow the steps below to create an example function.

info

This section assumes that you have the AWS CLI and a GitHub personal access token. If not, install the AWS CLI, learn how to configure the AWS CLI and learn how to obtain a GitHub personal access token.

  1. Create a service role for AWS Lambda executions:
aws iam create-role --role-name my-lambda-role --assume-role-policy-document '{"Version": "2012-10-17","Statement": [{ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}'
aws iam attach-role-policy --role-name my-lambda-role --policy-arn arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole

Note the role ARN (the Role.Arn field) in the output of the first command, as you will need it in subsequent steps.

  1. Create a directory named myfunction for the function code.

    mkdir myfunction
    cd myfunction

Within that directory, run the following commands to create a new Go module and add dependencies:

go mod init main
go get github.com/aws/aws-lambda-go/lambda

Within the same directory, create a file named lambda.go and fill it with the following code:

package main

import (
	"context"
	"encoding/json"
	"net/http"
	"os"

	"github.com/aws/aws-lambda-go/lambda"
)

func main() {
	lambda.Start(HandleRequest)
}

func HandleRequest(ctx context.Context) (interface{}, error) {
	token := os.Getenv("GITHUB_API_TOKEN")
	url := "https://api.github.com/repos/dagger/dagger/issues"
	req, _ := http.NewRequestWithContext(ctx, "GET", url, nil)
	req.Header.Set("Accept", "application/vnd.github+json")
	req.Header.Set("Authorization", "Bearer "+token)

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}

	defer resp.Body.Close()

	var res interface{}
	err = json.NewDecoder(resp.Body).Decode(&res)

	return res, err
}

Build the function:

GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o lambda lambda.go

This simple function performs an HTTP request to the GitHub API to return a list of issues from the Dagger GitHub repository. It expects to find a GitHub personal access token in the function environment and it uses this token for request authentication.

  1. Deploy the function to AWS Lambda. Replace the ROLE-ARN placeholder with the service role ARN obtained previously and the TOKEN placeholder with your GitHub API token.
zip function.zip lambda
aws lambda create-function --function-name myFunction --zip-file fileb://function.zip --runtime go1.x --handler lambda --timeout 10 --role ROLE-ARN
aws lambda update-function-configuration --function-name myFunction --environment Variables={GITHUB_API_TOKEN=TOKEN}
aws lambda add-permission --function-name myFunction --statement-id FunctionURLAllowPublicAccess --action lambda:InvokeFunctionUrl --principal "*" --function-url-auth-type NONE
aws lambda create-function-url-config --function-name myFunction --auth-type NONE

This sequence of commands creates a ZIP deployment archive, deploys it as a new AWS Lambda function named myFunction, and creates a publicly-accessible URL endpoint. The public URL endpoint is listed in the output of the last command.

  1. Browse to the public URL endpoint to test the AWS Lambda function. Confirm that it displays a JSON-encoded list of issues from the Dagger GitHub repository.