Automating AWS Lambda Version Cleanup with Node.js and AWS SDK

Introduction:

In the realm of serverless computing, AWS Lambda functions play a pivotal role. However, as your application grows, you might accumulate numerous versions of these functions, leading to potential storage issues.

AWS Lambda stores your function code in a private S3 bucket, with a generous allocation of 75 GB per account in each region. This space is shared by both Lambda functions and layers. Over time, repeated deployments can eat into this limit, risking deployment failures due to storage overflow.

Objective:

This article aims to simplify the process of maintaining your Lambda environment by automating the removal of the oldest Lambda function versions during each deployment using the AWS SDK.

Prerequisites:

Before we dive into the automation process, ensure you have Node.js set up on your machine. Additionally, you’ll need an AWS account with the necessary permissions to manage Lambda functions.

STEP 1

Start by setting up AWS CodePipeline for your project to enable automatic deployments. This pipeline will be the backbone of your deployment process, ensuring that updates are seamlessly pushed to your serverless application.

Please find the below link for reference:

Creating an Automated Deployment Pipeline - CodeCommit to Lambda

STEP 2

Next, create a scripts folder in your project’s root directory. Here, you’ll add a Node.js script named delete_old_versions.js. This script is designed to methodically clean up the oldest version of each Lambda function in your project.

const { LambdaClient, ListVersionsByFunctionCommand, DeleteFunctionCommand } =
 require('@aws-sdk/client-lambda');
const { readFileSync } = require('fs');
const { load } = require('js-yaml');
const defaultProvider = require('../defaultProvider');

// Replace these variables with your AWS credentials and region
const lambda = new LambdaClient({
    credentials: defaultProvider,
    region: process.env.LAMBDA_REGION
});
// Read the serverless.yml file to get the list of functions
const serverlessConfig = readFileSync('serverless.yml', 'utf8');
const serverlessYaml = load(serverlessConfig);
const functions = Object.keys(serverlessYaml.functions);
// Delete the oldest version for each function
async function deleteOldVersions() {
    const logFunctionName = 'deleteOldVersions Custom Function';
    for (let functionName of functions) {
        functionName = process.env.SERVICE_NAME + '-' +
           process.env.npm_config_stage + '-' + functionName;
        console.log(`[INFO] [${logFunctionName}] Deleting oldest 
           version of function: ${functionName}`);
        try {
            const versions = await lambda
   .send(new ListVersionsByFunctionCommand({ FunctionName: functionName }));
            // Filter out the $LATEST version if present
            const publishedVersions = versions.Versions.filter((version) =>
                version.Version !== '$LATEST');
            // Skip if there is only one version (or no published versions)
            if (publishedVersions.length > 1) {
                // Sort versions in ascending order
                publishedVersions.sort((a, b) => 
                      a.Version.localeCompare(b.Version));
                // Get the oldest version
                const oldestVersion = publishedVersions[0].Version;
                // Delete the oldest version
                await lambda.send(new DeleteFunctionCommand(
                { FunctionName: functionName, Qualifier: oldestVersion }));
                console.log(`[INFO] [${logFunctionName}] 
                        Deleted version ${oldestVersion} of function:
                        ${functionName}`);
            } else {
                console.log(`[INFO] [${logFunctionName}] 
                  Skipping deletion. Only one (or no published) version
                  found for function: ${functionName}`);
            }
        } catch (error) {
            console.error(`[ERROR] [${logFunctionName}] 
            Error deleting versions for function ${functionName}:`, error);
        }
    }
}
deleteOldVersions();

It leverages the AWS SDK, specifically commands like ListVersionsByFunctionCommand and DeleteFunctionCommand, to manage function versions. The script also reads from a ‘serverless.yml’ file to identify all the functions that need management.

Here’s a simplified version of what the script does:

1. Initializes the Lambda client with your AWS credentials and region.

2. Reads the serverless.yml file to list all the functions.

3. For each function, it lists all versions and removes the oldest, ensuring that the $LATEST version is preserved.

Remember to include the necessary dependencies in your project and authenticate with valid AWS credentials.

Sample serverless.yml file for reference:

service: ServiceName
plugins:
  - serverless-dotenv-plugin
  - serverless-offline
  - serverless-middleware
provider:
  name: aws
  stage: dev
  region: ap-southeast-1
  runtime: nodejs18.x
  timeout: 150
  tracing:
    apiGateway: true
    lambda: true
functions:
  function1:
  function2:
  function3:

STEP 3

Incorporate the cleanup script into your build process by adding it to the pre-build phase of your buildspec.yml file. This ensures that before each deployment, the script runs and cleans up old Lambda versions, keeping your environment lean and efficient.

version: 0.2
phases:
  install:
    runtime-versions:
      nodejs: 18
      python: 3.11
    commands:
      - npm install --force
      - npm install -g serverless
  pre_build:
    commands: 
      - npm run --stage=$STAGE delete-old-versions 
  build:
    commands:
      - python3 deploy.py $STAGE
cache:
  paths:
    - node_modules

npm run — stage=$STAGE delete-old-versions

The above command takes the environment variable specified in the build stage of the pipeline

It also runs the delete-old-versions command which is specified in the scripts of package.json file, which in turn runs the script provided in the delete_old_versions.js file.

// sample package.json file
{
  "name": "samplebackend",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "delete-old-versions": "node scripts/delete_old_versions.js",
    "test": "jest --runInBand --coverage --reporter=lcov --detectOpenHandles"
  },
  // dependencies, dev-dependencies are not shown here
}

Conclusion:

Integrating this Node.js script into your deployment pipeline, such as within an AWS CodeBuild project, automates the housekeeping of Lambda function versions. This not only aids in cost management by preventing unnecessary storage use but also ensures the smooth operation and performance of your serverless applications.

About the Author

Shravan Kumar is a skilled full stack senior developer adept at utilizing a wide range of technologies and methodologies to deliver high-quality software solutions across various industries. With 6 years of experience, he has worked on different functional environments and business climates. Beyond his technical prowess, he finds fulfillment in exploring his passions for travel, music, cricket, and marathons, while also expressing his creativity through singing, cricket, and Tamil poetry.

About CodeStax.Ai

At CodeStax.Ai, we stand at the nexus of innovation and enterprise solutions, offering technology partnerships that empower businesses to drive efficiency, innovation, and growth, harnessing the transformative power of no-code platforms and advanced AI integrations.

But the real magic? It’s our tech tribe behind the scenes. If you’ve got a knack for innovation and a passion for redefining the norm, we’ve got the perfect tech playground for you. CodeStax.Ai offers more than a job — it’s a journey into the very heart of what’s next. Join us, and be part of the revolution that’s redefining the enterprise tech landscape.