segmentation fault in Rust Lambda using rusoto_dynamodb

0

I have some code that starts out like this:

use flate2::read::GzDecoder;
use lambda_runtime::{run, service_fn, Error, LambdaEvent};
use rusoto_core::Region;
use rusoto_dynamodb::{AttributeValue, DynamoDb, DynamoDbClient, QueryInput};
use serde_json::{json, Value};
use std::fs::File;

#[tokio::main]
async fn main() -> Result<(), Error> {
    let func = service_fn(lambda_handler);
    run(func).await
}

async fn lambda_handler(event: LambdaEvent<Value>) -> Result<Value, Error> {

    let id = event.payload.get("id").and_then(|id| id.as_str()).unwrap();
    println!("Querying for id: {}", id);

    let region = Region::default();
    let dynamo_client = DynamoDbClient::new(region); // also tried Default::default() here, and Region::UsWest2
    println!("Dynamo client created");

    ... the rest you don't need to see, because the above println is never reached ...

This is the Cargo.toml:

[package]
name = "query_by_id"
version = "0.1.0"
edition = "2021"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

[profile.release]
debug = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
aws_lambda_events = "0.15.0"
lambda_runtime = "0.11.1"
rusoto_core = "0.48.0"
rusoto_dynamodb = "0.48.0"
rusoto_config = "0.48.0"
serde_json = "1.0.116"
flate2 = "1.0.30"
tokio = { version = "1.37.0", features = ["full"] }
tokio-util = { version = "0.7", features = ["compat"] }
futures = "0.3.17"
rusoto_credential = "0.48.0"

and I build it using this Dockerfile:

FROM public.ecr.aws/docker/library/rust:1.78-alpine
WORKDIR /build

# Setup Rust environment for MUSL target, pre-cache known dependencies
RUN apk add musl-dev openssl-dev zip make
ENV PATH="/root/.cargo/bin:${PATH}"
ENV CARGO_TARGET_DIR=/root/.cargo/target
RUN rustup target add x86_64-unknown-linux-musl

RUN apk add pkgconfig
ENV OPENSSL_DIR=/usr
ENV OPENSSL_LIB_DIR=/usr/lib
ENV OPENSSL_INCLUDE_DIR=/usr/include/openssl

RUN cargo install cargo-prefetch
RUN cargo prefetch $(sed -n '/[dependencies]/,/^$/p' Cargo.toml | awk '{print $1"@="$3}' | tr '\n' ' ')

# Build the Application
COPY Cargo.toml /build/
RUN mkdir src && echo "fn main() {}" > src/main.rs && cargo build --release && rm -rf src
COPY src/ /build/src/
RUN cargo build --release

# Place executable into path expected by CDK Construct
RUN mkdir -p bin && cp $CARGO_TARGET_DIR/release/bootstrap bin/bootstrap
RUN strip -s bin/bootstrap

and I deploy it using this CDK Construct:

        // Create a Lambda function for querying
        const queryFunction = new lambda.Function(this, 'QueryFunction', {
            runtime: lambda.Runtime.PROVIDED_AL2023,
            handler: 'bootstrap',
            code: lambda.Code.fromDockerBuild(path.join(__dirname, '..', 'query_by_id'), {
                file: 'Dockerfile',
                imagePath: '/build/bin/',
                outputPath: '/var/task',
            }),
            vpc,
            securityGroups: [lambdaSecurityGroup],
            vpcSubnets: {
                subnetType: ec2.SubnetType.PRIVATE_ISOLATED,
            },
            filesystem: accessPoint,
            timeout: cdk.Duration.seconds(300),
            memorySize: 1024,
            environment: {
                DYNAMODB_ENDPOINT_URL: `http://dynamodb.${this.region}.amazonaws.com`,
                EFS_PATH: '/mnt/efs',
            },
            role: lambdaRole,
        });

and here's what I get when I try to execute the Lambda:

START RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6 Version: $LATEST
Querying for id: 123456
RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6 Error: Runtime exited with error: signal: segmentation fault
Runtime.ExitError
END RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6
REPORT RequestId: 3f9cc289-019d-45fb-8c0f-ead8104a5fb6	Duration: 28.66 ms	Billed Duration: 56 ms	Memory Size: 1024 MB	Max Memory Used: 16 MB	Init Duration: 26.47 ms	

What could possibly cause this? I thought segfaults were "impossible" in Rust. Maybe be something wrong with the Lambda environment?

AlexR
gefragt vor 3 Monaten558 Aufrufe
1 Antwort
0
Akzeptierte Antwort

I'm new to Rust and didn't realize until now that rusoto_dynamodb is essentially unmaintained. Cross-compiling and static linking to musl-libc is also apparently no longer necessary, but still shows up in many blogs and tutorials. My problem was resolved after making these changes:

  • replace rusoto_dynamodb with aws-sdk-dynamodb
  • compile on Amazon Linux 2023 with normal gcc
  • remove the fn main() {} pre-fetch step (this caused another issue which was masked by the initial problem)

Updated Dockerfile:

FROM amazonlinux:2023
RUN yum install -q -y openssl-devel gcc python3 python3-pip zip make tar gzip wget
# install Rust
VOLUME /root/.cargo
ENV CARGO_HOME=/root/.cargo
ENV CARGO_TARGET_DIR=/root/.cargo/target
RUN curl https://sh.rustup.rs -sSf | sh -s -- -y
ENV PATH="/root/.cargo/bin:${PATH}"
# Build the Application
WORKDIR /build
COPY Cargo.toml /build/
COPY src/ /build/src/
RUN cargo build --release
# Place executable into path expected by CDK Construct
RUN mkdir -p bin && cp $CARGO_TARGET_DIR/*/bootstrap bin/bootstrap
RUN strip -s bin/bootstrap

Updated Cargo.toml:

[package]
name = "s3_cache"
version = "0.1.0"
edition = "2021"

[[bin]]
name = "bootstrap"
path = "src/main.rs"

[profile.release]
debug = true

[dependencies]
aws_lambda_events = "0.15.0"
lambda_runtime = "0.11.1"
aws-sdk-dynamodb = "1.32.0"
aws-sdk-s3 = "1.32.0"
aws-config = "1.5.1"
serde_json = "1.0.116"
serde ={ version = "1.0.116", features = ["derive"] }
flate2 = "1.0.30"
tokio = { version = "1.37.0", features = ["full"] }
tokio-util = { version = "0.7", features = ["compat"] }
futures = "0.3.17"
aws-types = "1.3.1"
csv = "1.1.6"
AlexR
beantwortet vor 3 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen