Lambda Integrations

• Some of the lambda integrations are as follows,

Synchronous Invocation

• Synchronous meaning awaiting for result of invocation.

• Lambda functions are synchronously invoked by,

  • ALB

  • AWS SDK

  • API Gateway

  • AWS CLI

  • CloudFront

  • S3 Batch

  • Step Functions

  • Amazon Cognito

  • Kinesis Data Firehose

  • Amazon Alexa

  • Amazon Lex

• Client side should handle the error for lambda invocation.

• DLQ setup is not useful for this type of invocations.

ALB

• Lambda function must be registered in a target group.

• This will invoke the Lambda function synchronously.

• When client sends request to ALB, ALB will converted the request to json based request that will be send to lambda function which will have information about,

  • ALB

  • Http request details like header, HTTP methods type, query-parameters, body, where it is base-64 encoded or not.

• Response from lambda which is in json will be converted to HTTP response, which will have information such as,

  • status code,

  • HTTP response details like headers, body, whether it is base-64 encoded.

• Multi-Header values are supported by ALB and should be explicitly enabled in attributes of ALB, and are shown as arrays within Lambda event and response objects.

Asynchronous Invocation

• Dont need to wait for result of lambda invocation.

• Speeds up the processing time.

• A typical asynchronous processing will work as follows,

  • Events or triggering operation will send the details to an event queue.

  • Once that event is received, lamdba will read the event from the queue and process it.

  • If failure occurs while lamdba execution, 3 retries would be attempted and process the request.

  • In case of exhausting maximum attempt of retries the event is passed to dead letter queue (i.e SQS/SNS or some other message brokers).

  • Ensure lambda function are idempotent, so that retries does not impact the processing or introduce duplication.

• Lamdba should be provided with asynchronous configurations.

• Following AWS services invoke lambda processing asynchronously,

  • S3

  • SNS

  • CloudWatch Events/EventBridge

  • Codecommit

  • CodePipeline

  • CloudWatch logs

  • AWS Simple Email Service

  • CloudFormation

  • Config

  • IoT

  • IoT Events

EventBridge

• This service will send events to the lambda provided target type is set to lambda.

• Can send events periodically or based on some schedule.

• Lambda once configured as target, will recieve the events and can be processed.

S3 Event Notifications

• S3 creates events when object is created, removed, restored.

  • Events can be filtered based on object names.

• Various AWS services can subscribe to these events to react and do asynchronous processing based on them.

• These events are usually send within a minute.

• Some times events for write operations are combined when two operations occur in quick succession or concurrently on same non-versioned object. To avoid this object versioning can be enabled.

• To use this one has to enable the event notification on the S3 bucket.

Event Source Mapping

• It applies to

  • Kinesis Data Stream

  • SQS and SQS FIFO queue

  • DynamoDB Streams

• Lambda functions is invoked synchronously.

Stream based

• It creates an iterator for each shard and process items in order.

• Start with new items, from the beginning or from timestamp.

• Processed items are not removed from the stream.

• Low traffic scenario may use batch window to accumulate records before processing.

• Parallel processing can be done for multiple batches at shard level.

  • 10 Batches can be processed per shard.

  • For each partition key, in-order processing is still guaranteed.

• In case of errors, an entire batch is re-processed untill the function succeeds or the items in the batch expire.

  • To ensure in-order processing, processing for the affected shard is paused untill the error is resolved.

• You can configure the event source mapping to,

  • discard old events

  • restrict number or retries

  • split the batch on error.

• Discarded events can be configured to go to Destination.

Scaling

• One lambda invocation per stream shard.

• In case of parallelization, upto 10 batches processed per shard simultaneously.

Queue based

• Event source mapping will poll SQS using Long Polling.

• Batch size can be specified (1 to 10 messages).

• Queue visibility time out should be 6x the timeout of lambda function.

• DLQ should be setup at SQS and not Lambda, to handle items that could not be processed.

• For lambda sychronous failures, Destination can be setup for failures.

• Lamdba function should be idempotent, as event source mapping might receive same item twice from the queue, even when no error occurs.

• Lambda delete the items from the queues, after they're processed successfully.

Scaling

• Standard

  • Adds 60 more instances per minute to scale up

  • Can process upto 1000 batches of messages simultaneously.

• FIFO

  • Message with same GroupID will be processed in order.

  • Lambda function scales to the number of active message groups.

Destination

• When aysnchronous event mapper or lamdba invocation fails, destination provides a mechanism to save the failures.

• It can be used to store result of both successful and failed asynchronous events.

• Destination can be,

  • SQS

  • SNS

  • Lambda

  • EventBridge Bus

• Unlike DLQ, which allow only asynchronous invocation failures to be send to SQS and SNS, destination provides more service integrations and both successful and failure response can be captured.

Resource Policy and IAM

• In synchronous invocation, the source should have resource based policy attached to invoke the lambda function.

• In asynchronous invocation, as lambda is pulling the message from source, it should have appropriate execution role.

Lambda@Edge

• Edge Function is a code written and attached to a CloudFront Distributions.

• Serverless

• Pay as you use

• Applicable to use following use cases,

  • Web Security and privacy

  • SEO

  • Dynamic Web Application at the edge.

  • Intelligent Routing Across Orgins and Data Centers.

  • Bot Mitigation at the Edge.

  • A/B Testing

  • User Authentication and Authorization

  • RealTime Image Transformation

  • User tracking and Analytics

  • User Prioritization

• Cloud Front provides two types of functions

  • Lambda@Edge

    • Scale to thousands of requests/second

    • Runtime support in NodeJs or python

    • Can be used to change viewer and origin request/response.

    • CloudFront will replicate the Lambda function written in one region.

    • Can access File system, Network Access

    • Can access request body

    • Execution time can be maximum upto 10 seconds

    • Maximum memory can be in the range of 128 MB to 10 GB.

  • CloudFront Functions

    • Light weight functions with run time support of JavaScript

    • For high-scale, latency-sensitive CDN customizations

    • Scales upto millions of requests/second

    • Use to change viewer request/response only.

    • Native feature to CloudFront and is managed within Cloud Front.

    • Cannot access File system, Network Access

    • Cannot access request body

    • Execution time can be maximum within 1 second

    • Maximum memory can be within 10 KB.

    • Application includes

      • URL rewrites or redirects

      • Header manipulation

      • Cache key formation using request attributes

      • Request authentication and authorization

Lambda At Edge Request Flow

VPC

• By default, Lambda function is launched outside your own VPC (i.e., in AWS owned VPC).

• So, it cannot access resources in your VPC (RDS, ElastiCache, internal ELB etc).

• However, Lambda can be deployed in your private VPC with subnet and the security group.

• When a Lambda is deployed in private VPC, behind the scenes it will create a private ENI (Elastic Network Interface) in your subnet, provided lambda has AWSLambdaVPCAccessExecutionRole execution role. And through this ENI the Lambda can connect to resources in private VPC.

Internet Access and Lambda

• By default, Lamdba in your VPC does not have access to public internet.

• Similarly, deploying a lambda in public subnet still does not give access to public IP or internet access.

• When a lambda is deployed in private subnet, it can access internet through a NAT gateway, which in turn will connect to internet Gateway.

• VPC endpoints can be used to access resources deployed in AWS cloud privately without any internet gateway.

File System mounting

• Lambda functions can access EFS file system, if they are running in same VPC as lambda function.

• Lambda can be mounted with EFS to local directory during initialization.

• Must leverage EFS Access points, however ensure to watch out for EFS connection limits as each function invocation creates a connection to EFS. Also ensure EFS connection burst limits.

• Refer this to get a good understanding of all storage options available for lambda.

CloudFormation and Lambda

• CloudFormation can be used to upload a lambda function.

• There are two ways,

  • Inline Code

    • Inline lambda code in CloudFormation template for simple functions.

    • Disadvantage of this way is we cannot include dependencies.

  • Zip File

    • This can be done through S3.

    • Refer the S3 zip location in CloudFormation template.

    • Note that S3 bucket with lambda code can be in different AWS account as well.

      • For this to work properly, execution role and S3 bucket policy should be configured.

      • S3 bucket policy should allow the other account as Principal to read.

      • Lambda function from the other account should have the relevant execution policy.

    • The code needs to be zipped and has been uploaded to S3 file location.

    • The CloudFormation template will now be able to refer this file in CFT.

Lambda Container Images

• Base image needed for the container should implement Lambda API runtime API.

• Upto 10 GB of lambda function as container image from ECR can be deployed in to the container.

• Test the containers locally using the Lambda Runtime Interface Emulator.

Lamdba Containerized Images

Best Practices of Lambda Container Images

• Use AWS provisioned Base Images

• Use Multi-Stage Builds

• Build from Stable to Frequently Changing

• Use Single Repository for Functions with Larger Layers

• Use them to upload large Lambda Functions, upto 10 GB.

Lambda version

• When a lambda function is published, we create a version.

• Versions have increasing version numbers, and are immutable.

• Versions get their own ARN.

• Each version of lambda function can be accessed.

Aliases

• Pointers to Lambda function versions.

• We can use this to have them point to different Lambda versions.

• They are mutable.

• These enable for canary deployment, by assigning weights to Lambda functions.

• Aliases enable stable configuration of our event tiggers/destinations.

• Like Lambda version, they also get their own ARN.

• Note, aliases cannot reference aliases.

Lambda & CodeDeploy

• It can help you traffic shift for Lambda aliases.

• The traffic shift weight percentage can be varied using Code deploy and this feture is integrated within SAM (Serverless Application Model) framework.

• Grow traffic every N minutes untill 100%,

  • Linear :

    • Linear10PercentEvery3Minutes`

    • Linear10PercentEvery10Minutes`

  • Canary

    • Canary10Percent5Minutes

    • Canary10Percent5Minutes

  • AllatOnce: Immediate

• Rollbacks can be implemented for failure after version upgrade using Pre and Post Traffic hooks to check health of the Lambda Function.

HTTP Endpoint for Lambda

• Lambda function can have unique dedicated endpoint, which never changes.

• Supports resource based policies & CORS configurations.

• Can only be accessed via public internet, its of the form

    https://<url-id>.lambda-url.<region>.on.aws // _(dual-stack IPv4 or IPv6)_

• Create and configure using AWS Console or AWS API.

• Can be enabled on latest Lambda function version ($LATEST) or Lambda function aliases, cannot be applied to function with versions other than ($LATEST).

• Cannot have an endpoint URL on already published Lambda endpoint.

Lambda Function URL security

• Resource-based policy

  • Authorize other accounts/ specific CIDR / IAM principals.

• Cross-Origin Resource Sharing (CORS)

• Authorization Type

  • NONE

    • Unauthenticated access, resource based policy is always in effect and must grant public access.

  • AWS_IAM: IAM is used to authenticate and authorize requests.

    • Both Principal's Identity-based Policy & Resource-based Policy are evaluated.

    • Principal must have lambda:InvokeFunctionUrl permissions.

    • Same account - Identity-based Policy OR Resource-based Policy will ALLOW the request.

    • Cross account - Identity-based Policy AND Resource-based Policy will ALLOW the request.

CodeGuru

• Gain insights to runtime performance of your Lambda functions using CodeGuru Profiler.

• It creates a Profiler Group for your lambda function.

• Activate from AWS Lambda console, and is available for Java and Python runtimes.

• Once activated, Lambda adds:

  • CodeGuru Profiler layer to your function.

  • Environment variables to your function.

  • AmazonCodeGuruProfilerAgentAccess policy to your function.