Access AWS Resources

Instance Metadata Service (IMDS)

  • Allows EC2 instances to learn about themselves without using an IAM role for that purpose.

  • URL is http://169.254.169.254/latest/metadata.

  • Using this feature one can extract many details other than IAM policy itself like,

    • IP address

    • IAM role name

    • Launch script

  • There are two versions

    • IMDS v1

      • Uses URL http://169.254.169.254/latest/metadata to retrieve the metadata directly.

    • IMDS v2

      • A more secure version of IMDS V1.

      • Uses token to retrieve the metadata information.

            $TOKEN = `curl -X PUT http://169.254.169.254/latest/api/token -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`

      • Use the token retrieved from above request to make the metadata request.

            curl http://169.254.169.254/latest/metadata/ -H "X-aws-ec2-metadata-token: $TOKEN"

      • Only this version is supported for Amazon Linux 2023.

AWS CLI

Introduction

Profiles

  • When there are multiple accounts that need to be connected to using AWS CLI, we can use profiles to isolate the accounts and use their respective credentrials to connect to AWS.

  • To create a profile use the command below,

        aws configure --profile <profile_name>

  • To query the AWS resources using CLI and profiles use the below command.

        aws [command] [option] --profile <profile_name>
    # Example
    aws s3 ls --profile dev

MFA

  • To use MFA and CLI tool, use the STS (Security Token Service) GetSessionToken API to get a temporary session, see below for example.

        aws sts get-session-token --serial-number <arn-of-mfa-device> --token-code <token-code-from-mfa> --duration-seconds <seconds>

    • You get the arn of mfa device when register one in your account.

    • Output of above command looks like below,

        {
        "Credentials": {
            "AccessKeyId": "ASIAIOSFODNN7EXAMPLE",
            "SecretAccessKey": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYzEXAMPLEKEY",
            "SessionToken": "AQoEXAMPLEH4aoAH0gNCAPyJxz4BlCFFxWNE1OPTgk5TthT+FvwqnKwRcOIfrRh3c/LTo6UDdyJwOOvEVPvLXCrrrUtdnniCEXAMPLE/IvU1dYUg2RVAJBanLiHb4IgRmpRV3zrkuWJOgQs8IZZaIv2BXIa2R4OlgkBN9bkUDNCJiBeb/AXlzBBko7b15fjrBs2+cTQtpZ3CYWFXG8C5zqx37wnOE49mRl/+OtkIKGO7fAE",
            "Expiration": "2020-05-19T18:06:10+00:00"
        }
    }

    • Ensure the token obtained from the above command is added to ~/.aws/credentials file against the key aws_session_token.

Note for all the above commands to work user account should have relevant access granted.

Credentials Chain Provider

CLI

  • The CLI looks for credentials in the following order, (precedence order is high to low)

    1. CLI options

    2. Environment Variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN).

    3. Default CLI credentials file (typically located at ~/.aws/credentials).

    4. CLI configuration file (typically located at ~/.aws/config).

    5. Container credentials for ECS task.

    6. Instance profile credentails for EC2 Instance Profile.

SDK

Icon

SDK Icon

About

  • The SDK looks for credentials in the following order, (precedence order is high to low)

    1. Command line option for cli options

    2. System properties (aws.accessKeyId and aws.secretKey).

    3. Environment Variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)

    4. Default credentials profile file (typically located at ~/.aws/credentials)

    5. AWS configuration file (typically located at ~/.aws/config)

    6. Container credentials for ECS containers

    7. Instance profile credentails for EC2 Instance

  • If no region is given, by default us-east-1 will be selected.

Best Practices

  • Never store credentials in code. Let them be inherited from the credentials chain.

  • Use IAM roles as per the requirement.

  • Use Named profiles or environment variables when working outside of AWS.

API Limits

Rate Limits

  • DescribeInstances API for EC2 has a limit 0f 100 request/second.

  • GetObject on S3 has a limit of 5500 GET request/second per prefix.

  • Can ask AWS to increase the limit as per the need.

Service Limits

  • Running On-Demand Standard instances: 1152 vCPU.

  • Can increase the limits as per need by asking AWS through a service ticket.

Service Quota Limits

  • These limits are enforced by AWS and can be increased by making a service quota API request programmatically.

Throttling Exception

  • This exception is raised when too many API calls are made and AWS intermittently raises this exception.

  • This exception is a sign to use exponential backoff while making request.

  • SDKs handle this behaviour inherently.

  • Using APIs directly would require explicit exponential backoff implementation.

    • For 5xx server errors should only be retried.

    • Dont implement retries for 4xx client errors.

Signing AWS Request API (Sig v4)

  • There are 3 ways of doing it

    • Authorization headers

      • Authorization

      • Signature

      • SignedHeaders

    • Query parameters

      • X-Amz-Algorithm

      • X-Amz-Credential

      • X-Amz-Date

      • X-Amz-Signature

    • Browser based uploads USING POST

Reference

Previous12_SDK_CLI_TipsNext13_CloudFront

Last updated