Workshop 15: Python boto3 API

Introduction to boto3

boto3 is the official AWS SDK for Python, enabling developers to interact programmatically with AWS services. Instead of using the CLI or console, you write Python scripts to automate tasks such as provisioning infrastructure, managing storage, publishing messages, and more. boto3 handles authentication, request signing, pagination, and retries, letting you focus on application logic.

In this workshop, you’ll set up boto3, configure credentials, and walk through common use cases: listing EC2 instances, creating and deleting buckets in S3, publishing to SNS, and interacting with DynamoDB. Each step includes code examples and detailed explanations to ensure you understand how the API calls map to AWS operations.

Prerequisites

• Python 3.7+ installed on your system
• AWS CLI configured with a profile or environment variables
• IAM user or role with programmatic access and appropriate permissions
• Basic familiarity with Python scripting

1. Install and Configure boto3

First, install boto3 and its dependency, botocore, using pip. Then verify that boto3 can locate your AWS credentials, either from the default profile or environment variables.

# Install boto3 using pip
pip install boto3

# Verify installation and credentials with a Python REPL
python - << 'EOF'
import boto3
from botocore.exceptions import NoCredentialsError
try:
    sts = boto3.client('sts')
    print("Account:", sts.get_caller_identity()['Account'])
except NoCredentialsError:
    print("Credentials not found")
EOF

Installation Explanation

The pip install command fetches boto3 and its dependencies. In the snippet, we import STS (Security Token Service) to validate that credentials are loaded correctly. The SDK searches for credentials in environment variables, shared config files, or instance metadata.

2. List EC2 Instances

Use the EC2 client to retrieve a list of all instances in a region. This illustrates paginated API calls and filtering.

import boto3

ec2 = boto3.client('ec2', region_name='us-east-1')

# Describe all instances
response = ec2.describe_instances()
for reservation in response['Reservations']:
    for instance in reservation['Instances']:
        print(instance['InstanceId'], instance.get('State', {}).get('Name'))

EC2 Listing Explanation

The describe_instances call returns nested structures: reservations and instances. We iterate through them to print each instance ID and its state. boto3 automatically handles request formatting and returns Python dictionaries.

3. Create and Delete an S3 Bucket

Demonstrate object storage management by creating a new bucket, listing buckets, and finally deleting it.

import boto3

s3 = boto3.client('s3')
bucket_name = 'workshop15-boto3-demo-unique'

# Create bucket
s3.create_bucket(Bucket=bucket_name)
print(f"Created bucket: {bucket_name}")

# List existing buckets
for b in s3.list_buckets()['Buckets']:
    print(b['Name'])

# Delete bucket
s3.delete_bucket(Bucket=bucket_name)
print(f"Deleted bucket: {bucket_name}")

S3 Bucket Explanation

The create_bucket method provisions an S3 bucket. We then call list_buckets to verify its creation. Finally, delete_bucket removes it. In production, ensure the bucket is empty before deletion or use delete_objects to purge contents.

4. Publish Messages to SNS

Use SNS to send notifications. This example publishes a message to an existing topic ARN.

import boto3

sns = boto3.client('sns')
topic_arn = 'arn:aws:sns:us-east-1:123456789012:Workshop13Topic'

response = sns.publish(
    TopicArn=topic_arn,
    Subject='boto3 Test',
    Message='Hello from boto3!'
)
print('MessageId:', response['MessageId'])

SNS Publishing Explanation

The publish method sends a message to all subscribers of the topic. The response contains a unique MessageId, useful for tracking and logging. Ensure your credentials have sns:Publish permission.

5. Interact with DynamoDB

Illustrate NoSQL database operations by creating a table, putting an item, retrieving it, and deleting the table.

import boto3
from botocore.exceptions import ClientError

ddb = boto3.client('dynamodb', region_name='us-east-1')
table_name = 'Workshop15Table'

# Create table
try:
    ddb.create_table(
        TableName=table_name,
        KeySchema=[{'AttributeName': 'Id', 'KeyType': 'HASH'}],
        AttributeDefinitions=[{'AttributeName': 'Id', 'AttributeType': 'S'}],
        ProvisionedThroughput={'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5}
    )
    waiter = ddb.get_waiter('table_exists')
    waiter.wait(TableName=table_name)
except ClientError as e:
    print('Table exists or error:', e)

# Put item
ddb.put_item(
    TableName=table_name,
    Item={'Id': {'S': '123'}, 'Message': {'S': 'Hello DynamoDB'}}
)

# Get item
resp = ddb.get_item(TableName=table_name, Key={'Id': {'S': '123'}})
print('Item:', resp.get('Item'))

# Delete table
ddb.delete_table(TableName=table_name)

DynamoDB Explanation

We create a simple table with a string primary key. boto3 provides a waiter to pause until the table is active. We then perform basic CRUD: put_item, get_item, and finally delete_table. Exception handling ensures idempotent runs.

6. Cleanup and Best Practices

Always clean up resources to avoid charges. Use try/except blocks to handle existing resources. For production scripts, implement retries with exponential backoff, use IAM roles instead of static keys, and leverage environment-specific profiles for isolation.

Next Steps

Explore AWS service-specific resource classes in boto3 resource API (e.g., boto3.resource('s3')) for higher-level abstractions. Integrate boto3 scripts into AWS Lambda for serverless automation, and consider using AWS SDK for Python in combination with frameworks like Chalice or AWS CDK for infrastructure as code in Python.