How to serve Django Static files in a production environment from a private s3 bucket using CloudFront.

5 min readApr 16, 2021

There are many tutorials on the internet on how to serve static files from a public s3 bucket, but did you know that you could serve static files from a PRIVATE s3 bucket using django-storages and boto3, and Amazon Cloudfront?

Static files will be visible on the browser anyway, so why serve it from a private bucket?

Anyone can make requests directly to your s3 bucket and you might get a hefty bill.
SOC2 certification fails on public buckets.

Create S3 Bucket

Go to https://s3.console.aws.amazon.com/s3/home
Click on Create Bucket.
Name your Bucket appropriately and select your preferred region.
Leave all other options at default and click on Create Bucket.

Create IAM User to access the Bucket:

Go to https://console.aws.amazon.com/iam/home
In the Navigation menu click on Users.
Click on ADD User.
Provide a Username and select access type as Programmatic Access.
Set permissions to the user by attaching existing policies directly. Provide the user with AmazonS3FullAccess

Click on Create User and Download the Access Keys.

Installation

Once you create the IAM user and the Bucket. you need to connect it to your Django Project.

Inside your project’s virtual Environment run these commands:

pip install boto3
pip install django-storages

Now add storages to your INSTALLED_APPS inside settings.py.
Now add the following to your settings.py

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, 'assets'), #path to static files
]

AWS_ACCESS_KEY_ID = os.environ.get("AWS_ACCESS_KEY_ID", "YOUR IAM ACCESS KEY ID")
AWS_SECRET_ACCESS_KEY = os.environ.get("AWS_SECRET_ACCESS_KEY", "YOUR IAM SECRET ACCESS KEY")
AWS_STORAGE_BUCKET_NAME = os.environ.get("AWS_STORAGE_BUCKET_NAME", "YOUR BUCKET NAME")
AWS_S3_REGION_NAME = 'us-east-2'
AWS_S3_SIGNATURE_VERSION = 's3v4'
AWS_DEFAULT_ACL = None
AWS_QUERYSTRING_EXPIRE = 120

One cannot directly access the objects inside a public bucket. To access our files we need to make use of aws’ CDN service, i.e. Amazon Cloudfront! You can use signed URLs that expire after a certain time interval to access your files on the production server.

Create the Distribution

Go to https://console.aws.amazon.com/cloudfront/home
Click on Create Distribution
Select your PRIVATE S3 bucket in the origin domain name
DO NOT make any changes in the Origin ID
Select Restrict bucket access as YES
When you select restrict bucket access as yes aws asks you if want to create a new OAI(origin access identity). One aws account can have up to 100 OAIs. You can use the same OAI for multiple distributions. So if you don't have an OAI, choose Create a New Identity.
Next, you have to provide read permissions to this OAI. You can choose Yes, Update Bucket Policy to automatically update the Bucket policies of the bucket in question or even manually choose to do the same.
Leave the remaining as default and click on Create Distribution. It takes a couple of minutes to Create your Distribution.

Make Changes in settings.py

Paste your Domain Name in your settings as:

AWS_S3_CUSTOM_DOMAIN = 'yourdomain.cloudfront.net'

Now, your STATIC_URL looks like this:

STATICFILES_LOCATION = 'static'  # staticfiles will be in 'static'
STATIC_URL = '//%s/%s/' % (AWS_S3_CUSTOM_DOMAIN,
STATICFILES_LOCATION)

You will have to create custom backends for the static files inside your Django app. So, in settings:

STATICFILES_STORAGE = 'app_name.filename.StaticStorage'

The backend file will look like this:

from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage


class StaticStorage(S3Boto3Storage):
    location = settings.STATICFILES_LOCATION

    def __init__(self, *args, **kwargs):
        kwargs['custom_domain'] = settings.AWS_S3_CUSTOM_DOMAIN
        super(StaticStorage, self).__init__(*args, **kwargs)

YAY! you have successfully routed your static files request through Cloudfront! But you are only halfway done… You will still get a 403 Forbidden if you try to access the static files at this point

Creating Signed URLs

To create Signed URLs we need to Create a Private — Public key pair and upload the public key to CloudFront.

Generating Private-Public key pair

Linux:

Linux Users can Generate a Private-public key pair simply by running these commands in the terminal.

openssl genrsa -out private_key.pem 2048

Above Command generates a Private Key

openssl rsa -pubout -in private_key.pem -out public_key.pem

The above command generates a public key for the generated private key.

Windows:

Windows users cannot run these commands directly in the Powershell/CMD.

Download the Binaries for openssl from this link.
Extract the ZIP files and navigate to bin directory.
Open CMD inside the ‘bin’ directory.
Run the command openssl in the CMD

Now run the following commands.

genrsa -out private_key.pem 2048

Above Command generates a Private Key

rsa -pubout -in private_key.pem -out public_key.pem

The above command generates a public key for the generated private key.

Both the private_key.pem and public_key.pem will be generated inside the ‘bin’ directory. You can use a text editor to view the files.

Upload Public Key to CloudFront:

Go to https://console.aws.amazon.com/cloudfront/
In the navigation menu, choose Public keys.
Choose Add public key.

Provide a key name of your choice
Paste the contents of the public_key.pem in the key value.
You can even add an optional comment and click ADD

The public key has been added to CloudFront.

Now back to settings.py and include the following:

AWS_CLOUDFRONT_KEY = os.environ.get('AWS_CLOUDFRONT_KEY', None).encode('ascii') 
AWS_CLOUDFRONT_KEY_ID = os.environ.get('AWS_CLOUDFRONT_KEY_ID', None)

AWS_CLOUDFRONT_KEY is the contents of private_key.pem you generated in the previous step.
AWS_CLOUDFRONT_KEY_ID is the ID of the Public Key uploaded to CloudFront.

Update the Environment Variables respectively and you are good to go! Now the Staticfiles URLs on your website will look something like this:

https://yourdomain.cloudfront.net/static/path/filename.ext?Expires=1618607085544&Signature=le0O~4ylD8y2yAv76ASel-BzWfEztVzJgTSUNBasDjpjgfdjB80QeRun6-pbvc-g7T1qKajKileamCTo6Fc9Ujr47ww8lfdszomZ4vgyMPe7y8s3TUmTdmWkx6BrCwKRKaQD83m0mx~wd7JxuXJtFOLZDxfsf83Pqh8dot~DOA8T4hmGdSgDs-en09IL2VENHfsdfc6Rq8xvATKiI7-uBxDfsdfN9dZbqIAd-AdrQgw-lSOyLOeA4xrCOC5zXoxT8kvWCosfsH0GiUL61nfuiORY430EuVT6ZTLx2vRCfqjVgVFXS56GZVfbCDdCR2nfsdfsSeZkp1B3kxO7208iR37EnQfsflltc-1pRQomy~HesfsOX-WFvfsdGw__&Key-Pair-Id=KA2FFEI03D1CLOSE9

Thank you for reading this. I hope this tutorial helped you with some concepts of AWS.