In my previous post, we looked at the configuration of Django EB environment database settings (including connecting your EB instance to an instance-independent RDS database option) but there are still some details we need to cover, namely the configuration settings for static files and adding a custom domain.
Let’s quickly recap the previously discussed steps to get to the point where we can focus on the “final touches.” For detailed explanations, be sure to look at the previous posts from this series (1, 2) whereas here, I’ll just list the steps which were covered there without diving into details.
First of all, let’s quickly set up a local Django project using the commands shown below (if you need to copy-paste the code shown below, you can grab it on GitHub):
Recap – Commands for setting up Django project on Ubuntu workstation
With that, we now have a virtual environment and a Django project created. Things to take note of above are that we are keeping the virtual environment and project folders separate and renaming the project folder to distinguish between the root project folder and the “project app” folder. The next step is to configure the Django project for EB deployment, which involves the steps shown below (GitHub link):
Recap – Commands for configuring Django project for EB deployment
After performing these steps, we have our Django app deployed to AWS EB. I did not include a recap of database-related configuration settings – for that you can refer to my previous post which covers this topic in detail, but these settings are not strictly necessary for us to continue with the steps we will be discussing in this post. The only comment I would like to make on database configuration options is that despite in my previous post I explained how you can configure your AWS EB Django app to use AWS RDS instance decoupled from EB app, you may now want to do so if you want to enjoy rapid provisioning and stick with pure EB way of doing things.
But like I just said, for our discussion here steps we briefly recapped above are sufficient to continue and discuss the configuration of S3 storage for static and media files and adding custom domain name for our AWS EB Django app (or the environment if we stick to EB terminology).
Static and media files. Any Django app needs to serve additional files such as images, JavaScript, or CSS, which are commonly known as static files in Django terminology. In addition to this Django app may need to store files uploaded by app users, and those referred to as media files. Let see how we can ensure the persistence of those files for our app.
By design AWS EB creates and destroys EB environment’s resources on “create” and “terminate” environment stages, respectively. That means that the database and static and media files added to our Django app when the environment was active will be deleted upon environment termination.
For the environment database, as it was explained in the previous post as soon as you configure EB environment instance you can leverage automatically created database snapshots for database data retention (default option).
In the case of static and media files those, by default, are stored on EC2 instance local storage and EC2 instances that are used by EB environments do not have persistent local storage. As per AWS documentation: “Elastic Beanstalk applications run on Amazon EC2 instances that have no persistent local storage. When the Amazon EC2 instances terminate, the local file system is not saved, and new Amazon EC2 instances start with a default file system.” During EB environment deployment S3 bucket gets created, but it is used only to store such things as application versions, source bundles, custom platforms, log files and saved configurations (see details here). To put it simply it is just a location to which you upload your Django app package before it gets deployed into EC2 instance created for your EB environment, and your Django app static and media files will reside on EC2 instance local storage which is not persistent. By the way, this EB environment-specific S3 bucket receives special policy to prevent its accidental deletion, and you will need to drop this policy in case you will want to delete it.
To ensure retention of EB environment’s static and media files we will need to configure our Django app to use S3 storage service as explained below.
First of all, we need to create an S3 storage unit known as an S3 bucket, which we can do in the Amazon S3 management console (https://console.aws.amazon.com/s3). If you are accessing S3 console after you already created your EB Django environment, then you will see an S3 bucket which was created upon EB environment creation for deployment purposes:
AWS S3 Console – Buckets
Let’s now create new S3 bucket for our static and media files starting with clicking on Create bucket button in S3 console:
AWS S3 Console – Create Bucket
That will open Create bucket page where you will need to fill in bucket name which should be globally unique and should not use spaces and uppercase letters and uncheck “Block all public access checkbox” as shown below (the region should be the same for your S3 bucket and EB environment):
AWS S3 Console – Create Bucket Wizard
Unchecking “Block all public access checkbox” will require you to mark an extra checkbox named “I acknowledge that the current settings might result in this bucket and the objects within becoming public”, which is a way for AWS to make sure that we understand that all data in that bucket will be publicly accessed with such settings. With this checkbox in place we just click on Create bucket button:
AWS S3 Console – Completing Create Bucket Wizard
Bucket gets created almost immediately and appears in the buckets list. We now need to access newly created bucket properties either by clicking on bucket’s name in the list or using “Go to bucket details” button from the green informational pop up (“Successfully created bucket”):
AWS S3 Console – Accessing Bucket Details
Once in the bucket, the properties we need to navigate to Permissions tab > Bucket Policy and paste 12 lines of code as shown below, making sure that bucket name on line 8 matches name of your bucket, and click on Save button:
AWS S3 Console – Adding Bucket policy
Providing that bucket name you specified in your policy was correct your policy will be saved, and we will get bunch of orange tags indicating that bucket now has public access:
AWS S3 Console – Public Access Bucket Policy applied
With this policy in place, we now should switch over to IAM console (https://console.aws.amazon.com/iam) and there click on Create New Group button under Access management > Groups:
AWS IAM Console – Create New Group
That will start Create New Group wizard for us where we will specify some descriptive group name (e.g. “manage-django-test-env-bucket”) on the first wizard step:
AWS IAM Console Create New Group Wizard – Setting Group Name
And further on attach the AmazonS3FullAccess IAM policy to our group:
AWS IAM Console Create New Group Wizard – Attaching Policy
We will finally click on Create Group button on Create New Group Wizard Review page:
AWS IAM Console Create New Group Wizard – Review and Create Group
Once a group has been created, we will need to add a user to it. For that, we will be moving over to IAM Console > Access management > Users and clicking on Add user button as shown below:
AWS IAM Console – Adding User
Once we clicked on Add user button, we will need to provide a descriptive username (e.g. “django-test-env-bucket-user”) and tick Programmatic access checkbox under Select AWS access type as shown below:
AWS IAM Console – Add User Wizard Page 1
On the next step of Add user wizard we just adding this user into our recently created group (“manage-django-test-env-bucket”):
AWS IAM Console – Add User Wizard Page 2
After selecting group by marking checkbox next to it, just complete Add user wizard, ignoring optional tagging (page 3 of the wizard) for now and clicking on Create user on the penultimate wizard’s page:
AWS IAM Console – Add User Wizard Page 4
After that, on the last wizard page, be sure to click on “Download .csv” button to download and save CSV file with user’s credentials (those will be required later on):
AWS IAM Console – Add User Wizard Page 5
We are now ready to adjust our Django project settings so that it will be using this newly created bucket to store static and media files. For that, let’s get back to our Ubuntu machine and install django-storages and boto3 packages into the virtual environment we use for our project using pip install django-storages boto3 command as shown below:
Installing django-storages and boto3 packages
After installing these packages, we should update our requirements.txt file to get them listed there. To do that we will need to switch to project directory first and then use pip freeze > requirements.txt to pipe all installed packages into requirements.txt file once again and next edit this file using nano editor as shown below:
Generating requirements.txt file in project directory
Contents of our requirememnts.txt file should look as follows (requirements on the screenshot below do not include database-specific packages we were discussing in the previous post as we omitted this part here as optional):
Contents of requirements.txt file
As we re-run pip freeze > requirements.txt command we just need to make sure we remove the following lines from the requirements.txt which will cause EB deployment to fail:
pkg-resources==0.0.0
pytz==2020.1
Our next step is to edit Django project settings.py file (remember that this one resides within project app directory inside of parent Django project directory) and adding storages into the list of installed apps along with variables defining bucket access details and files location as shown below (required information in the text form can be found on GitHub):
Adding S3 storage variables and settings into settings.py file
Just a few comments on the data we added into settings.py file. The first block represents bucket access details. There, AWS_STORAGE_BUCKET_NAME and AWS_S3_REGION_NAME can be grabbed from AWS S3 console from your bucket properties page as shown below (region name can be found within URL when you look at your bucket properties):
AWS S3 Console – Grabbing bucket name and region
AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY values can be retrieved from CSV file we downloaded and saved on create user stage. The rest of the settings are common and can be specified exactly as they have shown above.
We also need to create custom_storages.py file in the root of Django project directory which will contain S3Boto storage settings as shown below (custom_storage.py file content in text form):
Content of custom_storages.py file
We are now done with our storage configuration and can test it. If we now execute python manage.py collectstatic command as shown below it will be uploading our static files to the S3 bucket.
Running collectstatic after S3 storage configuration is completed
You will see that execution of this command takes a bit longer than before as we actually uploading data to S3, and providing everything was configured correctly you will see “N static files copied” message after command execution completion. You can also check/view your bucket contents after that – you will be able to see your static files there:
AWS S3 Console – Viewing bucket contents
If you strictly followed through the article you won’t see the “media” folder in newly configured bucked as our empty Django project does not have any media files yet. If you want to test media files upload you can try adding a new app with some model that uses image field type (models.ImageField), register it into the admin site and add an image through Django admin. That’s all it takes to configure S3 bucket as a static and media files storage for your EB hosted Django app.
Adding a custom domain name to your EB Django app. Let’s now have a look at adding a custom DNS name. There is an Amazon documentation covering this process, but I’ll briefly cover the steps involved in this process here to give you an overall idea about steps involved in it.
Adding custom domain to your EB Django environment includes two parts. The first part, performed from the AWS side, and there you will need to create Hosted Zone from AWS Route 53 console as shown below:
AWS Route 53 Console – Hosted zones
Once you clicked on Create Hosted Zone you just need to type in the domain name which you own and want to use for EB environment/Django app, leaving zone type as is (“Public Hosted Zone”) and click on Create button.
AWS Route 53 Console – Create Hosted Zone
With hosted zone created you will need to create two record sets for your domain: A record alias for the bare domain name and another one for www subdomain entry (i.e. yourdomain.com and www.yourdomain.com). You just toggle Alias radio-button to Yes and select your EB environment from Alias Target drop-down locating it under Elastic Beanstalk environments category as shown below:
AWS Route 53 Console – Creating Record Set
This is how your recordsets should look like once two alias record sets were added into your domain hosted zone:
AWS Route 53 Console – Domain hosted zone record sets added
With that, we have records for yourdomain.com and www.yourdomain.com pointing to the EB environment, and we now can move on to the second step which implies switching over to your domain name hosting and connecting your domain name to AWS. Exact UI will vary depending on where your domain name is hosted but what you’ll need to do stays the same – you just need to change your domain nameservers from those of your DNS hosting name servers to AWS nameservers.
I’ll illustrate this process for gandi.net DNS hosting. In this case, you just need to logon to gandi.net and select your domain in the DOMAIN section of the console, and there click on Nameservers tab and click Change on the change button as depicted below.
gandi.net DNS hosting – Changing Nameservers
Once clicked on Change you’ll need to switch your Nameservers to External and paste AWS name server names from your Route 53 hosted zone you’ve just created earlier (see screenshot below).
Adding AWS nameservers as external nameservers for domain hosted with gandi.net
Once you entered all four AWS name servers save the changes and once this update is fully applied and synced through (that may take some time) your EB environment will be accessible through your custom domain name.
In this post, we covered the configuration of AWS S3 storage for the EB Django environment along with adding a custom domain name. And with that, I think we are now fully covered all the basic aspects of AWS EB deployment for Django apps, except for maybe Elastic Load Balancer and SSL configuration but those can be considered as extra/advanced configuration options. I hope this series of posts was informative for you and I do have plans to cover some other Django related topics later on, including looking at other hosting platforms and options. As usual, if you have any questions, comments, or simply want to propose a topic for the next post – feel free to do so in the comments section below.
