Month: October 2013

Getting a Django app to use HTTPS on AWS Elastic Beanstalk

The site that I am building needs to be secure.  So I want all requests to go through HTTPS rather than HTTP.  This is simple to do with my normal toolset.   Doing this with Django deployed on Amazon’s Elastic Beanstalk was a whole different ordeal.  After much Googling, head scratching, and experimenting I finally found a configuration that works.

Here goes:

Phase 1 – Setup your certificate and open the HTTPS ports

  1. Deploy your Django app to AWS (see Amazon documentation or previous posts for how to do this).
  2. Create a certificate (self-signed is fine) and use the Amazon IAM tools to upload it.  You can find instructions here.
  3. Open the AWS Elastic Beanstalk admin console
  4. Click on your application, then click on ‘Configuration‘ in the left hand column
  5. Click on the gear icon in the ‘Load Balancing” box at the bottom of the page
  6. In “Secure listening port” select 443
  7. In “Protocol” below that, select HTTPS
  8. In “SSL certificate ID” select the certificate you uploaded in step 2.
  9. In Application health check URL: enter the path to some page in your site that will not be protected (i.e /about)
  10. You might want to note the Load Balancer ID at this time, you may need it to troubleshoot later.
  11. Click “Save
  12. The instance will restart

At this point your site should work if you type https://<mysite.com> instead of http://<mysite.com>.  If it isn’t connecting, then there are a few things to check out:

  1. Click on “Configuration” again in the left hand column
  2. Click on the gear icon in the “Instances” box
  3. Note the “EC2 security groups” id so you can find the same one in the next steps
  4. Go to the AWS EC2 admin console  https://console.aws.amazon.com/ec2/
  5. Click on “Security Groups” in the left hand column
  6. Find the security group id from step 3 and click on that
  7. Click on the “Inbound” tab
  8. Verify that there is a rule for both 80(HTTP) and 443(HTTPS).  If 443 is missing, add it and then click “Apply Rule Changes”
  9. Wait a minute and check the site again to see if you can access it using https.   If so, go to the next phase, otherwise continue here
  10. Click on “Load Balancers” in the left-hand column.
  11. Click on the Load Balancer id from Step 10 in the first section
  12. Click on the “Listeners” tab that appears
  13. Ensure there is an entry for HTTPS like below.  If anything is missing, correct it.  Ensure the SSL cert matches the one you uploaded earlier.
Load Balancer Protocol
Load Balancer Port
Instance Protocol
Instance Port
Cipher
SSL Certificate
Actions
HTTP
80
HTTP
80
N/A
N/A
HTTPS
443
HTTP
80
<your cert name>

Phase 2 – Configure Django to use HTTPS

By now your site is accessible through https, but it isn’t forcing users to use https.   Let’s do those steps now.

First let’s secure Django.  In the apps’ Django settings file, add the following:

SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')
SESSION_COOKIE_SECURE = True
CSRF_COOKIE_SECURE = True

The two ‘cookie’ settings tell Django not to accept or issue these if the request isn’t secure.  This effectively keeps users from signing in or posting data when using HTTP.

The SECURE_PROXY_SSL_HEADER is a little trickier to explain.  We setup the AWS Elastic Beanstalk Load Balancer to listen on port 443, but internally it talks to your app on port 80. This can be seen in the Load Balancer configuration above where port 443 redirects to port 80.   So traffic coming to your app is ALWAYS on port 80.  So how will it know if the user is actually using https?  Well, Amazon adds an HTTP header to the request it passes to your app if the original request is using https.   The name of that header is “X_Forwarded_Proto”.   By setting SECURE_PROXY_SSL_HEADER = (‘HTTP_X_FORWARDED_PROTO’, ‘https’) you’re telling Django to treat any request that contains that header as a https request.  Yes, the two headers have different names depending on if it is used in HTTP or in Django.

Go ahead and deploy those changes to the AWS server and wait a few minutes for it to restart.

If all went well, at this point you should be able to access your app on https and log into your app on https.   Logging in with http should fail and return the user to the login screen because the session cookie won’t be issued thanks to the settings you just deployed. Posting data using HTTP should also fail for CSRF not being secure (provided your app uses CSRF…and it should)

Phase 3 – Redirect HTTP user to HTTPS

We’re still not done.  We want to automatically switch the user to https if they access the app using http.  To accomplish that, we have to add some redirect rules to Apache to handle this so there’s one more configuration to change.

The steps below have been replaced with the steps in this post

  1. ssh into your EC2 instance (see previous posts for how to do this)
  2. cd /etc/httpd/conf.d/
  3. sudo vim wsgi.conf
  4. Add the following within the <VirtualHost>…</VirtualHost> marker
    RewriteEngine On
    RewriteCond %{HTTP:X-Forwarded-Proto} !https
    RewriteRule !/about https://%{SERVER_NAME}%{REQUEST_URI} [L,R=301]
  5. Change the /about to whatever page you decided to use back in Step 9 in the first section
  6. Save the file
  7. Return to your Elastic Beanstalk admin console and select Restart App Server(s) under the ‘Actions” menu.
  8. The servers will restart
  9. Go to your site using http://<mysite.com>, the browser should redirect automatically to https://<mysite.com>

Now everything should be working.  The only issue will be that wsgi.conf gets overwritten each time you deploy your app.  So you have to do Step 4 above each time.   I am still working out how to configure that to automatically happen. Stay tuned for an update on that. –UPDATE– New instructions can be found in this post: Updating WSGI automatically in Amazon AWS

Advertisements

Connecting to Amazon RDS

Once you can access your Amazon EC2 instance through SSH (see this post), getting access to your RDS instance is fairly simple.  Here’s how:

  1. Log into the Amazon RDS Admin Console
  2. Click on “Instances” in the left hand column
  3. Find the instance you want to access, it may be the only one listed.  If not, you can go to your Elastic Beanstalk admin console, click on your app, Configuration, RDS Instances and find the instance ID.
  4. Select and copy the Endpoint URL.  You’ll need this later.

Here you have a couple of choices:  The first option is to access the RDS instance through the SSH tunnel you created previously (more secure), or the second option is to open a port (less secure).

Option 1: Using the SSH tunnel

Many database tools have an option to use an SSH tunnel.  I’m using MySQLWorkbench, these steps may vary depending on your tool.

  1. You’re done with the Amazon RDS Admin Console. So you can close it.
  2. Open your database tool and create a new connection document
  3. Select “Standard TCP/IP over SSH” as the Connection Method
  4. Enter your EC2 Instance endpoint at the SSH hostname (see previous post for where to find this if you don’t have it)
  5. Enter “ec2-user” for the SSH Username
  6. Select the keypair you created for the SSH connection for “SSH Key File
  7. Enter the RDS Endpoint from step 4 in the first section as “MySQL Hostname
  8. Enter 3306 as the “MySQL Server Port” (this will differ if you aren’t using MySQL)
  9. Enter “ebroot” as the “Username
  10. Optionally, save your RDS password.
  11. Test and save the connection.

Option 2: Opening a port

  1. In the Amazon RDS Admin Console, note the ID of the link in the first “Security Groups” section, then click the link
  2. Find the group with the ID from the previous step and click on it.
  3. Select the “Inbound” tab in the section that appears
  4. Add a new rule for 3306 (MYSQL) or whatever database you are using.
  5. Specify your IP address as the Source.  You can also use 0.0.0.0/0 initially for testing but don’t leave it like that because it is a security risk.
  6. Click “Apply Rule Changes”

Whichever option you choose, you should now be able to access the RDS instance from your local machine.

Accessing an Amazon EC2 instance using SSH

Being able to have command line access to the server is important.   Amazon AWS allows this, but it isn’t setup by default.   Amazon provides instructions on how to do it, but this took me a while to configure correctly because a key piece isn’t documented.

First you need to create a key pair to secure your connection.

  1. Go the AWS EC2 admin console
  2. Click “Key Pairs” in the left hand column
  3. Click the “Create Key Pair” button on the top of the page
  4. Give it a name and click “Yes”
  5. The key pair will be downloaded to your desktop as a file called <keypair>.pem  It will be in your browser’s Dowloads folder.  You’ll need this file in a minute so find it!
  6. Click on Instances in the left hand column
  7. Find your App’s instance in the list (may be the only one) and click on it
  8. Note the “Security Groups” id in the Description section that appears
  9. Also select and copy the “Public DNS” URL.  You’ll need that later.
  10. Click on “Security Groups” in the left-hand column
  11. Find the group with the id from step 8 and click on it
  12. Click on the “Inbound” tab
  13. Add a rule for 22 (SSH) with the source as your local IP.  You can also use 0.0.0.0/0 if you are experimenting, but don’t leave it like that as it could be a security risk.
  14. Click “Apply Rule Changes
  15. Now here’s what’s not documented… Go to the AWS Elastic Beanstalk admin console
  16. Click on your app
  17. Click on “Configuration” in the left hand column
  18. Click on the gear icon in the “Instances” box
  19. Select your key pair in the “EC2 key pair” box.
  20. Click ‘Save
  21. The server will restart.

Now the server should allow you to access it on port 22 with the key pair you created.  Let’s try it!

  1. Open a terminal app and navigate to where ever your <keypair>.pem file is saved
  2. chmod 400 <keypair>.pem
  3. ssh-add <keypair>.pem    This will add your keypair to SSH so you don’t have to keep referring to it
  4. ssh ec2-user@ec2-xxx-xx-xxx-x.compute-1.amazonaws.com   –  Always use ec2-user as the username and use the public DNS that you copied in step 9 above.
  5. It should now work

Django. Unchained?

I am primarily a Java developer.  For many years now I have been designing and building large enterprise-scale Java web applications which run on dedicated IBM WebSphere application servers.  It’s fine, but it is also heavy-weight and resource intensive.

For my personal project I needed a web app that I could get up and running quickly, that looked good, and could be hosted on a scalable cloud-based environment.  After some research and experimentation I settled on Python and Django for the programming language and framework, MySQL for the relational database, Bootstrap for the CSS/Javascript framework and Amazon AWS Elastic Beanstalk for the hosting environment.

Getting started

First things first, I needed to install Django on my Mac.  Mac OSX already comes with a version of Python and Django installed.  To check, open a Terminal window and type:

python

If python is installed it should display the version information:

$ python
Python 2.6.7 (r267:88850, Oct 11 2012, 20:15:00)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.

 

There are other things such as brew, pip and virtualenv which make working with Python easier.  I’m working on a Mac with OS X at the moment so I followed instructions (mostly) that I found here.

Once those things were installed, Amazon provides instructions on how to setup a basic Django project for use in the AWS Elastic Beanstalk environment which are fairly straight-forward so there isn’t a need to duplicate those instructions here either.

If you follow the instructions at the two links above you should have a Django app up and running within an hour.

 

New toys in the toy box

So I’ve found myself with some extra time on my hands and decided to learn some new skills.  I’ve always wanted to try a lighter web development framework.  Java is fine, but once you start to pile on JEE with JPA, Spring, Struts, Hibernate, etc the whole thing quickly becomes unwieldy.

I wrote a site in PHP a few years ago and enjoyed the relative simplicity of it so I was looking for a language along those same lines.  I thought about learning Ruby on Rails but decided to wait on that one.  Instead I chose to learn Python and Django, with Bootstrap as the Javascript/CSS framework.  Django is mature and has a ton of 3rd party libraries that handle almost any need.  It seemed with this combination I could get a site up and running quickly (or faster than I could if I threw all my Java skills at the same problem).

Well, 2 days later and not only is the site running, I’ve deployed it to Amazon’s cloud-based hosting environment (AWS Elastic Beanstalk, S3 storage, Route 53 domain, etc.  It has all been an interesting technical adventure which I will hopefully document in more detail in this blog in the coming days and weeks.