Getting Started with Django and Heroku

I’ve written a few Django applications recently that were made up of basically the same components: a web server, an asynchronous worker, a Postgres SQL database, Amazon S3 storage, Mailgun E-Mail management, and Stripe credit card billing.

After setting up a django application once with the above components, it’s extremely easy to do it again. Here, I just wanted to document the process that it might help some of you all, my billions of readers.

Start a Django Project

Heroku already has some instructions for getting started with a Django application and a virtual environment, but I just wanted to go a step further with some of the additional add-ons you’ll probably want.

Additional Requirements

The standard requirements.txt from the Heroku template are as follows:

Django==1.7
dj-database-url==0.3.0
dj-static==0.0.6
gunicorn==19.1.1
psycopg2==2.5.1
static==0.4
wsgiref==0.1.2

I additionally include the following packages as a base:

django-storages==1.1.8
boto>=2.31.0,<3.0.0
stripe==1.19.0
requests==2.4.1
python-memcached==1.53
celery==2.5.5
  • Django Storages will simplify the process of transferring static files to Amazon S3
  • Boto is a handy tool to dynamically upload files to S3 which is useful if your end users will end up uploading any sort of files to your web application
  • Stripe is pretty much the most developer friendly credit card billing service
  • Requests is probably the best high level library for making server-side network calls
  • Python-memcached is useful as an easy backend for Django’s caching mechanism. Free add-ons can be found with Heroku to support a minimal memory requirement
  • Celery is used to consume tasks that are messaged via your message broker (I’m using RabbitMq). I ended up using Celery 2.5 instead of Celery 3.x because of some whacky configuration problems that I ended up giving up trying to solve (don’t judge, these are weekend/train home projects).

Download a CSS Template, Convert it to a Django Template

Good HTML and CSS take a long time, and if you’re primarily a backend developer, then you’re probably also not good at it. I like to just buy CSS templates from a reputable site and not worry about it. I generally start at http://www.mojo-themes.com/.

The next problem is that you’ve been handed raw HTML, and it takes a while to convert everything into a Django template. This will take time, but I have this small script that helps out for 99% of the cases converting static image/javascript/css URL’s into django static URL’s:

import sys


strs_to_replace = [
    "images/",
    "img/",
    "css/",
    "js/",
]
end_char = "\""


def clean_string(entire_string, str_to_replace):
    start_index = 0
    new_str = ""
    while start_index < len(entire_string):
        try:
            next_instance_index = entire_string.index(str_to_replace, start_index)
        except ValueError:
            new_str += entire_string[start_index:]
            break
        new_str += entire_string[start_index: next_instance_index]
        end_index = entire_string.index(end_char, next_instance_index)
        str_to_modify = entire_string[next_instance_index: end_index]
        str_to_modify = "%s%s%s" % ("{% static '", str_to_modify, "' %}")
        new_str += str_to_modify
        start_index = end_index
    return new_str


def make_static(target_file):
    with open(target_file, "rb") as read_file:
        new_html = read_file.read()
        for str_to_replace in strs_to_replace:
            new_html = clean_string(new_html, str_to_replace)
    with open("updated-%s" % target_file, "w+") as write_file:
        write_file.write(new_html)

if __name__ == "__main__":
    target_file = sys.argv[1]
    make_static(target_file)

Just run the above script against a file, and Django “{% static %}” tag will be placed in the appropriate locations. Just ensure that the top level folders for images, javascript, css, etc are placed in your Django static directory.

Then you’ll also need to add this to the top of the file:

{% load staticfiles %}

I generally just pick one page to work with, and from there use Backbone.js and underscore templates for additional rendering rather than go the standard server-side templating route espoused by Django.

Get your Django Application Working Locally

Get your app working locally. From your root directory where you started the project you should be able to:

foreman start web

And load pages from “http://localhost:5000”

Initialize a Git Repository

Create a repository on Github’s website and follow the directions to associated that repository with your new code. That can be done with:

  git init
  git add .
  git commit -m "base commit"
  git remote add origin https://github.com/{{ username }}/{{ repo-name }}
  git push origin master

Sign Up for Necessary Services

The rest of the tutorial assumes you have a Stripe Account, an Amazon S3 Account, and a MailGun Account. So if you don’t have those, just go sign up, it’s free.

Adjust Heroku Settings

You’ll end up running these commands:

heroku login
heroku create
git push heroku master
heroku ps:scale web=1
heroku apps:rename cool-new-app-name

The last step is not necessary, but eventually you’ll want to rename your app to something meaningful.

Get the essential add-ons

For the project requirements I listed initially, the Heroku add-ons ended up being Cloud AMQP and PostgreSQL database. So:

heroku addons:add cloudamqp
heroku addons:add heroku-postgresql

For the AMQP service, there should be a broker URL that gets output. Add that to your Heroku environment variables (next step).

Set environment variables

Don’t make the same mistake I did and leave any of your API keys in a file that gets pushed to a public git repository. It will get found by some nerds eating hot pockets in a basement, and fraudulent charges will be made, and you’ll have to convince Amazon support that it was not you that spun up 200 virtual machines in a matter of minutes across every possible region.

So keep your environment variables stored in ~/.bash_profile (Mac) or ~/.bashrc (Linux). For dev variables, such as your local database settings, your app will run seamlessly, and for things that should be kept secret such as API keys, you can keep these relatively secure.

I ended up setting the following variables:

AWS_ACCESS_KEY_ID
AWS_SECRET_ACCESS_KEY
CLOUDAMQP_URL
DATABASE_HOST
DATABASE_NAME
DATABASE_PASSWORD
DATABASE_USER
DJANGO_SECRET_KEY
HEROKU_POSTGRESQL_COBALT_URL
MAILGUN_API_KEY
STRIPE_LIVE_PUBLISHABLE_KEY
STRIPE_LIVE_SECRET_KEY
STRIPE_TEST_PUBLISHABLE_KEY
STRIPE_TEST_SECRET_KEY

You can run through each key with

heroku config:set AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID

or specify the raw value if it’s not common between your dev and prod system. For all of the database settings, you should be able to login to Heroku, view your add-ons for an app, and get the values from there.

Upload Static Files to Amazon

My settings files end up looking something like this:

AWS_ACCESS_KEY_ID = os.environ["AWS_ACCESS_KEY_ID"]
AWS_SECRET_ACCESS_KEY = os.environ["AWS_SECRET_ACCESS_KEY"]
AWS_STORAGE_BUCKET_NAME = "credit-serve-static"

if os.environ.get("I_AM_IN_DEV_ENV"):
    STATIC_URL = '/static/'
else:
    STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
    DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
    STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.amazonaws.com/'

You’ll also need to create an Amazon bucket (in this case “credit-serve-static”) in Amazon S3 before you can run a script to upload files. Once that’s configured, you can safely upload static files in your django directory to amazon with:

heroku run python manage.py collectstatic --noinput

Update Your DNS Manager to point to Heroku

Depending on where you bought your domain name, you’ll need to go into your domain name manager, and set the @ record to redirect to your www cname. Then set the www record to point to http://{ app-name }.herokuapp.com. Google around for your particular service and you should be able to find directions.

The End

This is only the beginning, but this should get you to a good starting point with reasonably solid infrastructure for a basic web application. From here you have the necessary support to send emails, upload files to Amazon (either for static purposes or dynamically from user-generated content), create asynchronous tasks, create a dynamic page, and save records to a database. Each of those, however, is another tutorial unto itself.