Deploying Django Application on Apache2 and Ubuntu

Posted by vlad on Jul 17, 2012 in Blog, Howto | 2 comments

Annoyed by tedious application deployment? Can’t remember the steps? Previous web developer did not leave notes on how he deployed stuff to production? Have no fear, here’s your detailed instructions.

Web application deployment is one of those tasks that are complex enough to take you half a day to do properly, and at the same time they are simple enough that when you finally figure out how to do it, you say, “Aha, so that’s how it’s done! I’m glad I know now!” And you happily forget about it, until some weeks or maybe months later you need to deploy another one, and you scratch the back of your head, and for the love of everything good, can’t remember why WSGI can’t import that %*!$^& module or something like that.

Well, no more. Here’s the latest, freshest guide to deploying your app to Ubuntu 10.04 LTS, Apache2, mod_wsgi, Python 2.6 or 2.7, MySQL, and Django 1.4, which hopefully will still work in at least one or two future versions.

Assumptions / Prerequisites

We’ll assume that you have an Ubuntu server with Apache2 installed. And MySQL (or PostgreSQL, but we won’t go into details on Postgres here – configuration is similar enough, no need to repeat what’s easy to google). We’ll assume that Apache2 is running under www-data user, and that you have your root password to MySQL.

We’ll assume that you have a way to access the server via SSH and that you can sudo.

And you have Python installed. Ubuntu 10.04 comes with Python 2.6 and does not intend to upgrade to 2.7, so if you want to upgrade, you’ll need to build Python 2.7 and mod_wsgi from sources.

We’ll assume that you’ve got all that, and also the sources of the app to a server directory.

Let’s verify our assumptions.

• Check that you have Python (and learn what version you have):

  $ python -V
  Python 2.6.5

  or (if you compiled and installed Python 2.7 by hand)

  $ python2.7 -V
  Python 2.7

• Check that you have MySQL:

  $ mysql -V
  mysql  Ver 14.14 Distrib 5.1.63, for debian-linux-gnu (i486) using readline 6.1

• See if you have Apache2 and who’s running it:

  $ ps -ef|grep apache
  root     30596     1  0 02:29 ?        00:00:01 /usr/sbin/apache2 -k restart
  www-data 32034 30596  0 03:45 ?        00:00:01 /usr/sbin/apache2 -k restart
  www-data 32035 30596  0 03:45 ?        00:00:01 /usr/sbin/apache2 -k restart
  www-data 32036 30596  0 03:45 ?        00:00:00 /usr/sbin/apache2 -k restart
  ...

• Finally, see that you can log in to MySQL as root:

  $ mysql -u root --password=<your password>

If there are no errors so far, we’re ready to start.

Set Up Database

During development, it is ok to use sqlite3 for your Django app. But in production, we’ll need something more serious. Let’s say you already have a database name, a user name, and a password in settings.py file. Do you remember if you’ve already set up the DB in the past, or was it still on TODO list? Let’s check:

$ mysql --batch --skip-column-names -u <username> --password=<userpass> -e "SHOW DATABASES LIKE '<dbname>'"

If it spits out the name of your database, you’ve already set it up. Otherwise, follow these three steps:

mysql -u root --password=<rootpass> -e "CREATE DATABASE <dbname>"
mysql -u root --password=<rootpass> -e "GRANT USAGE ON *.* TO <username>@localhost IDENTIFIED BY '<userpass>'"
mysql -u root --password=<rootpass> -e "GRANT ALL PRIVILEGES ON <dbname>.* TO <username>@localhost"',

Of course, if you plan to do everything by hand, it’s more convenient to log in to MySQL as root and do everything in its shell. But consider scripting this task, so that you don’t have to remember how to do it, ever again:

def check_mysql(dbname, username, userpass, rootpass):
    log.debug('Checking database %s...' % dbname)
    db_match = None
    try:
        db_match = subprocess.check_output('mysql --batch --skip-column-names '
                                           '-u %(username)s --password=%(userpass)s '
                                           '-e "SHOW DATABASES LIKE \'%(dbname)s\'"'
                                           % locals(), shell=True)
        log.debug('SHOW DATABASES said: %r' % db_match)
    except subprocess.CalledProcessError, e:
        log.debug('show databases returned %s' % e.returncode)
    if not db_match:
        commands = [
            'mysql -u root --password=%(rootpass)s -e "CREATE DATABASE %(dbname)s"',
            'mysql -u root --password=%(rootpass)s -e "GRANT USAGE ON *.* TO %(username)s@localhost IDENTIFIED BY \'%(userpass)s\'"',
            'mysql -u root --password=%(rootpass)s -e "GRANT ALL PRIVILEGES ON %(dbname)s.* TO %(username)s@localhost"',
        ]
        for cmd in commands:
            thecmd = cmd % locals()
            log.debug('Running command: %s' % thecmd)
            error = subprocess.call(thecmd, shell=True)
            if error:
                raise RuntimeError('Failed to set up database. Last command: %(thecmd)s. Error: %(error)s' % locals())

A word of caution: using “shell=True” with subprocess.call is not safe if your input comes from outside your script. I am using it in mine, because I am in total control of the input data, and if I go mad and start feeding my own script destructive input, I’ve clearly got bigger problems than server security.

Check Django Project Settings

Now that your database exists and is accessible to the Django app, let’s have a look at the app settings to make sure it is configured correctly.

A common practice for developing Django apps is to have a set of debug-only settings for development, and a set of production-only settings for deployed app. A convenient way to do that is to add something like this at the end of your settings.py file:

try:
    from localsettings import *
except:
    pass

Create a file localsettings.py in the same directory, but do not include it into source control and do not deploy it to the server. Instead, on the server, create a server-specific localsettings.py file. This way, you can use sqlite3 on dev machine, and a real database in production.

Your settings or localsettings now must contain correct DB connection info:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': 'your-db',
        'USER': 'your-db-user',
        'PASSWORD': 'your-db-pass',
        'HOST': '',
        'PORT': '',
    }
}
    

Since we are using a script to verify our database, why not also script this part? It will look something like this:

def check_djangosettings(settings_dir, dbname, username, userpass):
    settings_fname = p(j(settings_dir, 'settings.py'))
    settings = open(settings_fname)
    settings_text = settings.read()
    settings.close()
    if not ('from localsettings import *' in settings_text):
        # Need to fix settings module to import localsettings
        log.debug('Adding import localsettings to settings module: %s' % settings_fname)
        settings_text += ('\n'
                          'try:\n'
                          '    from localsettings import *\n'
                          'except:\n'
                          '    pass\n')
        f = open(settings_fname, 'w')
        f.write(settings_text)
        f.close()
    fname = p(j(settings_dir, 'localsettings.py'))
    if os.path.exists(fname):
        log.debug('Localsettings module already exists: %s' % fname)
        return
    log.debug('Creating localsettings module: %s' % fname)
    f = open(fname, 'w')
    f.write('''
DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.mysql',
        'NAME': '%(dbname)s',
        'USER': '%(username)s',
        'PASSWORD': '%(userpass)s',
        'HOST': '',
        'PORT': '',
    }
}
    ''' % locals())
    f.close()

In this function, we use the localsettings trick to create a server-specific config.

WSGI File

Starting with Django 1.4, “django-admin startproject” gives you a nice starting point for WSGI config file. However, it needs to augmented if we want it to be useful.

But first, mod_wsgi needs to know where to load your Python from, in case you are using virtualenv. Simply open /etc/apache2/mods-enabled/wsgi.conf and add a line that looks like this:

 WSGIPythonHome /path/to/my/virtualenv

Now, for the WSGI script. All you really need to do is add your project’s directory to sys.path:

import os, sys

CWD = os.path.abspath(os.path.normpath(os.path.dirname(__file__)))
PROJECT_DIR = os.path.dirname(CWD)

sys.path.append(PROJECT_DIR)

Of course, this should also be scripted:

def check_wsgi(project_dir, settings_dir):
    fname = p(j(settings_dir, 'wsgi.py'))
    if os.path.isfile(fname):
        return
    if project_dir == settings_dir:
        settings_module = 'settings'
    else:
        settings_module = '%s.settings' % os.path.basename(project_dir)
    site_packages = p(j(os.path.dirname(sys.executable),
                        '..',
                        'lib',
                        ('python%s.%s' % (sys.version_info.major, sys.version_info.minor)),
                        'site-packages'))
    f = open(fname, 'w')
    f.write('''
import os, sys
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "%(settings_module)s")

CWD = os.path.abspath(os.path.normpath(os.path.dirname(__file__)))
PROJECT_DIR = os.path.dirname(CWD)

sys.path.append('%(site_packages)s')
sys.path.append(PROJECT_DIR)

from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
    ''' % locals())
    f.close()

Database Schema

Time to update your DB structure! If it’s the first time, you’ll need to do:

python manage.py syncdb

And on every update to the schema, don’t forget to migrate (assuming you are using South, which you should):

python manage.py migrate --all

Let’s script it, too:

def check_db_schema(project_dir):
    '''SyncDB and migrations
    '''
    log.debug('Updating database schema from %s' % project_dir)
    cwd = os.getcwd()
    os.chdir(project_dir)
    for cmd in ('syncdb', 'migrate -all'):
        full_cmd = '%s manage.py %s' % (sys.executable, cmd)
        result = subprocess.call(full_cmd, shell=True)
        log.debug('%s returned: %s' % (full_cmd, result))
    os.chdir(cwd)

We’re getting close! So, at this point our database is in the correct shape, our web sever is good, and WSGI stuff is ready. Let’s start moving files to the right places.

Copying Application Files

On a server running multiple applications, a good practice is to have a directory somewhere outside /var/www to keep all of your Django applications. For each application we deploy, we’ll create a subdirectory, and put all its code, uploads, static data in there. (There are other sensible approaches – e.g. have a dedicated place for all media files and uploads for all apps, separate from sources, so YMMV).

Let’s name the subdirectories according to domain names of the applications – so, the app for my.domain.com will live under /path/to/apps/my_domain_com/. Copy the entire contents of your Django project, starting with the level where manage.py lives, into your my_domain_com directory and set ownership to www-data:www-data.

Python’s distutils has a handy function for “soft-copying” a directory tree, aptly named “copy_tree” – so,

from distutils.dir_util import copy_tree

copy_tree(source, destination)

To change ownership of a tree in Python, we do this:

pw_uid = pwd.getpwnam('www-data')
os.chown(dirname, pw_uid.pw_uid, pw_uid.pw_gid)

After you’ve set up the directory and copied the app code, you also want to create two more things within the deployed app’s filesytem: logs and (optionally) uploads. This way, you’ll always find the logs of your application easily, and you’ll have everything sitting in one neat place.

Now, all that’s left to do is to add a site configuration in Apache, and we’re done.

Apache2 Site Configuration

Create a file under /etc/apache2/sites-available/, named just like your app directory. The contents of the file should look a bit like this:

<VirtualHost *:80>
    ServerAdmin you@yoursite.com
    ServerName www.domain.com
    ServerAlias domain.com

    Alias /static/ /path/to/your/app/static/

    <Directory /path/to/your/app/static>
        Order deny,allow
        Allow from all
    </Directory>

    Alias /uploads/ /path/to/your/app/uploads/

    <Directory /path/to/your/app/uploads>
        Order deny,allow
        Allow from all
    </Directory>

    LogLevel warn
    ErrorLog  /path/to/your/app/logs/apache_error.log
    CustomLog /path/to/your/app/logs/apache_access.log combined

    WSGIDaemonProcess %(app_name)s user=www-data group=www-data threads=20 processes=2
    WSGIProcessGroup your-app-name

    WSGIScriptAlias / path-to-your-app-wsgi-script
</VirtualHost>

Of course, this step is also included in the full version of the script.

On to the final step!

Restart Apache

sudo apache2ctl restart

…and you are done!

The full script automating this process can be found here. It’s open-source, use and modify it to your delight!