How to setup Apache, mod_python and a reverse proxy to Lighttpd for Django on Ubuntu

I'll Django for food

Update: I don't recommend this setup anymore. Use the improved setup of Django with nginx, Apache and mod_wsgi.

It's October 2008 and there's no doubt now that serious web development requires working with frameworks, software that allows us, lazy coders, to forget about repetitive tasks and focus on the specifics of each project, the stuff that makes them truly unique.

It's pretty obvious too that Django, the Python based web framework, is attracting more developers and companies lately. Guido is a googler since 2005 and the recently launched Google App Engine uses Python and Django. Seriously, there's not a better time to jump into the Django train (no pun intended RoR guys).

Experienced programmers can start writing Django applications quickly thanks to the excellent documentation and the free Django book. I've also read and can recommend Practical Django Projects, by James Bennett, part of the Django team, and Learning Website Development with Django, by Ayman Hourieh, a very smart and young Google engineer.

Django is really a web framework for perfectionists with deadlines. I'm very near to one and have almost finished coding what will be my first Django based application for Facebook. Yes, I've got a few articles about that coming soon as well.

So, everything looks great under the Sun on Djangoland? Well, there's something that bothered me since I started a few months ago: deployment on a production environment. The documentation and most books get us up and running quickly with the included development server and then just refer us to the recommended Apache and mod_python settings for more.

Unfortunately, specially if you're used to the common Linux, Apache, MySQL and PHP setup, like I was, you may be need more detailed instructions and that's why I decided to write this tutorial on how to setup Apache, mod_python and a reverse proxy to Lighttpd for Django.

Ladies and gentlemen, welcome to the show

Let's first introduce the actors in our little play:

  • Apache, your old well known web server, not much to add about it.
  • mod_python, an Apache module to integrate Python into the web server, way better than using standard CGI.
  • Lighttpd, Lighty for its buddies, a speedy web server suitable for serving static files.
  • mod_proxy, an Apache module that can act as a gateway between Apache and Lighty, our two web servers.
  • And of course Python, Django and any other related modules.

I'll not discuss about database or DNS servers setup here but obviously you have to take care of that as well.

The plan

All bank robbers and web developers know how important a plan is. This is ours.

Apache and mod_python will take care of processing all the dynamic parts, passing all requests to your Django project. We'll do this from a url such as http://example.com/.

Now, since Apache requires more memory for each process and we want to optimize our use of server resources we'll use Lighty, which has a smaller footprint, to serve the static content. That includes media such as images and Flash movies as well as Javascript and CSS files.

And now the nice part: I can setup Lighty on a different domain, IP address or even TCP/IP port, together with Apache in the same box or in another dedicated server, and thanks to mod_proxy make the content appear like living under http://example.com/media/

Ideally Lighty content will never be accessed directly but just thru Apache via mod_proxy.

After everything is correctly configurated we will be able to visit, let's say, http://example.com/blog/, and get a list of entries from our database, courtesy of Apache, mod_python, a Django view function and some Django templates. That same page can also show some images, like /media/pictures/pic.jpg or /media/logo.png, and add styling via /media/css/style.css, these files will be under http://example.com/media but will be served by Lighty.

Sounds like a good plan, uh? Put your US president mask on and let's start.

We're in, let's get that vault open

I ran the commands in this tutorial on my Ubuntu 8.04 server and they should work on any other Debian based distros. Adapting to other flavors of Linux shouldn't be hard. You could use any text editor to change configuration files but I'd like to remind you that extremely handsome male coders and beautiful geek girls use vim.

Let's start by installing Apache and mod_python:

sudo apt-get install apache2
sudo apt-get install libapache2-mod-python

I've found that Ubuntu sometimes doesn't set a server name for Apache so I like to add one. Edit /etc/apache2/apache2.conf and add this line:

ServerName "your-server-name"

it should usually go near this one:

ServerRoot "/etc/apache2"

We won't really use that name, it's just to avoid a warning message when starting Apache. Bank robbers don't like being warned, remember?

While you're in /etc/apache2/apache2.conf make sure you have this line:

ServerSignature On

This will help us later to see if mod_python is loading correctly and can be set to Off after that.

Now let's define our IP addresses. If I open /etc/network/interfaces. I have something like this:

auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 192.168.0.192
netmask 255.255.255.0
gateway 192.168.0.1
auto eth0:1
iface eth0:1 inet static
address 192.168.0.193
netmask 255.255.255.0
gateway 192.168.0.1
auto eth0:2
iface eth0:2 inet static
address 192.168.0.180
netmask 255.255.255.0
gateway 192.168.0.1
auto eth0:3
iface eth0:3 inet static
address 192.168.0.181
netmask 255.255.255.0
gateway 192.168.0.1

I have one network adapter, eth0, with multiple IP addresses assigned, all of them are private for these examples but you should have a different setup and in a production server all IP's would be public. I can choose any combination of addresses and ports for setting up my two web servers, or just one IP with two different ports. These are my choices:

Apache on 192.168.0.180:80
Lighty on 192.168.0.181:8000

To resolve our example.com domain to the Apache IP address edit /etc/hosts and add this line:

192.168.0.180 example.com

This is just for a development server, in a production environment you need to setup your DNS server, probably BIND or some external provider.

And now edit /etc/apache2/ports.conf accordingly:

Listen 192.168.0.192:80
Listen 192.168.0.180:80
<IfModule mod_ssl.c>
Listen 443
</IfModule>

Notice I have 192.168.0.192 in the first line, not one of my choices, that's just to show that Apache could serve other sites as well, PHP sites for example, using a regular virtual hosts setup. There's no conflict with our Django settings. We could even serve multiple Django and PHP sites from the same IP.

Now let's enable the modules for mod_proxy in Apache:

sudo a2enmod proxy_connect
sudo a2enmod proxy_http

this will automatically enable a third pre-requisite module: proxy.

Let's configure the example.com site in Apache. As root create a file called /etc/apache2/sites-available/example.com that contains these lines:

<VirtualHost 192.168.0.180>
ServerName example.com
ServerAdmin alexis@example.com
DocumentRoot /home/alexis/example/
AddHandler mod_python .py
PythonHandler mod_python.publisher
PythonDebug On
</VirtualHost>

The file could be named anything but I like to use the domain name. The important thing is to store it in /etc/apache2/sites-available/. I'm keeping it simple for this tutorial but you can add other options, like logging, later.

Make sure you've created the document root directory and that is readable by the user that runs the Apache process. In my case the directory is /home/alexis/example. Notice this directory is empty.

Now enable the site:

sudo a2ensite example.com

That's all you need in Apache for Django. If you'll be serving PHP or SSL enabled sites you may need to install a few additional packages and enable other modules:

sudo a2enmod ssl
sudo a2enmod rewrite
sudo a2enmod suexec
sudo a2enmod include

but that's outside the scope of this tutorial.

To enable the changes you need to restart Apache:

sudo /etc/init.d/apache2 restart

If you launch your browser and go to http://example.com/anything you should see a 404 Not Found page and a message like the following on the footer:

Apache/2.2.8 (Ubuntu) mod_python/3.3.1 Python/2.5.2 PHP/5.2.4-2ubuntu5.3 with Suhosin-Patch mod_ssl/2.2.8 OpenSSL/0.9.8g Server at example.com Port 80

If you see the mod_python and Python versions then you're on track. Now one more test, create a file called test.py in your document directory, in my case /home/alexis/example, with this content:

def index(req):
return "Test successful, mod_python working";

If you visit http://example.com/test.py you should now see Test successful, mod_python working.

You're golden!

Because you may need multiple restarts, or simply reloads in some cases, it's a good idea to setup a few aliases in ~/.bashrc:

alias apreload='sudo /etc/init.d/apache2 reload'
alias aprestart='sudo /etc/init.d/apache2 restart'
alias lrestart='sudo /etc/init.d/lighttpd restart'

The two first lines are for Apache, the last one for Lighty, which we'll install next. Now the next time you start a shell session you can run this to restart Apache:

aprestart

Let's install Lighttpd:

sudo apt-get install lighttpd

The configuration is in /etc/lighttpd/lighttpd.conf. Let's first set the default IP and port for the main server:

server.bind = "192.168.0.181"
server.port = 80

I could use just that for serving my files but I prefer to setup virtual hosts. To do so remove the comment, a #, from the mod_evhost line in the server.modules block and then add something like this to the bottom part of the configuration file:

$SERVER["socket"] == "192.168.0.181:8000" {
server.document-root = "/home/alexis/example/media"
}

Of course this is simplified version of a virtual host setup. You could add log files and other options as well.

Create the directory /home/alexis/example/media, it should be readable by the Lighty user, and put some files there. Now restart Lighty:

lrestart

Now point your browser to http://192.168.0.181:8000 and you'll see the files you put in /home/alexis/example/media.

What we've got so far?

Ok, now we have a site with Apache and mod_python, http://example.com running from 192.168.0.180. This is the site that will be seen by all your visitors and will serve Django content.

We also have a Lighty site to serve static files at http://192.168.0.181:8000, although we'd never need to access it directly.

Got it? Ok, now get your wizard suit and let's continue with our plan.

Black magic mod_python

Time to turn the Lighty content served from http://192.168.0.181:8000 into http://example.com/media/. Edit /etc/apache2/sites-available/example.com and make it look like this:

<VirtualHost 192.168.0.180>
ServerName example.com
ServerAdmin alexis@example.com
DocumentRoot /home/alexis/example/
AddHandler mod_python .py
PythonHandler mod_python.publisher
PythonDebug On

ProxyRequests Off
ProxyPreserveHost On
ProxyPass /media/ http://192.168.0.181:8000/
ProxyPassReverse /media/ http://192.168.0.181:8000/
ProxyPass /admin-media/ http://192.168.0.181:8000/admin-media/
ProxyPassReverse /admin-media/ http://192.168.0.181:8000/admin-media

</VirtualHost>

The new lines are in bold text.

ProxyRequests Off is important to define this as a reverse proxy.

This one:

ProxyPass /media/ http://192.168.0.181:8000/

tells Apache: "if the browser requests content under /media/ get it from the server at http://192.168.0.181:8000", which happens to be our Lighty server.

and this:

ProxyPassReverse /media/ http://192.168.0.181:8000/

is needed to set the correct HTTP headers. Don't ask me what this means, I'm not quite sure.

The same logic is applied for the /admin-media directory, which we'll use for serving the Django admin media files, more about this later.

Now, to allow your visitors to use the proxy edit /etc/apache2/mods-available/proxy.conf as root and update the <Proxy *> block to look like this:

<Proxy *>
AddDefaultCharset off
Order deny,allow
Deny from all
Allow from all
</Proxy>

Notice the last line in the block must be Allow from all or you'll get Forbidden errors. If I understood mod_proxy documentation correctly that is not a security issue because we are using a reverse proxy setup. I think you could also get rid of the Deny from all because it's overriden on the next line but anyway, this works.

Now you can use your browser to load any file from http://example.com/media as if you were calling http://192.168.0.181:8000. Remember, you should see the files you put on /home/alexis/example/media.

And finally

Let's complete our virtual host settings in Apache to load our Django project. Make your /etc/apache2/sites-available/example.com looks like this:

<VirtualHost 192.168.0.180>
ServerName example.com
ServerAdmin alexis@example.com

<Location "/">
SetHandler python-program
PythonHandler django.core.handlers.modpython
SetEnv DJANGO_SETTINGS_MODULE project.settings
PythonPath "['/home/alexis/python-work', '/home/alexis/python-work/project'] + sys.path"
</Location>

ProxyRequests Off
ProxyPreserveHost On
ProxyPass /media/ http://192.168.0.181:8000/
ProxyPassReverse /media/ http://192.168.0.181:8000/
ProxyPass /admin-media/ http://192.168.0.181:8000/admin-media/
ProxyPassReverse /admin-media/ http://192.168.0.181:8000/admin-media
</VirtualHost>

Notice I've removed the line for the document root, we don't need it because all requests to this site will be passed to mod_python and Django. I also got rid of the lines we first used to test mod_python.

I've already written about PYTHONPATH for serving Django with Apache so go take a look it you need to refresh your memory on that topic.

Now reload Apache and you Django site should be working at http://example.com/, all dynamic data coming via Apache and mod_python and static files from Lighty via mod_proxy. Isn't life good with you?

Help!, my Django admin is naked

If you're using the Django administration application in your project you'll find it at http://example.com/admin. Go there now. Aren't you seeing your favorite pastels? Then you're missing the media files.

Remember we added /admin-media/ to our mod_proxy settings? That's to access static files for the Django administration site. You'll need a couple more steps to complete the process.

First go to the directory you created for serving media with Lighty, /home/alexis/example/media for me, and create a symbolic link to where the admin media files are stored:

ln -s /usr/lib/python2.5/site-packages/django/contrib/admin/media/ admin-media

Notice You don't need to sudo or be root for this command as the link is created in your personal directory.

Also, the location of the site-packages/django directory could be different on your system, to find where's yours run the classical one-liner on the Linux shell:

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

And now update the settings.py of your Django project with this line:

ADMIN_MEDIA_PREFIX = '/admin-media/'

Ready? Now go to http://example.com/admin and enjoy your favorite shades of gray and blue, and yeah, administer your site too.

The end

Maybe I just suck at setting up web servers, proxies and deploying Django applications and you already knew everything I've described here. If that's the case then you may want to add some suggestions or shortcuts to this guide, they are all welcome.

But if you are like me and were a little lost getting your Django project running on the wild I hope you got it all clear now. And of course, I'd love to hear from you.

And now the Murrow line: good night, and good luck!

Join the conversation

Thank you . Really nice

Thank you . Really nice tutorial and well written . Used it to set Apache + Lighttpd on Centos .

thanks

very very useful and well done, thanks a lot

I am recently move to Django

I am recently move to Django but still working with previous PHP stuff. I have a development server set by a vmware machine (ubuntu8) on my vista system. My question is how to configure so that I can use both Django and PHP work on the same web server (for development). I don't know how the tutorial can adapt to that case.

thanks in advance!
Roger

Nice tutorial. And how have

Nice tutorial.

And how have you solved the url rewrite problem? The urls in the html pages are invalid.

I'm not using this setup

I'm not using this setup anymore, I now work with Nginx and mod_wsgi. Take a look at that tutorial.

Keep your comments relevant, written in good English and don't spam. Let's create useful and valuable discussions. Markdown is welcome.

Add your comment