++++++++++++++++++
part 1: Background
++++++++++++++++++
..
CGI_, the Common Gateway Interface, was the original framework-neutral
protocol that allowed dynamic web content to flourish. It remains the
reference standard. CGI defines some twenty `environment variables`_ it
passes to the program, specifying the URL requested and other information.
Any HTTP POST variables are provided on standard input. The program writes
a complete response document with HTTP headers on standard output. The
document is usually HTML but can be any type.
..
Quixote
-------
Our goal is to run the Quixote Altdemo under the widest variety of WSGI
servers currently available. So let's have a quick overview of Quixote.
Quixote_ (2.0) is a web application framework whose main distinguishing
feature is that one class generally covers an entire URL directory rather
than a single servlet node, and any nodes/subdirectories are
methods/attributes of this object. Other frameworks generally search
actual filesystem directories for the appropriate leaf node, so servlets
are merely interpreted files in otherwise static directories. In Quixote,
static files are represented as ``StaticFile`` and ``StaticDirectory``
instances attached to attributes in the the parent Directory class, and
their physical location can be anywhere in the filesystem. It's possible
to overlay a static directory using the special method ``._q_lookup()``,
which matches any attribute not otherwise found. A Directory is any object
with a ``._q_traverse()`` method/function that can be called like this::
output_string = d._q_traverse(list_of_remaining_url_components)
Normally it delegates to subdirectory objects quasi-recursively, but it can
do anything it likes including emulating the subdirectory.
The Publisher is a second object that wraps the root Directory instance;
it's called by adapters as::
response = p.process_request(request)
The adapter must create a Quixote Request object and accept a Quixote
Response. In SCGI deployments, the adapter is a stub that converts the
SCGI input into a Request object, and converts the Response to a string.
The Publisher handles Session objects, exception trapping, etc.
Quixote has several support services we won't cover here: PTL, htmltext, an
elegant form class, and HTML utility functions. These all can be used
individually without the rest of Quixote, and won't be covered here.
..
The Application
---------------
Let's see how Quixote natively runs the Altdemo since this is our behavioral
standard for the other environments. Download Quixote from
http://www.mems-exchange.org/software/quixote/ . (This article is based on
Quixote 2.0.) Unpack, install, and start the Altdemo using the built-in HTTP
server::
$ mkdir /tmp/test
$ export PYTHONPATH=/tmp/test:$PYTHONPATH
$ tar xzvf Quixote-2.0.tar.gz
$ cd Quixote-2.0
$ python setup.py install --install-lib=/tmp/test
$ python /tmp/test/quixote/server/simple_server.py
--factory=quixote.demo.altdemo.create_publisher
The last command is all one line. ``--host=localhost
--port=8080`` are the defaults; override them if necessary.
Point your browser to **http://localhost:8080/** and play with the demo. This
is the experience we want to duplicate under WSGI. Press ctrl-C when you're
bored. Look at the source ($QUIXOTE/demo/altdemo.py) if you're curious. If
you get an AssertionError on ``assert path[:1] == '/'``, verify you remembered
the trailing slash.
Some tests may require a SCGI client, so put this in your cgi-bin directory::
#!/usr/bin/env python
"""cgi2scgi.py -- Send a request to an SCGI server.
"""
import os, socket, sys
HOST = 'localhost'
PORT = 3000
def scgi_request(content, headers_dict):
dic = {'SCRIPT_NAME': '', 'PATH_INFO': '/'}
dic.update(headers_dict)
headers = [
('CONTENT_LENGTH', len(content)),
('SCGI', 1)]
headers += dic.items()
headers = ["%s\0%s\0" % pair for pair in headers]
headers = ''.join(headers)
headers_len = len(headers)
return "%s:%s,%s" % (headers_len, headers, content)
def recieve_all(sock):
sio = StringIO()
while 1:
data = sock.recv(1024)
if not data:
break
sio.write(data)
return sio.getvalue()
def send_scgi(content, headers_dict):
data = scgi_request(content, headers_dict)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.send(data)
return recieve_all(sock)
print send_scgi(sys.stdin.read(), os.environ),
Next we'll have to fix a little bug in the Altdemo. Edit
$QUIXOTE/demo/altdemo.py and change ".." to "./" in lines 92
and 107. Otherwise the "go back" and "start over" hyperlinks will go too far.
Start the SCGI server::
$ python /tmp/test/quixote/server/scgi_server.py
--factory=quixote.demo.altdemo.create_publisher
(The default rendezvous point is ``--host=localhost --port=3000``.)
Go to "http://localhost/cgi-bin/cgi2scgi.py/" in your browser and try it out.
---
Paste partly depends on WSGI Utils, so let's look at that first. Download it
from (current version 0.5). Install
and run its demos::
$ tar xzvf 'WSGI Utils-0.5'
$ cd 'WSGI Utils-0.5'
$ python setup.py install --install-lib=/tmp/test
$ python examples/basicExample.py
Feel free to curse the developer for putting spaces in the filenames. This
example serves a visit counter using sessions on http://localhost:1088/ . (The
number will go up by 2 because the browser is fetching "/favicon.ico".)
$ python examples/testApp.py
This serves a static directory at http://localhost:1066/, a time server at
http://localhost:1066/test.py (press shift-Reload a few times), and the world's
first password-protected calculator at http://localhost:1066/calc.py (username
'user', password 'user').
--
QWIP and SWAP
-------------
Qwip and Swap are two libraries by Titus Brown. QWIP is a WSGI wrapper for
Quixote applications; SWAP is a SCGI-WSGI gateway. In case you missed it, QWIP
is the WSGI-to-Quixote module we've been looking for. But the current version
is for Quixote 1.x. While we wait for Titus to update it for 2.0, we'll make a
few tweaks so it's at least usable for us. Download the tarball from
http://www.idyll.org/~t/www-tools/wsgi/::
$ tar xzvf qwip-and-swap-30.11.04.tar.gz
$ cd qwip-and-swap-30.11.04
Edit **q_hello.py** and change it to::
from quixote.directory import Directory
from quixote.publish import Publisher
class MyDir(Directory):
_q_exports = ['']
def _q_index(self):
return "Hello, world! (via q_hello)"
--
Paste has a top-level executable [not named yet; it will merge the legacy
``paste-server`` and ``paste-setup``]. It has two modes: server and setup.
In server mode it reads a Python-syntax configuration file and launches the
specified HTTP server/gateway, application/framework, and middleware. either launches a server/gateway or
creates a stub directory for a new application.n server mode it launches a
server/gateway with your application. In setup mode it creates a stub directory
for a new application. A Python-syntax configuration file is read into a
dictionary and available to all parts of the server runtime. This tells
in a Subversion repository, so you'll have
to have the ``svn`` client installed. ``build-pkg`` is a shell script that
downloads and installs several Python packages Paste depends on::
$ svn co http://svn.w4py.org/Paste/trunk Paste
$ cd Paste
$ ./build-pkg
$ scripts/paste-setup create --template=webkit_zpt /tmp/paste_app
$ cd /tmp/paste_app
$ .../Paste/scripts/paste-server -v
Go to http://localhost:8080/ in your browser, and you'll see a dump of the WSGI
dictionary for the current request. There's a screenshot here__ in section
"Running the Application".
.. __: http://pythonpaste.org/docs/TodoTutorial.html
Don't install Paste via setup.py, per the author's recommendation. Also don't
call an application 'paste' because it seems to confuse Paste; I kept getting
an ImportError.
It looks like before anything else, Quixote needs a WSGI-to-application server,
a ``quixote.servers.wsgi_server``. I guess Titus Brown's QWIP provides this,
when I get to it. Paste can't help here since the problem is Quixote-specific,
but once this component is found Paste would provide some benefits.
The most interesting benefit is Paste's unified configuration module. Quixote
users already have to connect (1) a Directory into (2) the Publisher into (3) a
server (e.g., scgi_server.py). Currently users do this with the server's
``--factory`` argument as in the Altdemo above or in a custom top-level script.
WSGI adds a fourth component: the webserver-to-WSGI module. Since it has to be
Python and run in the same process, perhaps a ``--wsgi-server`` argument
could specify it.
But wait, there's more! If you use middleware you have to connect a chain of
middleware components to your application. Do we also need a ``--middleware``
argument for each component, or a ``--middleware-factory`` for all components?
Or make the ``--wsgi-server`` argument do double duty? Or do we look elsewhere
for a better configuration system?
Paste has a similar top-level server called ``wsgi-server``. Andrew Kuchling
has written a summary__ of it.
.. __: http://svn.w4py.org/Paste/trunk/docs/servers.txt
Several webserver interfaces have already been written, including our favorites
SCGI and CGI, Twisted, the standalone threaded HTTP server in WSGI Utils, and
an esoteric 'console' interface which "simulates a web application from the
console" for debugging.
Paste's structure and its use of middleware and webkit doesn't seem to be
documented or commented, but let's trace the code. wsgi-server calls
``paste.server.run_commandline()``. This parses the command-line options and
reads the config file, then chooses a webserver interface from the ``servers``
dictionary and runs ``server(conf, app)``. ``app`` was set by
``make_app(conf)``, which instantiates a ``paste.webkit.wsgiwebkit`` object
from a directory (akin to a Quixote Publisher+Directory). So webkit seems to
be hardwired into Paste at the moment; the 'webkit_dir' option is even
required! I assume a Quixote plugin would go around here.
I'm not sure I like ``paste.server``. In Quixote, the user chooses a webserver
module in ``quixote.servers`` (or their own third-party module) and wraps their
application in that. Each webserver module is a top-level program, or you can
have a custom script invoke it. In Paste, a single super-server chooses
the webserver based on the configuration, and this module has functions for all
supported webserver types. This means to add a new webserver type, you have to
add a function in one of Paste's modules, you can't just bring your own and
start it up. This seems to be a limitation. I also don't like the way the
servers and middleware are all dumped into the same package as the server with
ugly names like ``cgiserver.py``. That's a minor detail, but it will get worse
as Paste is expanded.
Paste has a surprising way of handling middleware. A middleware component is a
wrapper function, in other words a decorator. A middleware chain thus is a
stack of nested functions, with the application being the innermost wrapped
one. In other words, the application is middleware!
The webkit servlet directory is specified in the config file as::
publish_dir = "/tmp/paste_app/web"
``paste.server.make_app(conf)`` calls ``paste.wsgikit.webkit()`` on this simple
directory string. This sets up a fake Webware environment so global imports
will redirect back into Paste. We'll have to do the same thing so applications
can do::
import quixote
req = quixote.get_request()
as usual without invoking the "real" Quixote Publisher that is non-WSGI-aware. ``webkit()`` then activates four middlewares: a URL parser, a session
handler, an exception handler, and a "recursive" middleware whatever that
means. It optionally wraps each middleware level in ``paste.lint``, a
middleware that verifies its immediate child complies with the WSGI protocol.
(This is starting to sound like Marklar: "It optionally wraps each marklar in
marklar, a marklar that verifies its immediate marklar complies with the
marklar.")
For Quixote and other preconfigured publisher objects, use ``publish_app``
instead of ``publish_dir``. Since we don't have a WSGI-to-Quixote module yet,
we'll use a generic WSGI publisher instead. Create a new directory
**/tmp/test_app2** containing the following file **app.py**::
def app(environ, start_response):
start_response("200 OK", [])
s = "<html>You requested <strong>%s</strong></html>"
s %= % environ['PATH_INFO']
return [s]
and the following file **server.conf**:
from app import app as publish_app
verbose = True
server = 'wsgiutils'
debug = True
Chdir into that directory and run **.../Paste/scripts/paste-server**. In your
browser, go to any URL starting with **http://localhost:8080/**.
Paste currently has no configuration option to build a middleware chain, but
you can simply wrap your application in middleware before attaching it to
``publish_app``.
Ian has written a few blog entries which further explain Paste (aka WSGIKit):
- http://blog.ianbicking.org/what-is-wsgikit.html
- http://blog.ianbicking.org/what-can-wsgikit-do-for-you.html
Some older blog entries (2004) that discuss other WSGI modules and
implementation strategies:
- http://blog.ianbicking.org/wsgi-implementations.html
- http://blog.ianbicking.org/a-wsgi-stack.html