« One of the ways CherryPy 3 will rockSingle Table Inheritance certainly *sounds* evil »

CherryPy 3 directions


Permalink 08:15:15 pm, by fumanchu Email , 1363 words   English (US)
Categories: CherryPy

CherryPy 3 directions

I committed the first round of changes for CherryPy version 3 on Friday. It's nowhere near complete, but it hopefully can give hints about the future.

Before I dive into the meat, you should know I moved some things around:

  • _cphttptools is now called _cprequest
  • There's a new 'tools.py' module (see below).
  • All of the code in the /filters folder still exists, but it's all been moved into the /lib folder. The filters folder has been removed.
  • You can now call functions and instantiate objects in config files. For example: now = cherrypy.lib.httptools.HTTPDate()


In CherryPy 2.2, you're able to replace the page-handler-dispatch mechanism by using a custom Request class; that is, you would subclass _cphttptools.Request and override the main or mapPathToObject methods. That can be tedious, since you can't specify the Request class on a per-request basis; the Request object has already been formed by the time the URL has been parsed.

In CP 3, there's a new _cprequest.dispatch function, and each Request object calls it. If you don't like the way CP looks up page handlers by default, you can declare your own dispatcher in the config:

dispatcher = my.custom.dispatcher.function


dispatcher = my.custom.DispatchClass(blah)

The only requirement is that the right-hand-side be a callable: it takes a "path" argument and should return a page handler (a callable). The default Dispatcher also sets request.virtual_path, so unless you're also setting request.execute_main to False you should probably do the same.

Filtering is now Hooking

I had a good long look at filters in CP 2.2. Despite their name, they don't really "filter" anything; nothing "passes through them". Some of them modify cherrypy.request attributes, but just as many of them don't. They're not implemented as filters; instead, they're "hooks".

A "hook" usually means a place where callbacks are called, and CherryPy filters have always been called from a pre-determined set of hooks (e.g. before_request_body). So I went ahead and changed the terminology throughout the codebase.

But there's a much bigger change than just the name. People have been pining for CP to release its grip over both filter declaration (which filters are available) and filter invocation (CP 2 calls all filter methods whether enabled or not). These issues have largely been solved in the current trunk by moving control out of the global cherrypy.filters module and into each Request object. Every Request object now possesses a "hooks" attribute, a _cprequest.HookMap object. The HookMap class has the following attributes and methods:

  • callbacks: a dict of the form {hookpoint: [callback, ...]}. The "hookpoint" is one of our old filter method names, like "before_finalize".
  • failsafe: a list of hooknames that should run all their callbacks, even if some of those callbacks raise exceptions.
  • attach(point, callback, conf=None): allows you to attach a callback to be invoked by this request. Any code can do this, and can do it on the fly! See the new caching module for an example; if the request is served from cache before_main, then the logic which would cache the page handler output is never attached, and therefore never invoked.
  • run(point): runs all registered callbacks for the given hook point.
  • populate_from_config(): this is called automatically by the Request object, and searches for Tools which it can call to setup hooks. What's a Tool? Read on...


CherryPy has always included a number of extensions and libraries which help you design web applications more quickly. In addition, many people have designed their own extensions to CP, some as custom filters, some as decorators, some as base classes to be subclassed, some as WSGI middleware, custom Request objects, on*Start methods, etc., etc., etc.

I'd like to call all of these extensions "features" for the rest of this post. A "feature" in this sense is any function(s) or module which could be implemented in a variety of ways. If the feature should apply site-wide, you probably want to run it like a CP2-style filter, and perhaps declare its scope in the config dict/file. But if it only applies to a page handler or two, you might think a decorator would be more attractive syntax. Sometimes, you want to invoke the feature from inside the page handler, after you've inspected a certain header, or after a lookup has failed.

However, it often happens that implementing your feature to be used in one of these ways harms its use in another: if you make a lovely decorator out of your feature, chances are that you cannot just "plug it in" as a before_main handler and expect it to work. This was a big problem for CP-2; a lot of logic could be useful elsewhere, but wasn't available because it was "locked away" inside a filter or some other construct.

A Tool is my new term for "feature adapter". If you can write your feature as a normal Python function, with normal Python arguments instead of config.get calls, chances are it can be wrapped in a Tool in a single line of code:

cherrypy.tools.cool_stuff = cherrypy.tools.Tool('before_finalize', cool.stuff)

What does that line buy you?

  • Your function is registered in the CherryPy tool registry, so
  • Your function can be called from the tools namespace: tools.cool_stuff(*args, **kwargs).
  • Your function can be used as a decorator via @tools.cool_stuff.wrap(*args, **kwargs). Any arguments passed to wrap() get passed to your function whenever it is called.
  • Your function can be used as a hook and managed in config. Remember the populate function (above)? It scans through the current config, finds any items that start with "tools.", and checks to see that "tools.cool_stuff.on" is True. If it is, it takes all other "tools.cool_stuff.*" config entries and passes them as named arguments to your cool_stuff function, at the hook point you requested.

That is the "simple case", and there is sufficient room for very complex additions to that (grep for the setup method). If your feature needs to replace the page handler, for example (as caching, static, and xmlrpc do), there's a tools.MainTool class; when used as a decorator or a hook, it automatically skips the page handler for you if your function returns True (meaning "I've handled this request, thanks").

I plan to explore other Tool improvements in the near future:

  • Argument inspection is high on the list, so that decorators, etc get the same argspec as your original function. You might also be able to import tools and let your IDE auto-complete your config entries, which in my mind would cut down on reaching for manuals quite a bit. It would have to be optional, because IIRC Jython doesn't have an "inspect" module.
  • Other wrappers on the Tool class for...what? Base classes? WSGI middleware? custom Request objects? on*Start/Stop methods?
  • Look harder at the flags request.processRequestBody and request.execute_main. They're ugly. Devious thought: replace request.processRequestBody and request.main with default hooks.
  • "Tools" may not be the best name.
  • Other hook points are possible. Investigate using hooks in a more generic fashion.
  • Using a tool as a decorator effectively means that it is not overridable in config. This "feature lock" is something I've wanted for quite a while, but there may need to be some means of allowing config to override such features or their arguments. For example, a developer may want to insist that a "staticfilter" be in place, but not particularly care about the OS path to its resources.

There are other issues that need to be addressed in CherryPy 3, of course (separating the CP server and the HTTP server springs instantly to mind). But these changes should give a us a good basis for consolidation of a lot of code, and the freedom to use all our beautiful library logic in whatever way is most appropriate to each application and installation. I look forward to all your ideas and improvements.

1 comment

Comment from: Kevin Dangoor [Visitor] · http://www.blueskyonmars.com

This looks great, Bob! Quite a bit more flexible and, from what I understand of the 2.2 filter implementation, likely to perform better as well.

I haven't looked at this new code yet, but one good thing to have available (and I'm guessing it is) is CherryPy's dispatch mechanism. I've seen people talk of cases where they have a default method that looks up and object and then wants to continue CP-style traversal from there.

My other desire for the next major release of TG is to be able to attach an application (CP/TG or WSGI) at a point in the object hierarchy. Christian made a filter to do this, I believe, but I figured I'd mention it again in case there's special treatment that would make it work better.

04/25/06 @ 07:41

Leave a comment

Your email address will not be revealed on this site.

Your URL will be displayed.

Please enter the phrase "I am a real human." in the textbox above.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)
September 2020
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
    1 2 3 4 5
6 7 8 9 10 11 12
13 14 15 16 17 18 19
20 21 22 23 24 25 26
27 28 29 30      


The requested Blog doesn't exist any more!

XML Feeds

free blog tool