Archives for: October 2005

10/27/05

Permalink 01:08:47 am, by fumanchu Email , 1452 words   English (US)
Categories: CherryPy

URL-rewriting in CherryPy 2.1

There are a lot of reasons, and places, why a developer would want an original Request-URI to be treated as if it were another. CherryPy 2.1.0 has a (possibly bewildering) array of attributes, core code, and filters which either enable rewriting or are affected by it. Here's how I see the state of the art (this is not gospel--much is my opinion regarding design intent).

Features

First, some features which depend on rewriting:

  1. Generating URL's to spit back out in HTML.
  2. HTTP Redirects and their targets (new URL).
  3. Handler dispatch: mapping URI's to handler methods. Includes...
  4. Arbitrary mount points: allowing a deployer to mount an application at an arbitrary base URI.
  5. Config lookups, since the config map is keyed by URI (path only--no queryString or fragment).
  6. Logging: do you log the original URI or the rewritten one, or both? in different logs? in all messages?

Request Attributes

Now, cherrypy.request has the following attributes (grabbed straight from the book):

  • requestLine: This attribute is a string containing the first line of the raw HTTP request; for example, "GET /path/page HTTP/1.1".
  • method: This attribute is a string containing the HTTP request method, such as GET or POST.
  • path: This attribute is a string containing the path of the resource the client requested.
  • queryString: This attribute is a string containing the query string of the request (the part of the URL following '?').
  • protocol: This attribute is a string containing the HTTP protocol of the request in the form of HTTP/x.x

Let's take an example HTTP requestLine and see if we can't parse it out:

     DELETE /path/to/handler/?param=somevalue HTTP/1.1
     \____/ \_______________/ \_____________/ \______/
     method       path          queryString   protocol

Pretty straightforward; no overlaps. Note that if the Request-URI includes a scheme and host, that'll be stripped when path is formed.


There are a couple of other URI-related request attributes:

  • base: This attribute is a string containing the root URL of the server. By default, it is equal to scheme://headerMap['Host'].
  • browserUrl: This attribute is a string containing the URL the client requested. By default, it is equal to base + path, plus the queryString, if provided.

Since the requestLine doesn't always include the scheme or host (it may, rarely), these are obtained from other sources and joined into base. The browserUrl joins the base, the path, and the queryString to form a complete, absolute URI (what was hopefully in the Address bar of the end-user's web browser, if that's applicable).


Finally, we have these copies/substitutes for the functionality provided by path:

  • objectPath: This attribute is a string containing the path of the exposed method that will be called to handle this request. This is usually the same as cherrypy.request.path, but can be changed in a filter to change which method is actually called.
  • originalPath: This attribute is a string containing the original value of cherrypy.request.path, in case it is modified by a filter during the request.

The objectPath may be used to control dispatching, but there's nothing in the core that uses it that way. Since it's almost always None, dispatching usually falls back to the value of path. Once the handler dispatch is completed, then objectPath contains the route to the found handler, expressed as a path; in the above example, it might be "/path/to/handler/index" if an "index" function handles the request.

The originalPath is also an odd attribute. You would think that CherryPy core features, especially those which use or implement URI rewriting, would make use of this value. But none of them do. It gets set but never used.

How to rewrite in 2.1

Rewriting "base"

This is what the builtin baseUrlFilter does, so that an instance of CherryPy running behind Apache with mod_proxy or mod_rewrite can spit back out proper URI's in HTML, redirects, etc. As far as I can tell, this works well and has no issues with the rest of CherryPy. The only other value which overlaps with the value of base is browserUrl, which the filter also rewrites.

Rewriting "path"

Another way to rewrite is to use a filter that changes the value of path for you as early as possible. For example, I use a VirtualPathFilter which does this:

class VirtualPathFilter(object):
    """Filter that changes cherrypy.request.path, stripping a set prefix."""

    def onStartResource(self):
        if cherrypy.config.get('virtualPathFilter.on', False):
            prefix = cherrypy.config.get('virtualPathFilter.prefix', '')
            if prefix:
                path = cherrypy.request.path
                if path == prefix:
                    path = '/'
                elif path.startswith(prefix):
                    path = path[len(prefix):]
                cherrypy.request.path = path

This allows me to provide feature #4, arbitrary mount points. I write my application as if it were always mounted at /, but the deployer can then provide a virtualPathFilter.prefix to turn the URL /prefix/page?id=3 into /page?id=3.

Unfortunately, if the other pieces of CherryPy aren't written to support arbitrary mount points, then this scheme falls apart. And they aren't so written. I've just broken many of our other features:

  1. Generating URL's to spit back out in HTML. Broken. I now have to manually provide prefix to my HTML templates, or take on the nightmare of making every generated URL into a URL which is relative (e.g. "../../otherpage") to the current one.
  2. HTTP Redirects and their targets. Broken. I now have to manually provide prefix to each instance (or use relative URL's). But I can't control CherryPy's redirect instances! For example, when CherryPy tries to redirect index methods by adding a trailing slash to the requested URI, it uses the value of path, which I've rewritten.
  3. Handler dispatch: not broken.
  4. Arbitrary mount points: not broken.
  5. Config lookups. Broken? Some other filter which does a config lookup could run their onStartResource method before mine. Since my filter is user-defined, it is forced to run after all of the builtin ones; none of those currently perform config lookups, however. If any of the server.* config entries are specified somewhere other than "global", then we have the same issue. Finally, what's to stop a future CP developer from adding more such problems (as they fix other bugs)?
  6. Logging: the error.log and access.log will both use the original URI (from requestLine). Broken? or not? One? Both?

Rewriting "objectPath"

An alternative to rewriting the path is to use a filter that changes the value of objectPath instead, before the handler is looked up and called. For example, we could change VirtualPathFilter to do this instead:

class VirtualPathFilter(object):
    """Filter that changes cherrypy.request.objectPath, stripping a set prefix."""

    def beforeMain(self):
        if cherrypy.config.get('virtualPathFilter.on', False):
            prefix = cherrypy.config.get('virtualPathFilter.prefix', '')
            if prefix:
                path = cherrypy.request.path
                if path == prefix:
                    path = '/'
                elif path.startswith(prefix):
                    path = path[len(prefix):]
                cherrypy.request.objectPath = path
                                 ^^^^^^^^^^

Are there any side-effects to this approach?

  1. Generating URL's to spit back out in HTML: Broken. No change from rewriting path.
  2. HTTP Redirects and their targets: Broken. No change from rewriting path.
  3. Handler dispatch: not broken.
  4. Arbitrary mount points: not broken.
  5. Config lookups. Broken. A call to config.get() defaults to using path, which we haven't rewritten, which might seem all right until you try to deploy the app: every configMap key must be rewritten to prefix the mount point, and this must be done separately for each site. Some might call this an acceptable trade-off. I don't. ;)
  6. Logging: probably not considered broken.

Recommendations for CherryPy 2.2

We need to fix rewriting path or objectPath, or both. Let's try fixing objectPath:

  1. Generating URL's to spit back out in HTML: What to do about user code? Tell them to always use relative URL's? Not acceptable, really. A rewriting filter needs some way to "unrewrite" an arbitrary path, it seems. A prefix-only rewriter could do this, for example, but not a regex-rewriter. Maybe a prefix stripped from path could just be suffixed to base?
  2. HTTP Redirects and their targets: Same issue as #1. But also make the trailing-slash hack redirect by using browserUrl instead of (objectPath or path) + queryString. Note that HTTPRedirect already uses browserUrl.
  3. Handler dispatch: not broken.
  4. Arbitrary mount points: not broken.
  5. Config lookups. Make config.get try objectPath? There's a big problem there: objectPath might grow an extra "/index" or "/default" suffix halfway through the request process. So we'd have to separate the two concepts into a "searchPath" and a "foundPath". Even if we did that, we would still have the issue that path-rewriting does (user-defined filters run late). We would have to find a way to run a rewriting filter before most (all?) others. Maybe rewriting shouldn't be a filter at all—maybe it should be part of the fixed API, if only for prefixed mount points.
  6. Logging: probably not considered broken.

Seems we have our work cut out for us.

10/26/05

Permalink 03:35:00 pm, by fumanchu Email , 262 words   English (US)
Categories: CherryPy

Agenda topics for CherryPy 2.2 roadmap meeting

There will be an IRC meeting for the CherryPy 2.2 roadmap on Thursday, 6pm GMT. Things I want to discuss (most-important items first):

  • Merging the /requestobj branch to trunk. This branch merges the cherrypy.request object with the _cphttptools.Request object. This is just generally a good idea, and will make the code cleaner. It could also be a first step toward...
  • Allowing subclassing of the Request object in order to handle various dispatch schemes.
  • An API for HTTP-method dispatch, including ways to encourage developers of new apps to care about idempotency.
  • Ticket #356: Formalize server.environment as a set of config defaults
  • Multiple apps in a single process, including...
  • Arbitrary mount points for applications
  • Possible solutions to #362 (guaranteed filter methods). One such solution might involve...
  • Alternatives to the filter system--see if there's any way to declare customizations (which the app developer sees as frozen, and critical to the app) in code
  • Formalize the intended uses of request attributes (like browserUrl, base, path, querystring, etc.). There is some confusion in the codebase regarding when those should be, and are currently, rewritten. This will the inform...
  • Fixing the VirtualHostFilter (to inspect the Host header)
  • Better HTTP content-negotiation (Accept-* and Vary headers)
  • Better content caching (Expires header, etc)
  • Not really for discussion, but I want to get my mod_python WSGI gateway to work as well as mpcp on Unix

More as I think of them...

10/25/05

Permalink 11:06:09 pm, by fumanchu Email , 59 words   English (US)
Categories: IT, Python

Somebody's finally using text/x-json

Looks like there's a jsonserver for Zope 3 and one for Zope 2.7+ which are using the "text/x-json" content-type I mentioned earlier. Sigh; guess I won't be at the top of Google for that much longer. ;)

Permalink 01:33:07 pm, by fumanchu Email , 2897 words   English (US)
Categories: IT, Python, CherryPy

Customization vs handler dispatch in web application servers

[10/25/05 Update: changed some terms to make it more clear.]

At its most basic, a web-application server can be said to map a set of URI's to a set of handlers. From Roy Fielding's REST dissertation:

The resource is a conceptual mapping -- the server receives the identifier (which identifies the mapping) and applies it to its current mapping implementation (usually a combination of collection-specific deep tree traversal and/or hash tables) to find the currently responsible handler implementation and the handler implementation then selects the appropriate action+response based on the request content.

In an HTTP server, the "identifier" is the URI (which includes the query string, as I learned recently). The "handler implementation" is almost always a function in some programming language; for many HTTP servers written with scripting languages, these handlers will be written in the same language as the server. CherryPy 2.1, Django 1.0, and Quixote 2.3 are Python examples of this. mod_python 3.1 is an example of a Python web-application tool where the HTTP server is written in some other language (Apache, written in C). In a moment, we'll take a look at how each of these manages URI-to-handler mappings, which we'll call "dispatch".

Every web-application server, whether tied to a larger framework (Django) or not (CherryPy, Quixote, mod_python), must also address the need for customization. By "customization", I mean modifications to the per-request behavior of the server. I do not mean the behavior of the application, although the same techniques are often employed for both. I also do not mean end-user settings, which are properly stored in an application database. I also want to make it absolutely clear that I don't mean "data which exist in configuration files"—the concept of customization is distinct from the medium.

Now let's look at our four servers, and see how they manage URI-to-handler dispatch, and how they provide customization:

CherryPy

CherryPy takes the "deep tree traversal" route in order to map URI's to handlers. There is a cherrypy.root object which the developer creates, which always maps to the root URI "/". Subpaths are attached as attributes to the root object. Since the path portion of a URI is also heirarchical, there is a relatively straightforward mapping.

CherryPy allows some flexibility by providing a default method; if the mapper reaches the end of its search without finding a matching handler, it will then reverse direction, looking for a parent method named "default", which it then calls, passing any child path info as arguments. That is, a URI of "/path/to/parent/child/repr?color=red", if handled by cherrypy.root.path.to.parent.default, will be called as default("child", "repr", color="red").

CherryPy manages customization primarily via an internal Python dict (a key->value map); each key is a URI, and each value is another dict of (name, setting) pairs. This is often specified in an "ini-style" config file.

Django

Django might be said to epitomize the "hash tables" approach to handler dispatch, using an ordered set of regular expressions. The urlpatterns object is a tuple of tuples, where each inner tuple is of the form: ("pattern", "handler"). The pattern-handler pairs are evaluated in order until the URI matches a pattern, at which point the handler is looked up (converted from a string to the function which it identifies) and called. By using regular expressions, Django is free to map any set of URI's to a given handler.

Django keeps global customization data in a settings module; each global variable in that module can be used as a named value. Per-request customization, however, is managed entirely within the handlers, in code.

Quixote

Quixote has a mapping strategy apparently designed for maximum flexibility. Applications create a Publisher object, to which the server passes each HTTP request. The default Publisher will call self.root_directory._q_traverse(), passing the value of the PATH_INFO environment variable (split into chunks by each "/" in the URI). The _q_traverse method may then "do what it likes" with that path info; the common Directory object tries to map URI's to local methods, or to _q_traverse methods of successive child objects.

Quixote manages global customizations with a Config class; each attribute of that class is a constant which the server uses to customize its request-response process. The data can be read from a file (more properly, executed from a Python file). But like Django, per-request customization is managed entirely within the handlers, in code.

mod_python

Mod_python plugs right into Apache, and can control much more of the HTTP conversation than most of the other frameworks. Here, let's just talk about the PythonHandler directive; it's used as follows:

<Location /myapp>
    SetHandler python-program
    PythonHandler wsgiref.modpython_gateway::handler
</Location>

That is, the mapping between URI's and handlers is performed with Apache's Location, LocationMatch, Directory, Files, and similar directives. That's usually not the whole story, however; many modpython applications define few Handlers, or even just one, choosing instead to implement their own additional dispatch and customization layers within those handlers. I believe this tendency is one of the factors which have led so many Python web-framework developers, even the above three, to build on top of modpython as a deployment option.

The only generic, server-provided means of managing customization data for mod_python is the PythonOption key value directive (although other directives exist and may even be inspected; much of the customization in a modpython application is done entirely within Apache, or via other modules). Each PythonOption applies to the same set of URI's for which the given handler will be invoked.

Customization referents

All of the above designs, as described, have an additional detail in common: they can map multiple URI's to a given handler, but cannot (or tend not to) do the reverse: map a given URI to multiple handlers [1]. This is a surjective, not injective, mapping (click on the image to learn more):

Surjective, not injective, map. 1=D, 2=B, 3=A, 4=A

The central question then arises: is per-request customization data bound to the URI's, or to the handlers? Let's answer that for each of our four examples:

  • CherryPy customizations are definitely associated with URI's. Each section in the config dict is directly mapped from a URI key (note that a single key may match multiple URI's).
  • Django customizations are definitely associated with/defined by handlers. Any customization is written into the View objects themselves, in Python.
  • Quixote customizations are also definitely associated with handlers, just like in Django.
  • Mod_python customizations are associated with URI's; each PythonOption (or other Apache directive) applies to a given Location.

In the two frameworks (CherryPy and mod_python) where per-request customization data is associated more closely with URI's, the implementation is declarative as opposed to imperative; the server is free to use the data as it sees fit, in order to meet the perceived goal of the user. In the other two servers, Django and Quixote, the implementation is ad-hoc; developers may choose to use declarative implementations (for example, global constants), or they may "hard-code" the behavior.

This difference shouldn't be a surprise. Django is already a full-stack framework like Ruby on Rails or Spring. CherryPy, in contrast, was designed to (optionally) act as a base component for larger, "full-stack" frameworks like Subway or TurboGears. That is, CherryPy must be customizable both by end-applications and by intermediate frameworks. That would be difficult to achieve with imperative customization. Mod_python goes even further, since Django, Quixote, and even CherryPy can optionally use it to connect to Apache.

Access roles and customization

We need to pause, here, and make a distinction between application developers and application deployers. For many small applications, these two roles are played by the same person (who cannot understand why everyone is so picky about config architecture ;) ). But for larger applications or megaframeworks, the two are very distinct. Frequently, the following division of roles is expected:

ImperativeDeclarative
Server code Framework developer App developer
Application code App developer App developer
(often default values)
Config files Deployer
(a Programmer)
Deployer

Much of this is a direct result of the state of programming tools and languages. For example, "imperative server code" is the domain of framework developers, because only they have CVS/SVN privileges; if anyone else makes changes to that code, they fear losing their changes on the next update (although distributed RCS like Arch or Bazaar, can help ameliorate this a little). Similarly, config files exist outside the CVS/SVN of the application code, and are therefore the only domain of the deployer. But note that config files which use the same language as the application code are often assumed to be too difficult for non-programmers to use.

The Customization Maturity Vector

When developing applications (both new or existing), many developers tend to start with all behaviors embedded in imperative code. After a time, the developers notice a need for varying behaviors, and decide to provide a switch, in code, for it. This may take the form of constant values or subclassing or composition or some other pattern. Once the behavior set is reduced to a small number of variants, control over its customization may be placed in a config file. This results in a fairly predictable vector:

Imperative Code (IC) -> Declarative Code (DC) -> Declarative Text (DT)

Server and framework authors do the same, of course, preferring to start with imperative implementations in code, and moving slowly, but predictably, to declarative implementations in text. And they are right to move slowly; the decisions about where to store and retrieve such data are critical to proper isolation and encapsulation, key ingredients of multi-layered software.

However, what is often not addressed is that the different mechanisms not only implement access control, but directly affect program readability and server architecture, as well. For example, a server author who wishes to make a new, customizable feature available has several options. They may:

  1. IC: Provide a separate method for each possible behavior, expecting the application developer to call the correct one when required.
  2. IC: Provide a default method, which they expect to be overridden in a subclass (or otherwise replaced/superseded; some might call this Declarative, depending on the syntax).
  3. DC: Provide a standard location by which the application developer may declare a method to be called (plugin style).
  4. DC: Provide a method which is configured via a constant; the value may be placed in a variety of scopes.
  5. DT: Provide a method which is configured via a "config entry", whether in code or in a text file.

In my limited experience, their decision will most likely be motivated by the roles defined in the previous section, and by the "Maturity Vector" above. I'd like to hear from some other authors about their experience. But for now, let's move on to the architectural implications.

Issues for declarative mechanisms

CherryPy and Apache-configured-mod_python share a common weakness: customization data is stored in config files, in a declarative language which is not that of the application code. However, developers like to think of customizations as applying to handlers, not to URI's, and they often gripe when the effects of configuration files are divorced in time and space from their corresponding handler code.

For example, CherryPy has a plugin mechanism consisting of filters: classes with a set of methods which are called at various points in the request process. Until version 2.1, all filters were declared in code; each object on the cherrypy.root tree could define its own _cpFilterList attribute, a list of filter instances. Such filters apply to that object and its children, and therefore any URI's which map to that object or its children. CherryPy provided some filters in the standard distribution, but many were created by app developers to meet their specific needs.

Beginning in version 2.1, however, the builtin filters changed their declaration method, from an in-code list to configuration files, and therefore, changed from being associated with handlers to being associated with URI's. Interestingly, user-defined filters did not. Developers, especially CherryPy veterans, therefore, become confused between the two mechanisms. When configuration is bound to URI's instead of handlers, it is easy to delay the actual handler dispatch; in some cases, it may be delayed too long, limiting the customization which developers can perform, both in the handlers themselves and via any filter/plugin hooks provided by the server.

The effect is not limited to the filters themselves. Since some app developers see filters as a catch-all for customization needs, they place customizations in filters which don't really belong there, because they naively expect user-defined filters to be automatically declarable and configurable via the config file (they're not). On the other hand, filters are perceived to be black magic by many app developers, and some behaviors are hard-coded in handlers which belong more properly in filters (Python decorators seem to be a Strange Attractor for this).

Another problem arises because text-config-file declarations in both of these tools map to URI's, and not handlers. Server and app developers are at a loss in those rare cases when they need to allow deployers to specifically customize a handler as opposed to a URI. For example, a deployer for a site which internally redirects an arbitrary number of paths (e.g. http://www.site.com/~[user]/help -> /help) to a common, customizable handler would much rather write a single config entry, but the config file format forces them to write one entry for each virtual path.

Issues for imperative mechanisms

Django and Quixote, by contrast, seem to handle all per-request customization in code, imperatively. At least, there is no central, server-managed repository for declarative settings—any app developer could decide to implement their own; some do. Some simply expect customization to be done directly on the source files (example).

The first problem with this approach is that deployers (who are not programmers) have a significant psychological, if not educational, hurdle to overcome when configuring their copy of the application. [This isn't a religious treatise, so I'll stop there.]

The second problem is that it's difficult to really extend the framework itself. It's not expected, of course, that anyone would write a framework on top of Django, but neither is there any facility for extending per-request Django behavior, other than in imperative code. That is, if a site needed to gzip only some of their HTTP responses, a developer would have to implement that behavior, for each handler, in imperative code.

Finally, when customization is bound to handlers instead of URI's, the timing of the handler dispatch is of utmost importance; it must occur very early in the request-handling process, so that any per-request customization data can be available as soon as possible. Often, the mapping from URI to handler must be done absolutely first, so that the server itself has access to such data, even before calling the main handler. Django, for example, resolves the URI to the handler right away; the only serious action it takes first is to call global middleware (which has no per-request config). Quixote takes a hybrid approach; the _q_traverse methods act as "server code" (providing dispatch, and possibly configuration) but are instantiated in application objects.

Conclusion

The choice of where to store per-request customization data in a web application server is never a trivial one. It is constrained by project maturity and social expectations, as well as the architecture of both the server and each application. These concerns often compete; occasionally, they produce unresolvable conflicts.

When designing a web application server, the design of any configuration system is of utmost importance, and will affect the design of the entire server and any applications or frameworks which are built on the server. If a configuration system is not flexible, it may resist (or deny) applying the server to some application domains, or limit the extensibility of the server.

The customization system must also be designed to work in concert with the handler-dispatch mechanism. Customizations of all kinds should be analyzed and explicitly designed to be bound to either handlers or URI's. Pretending that handlers and URI's are synonymous will only hide implementation conflicts, delaying them from design time to deployment time.

Any feedback on this document is welcome. I'd like to learn a lot more about this from other server/framework authors' experiences, and from the analysis of any developers and/or deployers. Add your comments below, and I'll work on keeping this document updated.


[1] Mod_python has the best facilities for doing this, since many Apache handlers are designed to be run in series, or to cascade. An enterprising Quixote developer could, in theory, write a Publisher which called multiple handlers (but then, any of these tools' handlers could implement their own additional layer of dispatch, as well). But the vast majority of applications tend to remain surjective.

10/21/05

Permalink 03:37:20 pm, by fumanchu Email , 227 words   English (US)
Categories: CherryPy

Short list of things CherryPy should do

Sylvain just reminded me of one of Ryan Tomayko's early rants on HTTP and REST, On HTTP Abuse. It was probably the one post that jump-started my exploration of REST, which has been guiding my contributions to CherryPy.

In that post, he presented a short list of things which a "web framework" should provide:

For instance, which frameworks ...
  1. ... help implement content negotiation properly?
  2. ... provide facilities for implementing a smart caching strategy for dynamic content? (proper use of If-Modified-Since, Expires, Cache-Control, etc.)
  3. ... make dealing with media types easy?
  4. ... make dealing with character encodings easy?
  5. ... encourage the use of standard HTTP authentication schemes?
  6. ... have a sane mechanism for attaching different behavior to different verbs for a single resource?
  7. ... help ensure that URIs stay cool?
  8. ... make dealing with transfer encodings (gzip, compress, etc.) easy?
  9. ... help you use response status codes properly? (e.g. Nearly all dynamic content returns either a 200 or 500).

Even if CherryPy isn't a framework, it should do most of these. The latest release of CherryPy, version 2.1, addresses some of these (3, 4, 5, 8, and 9). The others are possible, but not as easy as they could be; 1, 2, and 6 are top priorities (for me, anyway) to work into version 2.2. Items 3 and 4 could use more work, too.

10/20/05

Permalink 11:12:57 pm, by fumanchu Email , 79 words   English (US)
Categories: IT

Firefox tip #34928509: nameless bookmarks

I just discovered I can set the name of a bookmark to nothing. This should free up a bunch of my Browser Toolbar real-estate, for sites that are nice enough to make a distinctive favicon. There's certainly no need for me to name the bookmark for Questionable Content as "QC", when the icon is already a black box with "QC" in white letters. I've also done away with names for Google, OneLook, and CherryPy.

Neat.

Permalink 09:00:23 pm, by fumanchu Email , 332 words   English (US)
Categories: Python, CherryPy, WSGI

Is CherryPy a web framework?

Guido van Rossum recently wrote:

Python, in its design philosophy, tries hard not to be a framework. (This sets it apart from Java, which is hostile to non-Java code.) Python tries to be helpful when you want to solve part of your problem using a different tool. It tries to work well even if Python is only a small part of your total solution. It tries to be agnostic of platform-specific frameworks, optionally working with them (e.g. fork and pipes on Unix) but not depending or relying on them. Even threads are quite optional to Python.

Oddly enough, this is how I feel about CherryPy, that it tries hard not to be a framework. It tries to be helpful, recognizing that it's most likely only part of your solution. It tries to be agnostic of templating and persistence systems, and has little to say about markup languages, content-types, site architecture, or RPC formats.

Guido was responding to Phillip J. Eby, who wrote:

A Pythonic framework shouldn't load you down with new management burdens and keep you from using other frameworks. It should make life easier, and make your code more interoperable, not less. Indeed, I've pretty much come to agreement with the part of the Python developer community that has says Frameworks Are Evil.

Not wanting to be Evil, I've tried to make CherryPy 2.1 a system which doesn't load you down with new management burdens. Instead, it exposes the functionality of HTTP by presenting it in a Pythonic way. I and many others think it makes life easier—CherryPy appears to have an underscore-shaped learning curve. ;) And as for interoperability, CherryPy was one of the first Python web-application servers to grow a WSGI interface.

I suppose that CherryPy will always have to bear the moniker of "framework", if only because it calls your code, instead of the other way around. But let's keep it a Pythonic framework as long as we can.

Permalink 08:45:35 pm, by fumanchu Email , 14 words   English (US)
Categories: CherryPy

CherryPy in epydoc format

Link: http://www.turbogears.org/docs/api/cherrypy-module.html

Thanks, Kevin! Can we get a new one when CP 2.1 final is released? Please? :)

10/19/05

Permalink 01:45:08 am, by fumanchu Email , 545 words   English (US)
Categories: General, Photography

Backpacking in Los Padres National Forest

Ryan (my fellow IT worker), Chris (his girlfriend), and I took a well-deserved five days and went backpacking on the Pine Ridge Trail in the Los Padres National Forest (Ventana Wilderness).

On Thursday, we left San Diego at 1:00 AM so that we could get a full day's hiking in. It's over 10 miles to Sykes Camp, plenty of time to reacquaint oneself with California's gorgeous trees:

Twisted trees on the Pine Ridge trail Entwined trees on the Pine Ridge trail Tall dark trees in a deep canyon

Once we arrived at Sykes, we partook of the hot springs. Here's Chris standing in a very small one:

Chris standing in a hot spring

The next day, we didn't move camp—just wandered up the Big Sur River in search of cool things. We found quite a lot of them, but I don't have any pictures to show you, and I probably would hide them from you if I had any. No sense making such a fantastic place too popular. Ryan found a turtle, maybe this pic will tide you over:

Floyd the turtle on Ryan's head

Saturday, we decided to hike on over to Cienega Camp, which sits above the North Fork of the Big Sur River. Here's a panorama of the North Fork valley:

Panorama of the Big Sur North Fork valley

That day...wasn't the best. Cienega is six miles away from Sykes, but the last three miles are a tough fight through thick brush. Wear pants if you ever try it. I didn't. ;) Once we got there, we found Cienega to be little more than a wide spot in the trail, so we turned around and backtracked the three miles to Redwood Creek Camp, which was much nicer:

Wall of redwoods at Redwood Creek Camp

Deep shot of redwoods

In the second picture, I hope you can see that the forest floor is a long way down. The "small" trees in the gaps are the same size as the near trees, just much further away.

Sunday, we continued our return trip, this time passing Sykes and stopping for the night at Barlow Flats Camp. Once there, Ryan and I decided to wade down the Big Sur River. I wanted to see the point where Logwood Creek fed out into the river, and once there, I cajoled Ryan into scrambling up its whole length until it rejoined the trail. Sure glad we did; this waterfall was 20 feet high, and fed into a pool at least 20 feet deep:

20 foot waterfall

This is only one of the many cool waterfalls we navigated on our way up.

In Barlow Flats Camp, unlike the other camps, the nearly-full moon was visible, and astoundingly bright. I spent a lot of time trying to get the perfect moon shots:

Moon behind a deciduous leaf Moon and Big Sur River

On Monday, we headed back to the parking lot, having traveled about 40 miles in all. Here's a last pic of the trail which I particularly liked, due to its spiral structure:

Trail with hint of a spiral

We left the park about 1:30 PM, stopped for pizza in Monterey, and tried to head home. After road closures, rain, flash floods, accidents, wrong turns, and too many leftover snacks, we hit San Diego about midnight! But I got a neat picture of the moon over Tehachapi. Note that, for all of these moon pictures, I used anywhere from a 4 to 15 second exposure, and never had a tripod—just me holding the camera as steady as I could.

Moon over Tehachapi, CA

To Dan, I just have to say, we should have gone to Los Padres instead of Kern last month. ;)

Permalink 12:37:37 am, by fumanchu Email , 159 words   English (US)
Categories: General, CherryPy

A severe shortfall in the motivational index

I can't seem to finish or maintain anything lately. I never can, but recently it's been more acute. Maybe it has to do with the pending release of CherryPy, which means we're in "feature freeze" mode; I miss writing the sweeping redesigns that I was able to shove into 2.1 before the beta.

But it's apparent in other areas, as well. I have about 80 hours of video transfer to DVD to do for various friends (and some personal projects). I haven't touched any of them in months. I also have a company to officially close, which is moving at a glacial pace. I've got a bunch of papers in triplicate that I need to mail off to the State, but can't seem to work up the drive to buy a large envelope for the purpose.

But then, having everything "done" is a much worse state. Maybe I should count my busy blessings.

Permalink 12:30:06 am, by fumanchu Email , 54 words   English (US)
Categories: Photography

Eunivision

This is just a fun panoramic shot I took from Eunice's driveway, showing the view of the whole El Cajon valley from her house (keep in mind that it's a distorted panorama; the extreme edges of the pic are actually in a line with each other, not at right angles as it might appear).

Panorama of El Cajon

Permalink 12:26:06 am, by fumanchu Email , 45 words   English (US)
Categories: General

This blog has moved recently

You might not have noticed, if you don't check every day, but this blog has moved recently. It was hosted at amorhq.net (owned by Amor Ministries), but now it's at aminus.org (owned by yours truly). Please update your aggregators, feedlists, links, and bookmarks.

10/07/05

Permalink 03:48:36 pm, by fumanchu Email , 232 words   English (US)
Categories: Dejavu

Dejavu has a new home

Dejavu, my pure-Python ORM, has a new Trac home at http://projects.amor.org/dejavu. As always, it's open to the public to download, use, or develop. It could use another contributor or two, and the snazzy new Trac front-end should help that immensely.

Just to remind you about the parts of Dejavu I think are cool:

  1. It completely hides the storage layer, so your code need never know you're using a database. It supports MS Access, MS SQL Server, PostgreSQL, MySQL, SQLite, and shelve, all transparently.
  2. Simple queries use simple syntax: Panda = sandbox.unit(zoo.Animal, Species='Ailurpoda', Name='Mei'). More powerful queries use the logic.Expression object and pure Python (no code strings!):
    red_filter = logic.Expression(lambda x: x.color == 'red')
    RedAnimals = sandbox.recall(zoo.Animal, red_filter)
    
  3. Dejavu has been designed from the ground up to be used in a multithreaded environment.
  4. New database adapters are easy to develop using the `db` module. The PostgreSQL module, for example, is 100 lines of code.
  5. Model-layer Units are clean and easy, using modern, built-in Python types. They are also extendable to custom property classes and custom types:
    from dejavu import Unit, UnitProperty
    
    class MissionTrip(dejavu.Unit):
        """A Mission Trip experience."""
        
        DirectoryID = TripGroupProperty(int, index=True)
        FirstDate = NotifyProperty(datetime.date)
        GroupName = UnitProperty()
        LastDate = NotifyProperty(datetime.date)
        AmountDue = UnitProperty(fixedpoint.FixedPoint)
    
  6. Persistent query-engines and analysis tools are included.

10/06/05

Permalink 05:52:07 pm, by fumanchu Email , 359 words   English (US)
Categories: CherryPy

Eat less, exercise more

Dave Warnock muses:

I am not about to argue that [TurboGears and Subway] should merge, instead I feel they can improve most by making sure that they each stay thin putting all the improvements they can back into the components eg CherryPy (which they both have been doing) and into the deployment elements (setuptools and paste).

That's a good point, and I think it's at the heart of what CherryPy is trying to be: non-fattening. So there's definitely going to be some pushback from CherryPy itself trying to "stay thin".

I often say that CherryPy is not a "web framework"; it is an "HTTP framework". That is, it doesn't try to provide tools for every facet of web development. Instead, it concentrates on wrapping HTTP up in a Pythonic way.

IMO, working to stay thin is an important factor in getting CherryPy "more exercise": it gets used in more meta/mega-frameworks like Subway and TurboGears precisely because it hasn't gobbled up every good idea, just because it's web-related, or even just because it's Python. For example, CherryPy 2.1 is deprecating the Aspect module that was in 2.0, because it isn't related to the HTTP-focus of CherryPy.

David goes on:

Another project that is the next level down from these frameworks but that is also moving fast is Quixote, I feel the differences between Quixote and CherryPy are also becoming smaller (shown by the recent blog posts on Python Web Controllers). Whether they could ever merge is a different matter. Probably not possible (or even desireable) for the moment.

I would have to agree. There's a decisive difference in architectural style between CherryPy and Quixote. That doesn't mean there aren't components that are common to each, and there are certainly some which are unique to each which deserve to be ported! If the Quixote coders are willing to give up all the method names starting with set_ and get_ we're ready to have a conversation about merging. ;)

10/04/05

Permalink 04:09:43 pm, by admin Email , 65 words   English (US)
Categories: CherryPy

The medusa called autoreload

This is what I spent my weekend working on (among other things). It's the "autoreload" functionality in CherryPy. It was so complicated that it took me 15 minutes to understand it again, anytime I got distracted; having the diagram makes it quicker, at least. They say the human brain can handle about 7 things simultaneously, and this snake-pit takes about 4 at a minimum:

Permalink 08:50:13 am, by admin Email , 54 words   English (US)
Categories: IT

Fractal frustration

Link: http://miksovsky.blogs.com/flowstate/2005/10/the_fractal_nat.html

Okay, coworkers, now you know: when I put my hands over my eyes and sigh heavily, it's because I'm on step 8. Actually, the more I learn about IT, the better I get at seeing step 8 coming when you're still on step 2. So please bear with me whenever I seem unduly frustrated at small requests.

October 2005
Sun Mon Tue Wed Thu Fri Sat
 << < Current> >>
            1
2 3 4 5 6 7 8
9 10 11 12 13 14 15
16 17 18 19 20 21 22
23 24 25 26 27 28 29
30 31          

Search

The requested Blog doesn't exist any more!

XML Feeds

powered by b2evolution