Categories: IT, Architecture, Linnaeus Award, Python, Cation, CherryPy, Dejavu, WHELPS, WSGI, Robotics and Engineering

Pages: << 1 ... 3 4 5 6 7 8 9 10 11 12 13 ... 17 >>


Permalink 12:30:17 am, by admin Email , 256 words   English (US)
Categories: Dejavu

Dejavu 1.4.0 (Python ORM) release

Dejavu 1.4.0

I'm extremely pleased to announce the release of Dejavu 1.4.0, a pure-Python Object-Relational Mapping library. Dejavu allows you to create, query, and manage persistent data using your existing knowledge of Python programming.


  • Data queries are expressed using pure Python; no SQL, no operator hacks, and no need to wrap code in strings.
  • Data may be transparently stored in PostgreSQL, MySQL, SQLite, Access, or SQL Server databases, as well as in flat files (using shelve), and caching proxies. You can create and combine custom storage systems for your own integration and performance needs.
  • Easy associations between Unit classes.
  • Full thread-safety for reliable use in web applications and other concurrent environments.
  • Views, sorting, cross-tabs, and other analysis tools.
  • Unit Collections, plus Engines and Rules to form powerful end-user query and reporting interfaces.

What's new in 1.4

  • Full LEFT, RIGHT, and INNER JOIN support (okay, one operator hack here).
  • Optimized and introspectable To-One and To-Many associations.
  • Arbitrary names for Unit.ID's (primary keys).
  • Support for multiple ID's (primary keys).
  • A new Schema class and other upgrade-management tools.
  • Default values for UnitProperties.
  • New logging hooks to help debug SQL and other storage issues.
  • Fixes to support Python 2.4 bytecode and other changes.
  • Inheritance support (all subclasses are recalled).
  • Vastly-improved test suite.
  • New recur module included, with a threaded Worker class.
  • Better support for update triggers.

Dejavu is in the Public Domain, and you may use it anywhere with no obligation whatsoever.

User documentation and a full Trac site are available at:


Permalink 11:10:35 pm, by fumanchu Email , 213 words   English (US)
Categories: Python, General, Dejavu, CherryPy

We're hiring, by the way

The job posting is pretty tame: we need a Python web developer. But I thought I'd add my personal point-of-view, and say that we really mean "developer" and not just "coder". You'd be responsible for producing working web apps, but that involves a lot of design work and architectural decision-making.

You would also be expected to contribute to the CherryPy HTTP framework and to Dejavu (my Python ORM), since I'm a core dev on both those projects and use them heavily already. In other words, if you have or want exposure to the full stack of modern web development challenges, this is the job for you. You'll be a full member of an IT team of 3 serving an energetic staff of 50.

You'll also get something that's hard to find in most programming jobs: warm fuzzies. We build homes for the poor in Mexico, simultaneously "building" the church in Mexico, the U.S., Canada, and elsewhere. We are not on the cutting-edge of world missions--we are defining that edge. If you've been thinking about "doing more for Jesus", but would rather write code than dig ditches in Uganda, give us a call (619-662-1200 ext 11).


Permalink 10:34:47 pm, by fumanchu Email , 54 words   English (US)
Categories: IT

There's no definition without culture

Adam Green recently wrote about "Web 2.0":

If it takes an essay or more than 5 or 6 bullet points to explain something, it is poorly defined.

I'll disagree with a counter-statement:

If it takes less than 5 or 6 bullet points to explain something, it's because the audience already knows what it is.


Permalink 08:49:20 pm, by fumanchu Email , 258 words   English (US)
Categories: IT

Win98 to the rescue

My, that was fun. Four hours of restoring a corrupted SAM on my mother's XP laptop—she has 46k dialup. :/ Here's the error message on boot:

lsass.exe - System Error "Security Accounts Manager initialization failed because of the following error: The handle is invalid. Error Status: 0xc0000008. Please click OK to shutdown the system and reboot into Safe Mode, check the event log for more detailed information."

The final resolution process:

  • On another computer, obtain NTFS4DOS, install it on a floppy, then burn the floppy contents onto a CD.
  • Boot the laptop from a Win98 CD. When prompted, choose "Run the computer with CD-ROM support."
  • Stay on the command line, pull out the Win98 CD, and insert the NTFS4DOS CD. Run NTFS4DOS.EXE.
  • NTFS4DOS loads, then waits for you to type "Yes" or "No" to the question, "Are you using this on a private computer?" Boring nagware, but a useful moment for us to swap CD's again—put the Win98 CD back in, so COMMAND.COM can be found on it.
  • Answer "Yes" to the nagware prompt.
  • Follow the instructions from Microsoft (method 2, step 3) to replace your corrupt or missing SAM with the one from windows\repair:

    copy c:\windows\system32\config\sam c:\windows\tmp\sam.bak
    delete c:\windows\system32\config\sam
    copy c:\windows\repair\sam c:\windows\system32\config\sam

Of course I considered Knoppix, but have you ever tried to download a 750MB ISO over dialup?


Permalink 11:34:47 am, by fumanchu Email , 106 words   English (US)
Categories: IT

Open source needs more pop quizzes

Glyph recently wrote:

I have also made no attempt to be fair, and I don't want to do this again any time soon, so if your favorite operating system got trashed here, don't bother to tell me that I'm not being fair. Think of it as a pop quiz!

Open source projects need more pop quizzes. There's a reason why you don't let developers run usability tests—it's far too easy to jump in with justification for why your product didn't work "just this once".

I'm just writing this to remember the "pop quiz" meme. Made me smirk. :)


Permalink 11:37:08 pm, by fumanchu Email , 226 words   English (US)
Categories: Dejavu

Dejavu 1.4 now in beta

After more than a year since 1.3 was released, I'm just about ready to officially release Dejavu 1.4! In addition to bugfixes, there are some major new features:

  • Sandbox.recall now returns a list (use xrecall to get an iterator).
  • Associations are now aware of whether they are to-one or to-many.
  • logic.Expressions can now take multiple positional arguments (so you can test multiple Units at once).
  • Improved multirecall, including full support for INNER and OUTER JOINs for all Storage Managers. Since the signatures for recall and multirecall now align, the "multirecall" name has been dropped; just call Sandbox.recall(classes, expr) whether you're querying a single class or multiple ones.
  • Units may now have arbitrary identifiers (primary keys).
  • Unit Properties have a new "default" attribute.
  • Simple inheritance is now supported; recalling one class will also recall its subclasses.
  • New Sandbox "magic recaller" methods, like inv = box.Invoice(13).
  • New Sandbox.view method, to retrieve persisted data without creating full Units.
  • A new Schema class to help manage changes to your model, and helper methods to sync database schemas.
  • New logging support.
  • A new test runner.
  • Python 2.4 fixes for codewalk, the test suite, and fixedpoint.

As you can see, a year's worth of work. ;) Feel free to kick the tires on all the new stuff. I should bless a release candidate in early January.


Permalink 02:08:17 pm, by fumanchu Email , 338 words   English (US)
Categories: Dejavu

Dejavu is adding schema versioning

I just dumped a first crack at a Schema class on the trunk. Test code is here (search for 'schema'), docs are here. I haven't written anything like this before, so if anyone has recommendations or warnings about the direction it's heading, now is the time to speak up (before 1.4 is officially released ;) )!

Basic design: there's a dejavu.Schema class which your app can subclass. Whenever you need to change the underlying database (or other persistence mechanism) schema of your app, you write a new upgrade_to_X method, where X is an incrementing version number. Each such method contains the commands which will upgrade an installation from (X - 1) to X.

At runtime, you call MySchema.upgrade(), and each deployment will run any upgrade_to_X methods that it hasn't yet run, in order. The "currently deployed version" number is stored in a magic DeployedVersion Unit.

The upgrade_to_X methods can choose to stay database-neutral and just use the (new) arena.add_property, drop_property, and rename_property methods. But because each Schema is application-specific, you can also write optimized instructions for your known StorageManagers. For example, say you need to change an int property to a string. The "database-neutral" way would be to have additional Arena methods for such tasks. Some of those methods may be added in the future, but nothing's stopping you now from writing non-portable SQL statements if you know your app is only deployed on, say, Postgres (but you should probably assert that before you execute the SQL statements).

Anyway, I'd be interested to hear from anyone else who has written database-versioning tools. Save me from a pitfall if you can. :) Have fun with the new Schema class and let's see if there are a couple of other common methods (like add_column) that should go into the Arena and the StorageManagers.


Permalink 11:23:32 am, by fumanchu Email , 103 words   English (US)
Categories: WSGI

WSGI gateway for mod_python status fix

After much woe, I think I finally tracked down the status problems I was having with (which is now available on my "misc" Trac site). It should now correctly handle redirects, 404's, and .css and .js content. I think it also fixed my earlier "delayed content" problem.

I hereby nominate mod_python's status API for the "One Obvious Way To Do It" booby prize. Having req.status, a return value/status, and the option to raise a status makes far too many combinations.


Permalink 11:42:58 am, by fumanchu Email , 695 words   English (US)
Categories: CherryPy

What will CherryPy 3 look like?

The correct answer is: "nobody knows". But here are some ideas I've been kicking around the ol' cranium lately...

[09:32] *** now talking in #cherrypy
[10:22] <Lawouach> where to start
[10:22] <Lawouach> what's your basic idea toward 3.0?
[10:22] <@fumanchu> oh, I have so many ;)
[10:22] <Lawouach> lol
[10:22] <Lawouach> say big general ones :)
[10:22] <Lawouach> not details per se
[10:23] <@fumanchu> 1) make CP have a kick-butt,
    non-CP-specific toolkit (lib/httptools), that is SO
    good that Quixote, Django, et al can't *help* but
    decide to use it instead of their own server processes
[10:24] <@fumanchu> even if they don't like the way CP
    maps handlers to URL's, for example
[10:24] <@fumanchu> they should be able to build a
    server with the behavior they like out of lib/httptools
[10:25] <Lawouach> we want to be lib that rule them all :)
[10:25] <@fumanchu> yup
[10:26] <Lawouach> i agree as long as we don't become a
    framework on our own, but i already know it's not what
    you intend :)
[10:26] <@fumanchu> right
[10:26] <@fumanchu> it's an anti-framework approach
[10:26] <@fumanchu> we make writing-a-web-framework
    into a weekend's work
[10:27] <@fumanchu> take some from column A; try all of column B
[10:27] <Lawouach> do you want to stay very low-level
    (aka HTTP wrapper level) or make it a bit higher level
    and provide functions such as the bast_match() we were
    talking about last week?
[10:27] <@fumanchu> best_match would be fine as long
    as it doesn't depend upon cherrypy
[10:28] <Lawouach> right, this was a bad example
[10:28] <Lawouach> but basically where httptools should stop?
[10:28] <@fumanchu> I think that can be open-ended
[10:28] <Lawouach> i think we should keep the level
    you've been doing till now
[10:29] <@fumanchu> 2) then, by pulling a ton of code
    out of _cphttptools (putting it in lib/httptools instead),
    I want to see if we can get the Request and Response
    objects down to a tiny size
[10:34] <@fumanchu> the trunk version of _cphttptools
    is already 60% of its 2.1 size
[10:35] <Lawouach> right. hmmm
[10:37] <@fumanchu> and a *lot* of what's left is very OO
[10:38] <@fumanchu> so, one idea I'm toying with: allow
    developers to use their own subclasses of Request
    and Response
[10:40] <@fumanchu> if we make it super-easy to use custom
    Request subclasses, then they will want to start
[10:40] <@fumanchu> take out the filter logic, and becomes:
def _run(self, requestLine, headers, rfile):

    self.headers = list(headers)
    self.headerMap = httptools.HeaderMap()
    self.simpleCookie = Cookie.SimpleCookie()
    self.rfile = rfile

    except cherrypy.RequestHandled:
    except (cherrypy.HTTPRedirect, cherrypy.HTTPError), inst:
[10:40] <Lawouach> regarding the subclassing of request
    and response, i'm know that it could interest very
    much the guys behind itools
[10:40] <@fumanchu> yes
[10:40] <@fumanchu> and Ben Bangert (routes)
[10:41] <@fumanchu> anyway, if is *that* simple,
    then who needs filters?
[10:41] <@fumanchu> just code them procedurally into your method
[10:43] <@fumanchu> looking over the filters that are built in...
[10:44] <@fumanchu> I think that half could be done just as
    easily as lib/httptools functions
[10:44] <@fumanchu> and half could be "always on"
[10:44] <@fumanchu> (if we continue to improve them, like
    encodingfilter, to meet the HTP spec)
[10:44] <@fumanchu> HTTP
[10:44] <Lawouach> that's my white cheap :) (i don't think this
    expression exists so i make it up!)
[10:45] <Lawouach> i really want CP to be HTTP conditionnaly compliant
    at least :)
[10:45] <Lawouach> and maybe in CP 4.0 to be unconditionnaly compliant!
[10:45] <Lawouach> :p
[10:45] <@fumanchu> I completely agree
[10:46] <@fumanchu> anyway, I want to stress that I'm still playing
    with these ideas
[10:46] <@fumanchu> nothing's set in stone
[10:47] <Lawouach> since you've be proposing them a while back,
    i've been a great fan of them
[10:47] <@fumanchu> and trying to implement them will turn up
    lots of problems, I'm sure
[10:47] <@fumanchu> oh, well thanks
[10:47] <Lawouach> that's why i don't have so many different
    things to bring for cp 3.0
[10:51] <@fumanchu> one of the nice things about these ideas
    for 3.0 is that the bulk of the work can be done within
    the 2.x branch


Permalink 09:27:43 pm, by fumanchu Email , 302 words   English (US)
Categories: Python

Oh so very stumped

Dear lazyweb,

After 6 hours, I am utterly stumped. I've got an application built with a popular Python web application server, via mod_python, and keep seeing data bleed from one request to the next. That is, if I:

  1. Request a page that has a css <link>,
  2. Request a non-existent jjj.css file (resulting in a 404), and then
  3. Request a different, non-existent mmm.css file (another 404), I see this in the third window:


The requested URL /jjj.css was not found on this server.

Apache/2.0.55 (Win32) mod_ssl/2.0.55 OpenSSL/0.9.8a mod_python/3.2.2b Python/2.4.2 mod_auth_sspi/1.0.2 Server at Port 443
HTTP/1.1 404 Not Found Date: Tue, 22 Nov 2005 01:57:37 GMT Server: Apache/2.0.55 (Win32) mod_ssl/2.0.55 OpenSSL/0.9.8a mod_python/3.2.2b Python/2.4.2 mod_auth_sspi/1.0.2 Content-Length: 371 Keep-Alive: timeout=15, max=94 Connection: Keep-Alive Content-Type: text/html; charset=iso-8859-1

Not Found

The requested URL /mmm.css was not found on this server.

Apache/2.0.55 (Win32) mod_ssl/2.0.55 OpenSSL/0.9.8a mod_python/3.2.2b Python/2.4.2 mod_auth_sspi/1.0.2 Server at Port 443

The body of request #2 is present in request #3, and so are the headers of request #3! Frightening.

This happens reliably with both Firefox and IE. It happens whether I use HTTPS or not. It happens whether I use authentication or not. It happens when I strip the modpython gateway-for-WSGI I wrote down to 80 lines.

It stops happening when I use CherryPy's builtin WSGI server, so I don't think any part of CP is to blame, which leaves a bug in mod_python or Apache2. I'm particularly inclined to blame them because, although CherryPy and Apache itself log both the missing responses as 404, Ethereal shows me that the actual third response, as received by the client, has a 200 response code!

So I'm stumped. Any solutions, pointers, or flights of debugging fantasy accepted.

<< 1 ... 3 4 5 6 7 8 9 10 11 12 13 ... 17 >>

August 2019
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31


The requested Blog doesn't exist any more!

XML Feeds

free blog software