Jon Udell wrote about the Zen of automation, including
Either way, what’s missing is the sensitive feedback. To automate a task I turn on the macro recorder, self-consciously perform the task, analyze it, fiddle around with the script, and then package it for reuse. So long as the context remains the same, the script will continue to work. Of course the context never does remain the same, and I’ll end up adjusting the data sources read by my account exporter and tweaking the charts created by my log visualizer.
In my opinion, automation will rarely, if ever, have a Zen. To use Jon's terms, it's not that the context changes; it's that the tool has become the new context. When the original problem morphs into (or is replaced by) another, there is a choice to be made regarding whether the tool-we-built-to-solve-the-original-problem should adapt or be replaced. It's the classic "when all you have is a hammer..." problem.
It only takes a light touch to get the kid on the bicycle to sense and react to my cues, but I have to clobber my software to make it pay attention. Getting applications and services to share a common message bus should help.
Given the above, I don't see how a "common message bus" will help unless the tools themselves learn and adapt. You're asking a screwdriver to magically become a hammer.
The physical world answers the need for adaptation in a number of ways:
I'm sure there are others.
For the glue-language scripting that Jon's talking about, I see developers choosing response 3 and 4, mostly. Users (as opposed to developers) will often choose 1 and 2--they force the task to fit the tool, since tools are expensive to make, and tool-making isn't something they do. The feedback Jon's seeking should be between developers and users, not developers and their tools. You don't train the bicycle; you train the child to ride the bicycle better, more confidently. You train the child to cry less when they crash.
Andre Stubbe and Markus Lerner - Univ of Berlin. Project homepage
Check out the Quicktime videos.
I did a boneheaded thing. I sent a bunch of you an announcement about this new blog of mine, even though our network, at the time, looked like this:
It's being put back together, slowly. So if you can't get to this page during the rest of this week, just try again the next day. If you have problems next week, let me know right away.
Dejavu's domain objects (those objects which are part of your model, and which you probably want to persist) all subclass from
dejavu.Unit. When you create a new Unit, you declare that it should be persisted by calling
Unit.memorize(). Plenty of other ORM's (Object-Relational Mappers) don't work this way; instead, every object is automatically persisted unless you specify otherwise. Why doesn't Dejavu do it automatically?
In the most common case, Dejavu uses the Model which you design to create and populate a database. Your Unit classes become tables, Unit Properties translate into columns, and each Unit instance is persisted as a row. In this case, your write the Model and let it drive the database design to match. In some cases, however, you need to integrate with an existing database. Dejavu has been designed to support this, as well (although it requires more work).
Often, a pre-existent database will possesses validation checks, from type and value constraints to referential integrity enforcement. Such guards often require that data be collected and pre-processed by your application before submitting it for persistence; that is, you must get "everything right" before you add a new row to a table. Since these requirements are application-specific, it would be impossible for Dejavu to guess when the object is "ready to persist". As the Zen says, "In the face of ambiguity, refuse the temptation to guess."
It is entirely possible for you to write a subclass of
dejavu.Unit which, at the end of
memorize for you. It would have been much harder and more confusing to do the opposite. If you find yourself not needing the flexibility which explicit calls to
memorize provide (and remembering to call
memorize becomes a burden), feel free to use such a subclass.
Further, Dejavu is designed to work with multiple stores, which may be determined dynamically. I had a use case, for example, to manage Transaction objects, where Income Transactions were placed in one store, and Expense Transactions were placed in another store. A custom Storage Manager proxied the two stores, and decided in which store to persist each Transaction Unit based on the
is_expense attribute. That attribute might not be known at the time an automatic mechanism persisted the object. To make matters worse, the Income store used an autoincrementing ID field, a fact over which I had no control, so I couldn't simply migrate a Transaction from one store to the other as the attribute changed.
Another approach would have been to simply have two separate Unit classes, IncomeTransaction and ExpenseTransaction. However, this would have broken encapsulation--the storage requirements would be intruding on the model design. I very much want the Model to migrate seamlessly on the day when we ditch the Income store, and the integrated Transaction class fits the company's mental and behavioral model better.
All that said, the decision comes down to "explicit is better than implicit". Since Dejavu is a library, it's much easier to provide the functionality in an explicit manner, and allow application developers to make it implicit if they see fit. If the behavior is performed implicitly, "behind the scenes", it is much more difficult to allow developers to then make that explicit when they need to; you end up either exposing class internals, or forcing the developer to reproduce your internal logic, often incompletely.
Update: I stumbled onto Mike Spille's blog, which talks a bit more (and better) about middleware versus libraries.
Ian Bicking recently promoted the idea of a WSGI reference library, to possibly include the following components (among others):
Not being the most careful reader in the world, I was thrown by the phrase, "...collaborating on a ... library of WSGI middleware"; I read the list as if he meant each piece would be a middleware component! Of course he did not intend that. Many of the items in the list are WSGI applications, which sit at the end of the software stack.
Some of the items in the list are, in fact, paraware; that is, they parallel the main application. Traditional programming libraries/toolkits are a common example of paraware. They provide functionality by supplying input and output hooks, which are supplied and consumed by the main application:
mylib.set_value(3) mylib.munge(obj) result = mylib.get('f')
Middleware, on the other hand, handles/munges a content stream, and sits between at least two other components in a software stack. Middleware is a nasty thing in many environments, because each middleware component must manage I/O of all shared objects, in two directions (both its caller and the next component in the stack). In Python, however (and specifically WSGI), the shared objects are all on the same heap, and can all be passed by reference.
I see problems with writing most of these components as middleware. WSGI has a shot at being ubiquitous because it enforces a set of interfaces and a data model; this same enforcement, however, can also be a liability, since WSGI is not yet ubiquitous. As a developer of a web framework, I have a dilemma: I need to provide the same functionality whether my users use WSGI or not. This means I need to write such components as libraries (so they can be used as paraware) and then wrap them with WSGI boilerplate (so they can be used as middleware). This leads to serious code smell. WSGI's callback structure is complicated enough without me introducing library-code wrappers. Perhaps what we need are generic pieces of WSGI middleware which you can init with a callback from your library code. Hmmm.
I've been meaning for a while now to investigate breaking my Cation app framework down into a set of libraries (instead of the monolithic framework it is today). You can see from the dearth of recent checkins that I haven't done any of it yet. Many of those could be added to a WSGI library (some are already on Ian's list). Here are the ones I'd be most interested in writing:
I'd like to do this myself because Cation keeps a list of application developers (usernames), and shows full tracebacks in the browser to developers. Ordinary users get a "pretty" error message, and the full traceback goes into the log only. I'm pretty sure a standard library version wouldn't do that. Integrating the usernames into the error handling logic leads me to want to provide this as paraware, since middleware components are usually not expected to interoperate.
This isn't WSGI-specific, and shouldn't be a candidate for WSGI. But it's something I'd like to rewrite in more of a library style, instead of a framework.
For example, this would assist a WSGI application in fulfilling a request to shut down--each active web request (thread) could be sent a shutdown message and kill itself gracefully from outside the application itself.
Since HTML form values are always received as strings, a standard (but overridable) way to convert them into Python values would be helpful. In the other direction, values need to be coerced to strings, put in the encoding of the server (or of the page), and often quoted safely. Again, this would probably need enough customizability that it would be a poor candidate for middleware, but a good candidate for a set of library calls.
Classic middleware, meeting a need orthogonal to the actual content delivery, and not needing customization or context.
Something that might on occasion need to be specialized, but ultimately a commodity for 90+% of cases. The standard implementation would be nothing more than a pretty interface over simple (but secure) file management.
That's enough for the next year or so Pity I have so many other projects to work on simultaneously.
Ned nails it. If someone, somewhere has been using your software in production for a while (a year? years?), your core functionality is 1.0. Branch and fork all you want from there, but please let us, your users, know it's worth a try.
I particularly don't understand why this trend seems to apply to development libraries more than anything else. Library users are developers--give them some credit. They'll find and fix the broken bits on their own.
In an attempt to automate our backups (my PFY was doing manual DVD burning every day), we bought a Dell Poweredge SC420 with a pair of 250G SATA drives and no OS. It'll go either to our disaster hotsite, or a colo. The only thing the box will be doing is rsync over ssh. Our various nix and Windows servers (via cwrsync) will connect on a schedule and back up various top-level directories. Testing on a different server over the 'Net showed typical backup times of 5 to 10 minutes for one such directory, depending on how much had changed overnight. We expect to have backup traffic for about 1 hour each night, up to a max of 4 hours (rarely). Email will, of course, be the killer. We may go back to CD's/DVD's for that.
At any rate, here are the steps I went through to set up the Dell server:
vi /etc/grub/menu.lst Change the lines starting with "kernel": append the text " acpi=off".
cd / mkdir /data1 mkdir /data2 vi /etc/fstab add the lines: /dev/sda1 /data1 ext3 defaults,errors=remount-ro 0 2 /dev/sdb1 /data2 ext3 defaults,errors=remount-ro 0 2
cd /etc/rcS.d mv S36discover S29discover
apt-get install rsync vi /etc/default/rsync (set RSYNC_ENABLE=true) vi /etc/rsyncd.conf (see online docs) /etc/init.d/rsync start
vi /etc/ssh/sshd_config PermitRootLogin no RSAAuthentication no
Another Scott on IWETHEY asked me to expand on why I chose b2evolution for the blog software here, especially in relation to this post. I'm awfully bad at recording my decision-making processes, but I'll try.
Lots of blogs I examined failed one of our requirements outright, or at least offended my sensibilities :
Meh. That's enough for now. b2evo has had its own quirks, but the problems have been surmountable with a minimum of effort. I think it will serve us well enough.
Yes, this really is the first post.
My company, Amor Ministries, has been talking about staff websites since...well, forever. We've toyed with various ideas and CMS systems. I even wrote one myself for HTML editing called Tibia. However, nothing seemed to solidify--the burden on the authors is usually too great.
Last week, I floated the concept of blogs past Alon, our Development Team leader, and for the first time, his desires and the available tech started to gel very nicely. We made a short laundry list of desirable features for blog software, and the result will be built here.
I went with b2evolution because:
The first features I'm looking to write (and contribute back to b2evo, if they'll answer my email):
|<< <||> >>|