Category: Dejavu

Pages: << 1 2 3


Permalink 03:56:35 pm, by fumanchu Email , 368 words   English (US)
Categories: IT, Dejavu

Trigger unhappy

Bah. I can't design my way out of a wet paper bag when I get rushed, and I've been awfully rushed the last month or so.

The problem is update triggers. Our business (like every other) has tons of things that should happen when data changes. For example, when one of our clients decides to arrive a day earlier, that affects lots of business decisions. The people who make those decisions need to be notified, usually by email. Other, dependent business objects need to have their data updated. The very act of changing the date needs to be logged.

OK, that's not the real problem yet. The real problem is finding out when to do all of these activities. My naive first approach was to simply perform all of these side-effects whenever the FirstDate property of a MissionTrip unit changes:

def __set__(self, unit, value):
    if value != unit._properties[self.key]:
        unit._properties[self.key] = value
        self.fire_triggers(unit, value)

This is straightforward, but not very flexible. I put myself in a corner recently over this: the FirstDate is calculated based on a set of TripDate objects. When a user submits the MissionTrip web page, they might add three new TripDate objects—each one triggers a recalculation of the FirstDate property, and I ended up with three separate records in the log, along with three separate email notifications!

The local fix would be to not put a trigger on the TripDate objects, and to manually recalc the FirstDate. Wherever needed. There goes the whole point of objects.

What would be better would be a trigger that could fire at the end of the session (say, the web-page submit) instead of at the point of modification. But I can't figure out a clean way to do that. I worry that I won't remember to call session.cascadeAllTriggers() at the end of some submit handler, and I won't notice that fact for some time, since all of those activities are by nature side-effects. Not to mention the corner cases where I need to fire triggers before the end of the session.

Hmmm. I'd like something more declarative than imperative, I think.

What a messy part of domain modeling. :(


Permalink 08:49:45 pm, by fumanchu Email , 171 words   English (US)
Categories: Dejavu

Bug in

There was a bug in until today (revision 69). You should probably upgrade from SVN if you're using dejavu.

The bug manifests whenever you have a logic.Expression that has a function call which cannot be represented in SQL. For example, I hit it with the Expression: lambda x, **kw: x.FirstDate <['Year'], 12, 1).

There are two parts to explaining why this went unnoticed for so long. First, all of my uses of had been constants so far. When they are made into Expressions, the early binder turns the whole .date constructor call into a LOAD_CONST. Consequently, I've never written a dispatcher for the constructor, which is a function. So db.SQLDecompiler.visit_CALL_FUNCTION had no concrete function object to call, and nowhere to dispatch to. When both of those most-common cases fell through, the decompiler stack should have gotten a cannot_represent entry appended to it; instead, that entry was overwriting the top-of-stack. Ugly and oh-so-fun to track down.


Permalink 03:33:21 pm, by fumanchu Email , 344 words   English (US)
Categories: IT, Python, Dejavu, Cation

Where to put the locks

I've got to add access control to "Mission Control" (our main business application), and I cannot decide where to do it.

There are two main components: a UI (built on a UI framework, Cation), and a Model layer full of business objects (built on an Object-Relational Mapper, Dejavu).

Part of me wants to build the controls into the business objects themselves, since that's really what needs to be protected. I don't care if someone can access a page if they can't see any data on it (or add any data through it). It wouldn't be difficult to enforce, because all the business objects already subclass dejavu.Unit, the base class for the ORM. The downside is that Dejavu doesn't currently have any concept of users or access controls. Any user context, therefore, would have to be passed into Dejavu from outside. This wouldn't be too hard, since every Dejavu session checks out its own Sandbox object--I could just stick a reference to the access-control library into the Sandbox. I'd probably end up leaving the default Sandbox as is, and write a new SecureSandbox class, then decide between the two via a config parameter.

Writing the locks into the Model would be nice, because, in general, those change less often than the View. When new features are added, having the locks in the View code means a higher probability of forgetting to test for permission.

It would also be nice for management by end-user admins--they tend to understand "you can see Mission Trips but not Projects" better than long lists of event descriptions like "Show the Zip Code of the Primary Addressee on a Group Scheduling Report (but not a Group Scheduling Calendar)".

But another part of me wants to put the locks in the View. It would allow finer-grained permissions, and allow locks on behaviors which don't involve persisted business objects. It would also be better-integrated, since the View layer already interacts with authentication code in the webserver. There's also a lot of precedent, in that most ORM's don't manage permissions.



Permalink 03:59:40 pm, by fumanchu Email , 496 words   English (US)
Categories: Dejavu

Why doesn't Dejavu provide a "unique" property?

On c.l.p., Leif K-Brooks wrote:

My ideal solution would be an object database (or object-relational mapper, I guess) which provided total transparency in all but a few places, built-in indexing, built-in features for handling schema changes, the ability to create attributes which are required to be unique among other instances, and built-in querying with pretty syntax.

Dejavu meets a bunch of those needs (transparency, indexing, very pretty querying--no schema change tools though), but my initial reaction to reading his list was to say to myself, "Dejavu doesn't really provide 'attributes which are required to be unique'."

Well, it turns out it does, it's just that they're limited: the 'ID' attribute is the only one which can be required to be unique. They do not need to be globally unique--you could have a Customer object with an ID of 23 and a Purchase object with an ID of 23. But you can't have two Customer objects, each with the same ID of 23.

Well, that's not quite true. Rather than having a ton of explicit guards sprinkled throughout the code, the 'requirement to be unique' is enforced by the fact that Sandboxes will simply overwrite one Unit with another if they have the same ID. It's just that this is done lazily, so far a brief period of time, you could have two objects with the same ID.

  1. Create an instance A, either with or without an ID.
  2. Memorize it.
  3. Create an instance B, explicitly giving it the same ID as A.

When you memorize B, your Storage Manager may allocate space for instance B. But you'll only be able to recall either A or B (provider-dependent).

The other built-in guard is the UnitSequencer, which provides ID values when you don't. If you provide your own ID, then the sequencer doesn't get called. Right now, there are two sequencers:

  • UnitSequencerInteger: this one will supply increasing integers. [hmmm... I should put a mutex in there]
  • UnitSequencerNull: this one just raises NotImplementedError, so you need to always supply your own ID's.

I suppose I could rewrite the Null one to guarantee uniqueness, in order to avoid the corner case I outlined above. I just haven't had the need. I'm a conscientious coder, and haven't needed any more checks than these two.

The final issue for (future) users of Dejavu might be that they need some attribute other than ID to be unique. But I've never run into that, and am reticent to add it unless someone asks for it. So ask, but be prepared to convince me that that's a better approach than redesigning your model, or using a custom StorageManager to fake it, or both.


Permalink 03:58:55 pm, by fumanchu Email , 572 words   English (US)
Categories: IT, Dejavu

Explicit Domain object persistence

Dejavu's domain objects (those objects which are part of your model, and which you probably want to persist) all subclass from dejavu.Unit. When you create a new Unit, you declare that it should be persisted by calling Unit.memorize(). Plenty of other ORM's (Object-Relational Mappers) don't work this way; instead, every object is automatically persisted unless you specify otherwise. Why doesn't Dejavu do it automatically?

Legacy Database Design

In the most common case, Dejavu uses the Model which you design to create and populate a database. Your Unit classes become tables, Unit Properties translate into columns, and each Unit instance is persisted as a row. In this case, your write the Model and let it drive the database design to match. In some cases, however, you need to integrate with an existing database. Dejavu has been designed to support this, as well (although it requires more work).

Often, a pre-existent database will possesses validation checks, from type and value constraints to referential integrity enforcement. Such guards often require that data be collected and pre-processed by your application before submitting it for persistence; that is, you must get "everything right" before you add a new row to a table. Since these requirements are application-specific, it would be impossible for Dejavu to guess when the object is "ready to persist". As the Zen says, "In the face of ambiguity, refuse the temptation to guess."

It is entirely possible for you to write a subclass of dejavu.Unit which, at the end of __init__, calls memorize for you. It would have been much harder and more confusing to do the opposite. If you find yourself not needing the flexibility which explicit calls to memorize provide (and remembering to call memorize becomes a burden), feel free to use such a subclass.

Multiple stores

Further, Dejavu is designed to work with multiple stores, which may be determined dynamically. I had a use case, for example, to manage Transaction objects, where Income Transactions were placed in one store, and Expense Transactions were placed in another store. A custom Storage Manager proxied the two stores, and decided in which store to persist each Transaction Unit based on the is_expense attribute. That attribute might not be known at the time an automatic mechanism persisted the object. To make matters worse, the Income store used an autoincrementing ID field, a fact over which I had no control, so I couldn't simply migrate a Transaction from one store to the other as the attribute changed.

Another approach would have been to simply have two separate Unit classes, IncomeTransaction and ExpenseTransaction. However, this would have broken encapsulation--the storage requirements would be intruding on the model design. I very much want the Model to migrate seamlessly on the day when we ditch the Income store, and the integrated Transaction class fits the company's mental and behavioral model better.

All that said, the decision comes down to "explicit is better than implicit". Since Dejavu is a library, it's much easier to provide the functionality in an explicit manner, and allow application developers to make it implicit if they see fit. If the behavior is performed implicitly, "behind the scenes", it is much more difficult to allow developers to then make that explicit when they need to; you end up either exposing class internals, or forcing the developer to reproduce your internal logic, often incompletely.

<< 1 2 3

October 2017
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31        


The requested Blog doesn't exist any more!

XML Feeds

free blog tool