« Epic [FAIL]Xtremely Quick Development System »

LINQ in Python

04/22/08

Permalink 11:58:01 am, by fumanchu Email , 105 words   English (US)
Categories: IT, Python, Dejavu

LINQ in Python

Chui's counterpoint pines:

There are some interesting ideas raised in LINQ that even Python developers ought to explore and consider adopting in a future Python.

Python had all this before LINQ in Dejavu and now Geniusql, and more pythonically, to boot. Instead of:

var AnIQueryable = from Customer in db.Customers where
    Customer.FirstName.StartsWith("m") select Customer;

you can write:

m_names = Customer.select(
    lambda cust: cust.FirstName.startswith("m"))

and instead of:

var AverageRuns =(from Master in this.db.Masters
    select Master.Runs).Average()

you can write:

avgruns = Masters.select(lambda m: avg(m.Runs))

14 comments

Comment from: Jonathan Ellis [Visitor] · http://spyced.blogspot.com

Speaking of Geniusql, any updates or plans for a new public release?

04/22/08 @ 13:19
Comment from: Larry Hastings [Visitor]

While Geniusql provides a lovely syntax, I would be surprised if it offered the other major advantage of LINQ: incorporating the filtering code into the SQL statement run on the database. I suspect, for example, that the lambda expressions and generator expressions from your samples will run in Python, wheras in the C# example the 'where Customer.FirstName.StartsWith("m")' will actually be woven into the SQL SELECT statement. This is way faster, as it means you don't have to propogate all the data records from the db into Python (and likely over the network!).

04/22/08 @ 13:30
Comment from: fumanchu [Member] Email

Jonathan, Geniusql is pretty much ready for a 1.0 RC release; I just need time to package it.

Larry, prepare to be surprised. :)

http://www.aminus.net/geniusql/chrome/common/doc/trunk/managing.html#expressions

04/22/08 @ 14:35
Comment from: Luke Plant [Visitor] · http://lukeplant.me.uk/

@Larry: I'm afraid you suspicions are wrong, from what I can tell from the documentation. It uses bytecode hacks to disassemble the expressions and generate SQL for the backend. Which is quite neat or scary voodoo magic, or both, depending on your point of view.

04/22/08 @ 14:53
Comment from: Lakin Wecker [Visitor] Email · http://lakin.weckers.net

Larry,

Actually, the lambda statements in GeniuSQL and DejaVu provide the same benefit: the lamdba expressions are compiled and woven into the SQL clauses that are sent to the database. So I guess you should be surprised, although I don't understand why. ORMs in python have had this ability (specifying SQL clauses using an elegant syntax) for a long time.

04/22/08 @ 19:46
Comment from: Christian Wyglendowski [Visitor] · http://www.dowski.com

Larry: Nope. Depending on the backend in use (it doesn't have to be SQL), the Python lambda is converted into optimized SQL code.

04/22/08 @ 21:46
Comment from: Simon Davy [Visitor]

I'm curious, how does Genuisql fare in terms of complexity on non trival queries? Say, computing an aggregate on a 3 way join, including a view?

I havn't used Genuisql, but I have used SQLObject and SQLALchemy, and LINQ is certainly competitive (in terms of readability/brevity) with them, especially in more complex queries.

Plus, another thing I like about LINQ is that the same primitives can be used to query SQL, XML, Amazon, Google, or whatever. And it gives c# something approaching general list comprehensions/generators.

04/23/08 @ 02:12
Comment from: fumanchu [Member] Email

Simon,

Views are treated just like Tables in queries, so there's no difference there. As far as aggregates over joins, the GROUP BY clause is automatically calculated as the set of all output columns which are not aggregates (although this can be manually overridden).

04/23/08 @ 16:29
Comment from: rgz [Visitor]

I didn't know these libraries used the bytecode to produce optimized results.

But...

where Customer.FirstName.StartsWith("m")

is better than...

select(lambda Customer: Customer.FirstName.startswith("m"))


The thing that many pythonists have noted is that we could have this with some extensions to the iterator protocol. But well, it may be included in python 4000.

04/23/08 @ 17:25
Comment from: forestcell [Visitor]

It is a good idea!!!

05/23/08 @ 00:31
Comment from: Chris [Visitor]

Brilliant work. The syntax is soooo much more natural than other orms. I hope to use this in my project.

But what do they mean by 'bytecode hacks' instead of core AST?

Could this query syntax sit on top of SQLAlchemy?

07/07/09 @ 14:14
Comment from: Tim [Visitor] Email

What I think LINQ has provided that is a bit different than the alternatives being discussed here is a common API on enumerations that works regardless of whether the data is in a database, a file, in memory or anywhere else. I don't personally like LINQ query syntax, and I don't use it, but I use the LINQ extension methods directly very often. I think the distinction between:

Customer.select

and

source.Customers

is quite important as the developer may be thinking quite differently in each. In the first, I am thinking that I'm grabbing objects from some store that is linked with the class Customer via an ORM. In the latter, I may think the same way, but I personally like to think of it as "I've got this object that has many customers and now I can play with them". That collection may be backed in a database, or it may just be a collection.

None of this is not already accomplished either in Python (generators or list comprehensions) or via the many community modules that address object-relational mapping or extend what itertools already gives you. However, what I can't find in Python is a unified API regardless of what the underlying mechanism of keeping my data is. To me, it doesn't matter if I'm calling into a for loop or generating a monster SQL statement. I'd rather write the same code regardless of which of these is going to happen under the hood.

Specifically when looking at LINQ as an ORM though, I do agree that the Geniusql example in the post is quite a bit nicer than the LINQ query syntax.

08/12/10 @ 09:38
Comment from: fumanchu [Member] Email

Tim wrote:
"To me, it doesn't matter if I'm calling into a for loop or generating a monster SQL statement. I'd rather write the same code regardless of which of these is going to happen under the hood."

Me too. But I want some control over which is going to happen, because the performance characteristics of each approach vary so widely. That's one reason I went with lambdas for GeniuSQL: you can take that lambda "f" and run [a for a in B if f(a)] in pure Python and get the same result as if you passed it to B.select(f).

With Dejavu's federated storage, you can declare B is stored in RAM and type B.select(f) everywhere.

08/12/10 @ 10:31
Comment from: tc [Visitor] Email

I haven't tried this stuff in Python yet, but I'm writing lot of C# code and use LINQ all the time. I really like LINQ, you can express your needs much easier and more conceise than before (in previous versions of C#), but I need to comment on 2 things that were touched in previous comments.
1) Although the LINQ API seems like it is common for DB backed storage, in-memory collecionts and XML the truth is that those APIs are NOT UNIVERSAL, they are different in terms of what is supported and one will learn that the hard way :) Basically each storage has its specific characteristics so it is not possible to develop single common API that is powerfull at the same time.
2) Although complex LINQ querries are easy to write and express in C#, which I'm not sure of when it comes to Python syntax, still I would not take this as a huge advantage (rather just a little) cause the problem is you don't want to put complex and smart LINQ querries in your code cause if they are too tight they'll become hard to maintain.

Cheers,
Pete

02/07/11 @ 04:51

Leave a comment


Your email address will not be revealed on this site.

Your URL will be displayed.

Please enter the phrase "I am a real human." in the textbox above.
(Line breaks become <br />)
(Name, email & website)
(Allow users to contact you through a message form (your email will not be revealed.)
August 2014
Sun Mon Tue Wed Thu Fri Sat
 << <   > >>
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            

Search

The requested Blog doesn't exist any more!

XML Feeds

multi-blog