Parallelizing L2S Entity Retrieval

Posted by MarkB on Stack Overflow See other posts from Stack Overflow or by MarkB
Published on 2010-04-24T15:56:05Z Indexed on 2010/04/24 16:03 UTC
Read the original article Hit count: 191

Assuming a typical domain entity approach with SQL Server and a dbml/L2S DAL with a logic layer on top of that:

In situations where lazy loading is not an option, I have settled on a convention where getting a list of entities does not also get each item's child entities (no loading), but getting a single entity does (eager loading).

Since getting a single entity also gets children, it causes a cascading effect in which each child then gets its children too. This sounds bad, but as long as the model is not too deep, I usually don't see performance problems that outweigh the benefits of the ease of use.

So if I want to get a list in which each of the items is fully hydrated with children, I combine the GetList and GetItem methods. So I'll get a list and then loop through it getting each item with the full cascade. Even this is generally acceptable in many of the projects I've worked on - but I have recently encountered situations with larger models and/or more data in which it needs to be more efficient.

I've found that partitioning the loop and executing it on multiple threads yields excellent results. In my first experiment with a list of 50 items from one particular project, I did 5 threads of 10 items each and got a 3X improvement in time.

Of course, the mileage will vary depending on the project but all else being equal this is clearly a big opportunity. However, before I go further, I was wondering what others have done that have already been through this. What are some good approaches to parallelizing this type of thing?

© Stack Overflow or respective owner

Related posts about l2s

Related posts about domain-driven-design