Tuesday, May 29, 2007

Defining the Direction of LINQ to Entities/EDM

The ADO.NET team today posted an invitation targeted at "experienced IT developers" among Tech*Ed 2007 attendees to "attend a focus group on LINQ/EDM ... to help define the direction of LINQ to Entities." I'm not attending Tech*Ed because Microsoft moved it from New Orleans, a city I love almost as much as San Francisco, to Orlando, which I detest. (I'd rather travel to Flint, Michigan, than Orlando, Florida.)

Update 6/7/2007: LLBLGen Pro's Frans Bouma posted SqlServer 2008: Does it or doesn't it have the Entity Framework?, which supports my request for read-only foreign key values (#9) and a comparison of the Entity Framework's current and short-term future feature set with other popular object/relational mapping (O/RM) tools (#2). Ayende (Oren Eini) replied with Phsycic Debugging from Blogs, which contains an analysis of why it might be difficult for the ADO.NET team to deliver foreign key values without retrieving the corresponding object.

Update 6/2/2007: ADO.NET architect, Mike Pizzo responded in detail to my feedback in this comment.

I won't be able to join the focus group, so I'll add here the two cents I would have provided there. Following are what I consider the more important actions required to make the Entity Framework, Entity Design Model, Entity Client, and LINQ to Entities a competitive O/RM tool:

  1. Come clean with positioning of EF/EDM versus LINQ to SQL by releasing the retracted white paper that compared features of these technologies. (Julie Lerman mentioned an ADO.NET Team post that "addresses one of the biggest questions about querying against the Entity Framework - what are the pros & cons of Entity SQL vs. LINQ" that's disappeared. She says, "In discussing this, the post also lays out a lot of great background info.")
  2. Provide a similar white paper comparing the projected feature set of EF v1.0 and, perhaps, EF vNext with that of NHibernate, NPersist, and commercial OR/M tools such as LLBLGen Pro, Wilson ORMapper for .NET, et al.
  3. Recommit to Mike Pizzo's fervent promise of RTM with a graphical EDM Designer in the first half of 2008, not "in the Katmai timeframe."
  4. Deal openly with Mats Helander's contention that Microsoft is crying "O/R wolf" and won't release the EF/EDM because of the "bad press" ensuing from tools that "don't work or underperform."
  5. Respond formally to the concerns about lack of persistence ignorance and other issues voiced by the "O/RM Mafia" at the last MVP Summit in Redmond.
  6. Provide a detailed development path to and timeline for the version that enables generating the database schema from a domain model, rather than developing the domain model from the database schema.
  7. Eliminate the inability to navigate associations having composite primary keys. (LINQ to SQL doesn't have a problem with composite primary keys).
  8. Implement new object addition and all other advertised features that throw "Not yet implemented" exceptions.
  9. Provide read-only access to foreign key values.
  10. Define the support for n-tier architectures and provide non-trivial, loosely-coupled sample projects with WCF.
  11. Deliver a log feature similar to LINQ to SQL's that captures the [not just T-]SQL sent to the server.
  12. Provide a more friendly method of responding to concurrency exceptions, such as that provided by LINQ to SQL.
  13. Commit to writing an Oracle managed provider if Oracle is reluctant to do so. Developers need proof that EF/EDM/LINQ to Entities is database-agnostic. [Oracle is now on board and participating in a Tech*Ed 2007 chalktalk about EntityClient providers.]
  14. Let developers know when they can expect the first CTP and the approximate schedule of CTPs/betas to follow.*
  15. Provide detailed change lists for each CTP.
  16. Accelerate work on adding DML constructs (INSERT, UPDATE, DELETE) to Entity SQL/Entity Client.
  17. Provide an analysis of EF/EDM in terms of Martin Fowler's Patterns of Enterprise Application Development and compliance with domain-driven design (DDD) and test-driven development practices as Ian Cooper did for LINQ to SQL in his Being Ignorant with LINQ to SQL essay.

Update 6/14/2007: Added item 17 as promised in Ian Cooper Takes on DDD, TDD and PI with LINQ to SQL.

I'll update this list as I think of additional issues or missing features. If you have an EF bug you'd like squashed or a feature you'd like to see in EF/EDM/LINQ to Entities, please leave a comment.

* Note: ADO.NET's Zlatko Michailov said on May 21, 2007, "The CTP is scheduled for June" in this ADO.NET Orcas forum thread. However, future CTP drop schedules should be posted in the ADO.NET Team blog for better visibility to interested parties via syndication.

P.S. The sign-up site includes this sentence about the focus group session: "This session will describe our current thinking about the LINQ/EDM APIs we will offer with the Orcas release of Visual Studio." [Emphasis added.]

Last I heard, EF/EDM/LINQ to Entities had been removed from the Orcas release.

Update 5/30/2007: Added five topics.

2 comments:

Michael Pizzo said...

Hi Roger;

Sorry you won’t be at TechEd. Florida in June is not my first choice either, but there you have it. In the meantime, thanks for the great feedback. A few comments below…

1. Entity Framework and LINQ to SQL.

I honestly don’t know what paper Julie is referring to, but note that she describes it as comparing Entity SQL versus LINQ (i.e., when to use a textual ESQL query versus a typed LINQ query, both of which are supported within the Entity Framework), not the Entity Framework versus LINQ to SQL.

The bottom line on Entity Framework and LINQ to SQL was that they are each designed around a different set of priorities; LINQ to SQL is designed around rapid development, with direct (1:1) mapping, while the Entity Framework is designed around exposing a rich conceptual model with flexible mappings to extended ADO.NET Data Providers. We considered not shipping one or the other of these to reduce confusion, but ultimately decided that neither stack fully addressed the customer commitments and expectations set by the other. So we will ship both; LINQ to SQL in Orcas and the Entity Framework in an update to .NET Framework 3.5 in first half 2008. Over time we will look to customers to help us decide whether it makes sense to continue to invest in enhancing both stacks or merging into a single stack over time.

Being clear on the differences *today* between the two stacks is a good idea, and I’ll add it to my list of future BLOG topics.

2. Provide a similar white paper comparing the projected feature set of EF v1.0 and, perhaps, EF vNext with that of NHibernate, NPersist, and commercial OR/M tools such as LLBLGen Pro, Wilson ORMapper for .NET, et al.

Another good suggestion. I just spoke to our Community Coordinator about this, and it’s been on her list for a while, she’s just looking to find someone with expertise in the different features to write the article (any volunteers?)

3. Recommit to Mike Pizzo's fervent promise of RTM with a graphical EDM Designer in the first half of 2008, not "in the Katmai timeframe."

Sorry for the confusion around dates. Orcas, Katmai, and the Entity Framework are all being released as part of a single marketing wave. Apparently we get more when we announce things together, even if they don’t release at the same time. Whatever. Anyway, the Entity Framework has no dependencies on Katmai, and ships as an update to the .NET Framework, not as part of Katmai, so we’re not dependent on Katmai to ship.

4. Deal openly with Mats Helander's contention that Microsoft is crying "O/R wolf" and won't release the EF/EDM because of the "bad press" ensuing from tools that "don't work or underperform."

It’s an undeniable fact that Microsoft has fumbled the ball repeatedly when it comes to delivering an O/R solution. The biggest reason we didn’t ship ObjectSpaces was not technical, but rather a lack of recognition at a management level of the importance of providing such a solution. The only way for ObjectSpaces to gain traction was by aligning with something bigger (i.e., MBF or WinFS). The biggest thing that’s changed (thanks largely to the success of Hibernate) is that management now gets the importance of a good O/R solution. The support for LINQ, LINQ to SQL, and the Entity Framework at the executive level is what ObjectSpaces never enjoyed, and what gives me confidence that we will finally shed the albatross that’s been hanging around our neck since .NET Framework 1.0.

5. Respond formally to the concerns about lack of persistence ignorance and other issues voiced by the "O/RM Mafia" at the last MVP Summit in Redmond.

The “O/RM Mafia” has been extremely effective in influencing our Object Services Dev Lead. I’m not sure what they have over him, but he’s totally bought into the idea of persistence ignorance (he’s said more than once that if only he’d “gotten” it a year ago, we would have gone a very different direction). Though I can’t say we’ll reach full persistence ignorance for V1, we are actively working on removing dependencies on domain classes in our remaining milestones.

6. Provide a detailed development path to and timeline for the version that enables generating the database schema from a domain model, rather than developing the domain model from the database schema.

I can’t give a detailed development plan and timeline for generating database schema from a domain model, but I can say that it’s currently a hot topic. The problem with doing database generation is that Data Definition (DDL) is even less interoperable than Data Manipulation (DML) between different backends, so while we could do a one-off implementation for SQL Server, we really want to keep the Entity Framework provider-neutral. We need to work with provider writers to define a common representation and semantics that our tools can use across different backends. This is one of the topics I plan on discussing with provider writers at TechEd next week.

7. Eliminate the inability to navigate associations having composite primary keys. (LINQ to SQL doesn't have a problem with composite primary keys).

If you are referring to the Beta 1 limitation that a column that was part of a primary key could not participate in a foreign key, that has been fixed.

8. Implement new object addition and all other advertised features that throw “Not yet implemented” exceptions.

Can you describe how you’re trying to add a new object that throws the NYI exception? Clearly it’s possible to add new objects in the common case using the Entity Framework; not sure what scenario you’re hitting…

9. Provide read-only access to foreign key values.

This is actually a feature I’m fighting to get into our final milestone for V1. Can you describe the scenarios where this is used? Do you need the ability to query on the foreign key value, or simply expose it on the domain object?

10. Define the support for n-tier architectures and provide non-trivial, loosely-coupled sample projects with WCF.

I would love to see us do some more in-depth sample projects with WCF.

11. Deliver a log feature similar to LINQ to SQL’s that captures the [not just T-]SQL sent to the server.

We definitely want to provide more visibility into the generated SQL, but are trying to determine the best way to do this. ADO.NET has a tracing mechanism called “BID” that seems the right fit for tracing generated SQL. We’d also like to do a better job of visualizing it during debugging. Is there any interest in seeing the EntitySQL generated through LINQ/ObjectQuery, or do you really just want to see the store-generated SQL (which is a little harder for us, since we have to get it from the provider)?

12. Provide a more friendly method of responding to concurrency exceptions, such as that provided by LINQ to SQL.

I’ll look into this.

13. Commit to writing an Oracle managed provider if Oracle is reluctant to do so. Developers need proof that EF/EDM/LINQ to Entities is database-agnostic.

We have had two “ProviderFests” where vendors have come to Redmond for a week at a time to extend and test their providers with the latest Entity Framework bits and give us feedback. SQLite has already posted a version of their provider that works with the Entity Framework beta, and IBM, DataDirect, Oracle and MySQL will all join me for a chalk talk at TechEd on Friday to discuss their experiences extending their providers for the Entity Framework.

14. Let developers know when they can expect the first CTP and the approximate schedule of CTPs/betas to follow.

The first CTP of the Entity Framework outside of Orcas will be in June, and we will have CTPs/betas aligned with each of the Orcas Betas & RTM.

15. Provide detailed change lists for each CTP.

Yes; we should do this…

Anonymous said...

Re Mike's response.... WOW! :-)

Julie