LINQ to SQL:Entity Framework::REST:SOAP?

There's a current controversy regarding the need for—and desirability of—two object/relational mapping (O/RM) tools in ADO.NET vNext: LINQ to SQL and the ADO.NET Entity Framework and its components. Based primarily on blog posts and comments, the debate appears to center on the following contentions:

1. Entity Framework's three XML mapping files and Object Services layer are overly complex for the needs of the vast majority of .NET developers.
2. Entity Framework is over-reaching by attempting to extend the Entity Data Model to "Replication, Reporting Services, BI, Integration Services, etc.", and remoting as well as lighter-weight incubator applications such as "Project Astoria" and, especially, "Project Jasper."
3. LINQ to SQL satisfies the needs of the vast majority of .NET developers.
4. Entity Framework doesn't provide "Persistence Ignorance"; that is, the persistence layer is limited to relational databases and isn't transparent to the object layer. The current version is targeted on database-to-model implementation rather than the model-to-database approach preferred by domain-driven modeling proponents.
5. Table per concrete object and table per type inheritance models and many-to-many relationships can be added to LINQ to SQL post-Orcas.
6. Entity Framework and LINQ to Entities reduce Microsoft resources devoted to LINQ to SQL, including opening the LINQ to SQL provider model to other database vendors and delivering a once-promised provider SDK.
7. LINQ to SQL wasn't invented by the ADO.NET Team, who developed the Entity Framework in parallel with LINQ to SQL (then DLinq). When the ADO.NET folks got control of LINQ to SQL, they hobbled it with an artificial SQL Server-only limitation.

The preceding contentions closely parallel those in the REST vs. SOAP Web services controversy that's continued since SOAP was Dave Winer's XMLRPC, but began in earnest in mid-2002. Here are some of the common points raised:

1. The SOAP and WSDL specifications are overly complex for almost everyone's needs.
2. SOAP has spawned Web Services (WS*) specifications to support security (WS-Security, WS-SecurityPolicy, WS-SecureConversation, WS-Trust, WS-Federation), messaging (WS-Addressing, WS-Enumeration, WS-Eventing, WS-Transfer, WS-ReliableMessaging), and transactions (WS-Coordination, WS-BusinessActivity, and WS-AtomicTransaction). The WS* specifications are almost impossible to read, understand, and/or implement.
3. REST doesn't support security, messaging, or transactions and most network admins don't permit invoking HTTP's PUT and DELETE verbs.
4. REST satisfies the needs of the vast majority of programmers, content providers, and users.
5. REST can be upgraded with extensions to conventional URI syntax, such as those proposed for Microsoft's "Project Astoria" incubator project.
6. SOAP requires frameworks and/or toolkits, such as Web servers, application servers, Windows Communication Framework, Visual Studio, and/or Eclipse, which developers might need to purchase. Even if open-source, developers must devote substantial time and effort to master them.
7. SOAP and the WS* specifications were subsumed by Microsoft, IBM and, to a lesser extent, BEA and other vendors, who sell servers, SOAP toolkits, frameworks, development environments, security certificates, and/or consulting service. Intraoperability (and some interoperability) issues lead to vendor lock-in.

Discussions re LINQ to SQL vs. Entity Framework

Paul Wilson, the developer of the commercial (US$50) Wilson ORMapper, said on April 29, 2007 Great News: Only One O/RM Shipping in Orcas:

I almost can't believe the great news that came from MS today. That's right, the ADO.NET Entity Framework has been delayed ! I bet most of you think I'm being sarcastic, but I am serious. I've got at least three reasons why this is good news to me.

First, this gives Linq to Sql a better opportunity to thrive. Linq to Sql is the "simpler" OR/M that's looking good enough for the vast majority of cases, while ADO.NET EF is far more complex -- and yet most gurus only wanted to talk about the EF.

Next, since ADO.NET EF is so complicated, it absolutely must have a great designer ship concurrently, which was not the plan. MS has apparently accepted this feedback since this is at least the publicly given rationale for the delay in shipping the EF.

Finally, and this one may not pan out, but it is my own hope that ADO.NET EF is being re-aligned somewhat with Linq to Sql. These two O/RMs are similar enough to share at least some code, and I believe that some of the MS guys have hinted at this too.

So I'm happy that at first there will be one O/RM -- Linq to Sql. The gurus may be disappointed, but the vast majority of MS devs will be new to O/RMs anyhow, and Linq to Sql will be good enough. Very much like my simple WilsonORMapper has been so widely used.

Ian Cooper's convincing April 29, 2007 LINQ to Entities and Occam's Razor post says, in part:

[T]he decision not to support querying to relational stores other than Microsoft SQL Server from LINQ to SQL seems to be deliberately crippling that product for no good reason. The provider model should be exposed and other implementations encouraged. Isn't that the value of LINQ? This feels like an internal turf war over who provides the standard data access layer to .NET developers. As a customer I am not interested in your internal disputes but in solving my customer's problems with your tools. LINQ To SQL should be capable for simple mapping scenarios across backend types and the provider model should be opened up for that. Do not force us to swallow the whole LINQ To Entities pill, just to get support for multiple backends. This is not a reasonable decision.

Similarly I would hope that later releases of LINQ to SQL could be enhanced to support many-to-many associations and table per concrete classes inheritance strategies. Why force an upgrade to LINQ To Entities to obtain these feature sets which are common to many ORMs which do not have an additional abstraction layer? I do not see any technical reason for the Entity Framework being a requirement for these to work. NHibernate and Wilson for example, already provide these capabilities on the existing ADO.NET framework.

Experience with many MS developers suggest that if you give an 'upgrade path' to LINQ to Entities you will implicitly degrade LINQ to SQL as being a toy. It is not; unless you make it so.

Kevin Hoffman compares LINQ to Entities with LINQ to SQL in this August 17, 2007 LINQ to Entities vs. LINQ to SQL - What should I use and when? post, which waxes enthusiastic about the Entity Framework, its conceptual data model, and LINQ to Entities:

LINQ to Entities allows me to write C# LINQ queries that run against a conceptual data model. This is huge. I cannot stress enough to you how unbelievably forward-thinking and useful it is to be able to run queries against a conceptual object model as opposed to a database schema. The issue that a lot of folks have is in trying to decide if they need to use the Entity Framework or if they are fine with plain-vanilla LINQ to SQL (formerly called DLINQ).

Kevin complains about the inability to use foreign key fields as discriminator columns in ADO.NET Entity Framework - Entity Inheritance Woes of October 10, 2006, and the initial delay to the release of the EDM Designer in Microsoft finally shows off their EDM designer... but it won't ship with Orcas?!? (March 28, 2007) and Who ships a modeling tool without a visual designer? Microsoft (April 24, 2007).

Julie Lerman also compares the two technologies in LINQ to SQL vs. Entity Framework (April 4, 2007).

Entity Framework and LINQ to SQL Background

"Project Jasper" demonstrates the resource hit for implementing the Entity Data Model (EDM) dynamically. In Sam Drucker and Shyam Pather's DEV18 - Rapidly Building Data Driven Web Pages with Dynamic ADO.NET MIX07 video it took about 30 seconds (19:17 to 19:47) to initialize and cache the DynamicContext object, which represents the EDM, for a short list of blog entries. (Jasper's DynamicContext object corresponds to LINQ to SQL's DataContext object.)

Danny Simmons' May 3, 2007 Where’s the model? Is it the code or something else? post takes on Jeffrey Palermo's concern with substituting the Entity Data Model (defined by its XML files) for the domain model (defined by its code). Danny says:

We do talk about the EDM for an application as "the model", but that's just a verbal short-hand for "data model", which I believe is a critical distinction. We are explicitly NOT trying to define behavior in the EDM. The code is the right way to define that. We are, however, trying to define enough about the structure and constraints on the data that we can provide a rich set of services in a general fashion that still honor those constraints. ...

Once we start generating code from the EDM, things definitely become blurred a bit, but the explicit goal of the generated code is not to replace your object model—it's to save you some typing of repetitive things when you are building that object model. Further, the intent is that we would not require this generated code—you can always build your model completely on your own and just align the EDM with the shape of that model so that you can take advantage of the EDM-enabled services we provide (right now that means persistence/query, but eventually that means other things like reporting, replication, remoting, etc.).

For more about the EDM Designer delivery delay see Déjà Vu All Over Again: Entity Framework Cut from Orcas.

Discussions re REST vs. SOAP

Mike Champion's May 1, 2007 Reporting for duty on WS-Deathstar post, which announced that he's moving from the "core XML team in SQL Data Programmability ... to a position in the Connected Systems Division's Interoperability unit." His responsibilities there "include representing Microsoft on the W3C, helping with web services standards partnerships, and generally helping the world understand the method behind the apparent WS-Madness, or at least the subset of it that Microsoft endorses."

In his LINQ to XML swansong, Mike says:

I've always been annoyed by the zealotry manifested in the "SOAP vs REST" debate. The overhyped WS marketing and the RESTifarian pushback in 2001-2002 or so had the effect of people "flipping the bozo bit" on each other wholesale. We're just now seeing some exploration of the sensible middle position -- the native Web technologies are powerful and deserve first class support in the services-oriented tools and specs, and at least some of the WS technologies are useful on the Web as well as the enterprise, especially in the realm of identity and security. People on the WS side are finally understanding the the potential of the SOAP GET binding, and maybe the REST folks are starting to understand the oft-maligned WS-Transfer ("HTTP over SOAP over HTTP") is not quite so pointless in a multi-hop, multi-protocol world.

Update 5/24/2007: Jon Udell reviews the recently published RESTful Web Services by Leonard Richardson and Sam Ruby, who also are subjects of Jon's interview for a forthcoming IT Conversations show. Jon quotes the authors:

“We’re writing a book,” the authors say in their web introduction, “to codify the folklore, define what’s been left undefined, and try to move past the theological arguments.”

According to Jon, the book proposes use of Resource-Oriented Architecture (ROA) for Web services, wherein:

• Data are organized as sets of resources
• Resources are addressable
• An application presents a broad surface area of addressable resources
• Representations of resources are densely interconnected,

when the "security protocols and complex coordinated workflows" provided by Service-Oriented Architecture (SOA), such as SOAP and implementation of the WS* specifications aren't required.

Tim Ewald's April 26, 2007 I finally get REST. Wow. post details his decision to substitute REST for SOAP says, inter alia:

It's depressing to think that SOAP started just about 10 years ago and that now that everything is said and done, we built RPC again. I know SOAP is really an XML messaging protocol, you can do oneway async stuff, etc, etc, but let's face it. The tools make the technology and the tools (and the examples and the advice you get) point at RPC. And we know what the problems with RPC are. If you want to build something that is genuinely loosely-coupled, RPC is a pretty hard path to take.

That realization would have gotten me down if not for the fact that something else jazzed me up an hour or so later. I was in the process of considering the alternatives when I finally understood REST. And wow, it was eye-opening. REST is often positioned as CRUD operations against entities identified by URIs. Then it is dismissed as to simplistic to be useful. You can't build with just CRUD, the reasoning goes, just think about why we write sprocs. I've been down that path any number of times and always ended up in the same place. But I had it all wrong.

Tim was a staunch SOAP Web services proponent/evangelist and wrote MSDN's "House of Web Services" column while a Microsoft employee.

Don Box, one of the initial contributors to SOAP 1.0 and a power behind WCF, makes these recommendations for choosing SOAP, REST with POX (plain old XML), or both in his February 17, 2006 Pragmatics post:

1. If you want a great experience for .NET/Java devs, you’ll typically publish schemas (through WSDL) and support SOAP.
2. If you want a great experience for LAMP folks, you’ll support POX messages and will provide a non-XSD description of your formats.
3. If you want to reach both audiences, you’ll do both #1 and #2.
4. If you want to reach both audiences before your competition does, you'll avoid indulging in religious debates and ship something.

Don and Steve Maine presented DEV03 - Navigating the Programmable Web at MIX07. Here's the description from the MIX07 sessions page:

Abstract: RSS. ATOM. JSON. POX. REST. WS-*. What are all these terms, and how do they impact the daily life of a developer trying to navigate today's programmable Web? Join us as we explore how to consume and create Web services using a variety of different formats and protocols. Using popular services (Flickr, GData, and Amazon S3) as case studies, we look at what it takes to program against these services using the Microsoft platform today and how that will change in the future.

It's clear that the SOAP folks understand the need for RESTful Web services. Following are some highlights from the session's slides:

• It's the URI, Stupid! Query, Structured Expression, or Opaque Identifier. Orcas provides System.UriTemplate to support URI template syntax.
• GET is more Important than you think. Most of the world is GET; the rest is relative chaos. Orcas adds GET support to WCF, as well POST, PUT, and DELETE.
• WebHttpClient is a speculative feature, "which we hope will show up in an interesting vehicle very soon."
• Formats Matter: POX (Plain Old XML), JSON (JavaScript Object Notation), RSS/ATOM, and SOAP.
• RSS/ATOM constrain POX by adding list structure.
• SOAP constrains POX by adding a header processing model.
• WCF's WebGet/WebInvoke in Orcas support XML, JSON, or opaque binary formats (e.g., JPEG, PNG); WCF 1.0 supported POX and SOAP.
• WCF's SyndicationFeed/SyndicationItem provide a programming model for RSS/ATOM that integrates with the .NET serialization stack.
• Astoria is built on top of the technology presented in the session.
• "We could have put a SOAP wrapper on any of these services, but we didn't. We wanted reach."

The initial Astoria CTP doesn't support RSS/ATOM formats. Obviously, a later CTP should.

James Governor's February 15, 2005 SOAP is boring, wake up Big Vendors or get niched post, says:

Yesterday my partner Stephen issued a wake up call for middleware and tools vendors - give developers what they want, not what you think they should have.

• I have yet to hear any vendor - not one - talk to me about how they help developers design, implement and consume RESTian web services.

I take my hat off to Robert McMillan at Network World for writing this story two years ago…

One big question is why haven’t IBM and Microsoft responded? The obvious answer is vested interest. When you have “bet the company” on a technology stack its kind of a drag to have to respond to something else. Its interesting that in a week when the bug guys, including Gartner, have trumpeted the arrival of UDDI 3.0, the world is quietly getting on with more interesting projects. ...

Whats a web service? Still a great question. But anyone that defines a Web Service using SOAP in the definition is missing out on where the action is. Distinctions between enterprise and “consumer” are breaking down. REST is evidently where that convergence is being played out, not WS-I.

Governor's challenge to IBM and Microsoft raised a flurry of responses. For example, Dare Obasanjo wrote his Understanding the Place of POX, SOAP and WS-* in Building XML Web Services the same day. Be sure to read the comments and responses by Stephen O'Grady, James Governor, XML Team, Brain.Save, and, especialy, Mike Taulty.

Amit Asaravala delivers a reasoned argument for RESTful Web services in his October 21, 2002 Giving SOAP a REST post on DevX:

Many developers will be surprised to learn that SOAP isn't the only game in town for Web services interfacing. REST offers a perfectly good solution for the majority of implementations, with greater flexibility and lower overhead. Developers need to stop reaching immediately for SOAP and start choosing the right technology for the application.

Joel Spolsky says in his April 25, 2002 Joel on Software post about "Soap Backlash":

The real problem with SOAP is that it's a completely inadequate remoting system. It doesn't have events (which makes it useless for large classes of applications). It doesn't support references (ditto). It has about 10 years to go before it gets to where CORBA and DCOM were 10 years ago. And we're supposed to be all excited about this because we can sneak through the firewall. Gee, I wonder what the firewall vendors are working on these days? As soon as they're done, we're even further back than we started.

REST and SOAP Background

Roy T. Fielding, whom Jon Udell calls "the primary architect of HTTP" and who's a co-founder of the Apache Software Foundation, is credited with the definition of the Representational State Transfer (REST) architectural style in his University of California Irvine doctoral dissertation, Architectural Styles and the Design of Network-based Software Architectures (2000). Listen to Jon's August 25, 2006 podcast, A conversation with Roy Fielding about HTTP, REST, WebDAV, JSR 170, and Waka.

"Project Astoria," introduced at MIX07 as an incubator project by Pablo Castro in his XD006 - Accessing Data Services in the Cloud session, appears to me to be Microsoft's first semi-official foray into RESTful Web services.

My January 31, 2003 "Soap Extension Soup" article for XML & Web Services Online discussed the flurry of WS* specifications foisted on the Web services development community in 2002H2. It took the moribund Web Services Interoperability Organization (ws-i.org) three years to come up with their Basic Security Profile 1.0 recommendation. (OASIS published the Web Services Security v1.0 spec in March 2004.) I was overly optimistic in saying:

If OASIS produces a final WS-Security spec by the end of 2003, WS-I could easily take another year to produce a WS-Security 1.0 profile, test tools, and PKI-based sample applications. [Emphasis added]

Conclusion: Entity Framework clearly has the potential to become another SOAP. Hopefully, Microsoft's ADO.NET team will avoid that trap.

Technorati tags: LINQ, LINQ to SQL, Entity Framework, REST, SOAP, Entity Data Model, LINQ to Entities, Entity SQL, eSQL