Link and Entity Framework Posts for 12/31/2007+

Happy New Year! This post is updated daily or more often, depending on availability of articles.

Ben Hall Describes Inserting LINQ to SQL Data with Stored Procedures

Others, including Scott Guthrie, have explained how to substitute stored procedures for dynamic SQL when performing INSERT, UPDATE, and DELETE operations with LINQ to SQL. In his January 4, 2008 Custom Insert logic with Linq to SQL post, Ben explains the process for the INSERT method. It's reasonable to expect that posts for UPDATE and DELETE will follow.

Added: 1/5/2008

David Hayden Adds Custom Dynamic Data Fields to His Tutorial

ASP.NET Dynamic Data Websites - Creating Custom Dynamic Data Fields of January 4, 2008 is the fourth member of David's series. (See the "David Hayden Extends His ADO.NET Dynamic Data Validation Tutorial" topic below for previous members.) A standard set of default dynamic data controls is present by default; David adds the AJAX Control Toolkit's CalendarExtender to enter or edit data values.

Added: 1/4/2008

Danny Simmons Updates Entity Framework FAQ to Version 0.3

Danny's Entity Framework FAQ (Version 0.3 of 1/4/2008) has nine new entries:

• 7.5 Does the EDM support guids?
• 7.6 Does the EDM/EF support enums?
• 10.3 Can you use a guid property as part of an entity key?
• 10.4 Can I use a server-generated guid as my entity key?
• 18.2 How can I do a filtered load of a relationship?
• 18.3 How can I see the SQL which will be sent to the backend database as a result of my query?
• 20.2 Is it possible to serialize a graph of related entities? Why am I only getting one entity at a time when I do XML serialization?
• 20.3 Is it possible to serialize the ObjectContext?
• 20.4 How do I serialize derived entity types over a WCF service with DataContract serialization?

The additions to sections 18 and 20 clarify significant issues and are very welcome. (See Controlling the Depth and Order of EntitySets for 1:Many Associations of December 20, 2008 for more details.)

Added: 1/4/2008

2008 Predictions for LINQ, Entity Framework, et al. in Redmond Developer News

The cover story for the January 2008 issue of Redmond Developer News is "Peering into the Future: Experts predict what developers should expect in 2008" by founding editor Michael Desmond. Contributors include Mary Jo Foley, Rockford Lhotka, Jim Payne, Peter Varhol, Johannes Ullrich, William F. Zachmann, and me.

I read the tea leaves for Silverlight 2.0, ASP.NET Data Services, SubSonic, Volta, the "LINQ Blizzard," ADO.NET Data Services, the Dynamic Language Runtime (DLR). If you're interested in these topics primarily, ^F for jennings.

January 1, 2008 is my first anniversary writing for 1105 Media's Redmond Media Group (RMG), who publish Visual Studio Magazine, Redmond Magazine, Redmond Channel Partner, and Redmond Developer News. In honor of the "event," I've posted an updated Visual Studio Magazine Article Retrospective, which displays four years of covers based on stories I've written for VSM.

Added: 1/4/2008

The Exposing Arbitrary Data [from ANDS] Quartet by Mike Taulty

Mike tests ADO.NET Data Services (ANDS) with the Entity Framework and SQL Server as well as in-memory objects as data sources in these four posts of January 3, 2008:

ADO.NET Data Services - Exposing Arbitrary Data (1) follows the creation of two services (EFService.svc and MemoryService.svc) and use of WebDataGen.exe to create client proxies for the services. As was the case with LINQ to SQL as the data source, you must hack the classes for MemoryService to add missing associations.

ADO.NET Data Services - Exposing Arbitrary Data (2) describes how to implement IUpdatable.Add(), .SaveChanges(), .Attach(), and AddResourceToCollection() for inserting data into MemoryService.

ADO.NET Data Services - Exposing Arbitrary Data (3) provides code to delete an author and associated books with IUpdatable.Delete() on the BookContext class.

ADO.NET Data Services - Exposing Arbitrary Data (4) shows code to update an author and move a book from one author to another.

Mike concludes that IUpdatable needs documentation. (I agree). The examples would have been more interesting if Mike had included unit tests in the code.

If you're interested in duplicating Mike's tests with more complex (i.e., Northwind) in-memory objects, download the JSON vs. XML DataContract Serialization Test Harness, which includes classes and object initializers for 20 Order objects and their associated Customer and Order Details objects.

Added: 1/3/2008

Guy Burstein Joins Microsoft Israel's Developer & Platform Adoption Group

His I Started Working at Microsoft post of January 3, 2008 says:

In my new job, I will do what I like to do most, which is to keep the Israeli developers community updated with Microsoft's latest technologies, and help customers to choose the best of Microsoft has to offer.

Hopefully, Guy will continue his LINQ and Astoria-related blogging.

Added: 1/3/2008

Ben Hall Reviews the LINQ Quickly Book

Ben's Linq Quickly: Book Review of January 2, 2008 seems to me to damn the book by faint praise. Ben notes that the book covers LINQ to XSD but his review doesn't mention that LINQ to XSD development stopped at Orcas Beta 1 (see LINQ to XSD Redux and LINQ to Stored XML Coming of December 2, 2007.)

Ben picked up the same .NET Framework version number error (3.0 instead of 3.5) that I did in "Book Review: LINQ Quickly (Packt Publishing)" from LINQ and Entity Framework Posts for 12/17/2007+.

In my view, the book should have been titled LINQ Written Quickly.

Added: 1/3/2008

Marcelo Lopez Ruiz Posts Simple ADO.NET Data Services Hosting Code

If you prefer self-hosting an ADO.NET Data Service on a fixed TCP port instead of using the Cassini Web server, Marcelo's Fast, simple test app for ADO.NET Data Services post of January 2, 2008 offers a minimalist approach to exposing in-memory data with a C# console app.

Added: 1/3/2008

Mike Taulty Discovers that NOT NULL Foreign Key Fields Prevent ADO.NET Data Service POST Operations

The Northwind sample database's religious avoidance of applying the NOT NULL attribute to foreign key fields points out a problem (bug?) in .NET Data Services' December CTP.

Mike's ADO.NET Data Services - Inserting Related Entities post of January 2, 2008 demonstrates that you can't execute POST (INSERT) operations with objects that have associated (child) objects and the NOT NULL attribute applied to the foreign key value for the child object when you use LINQ to SQL as the data source.

Failure to provide an appropriate foreign key value violates relational integrity rules and should throw an error for a missing foreign key value.

If you apply the NOT NULL attribute to the foreign key, you receive a server-side error when attempting the POST. Mike says this occurs because he believes the framework:

1. Insert[s] the book
2. Update[s] the book to set the author id

However, this sequence should throw an error because step 1 violates referential integrity. The QuickStart example, which uses Product and Category objects, appears to insert a new Product object without a problem.

Something strange seems to be going on here.

Added: 1/2/2008

Mike Taulty Discovers You Can't Trace ADO.NET Data Services' Messages

Because Astoria uses HttpWebRequest directly, not via WCF and HTTP tracing with proxies isn't reliable.

So Mike shows you in his January 2 ADO.NET Data Services - Tracing Messages from Client to Service post how to create a partial class to add a Debugger.Break() to the OnSavingChanges event handler to enable inspecting the Object Context's ObjectStateManager at this point.

Added: 1/2/2008

Marcello Lopez Ruiz Explains How to Intercept HTTP Internal Server Errors from ADO.NET Data Services

Andy Conrad exposed another ADO.NET "stealth blogger" in his Server errors and the Astoria client of New Years' Eve. This time he's Marcelo Lopez Ruiz, who started blogging in November 2004 when he was an Avalon Test Lead. He joined the ADO.NET team in September 2006 and moved to Project Astoria in July 2007. Marcello's first ADO.NET Data Services blog with sample code is Getting ADO.NET Data Service error details on client of December 31, 2007.

Marcello shows you how to write a simple console app that uses the WebException in conjunction with HttpWebResponse class. Andy says:

One thing I regret that we didn't get into the Astoria December CTP was the ability to show detailed Astoria server errors to remote clients.  Something like what ASP.NET supports.  That way the developer can see what errors occurred on the server via the web browser.  Currently, one has to have a debugger attached to the server process to get the equivalent information.  This is particularity interesting in cases where the Astoria server code is not the component throwing the exception - for example when the Queryable provider throws an exception.

In addition, we are also looking at adding a way for providers to map specific exceptions to HTTP errors which could be used by provider writers to specify runtime HTTP errors instead of the generic "Internal Server Error".  For example, if there was a database constraint error when inserting some data.  Hopefully, this feature will make it into the next beta.

Added: 1/1/2008

David Hayden Extends His ADO.NET Dynamic Data Validation Tutorial

In a New Year's Day ASP.NET Dynamic Data and Validation Using LINQ To SQL OnPropertyChanging Partial Methods post, Dave describes how to take advantage of LINQ to SQL's OnPropertyChanging partial methods for data validations. His earlier posts on this topic include:

• ASP.NET Dynamic Data Website Getting Started - ASP.NET RAD Development for Data-Driven Websites (see below)
• ASP.NET Dynamic Data Websites - Customizing Pages and Validation

Added: 1/1/2008

Converting SQL to LINQ, Part 6: Joins (Bill Horst)

The sixth in the Converting SQL to LINQ series, posted December 31, 2007, tackles LINQ's VB 9.0 expression syntax for cross (Cartesian), inner, natural, and outer joins.

Jonathan Carter Continues His 11-Part ADO.NET Data Services Series

Jon has reached the seventh in his ADO.NET Data Services Training Series that began on December 12, 2007. Jon's a Microsoft evangelist for the .NET Framework and Visual Studio. In an earlier avatar, he was a writer for numerous magazines, so his episodes are more like stories than tutorials. The protagonist is ...

Phillip, who runs a cell phone resale company, and is looking to ramp up his business. Phillip has an extensive database of cell phones, carriers, and manufacturers. Lots of time and effort has been put into the population and maintenance of his precious data. One day he realizes that all that time he put into his database should be making him more money, so he decides to provide access to his data store to other companies looking to build a web presence but don’t have the resources Phillip has.

Here are the "chapters," with links to those complete as of the latest update:

ADO.NET Data Services Part 1: Services (December 12, 2007)

ADO.NET Data Services Part 1: Services (contd.) (December 12, 2007)

ADO.NET Data Services Part 2: Service Addressing (December 13, 2007)

ADO.NET Data Services Part 3: Query Parameters (December 14, 2007)

ADO.NET Data Services Part 4: Clients (December 15, 2007)

ADO.NET Data Services Part 5: LINQ (December 17, 2007)

ADO.NET Data Services Part 6: AJAX (December 19, 2007)

ADO.NET Data Services Part 7: Data Modification- Client (December 30, 2007)

ADO.NET Data Services Part 8: Service Operations

ADO.NET Data Services Part 9: Interceptors

ADO.NET Data Services Part 10: Open Objects

ADO.NET Data Services Part 11: Service Hosting

Added and Updated: 1/1/2008

David Hayden on Customizing ASP.NET Dynamic Data Web Sites

In his ASP.NET Dynamic Data Websites - Customizing Pages and Validation post of December 30, 2007, David shows you how Page Templates work, how to customize a ListDetails page, and how validate changes to LINQ to SQL objects' property values.

Free Introducing Microsoft LINQ eBook from Microsoft Press

Introducing Microsoft LINQ is one of the following three eBooks that Microsoft Press is offering as a no-charge dowload:

• Introducing Microsoft LINQ by Paolo Pialorsi and Marco Russo (ISBN: 9780735623910)
  This practical guide covers Language Integrated Query (LINQ) syntax fundamentals, LINQ to ADO.NET, and LINQ to XML. The e-book includes the entire contents of this printed book!
• Introducing Microsoft ASP.NET AJAX by Dino Esposito (ISBN: 9780735624139)
  Learn about the February 2007 release of ASP.NET AJAX Extensions 1.0, including an overview and the control toolkit.
• Introducing Microsoft Silverlight 1.0 by Laurence Moroney (ISBN: 9780735625396)
  Learn how to use Silverlight to simplify the way you implement compelling user experiences for the Web. Discover how to support an object-oriented program model with JavaScript.      

Bart de Smet: "Essential LINQ to Objects" in a 19:29 Video Segment

Bart gives a rundown of his early A custom implementation of the .NET Standard Query Operators from LINQ CodePlex project, which implements all LINQ Standard Query Operators (SQOs) in C# 3.0 code with this 20-minute video segment, Essential LINQ to Objects produced for Microsoft BeLux (requires registration.)

Note: The Silverlight video wouldn't play for me with Windows Server 2003 or Vista on 12/31/2007. I'll remove this note when/if the problem goes away.

Pablo Castro Discusses the Idempotence of .NET Data Services

Wikipedia defines idempotence as describing "the property of operations in mathematics and computer science that yield the same result after the operation is applied multiple times." Idempotence becomes an issue in loosely coupled systems that insert data, such as a SOAP or REST Web service that adds a new row to a table by an Insert method or POST operation.

If the underlying tables have identity columns as their primary key, such as the Northwind Orders, Employees, Products, Suppliers, and Shippers tables, they fail the idempotence test because a client that repeats the Insert invocation or POST operation for some reason inserts a new row for each repetition.

A table that relies on the client to supply the primary key value (such as Customers) can be idempotent if server-side business logic or the RDBMS itself refuses repeated updates of the same row and doesn't modify the table in the process.

Pablo's brief Idempotence on HTTP operations post of December 22, 2007 describes the idempotence of .NET Data Services' GET (SELECT), PUT (UPDATE), POST (INSERT) and DELETE operations, as well as its dependence on business rules or the underlying RDBMS's table configuration.

Note: Missed in the December 20+, 2007 post.

Rob Conery Compares NHibernate, LINQ to SQL, and Subsonic as O/RM Tools

Rob's ASP.NET MVC: Choosing Your Data Access Method post of December 14, 2007 is an admittedly biased comparison of the pro, cons and learning curves for three free data access and object persistence tools, two of which (NHibernate and Subsonic) are open source.

The primary topics of the comparison are:

• Domain objects, entities and data access patterns (Active Record, Repository, or Data Broker
• Application and its O/RM tool lifetimes (will support for the O/RM tool be available through the application's full lifecycle)
• Product "vision and approach" or "Why they made it?"
• Performance
• Ease of use

The post's 32 comments are from NHibernate and Subsonic proponents primarily and are well worth reading. I agree with one commenter (Elmar) that Entity Framework/Entity SQL/LINQ to Entities is more comparable to NHibernate than LINQ to SQL.

Now that ASP.NET MVC is out the door, hopefully Rob will have time to work on LINQ to Subsonic. (See SubSonic will be the "Convention-Driven Toolset for Microsoft’s New MVC Framework" for ASP.NET of November 11, 2007 and ASP.NET 3.5 Extensions Preview Released for more background on LINQ, MVC and Subsonic.)

Added: 1/1/2008; missed in the December 13+, 2007 post.

Technorati Tags: ADO.NET Data Services,Astoria Project,LINQ,LINQ to Objects,LINQ to SQL,Idempotence,ASP.NET Dynamic Data,Subsonic,NHibernate

JSON vs. XML DataContract Serialization: Download Test Harness

Rick Strahl's DataContractJsonSerializer in .NET 3.5 post of December 29, 2007 describes the .NET Framework 3.5's new DataContractJsonSerializer (DCJS) class from the System.Runtime.Serialization.Json.DataContractJsonSerializer namespace, which contains the serializer and related objects for the lightweight JavaScript Object Notation (JSON) transport (RFC 4627). JSON is one of ADO.NET Data Service's two wire formats, as noted in the "Julie Lerman: Astoria to Become ADO.NET Data Services" topic of Link and Entity Framework Posts for 12/10/2007+. (The Astoria Team intends to add plain old XML (POX) format to v1.0 by RTW.)

Quick Summary: JSON serialization with DCJS requires about 25% to about 40% fewer bytes than the XML DataContractSerializer (DCS) class to represent moderately complex objects, such as Northwind Order and their their associated Order_Detail objects, and takes about 25% longer than the DCS class to serialize and deserialize.

However, Windows Communication Foundation (WCF) serializes JSON streams to an internal XML Infoset wire format that requires about 50% more bytes than DCS to hydrate an object across a process boundary. Adding type="JSONDataType" attributes to each member's element contributes the additional bytes.

The added time to serialize a moderate-sized object collection and the dramatic increase in the size of WCF messages indicates that JSON serialization isn't as lightweight as one would expect from its simple syntax.

You can download the VB 9.0 code for the JsonSerializationWinVB.sln test harness project  here. The project requires Visual Basic 2008 Express or Visual Studio 2008 Standard or higher but doesn't need a database connection. (Mock generic data objects are provided.)

The Test Harness

Rick's sample code uses simple, lightweight objects and I wanted to verify whether the DCJS class actually was lighter in weight and as performant or better than the .NET Fx 3.0's DataContractSerializer (DCS) class with more complex classes containing a variety of data types, including Nullable<T>, and generic collections. So I created the following two classes with the LINQ In-Memory Object Generation (LIMOG) utility that's described in my Serializing Object Graphs Without and With References post of November 21, 2007.

Here are C# versions of the class definitions for the Order and Order_Detail objects used in the tests:

[DataContract(Name = "Order", Namespace = "")] 
public class Order { 
    [DataMember(Name = "OrderID", Order = 1)] 
    public int OrderID { get; set; } 
    [DataMember(Name = "CustomerID", Order = 2)] 
    public string CustomerID { get; set; } 
    [DataMember(Name = "EmployeeID", Order = 3)] 
    public int? EmployeeID { get; set; } 
    [DataMember(Name = "OrderDate", Order = 4)] 
    public DateTime? OrderDate { get; set; } 
    [DataMember(Name = "RequiredDate", Order = 5)] 
    public DateTime? RequiredDate { get; set; } 
    [DataMember(Name = "ShippedDate", Order = 6)] 
    public DateTime? ShippedDate { get; set; } 
    [DataMember(Name = "ShipVia", Order = 7)] 
    public int? ShipVia { get; set; } 
    [DataMember(Name = "Freight", Order = 8)] 
    public decimal? Freight { get; set; } 
    [DataMember(Name = "ShipName", Order = 9)] 
    public string ShipName { get; set; } 
    [DataMember(Name = "ShipAddress", Order = 10)] 
    public string ShipAddress { get; set; } 
    [DataMember(Name = "ShipCity", Order = 11)] 
    public string ShipCity { get; set; } 
    [DataMember(Name = "ShipRegion", Order = 12)] 
    public string ShipRegion { get; set; } 
    [DataMember(Name = "ShipPostalCode", Order = 13)] 
    public string ShipPostalCode { get; set; } 
    [DataMember(Name = "ShipCountry", Order = 14)] 
    public string ShipCountry { get; set; } 
    [DataMember(Name = "Order_Details", Order = 15)] 
    public List<Order_Detail> Order_Details { get; set; } 
} 

[DataContract(Name = "Order_Detail", Namespace = "")]
public class Order_Detail {
    [DataMember(Name = "OrderID", Order = 1)]
    public int OrderID { get; set; }
    [DataMember(Name = "ProductID", Order = 2)]
    public int ProductID { get; set; }
    [DataMember(Name = "UnitPrice", Order = 3)]
    public decimal UnitPrice { get; set; }
    [DataMember(Name = "Quantity", Order = 4)]
    public short Quantity { get; set; }
    [DataMember(Name = "Discount", Order = 5)]
    public float Discount { get; set; }
}

LIMOG adds the Name, Namespace, and Order attribute/value pairs; DCJS processes Name and Order attribute values and ignores Namespace values. I wrote custom SQL commands to generate object initializers for Northwind's last 20 Orders rows with their associated Order Details and Customer rows. A few of the orders are missing Nullable<DateTime> ShippedDate values.

Following are the test harness's four operating modes. Click the image for a full-size capture.

1. JSON (DCJS) Serialization with Parent (Order) Objects Only 

Representative deserialized values appear below the column headers.

2. JSON (DCJS) Serialization with Parent and Child  (Order_Detail) Objects

LINQ queries populate the 1:many Order_Details associations.

DateTime values serialize as "/Date(MillsecondsSince1970-01-01T00:00:00.000 +/- HHMM)/" strings, where HHMM is the offset from GMT in hours and minutes. (RFC 4627 doesn't specify a serialization format for date/time values, which has resulted in free-form encoding by various implementers. Stand-Alone JSON Serialization covers the current .NET implementation for ASP.NET AJAX services created in WCF.)

3. XML (DCS) Serialization with Parent (Order) Objects Only

4. XML (DCS) Serialization with Parent and Child (Order_Detail) Objects

XML Infoset Wire Format for WCF Implementations: Parent (Order) Objects

WCF implementations serialize JSON streams to a "internal" XML Infoset format defined by the Mapping Between JSON and XML white paper. The test harness uses the JsonReaderWriterFactory's CreateJsonReader method to instantiate an XmlDictionaryReader object whose ReadOuterXml method delivers the wire format as a string.

The semi-formatted text of the Infoset follows the report of its length below the deserialized data:

Note: I could find no documentation for the JsonReaderWriterFactory's CreateJsonReader method or creating the required XmlDictionaryReader object; I believe that this project contains the only publicly available example. (Search Google for CreateJsonReader.) Here's an excerpt from the test harness code that's based in part on Rick Strahl's example:

Dim ser As DataContractJsonSerializer = _ 
   New DataContractJsonSerializer(GetType(List(Of Order))) 
Dim ms1 As MemoryStream = New MemoryStream() 
ser.WriteObject(ms1, lstOrders) 
Dim json As String = Encoding.UTF8.GetString(ms1.ToArray()) 
txtBytes.Text = json.Length().ToString() 
ms1.Close() 
Dim ms2 As MemoryStream = New MemoryStream(Encoding.UTF8.GetBytes(json)) 
' Deserialize into generic List 
ser = New DataContractJsonSerializer(GetType(List(Of Order))) 
Dim jsonOrders As List(Of Order) = _ 
    TryCast(ser.ReadObject(ms2), List(Of Order)) 
' Serialize JSON MemoryStream to WCS XML wire format 
Dim xdrJson As XmlDictionaryReader = _ 
    JsonReaderWriterFactory.CreateJsonReader(ms2, _ 
    XmlDictionaryReaderQuotas.Max) 
xdrJson.Read() 
Dim xml As String = xdrJson.ReadOuterXml() 
xdrJson.Close() 
ms2.Close() 

XML Infoset Wire Format for WCF Implementations: Parent and Child (Order_Detail) Objects

As Rick mentioned in a comment to his post:

@Roger - awesome work checking out the perf considerations. I suppose it's to be expected that JSON is slower since the parsing of the JSON objects is necessarily a bit more complex than parsing the more XML so I'm not terribly surprised that it's slower than XML.

OTOH, you're not likely to use this on two way WCF communications, but only for AJAX/REST scenarios coming from a browser most likely and in that scenario there's not much choice since JSON is so much easier to use on the client than XML.

Atom10FeedFormatter<TSyndicationFeed> and Atom10ItemFormatter<TSyndicationItem> generic objects (for classes derived from SyndicationFeed) are likely to be used for two-way WCF communications with ADO.NET data services. Magnus Mårtensson shows you in his Create your own Syndication Feeds with .NET Framework 3.5 post of November 22, 2007 how to use these two objects (and their RSS 2.0 equivalents) to create and serialize SyndicationFeeds. Guy Burstein offers a similar demonstration in his How To: Create a RSS Syndication Service with WCF post of December 3, 2007.

Added: 1/1/2008

Size of Serialized Messages and Serialize/Deserialize Execution Times

Following is a summary table of the data collected for the four operating modes. Message sizes don't include whitespace. Execution times are the average of five tests conducted by reopening the form for each execution (to eliminate the effects of data caching.)

		Parent			Child
Serializer	Time, s.	Size, Bytes	XML, Bytes	Time, s.	Size, Bytes	XML, Bytes
JDCS	0.416	7,559	14,942	0.467	13,253	31,253
DCS	0.352	9,955	N/A	0.412	21,444	N/A

The preceding data was for a Gateway S-5200D with a dual-core Pentium 2.6 GHz processor running Windows Vista Premium as a virtual client with 1GB RAM assigned under a Window 2005 R2 Virtual Server.

Download the VB 9.0 source code for the JsonSerializationWinVB.sln test harness project  here and give it a try on your development machine.

Updated 12/31/2007: Minor edits and additions

Technorati Tags: JSON,JsonDataContractSerializer,XML,DataContract Serialization,AJAX,Serialization,ADO.NET Data Services,Project Astoria,LINQ,LINQ to Objects,Mock Data

LINQ and Entity Framework Posts for 12/24/2007+

Note: Happy New Year! There probably won't be many updates to this linkblog post.

JSON vs. XML DataContract Serialization: Downloadable Test Harness

This VB 9.0 test harness compares the message size and performance of moderately complex objects serialized and deserialized by the NET Framework 3.5's new DataContractJsonSerializer (DCJS) class and the .NET Fx 3.0's XML-based DataContractSerializer (DCS) class.

Here's a screen capture of the test harness's second mode, which displays the formattted JSON string for Northwind Order and Order_Detail objects:

Click the image for a full-size screen capture.

You can download the VB 9.0 code for the JsonSerializationWinVB.sln test harness project  here. The project requires Visual Basic 2008 Express or Visual Studio 2008 Standard or higher but doesn't need a database connection. (Mock generic data objects are provided.) The code is based on Rick Strahl's original version described below.

Rick Strahl Dissects .NET Framework 3.5's JSON Serializer

Rick's DataContractJsonSerializer in .NET 3.5 post of December 29, 2007 delves into the System.Runtime.Serialization.Json.DataContractJsonSerializer namespace, which contains the serializer and related objects for the lightweight JavaScript Object Notation (JSON) transport (RFC 4627). Rick observes that this serializer produces "Plain Old JSON" (POJ?) text without the proprietary __type="TypeName" attribute that ASP.NET 2.0 AJAX Extensions introduced.

To run Rick's code, you must add references to System.Runtime.Serialization (v3.0 for WCF) and System.Service.Model.Web (v3.5 for JSON), and the following directives:

using System.Runtime.Serialization.Json;
using System.Collections.Generic;
using System.IO;
using System.Text;

Add a using System.Runtime.Serialization; directive if you want to decorate classes with [DataContract] instead of [Serializable] attributes and add the [DataMember] attribute to public fields or properties you want to serialize.

Added: 12/30/2007

Paul Yuknewicz: ASP.NET LinqDataSource Guided Tour

The VB Team's LINQ for the Web Using VB (By Paul Yuknewicz) post provides an illustrated guided tour of the LinqDataSource component and LINQ to SQL behind a simple ASP.NET 3.5 Web site. The post is similar to Scott Guthrie's LINQ to SQL: Part 5 - Binding UI using the ASP.NET LinqDataSource Control post of July 16, 2007 except that VB substitutes for C#.

Added: 12/28/2007

Paul Vick: Visual Basic 9.0 Spec (Almost) Cooked

Paul Vick's (Almost) final VB 9.0 language specification posted article of December 27, 2007 (almost) says it all. You can download the latest edition (sans final copy edits) here. According to Paul, the spec covers the following new VB 9.0 features:

• Friend assemblies (InternalsVisibleTo)
• Relaxed delegates
• Local type inferencing
• Anonymous types
• Extension methods
• Nullable types
• Ternary operator
• Query expressions
• Object initializers
• Expression trees
• Lambda expressions
• Generic type inferencing
• Partial methods
• XML Members
• XML Literals
• XML Namespaces

Added: 12/27/2007

.NET Rocks: Joe Duffy on the Parallel Extensions and PLINQ

Carl and Richard interview Joe Duffy, the PM behind the Task Parallel Library (TPL), Parallel Extensions for .NET (PFX), and Parallel LINQ (PLINQ), about parallel (concurrent) programming and multi-core processors in Joe Duffy on the Task Parallel Library of December 24, 2007.  See the "Parallel Extensions" topics in LINQ and Entity Framework Posts for 11/26/2007+ and my PLINQ Gathers Momentum post of December 8, 2006 for more background.

Note: The PFX Team now has a Parallel Programming with .NET blog that has several posts on the use of PLINQ. Links to this blog will be added to future posts.

Added: 12/27/2007

Mike Taulty Discovers that WebDataGen.exe Doesn't Recognize LINQ to SQL Associations

Mike's ADO.NET Data Services - LINQ to SQL and Associations post of December 24, 2007 points out that when you run WebDataGen.exe against the URL of a .NET Data Source based on LINQ to SQL associations are ignored. The metadata that exposes associations as a Property of the String data type. Using an Entity Framework data source designates associations with a typed NavigationProperty and a corresponding Association element group.

This obviously prevents writing LINQ to REST queries that traverse associations. Defining LINQ to SQL entities with hand-written classes, as outlined in Andy Conrad's Linq to REST post of December 10, 2007, enables associations. You can then substitute LINQ queries, such as

var query2 = from c in context.CreateQuery<Categories>("Categories")
             where c.CategoryID == 1
             select new { c.CategoryID, c.CategoryName, c.Products };

for WebDataQueries like

WebDataQuery<Categories> query =
    context.CreateQuery<Categories>("/Categories(1)?$expand=Products");

However, I've found that any attempt to filter collections with Where criteria based on child member values, such as

var query = from c in context.CreateQuery<Customer>("Customers")
            where c.Country == "USA"
            && c.Orders.Any(o => o.Freight > 50)
            orderby c.CompanyName
            select c;

fail with a 400 - Bad Request error because the query string isn't composed properly. (Commenting the Freight amount test enables the query to execute.)

It's not clear to me how restricted LINQ to REST query syntax will be at RTM. It's likely that many Astoria users will load large chunks of data into memory-resident IEnumerable() sequences (probably generic Lists) and query them with client side LINQ to Objects expressions.

Mike Taulty Suggests Cure for Error 7005: 'Namespace' Attribute Is Invalid

The solution in Mike's ADO.NET Data Services and CLR Namespaces post of December 24, 2007 is to "put the types that you're exposing (i.e. the T that you have fed into WebDataService<T>) into a CLR namespace."

So far, I haven't encountered that exception.

A Christmas Present from Milan Negovan: A LINQ Cheat Sheet

You can read and download the two-page Cheat Sheet for Standard Query Operators (method call syntax) at Milan's ASP.NET Resources site.

Mike Flasko Proposes Eliminating Parent Key Duplication with Deep Addressing

Mike Flasko's Design Notes: URI Containment in Astoria post of December 21, 2007 (missing from previous linkblog post) observers that parent keys are duplicated in following case b:

a) entities are always addressable through top-level entitysets, e.g. /OrderLines(1,2)

b) deep addressing through container has redundant information, e.g. /Orders(1)/OrderLines(1,2)  (the container id, “1”, is repeated)

The plan is to annotate the CSDL and CLR object model to indicate a parent/child relationship to eliminate the repetition.

Technorati Tags: ADO.NET Data Services,Project Astoria,LINQ to REST,LINQ to Objects,LINQ to SQL,LinqDataSource,LINQ,PLINQ,Task Parallel Library,TPL,Parallex Extensions for .NET,PFX,JSON,XML,DataContract Serialization,JsonDataContractSerializer,Serialization

Two Out of Three Connect Bug Reports for VS 2008 Unacknowledged After Two Months

Joe Wilcox's Microsoft's 12 Gifts of Christmas of December 21, 2007 began with:

12. Connect 3.0 beta. The list begins with a little lump of coal in the Christmas stocking. Microsoft is testing a new Connect Web site. It's time for the Microsoft beta tester revolt, because the new site is butt-ugly. No thanks, Santa!

I used Microsoft Connect extensively during the VS 2005 and 2008 beta programs so I thought I'd give the new design the once-over. I agree with Joe that the home page's design is ugly, but you aren't forced into a search for duplicate reports opening the bug report and I see some usability advantages in the dashboard's right-hand list. Here's a screen capture of what came up for me:

Click image for full-size screen capture.

What struck me (other than the similarity of the orange hues to the OakLeaf site's) when I looked closely at the dashboard was my last three LINQ to SQL bug reports of mid-October that appeared in the Your Watchlist column. I hadn't checked on the bug reports for a few weeks, so I thought I'd see what the VS 2008 Team's response was:

• 306378 of 10/23/2007: Deleting a LINQ to SQL EntitySet Member in a Bound DataGridView Orphans Table Records was still active with no response from Microsoft.
• 305828 of 10/19/2007: Change to LINQ to SQL EntitySet Member Doesn't Mark Root Object Dirty was closed as by design on 10/23/2007 by Alex Turner, Program Manager, Visual C#.
• 304732 of 10/15/2007: LINQ to SQL's GetOriginalEntityState Doesn't Include EntitySet or Entity Ref Data for Related Entities, which included a complete test harness to reproduce the bug, was still active with no response from Microsoft.

Despite the closeness of the bug reports to the RTM date, I think failing to respond at all to what seem to me to be two significant bugs is inexcusable.

Note: A suggestion for LINQ to SQL, Provide a DataContext.AllowDataExplosions Property for Eager Loading with Multiple 1:N Joins, I made also was in the Your Suggestions region and Dinesh Kulkarney had responded to it.

A cosmetic makeover to Microsoft Connect won't deliver any significant benefits to developers or Microsoft's development teams. Prompt and detailed responses to bug reports are what make developers participate actively in beta programs.

The Connect folks are stomping ants while the elephants are running overhead.

Technorati Tags: Microsoft Connect, Bug Reports, LINQ to SQL, LINQ, Visual Studio 2008, VS 2008 Beta 2, VS 2008

LINQ and Entity Framework Posts for 12/20/2007+

Note: There are two linkblog posts this week because of the high frequency of relevant posts/stories.

Dare Obasanjo Analyzes Amazon's SimpleDB

Dare Obasanjo weighs in with his Amazon SimpleDB: The Good, the Bad and the Ugly post of December 21, 2007. He likes "Comoditizing hosted services and getting people to think outside the relational database box," dislikes "Eventual Consistency and Data Values are Weakly Typed," and detests "Web Interfaces, that Claim to be RESTful but Aren’t." I agree with Dare on all topics except eventual consistency. GET requests for data updates are atrocious.

My original Amazon Announces Beta of SimpleDB Web Services in the Cloud post has had gone through several updates since it's December 14, 2007 posting.

Matt Sollers Overcomes LINQ to SQL's Challenges and Limitations

Matt's December 21, 2007 Compact_LINQ contribution to CodeProject covers the following topics, some of which have received scant coverage by others:

• Introduction
• LINQ to SQL
• Grounds for Determination (The Sample)
• LINQ and SQL Server Compact Edition
• Working with Enumerations in LINQ to SQL
• Field Subset Challenges with LINQ to SQL
• Change Tracking Challenges with LINQ to SQL
• Disconnected Challenges with LINQ to SQL
• Pre-fetching Challenges with LINQ to SQL
• Other Points of Interest
• Summing Up
• References and Resources

His sample time-tracking project, which uses SQL Server Compact Edition (SSCE) as the LINQ to SQL data source, is available for download.

Added: 12/23/2007

Two Out of Three Connect Bug Reports for VS 2008 Unacknowledged After Two Months

A link to a proposed redesign of the Microsoft Connect bug-tracking service reminded me that the VS 2008 team hadn't replied to two of my three latest bug reports in about two months.

Prompt acknowledgment of and eventual reply to bug reports is far more important than adding color and illustrations to the pages.

Added: 12/23/2007

Danny Simmons Answers My EntityCollection Depth and Serialized EntityReferences Issues

Danny's Filtered association loading and re-creating an entity graph across a web service boundary... post of December 21, 2007 answers the issues I raised in my Controlling the Depth and Order of EntitySets for 1:Many Associations post of December 20, 2007:

He explains that you can control the depth of associated EntitySets by using the EntityCollection.Attach() method with a predicate, as in:

customer.Orders.Attach(customer.Orders.CreateSourceQuery().Where(order => order.Date >= new DateTime(2007, 1, 1)));

as well as with three other approaches. The EntityCollection.CreateSourceQuery Method help topic says:

Returns an object query that when executed returns the same set of objects that exist in the current collection.

but doesn't provide sample code. A search on 'CreateSourceQuery' returns valid hits to online help and Danny's post only, so there doesn't appear to be any sample code for the method, which also has an EntityReference counterpart.

It appears to be possible to take advantage of the ObjectStateManager's stubs to regenerate EntityReferences on the client side of a Web service in a three-step process:

1. Serialize and transport the target and reference entities with their key to the client
2. Deserialize and attach the target entities to the target entities
3. Deserialize and attach the reference entities to the target entities' stubs

Danny's general-purpose container sample will use the above approach.

Thanks for taking the time for such a detailed reply, Danny. I'm in the process of updating my post now.

Added: 12/23/2007

Scott Hunter Catalogs ASP.NET Dynamic Data Features

In his Breakdown of Dynamic Data Features of December 21, 2007, Scott provides a detailed, outline-style listing of the features that ASP.NET delivers. He emphasizes that most Dynamic Data features aren't scaffolding-dependent.

Added: 12/23/2007

Mike Taulty Explores Filtering Across Astoria Associations

His ADO.NET Data Services - Filtering Across Assocations post of 21 December 2007 shows the syntax for filtering many:1 associations between Northwind Customer and Order entities:

WebDataService.svc/Orders?$filter=(Customers/Country)%20eq%20('UK')

With an ASP.NET service and default de-pluralized LINQ to SQL entity names, my corresponding query looks like this:

http://localhost:49218/Northwind.svc/Orders?$filter=(Customer/Country)%20eq%20('UK')

Mike asks "How to navigate the other way foxed me a lot more (is this stuff written down somewhere? :-))."

Here are 1:many association examples from the ADO.NET Data Services' Simple Addressing Scheme for Data with Uniform URLs QuickStarts page:

http://myserver/data.svc/Customers('ALFKI')/Orders?$filter=Active eq true

http://myserver/data.svc/Customers('ALFKI')/Orders?$filter=Active eq true and (year(OrderDate) eq 2007)

Obviously, the preceding queries weren't tested because Northwind.Orders doesn't have an Active field and the year() function throws the following error with LINQ to SQL or EF as the service's data source:

No applicable function found for 'year' at position 1 with the specified arguments. The functions considered are: year(System.DateTime).

However, the following URI filters orders from ALFKI with Freight >  $50:

http://localhost:49218/Northwind.svc/Customers('ALFKI')/Orders?$filter=Freight%20gt%2050

The following returns all Products and their Category for each Supplier:

http://localhost:49218/Northwind.svc/Suppliers?$expand=Products/Category

(Pablo Castro says there's no support for $expand by the LINQ API.)

So far, I haven't been able to make LINQ to REST joins work with either EF or LINQ to SQL service data source. This LINQ expression

var query = from c in context.CreateQuery<Customer>("Customers")
            join o in context.CreateQuery<Order>("Orders")
            on c.CustomerID equals o.CustomerID
            where c.Country == "USA" && o.Freight > 50
            orderby c.CompanyName
            select c;

compiles without complaint, but creates the following (suspicious) query string:

$filter=((ShipCountry)%20eq%20('USA'))%20and%20((Freight)%20gt%20(100))&$orderby=OrderDate

that throws this runtime error:

Can only specify query options (OrderBy, Where, Take, Skip) after last projection.

I'm working in the blind on more complex LINQ to REST queries.

Added: 12/23/2007

Frans Bouma Tackles OrderBy, Aggregates, and GroupBy for LINQ to LLBLGen Pro

In his December 21, 2007 Developing Linq to LLBLGen Pro, part 10 post, Frans continues the description of his quest to LINQ-enable LLBLGen Pro with more Standard Query Operators (SQOs)—OrderBy(), ThenBy()OrderByDescending(), ThenByDescending(), Sum(), Avg(), Min(), Max(), Count(), LongCount(), and GroupBy().

It took three weeks for Frans to get GroupBy() to work with aggregates. Read his article to see why.

Added: 12/21/2007

Marcin Dobosz Describes ASP.NET Dynamic Data Foreign Key Bug and Fix

Marcin's Dynamic Data December preview context caching bug fix post of December 20, 2007 describes a problem relating to updating drop-down lists of foreign key filters and relation columns, and points to a fix for the problem.

Added: 12/21/2007

Guy Burstein and ADO.NET Data Services Update Issue with LINQ to SQL

Guy's ADO.Net Data Services Feedback: DataContextServiceProvider Required for LINQ to SQL support post of December 20, 2007 confirms Mike Taulty's discovery that LINQ to SQL's DataContext must support IUpdatable() in order to update an Astoria data source (see Below). Guy concludes:

Using ADO.Net Entity Framework as the service data context gives you update support for free. If you are using LINQ to SQL, you can implement IUpdatable yourself, as Mike has done here. I think that the product team should consider adding a DataContextServiceProvider class that uses the LINQ to SQL DataContext capabilities for updating the store.

I agree.

Added: 12/21/2007

Controlling the Depth and Order of EntitySets for 1:Many Associations

LINQ to SQL offers the DataLoadOptions.AssociateWith() method, which accepts a lambda expression as its argument that lets you apply LINQ Standard Query Operators to filter and order the members of EntitySets eager-loaded for association properties. If the Entity Framework (EF) has a corresponding feature, I haven't been able to find it.

Another Entity Framework issue that bothers me is the loss of EntityRefs when serializing entities with WCF's DataContractSerializer but the NetDataContractSerializer that can handle cyclic relationships (cycles) isn't interoperable with most other languages and platforms. LINQ to SQL has the same problem but, in my opinion EF requires fully serializable object graphs to play in the enterprise object/relational mapping (O/RM) tool market.

Read Controlling the Depth and Order of EntitySets for 1:Many Associations for more details and enlighten me on how to control EF EntitySet depth and order.

Update 12/23/2007: These issues have been resolved, at least in part. See "Danny Simmons Answers My EntityCollection Depth and Serialized EntityReferences Issues" above.

"Visual Studio 2008 Kicks Off" Quartet in Visual Studio Magazine's January 2008 Issue

Vol. 1 "Visual Studio 2008 Kicks Off"

The cover story for VSM's January 2008 issue is the first of my four articles about VS 2008's debut. Here's the deck:

Visual Studio 2008 is all about integrating new .NET Framework 3.x Web and smart-client technology, increasing developer productivity, and managing the application lifecycle. Find out how VS 2008's greatly expanded feature set and new .NET Fx 3.5 namespaces can boost your career as a professional developer and add to your programming enjoyment.

Vol. 2 "VS 2008 Kicks Off: The MIA Features"

Visual Studio 2008 is an ambitious new release and it includes a slew of new language features and tools that were required to get LINQ up and running by itself and with SQL Server 200x. Of course, not everything planned made it into the product. Here's a description of the elements that were pared back and what their status is now.

Vol 3 "VS 2008 Kicks Off: The Road Ahead"

Microsoft was already well on its way to creating the next version of Visual Studio (code-named "Hawaii") when it released VS 2008. Learn what the future holds for .NET developers.

Vol 4 "VS 2008 Kicks Off: It Comes in Many Shapes and Sizes"

Visual Studio 2008 stock-keeping unit (SKU) names for the Express, Standard, and Professional editions remain the same as for Visual Studio 2005 except for the year. Visual Studio 2005 Team Edition SKUs have been renamed to Visual Studio Team System 2008 Edition Name.

Calling the last member of the quartet an article or volume instead of a sidebar is a stretch but I've always been envious of Lawrence Durrell.

Bill McCarthy and Kathleen Dollard round out the VS 2008 Kicks Off topic with A Guide To VB 2008 and Exploring C# 3.0 articles.

Danny Simmons: An Essay on Data-Centric Web Services with Entity Framework

Part 1, Why are data-centric web services so hard anyway? of December 19, 2007, "begin[s] a discussion about creating web services which are built on top of the Entity Framework and enable the exchange of entity data," analyzes competing requirements (such as the flexibility of service operations and interoperability) and describes tactical problems (like graph serialization and change tracking.)

At present, graph serialization seems to me to be more a strategic morass than a tactical problem.

Part 2, So they're hard, but what if I need them[?] ... of the same date, discusses the challenges of designing and implementing data-centric Web services with the Entity Framework:

1. The traditional SOA / DTO approach.
2. Constrain your scenarios and use standardized operations. 
3. With graph-serialization you could roll-your own SOA system without separate DTOs.
4. General-purpose container object.
5. Automatically generate DTOs from a declarative description of the contract.

I'm eagerly awaiting a concrete implementation of the abstract concept of challenge #4. #5 seems to me to be a long way out.

If you're interested in the issues that confront Web service architects dealing with O/RM tools as the data access layer, don't miss these two posts!

Diego Vega Lists Differences Between the Capabilities of eSQL and LINQ to Entities

In Some differences between ESQL and LINQ to Entities capabilities of December 20, 2007, Diego separates differences into these categories:

• What is almost the same
• What is somewhat different
• What is way different

John Papa's related Choosing an API to Talk to the Entity Framework post of the same date has recommendations for choosing from these options:

1. Entity SQL with Entity Client
2. Entity SQL with Object Services
3. LINQ with Object Services

Zlatko Michailov Continues His Treatise on the Edm.Length() Canonical Function

Zlatko's The Edm.Length() Canonical Function over SqlClient of December 19, 2007 concludes that the SQL Server team needs to add a new string length function to eliminate the conflicts between T-SQL's and EF's definition of string length for varchar strings with trailing spaces.

Mike Taulty Describes Problems with Disabling LINQ to SQL's Optimistic Concurrency

Sometime you might want to enable "last entry wins" (i.e., no concurrency management) for a LINQ to SQL projects. Mike's LINQ to SQL - Tailoring the Mapping at Runtime post of December 20, 2007 shows just how difficult that can be.

Erick Thompson Contrasts Two Packaging Namespaces

In What is the Difference Between the System.IO.Packaging and Microsoft.Office.DocumentFormat.OpenXml.Packaging Namespaces? of December 20, 2007, Erick explains:

There are two separate namespaces, each containing a number of classes, which you can use to open and get at the contents of Open XML documents.

Erick notes that objects from the OpenXml.Packaging namespace are strongly typed and require less code than those of System.IO.Packaging.

Guy Burstein Continues His ADO.NET Data Services Series

ADO.Net Data Services Part 3 - Querying a Web Data Service and WebDataGen.exe of December 19, 2007 shows you how to build a WCF client to query the Web Services built in ADO.Net Data Services Part 2 - Using Service Operations with WebGet.

Technorati Tags: Entity Framework, Entity Framework Beta 3, LINQ, LINQ to SQL, LINQ to XML, LINQ to Rest, OOXML, Web Services, SOA, WCF, Windows Communication Foundation, Serialization, ADO.NET Data Services, Astoria Project, LINQ to LLBLGen Pro, ASP.NET Dynamic Data, SQL Server Compact Edition, SSCE

Controlling the Depth and Order of EntitySets for 1:Many Associations

Updated 12/23/2007: See the "Danny Simmons Answers My EntityCollection Depth and Serialized EntityReferences Issues" topic of the LINQ and Entity Framework Posts for 12/20/2007+ post for more details about the issues raised in this post's "Where's the Leash on EntitySet Depth in the Entity Framework?" Many thanks to Danny for his detailed, illustrated answers to the issues I raised.

Service-oriented architecture (SOA) with data-centric Web services having an object/relational mapping (O/RM) tool as the data access layer (DAL) requires the data transfer object (DTO) to eager-load EntitySets for 1:many associations. Lazy loading is proscribed because that would require the service consumer to exchange messages with the service for each associated entity or, perhaps, groups of entities.

Real-world data potentially involves business objects having 1:many associations with EntitySets that could contain thousands or even millions of rows: for example, all Order entities associated with a Customer or Salesperson entity for a large enterprise. Obviously, just serializing an entity having an EntitySet property containing several thousand entities could bring the service to its knees. It's obvious that a practical O/RM tool must provide a practical method to control the number (depth) of sub-entities in the EntitySet returned with the base entity. If fewer than all associated entities are returned, the tool must (in most cases) enable ordering the set before applying the filter.

The classic example of the requirement for ordering the members of an EntitySet property is returning orders for a customer or set of related customers to a customer service agent. To minimize retries, the business policy might be to return the last five orders for customers on the first request, which corresponds to a T-SQL SELECT TOP(5) * FROM Orders WHERE CustomerID = 'RATTC' ORDER BY OrderID DESC query.

Update 12/21/2007: Preceding query completed by adding ORDER BY OrderID DESC and other minor clarifications and additions.

The LINQ to SQL team anticipated this requirement by providing the DataLoadOptions.AssociateWith() method, which accepts a lambda expression as its argument. For example

dlOptions.AssociateWith<Customer>(Customer c =>  _
    c.Orders.OrderByDescending(Order o => o.OrderID).Take(5))

or

dlOptions.AssociateWith(Of Customer)(Function(c As Customer) _
    c.Orders.OrderByDescending(Function(o As Order) o.OrderID).Take(5))

added to the DataContext.LoadOptions collection generates the following T-SQL statement:

SELECT TOP (5) [t0].[OrderID], [t0].[CustomerID], [t0].[EmployeeID], [t0].[OrderDate], [t0].[RequiredDate], [t0].[ShippedDate], [t0].[ShipVia], [t0].[Freight], [t0].[ShipName], [t0].[ShipAddress], [t0].[ShipCity], [t0].[ShipRegion], [t0].[ShipPostalCode], [t0].[ShipCountry]
FROM [dbo].[Orders] AS [t0]
WHERE [t0].[CustomerID] = ((
    SELECT [t2].[CustomerID]
    FROM (
        SELECT TOP (1) [t1].[CustomerID]
        FROM [dbo].[Customers] AS [t1]
        WHERE [t1].[CustomerID] = @p0
        ) AS [t2]
    ))
ORDER BY [t0].[OrderID] DESC

which returns the last five Orders rows for the Customers row with CustomerID =[=] 'RATTC'. The preceding query won't win any performance awards and each Customer entity requires a separate query, but it's certainly a win over returning and throwing away thousands of Order and Order_Detail entities.

Alternatively, you can overload some methods in the code-generated DataContext classes with a partial class file and substitute a stored procedure (with optional parameters) for the lambda expression.

You get even a bigger win if you need to limit members of lookup list collections, such as  Product EntityRefs for eager-loaded Order_Detail entities only. This filter is especially important for organizations with a very large number of different products. However, serializing many:1 and 1:many associations creates cyclic relationships that binary serialization can handle but requires enabling the DataContractSerializer.PreserveObjectReferences property with a cumbersome shim/hack for XML serialization.

Unfortunately, there are no commonly observed standards for serializing and deserializing the ID/IDREF attributes in XML Infosets required to handle bidirectional references, so Microsoft's XML team purposely made it difficult to set DCS's PreserveObjectReferences property to true.

As mentioned in my Serializing Object Graphs Without and With References post of November 21, 2007:

An alternative to [the DataContractSerializer] (DCS) is the NetDataContractSerializer (NDCS), which can serialize graphs with 1:many and many:1 (bidirectional) associations by preserving association references in ID/IDREF pairs of Id and Ref values. NDCS delivers graph fidelity at the expense of interoperability because it requires clients to support CLR types. A factor in lack of interoperability with other vendors' Web service toolkits is the complexity of schemas generated by NDCS and Microsoft's patent that appears to cover NDCS's method of serializing/deserializing objects.

Controversy still seeths over the comparative benefits of serializing the object graph or maintaining interoperability by serializing a object tree and ignoring many:1 associations. It's possible to do this if you load the EntityRefs in a second step and have foreign key values with which to regenerate the EntityRef associations on the client. LINQ to SQL entities expose foreign key values; serializable EntityKeys in Entity Framework (EF) Beta 3 and later might make client-side regeneration of EntityReferences possible.

Where's the Leash on EntitySet Depth in the Entity Framework?

Danny Simmons brought up the issue of selective serialization of EF's association EntityRefs in his Question: Deep serialization of an object graph--how deep should it go? post of November 20, 2007. Here's an excerpt from his post:

1) The entire graph connected to the customer is returned every time. If you are building a stateless webservice, then you would likely construct a new ObjectContext instance each time the method is called, retrieve from the DB just those entities you want to return and then return them. In this scenario, returning the entire graph every time works just fine because the entire graph contains exactly what you want to return.

I had expected the ObjectQuery.Include() and ObjectContext.EntityCollection.Load() methods to provide an overload that delivered the equivalent to LINQ to SQL's DataLoadOptions.AssociateWith() method by Beta 3. However, a search of the Beta 3 documentation, such as Shaping Query Results (Entity Framework), the ADO.NET (Prerelease) forum, or anyone's blog (including Julie Lerman's). Perhaps I'm not using the correct search terms.

All sample code I've seen for the the ObjectQuery.Include() and ObjectContext.Entity.Load() method refers to "all" order line items, which is de rigueur, or a single salesperson's few orders. In all examples I've seen, including messages the forum, the argument is an quoted entity (table) name.

Disassembling ObjectQuery.Include() with Reflector gives the following:

public ObjectQuery<T> Include(string path)
{
    EntityUtil.CheckStringArgument(path, "path");
    ObjectQueryImplementation<T> queryImpl = this._impl.Clone();
    queryImpl.EnsureSpan().Include(path);
    return new ObjectQuery<T>(queryImpl);
}

Parameters: path

Dot-separated list of navigation properties that describe the query path in the graph that should be eagerly fetched. For collection types, the navigation property is lifted so it looks like a member of the collection.

Return Value: A new ObjectQuery<> with the defined query path.

The function's signature indicates to me that it's restricted to a particular entity that can be tracked along the navigation path from the base entity.

Note: Zlatko Michailov refers to ObjectQuery.Include() as an "exact-entity" operator in this answer in the forum:

Include() is an “exact-entity” operator of ObjectQuery<T>. (Reference: http://msdn2.microsoft.com/en-us/library/bb738708(VS.90).aspx) It denotes a navigation property that you want to dereference and materialize as part of your resulting entity collection. That’s why the “T” in the ObjectQuery<T> must be an entity type, i.e. it may not be DbDataRecord.

A Google search on "exact-entity" objectquery and "exact-entity" "entity framework" returns only the preceding forum answer as a valid hit.

NHibernate and the Castle Project use MultiQueries such as the following to specify the depth of EntitySets:

session.CreateMultiQuery()
.Add("from Blog b left join fetch b.Posts where b.Id = 1")
.Add("from Post p left join fetch p.Comments where p.Blog.Id = 1")
.List();

(From a comment to Oren Eini's Thinking about the Entity Framework post of May 16, 2007. There's a better example in Oren's Efficently loading deep object graphs post of June 20, 2007.)

It's my conclusion that without a workable (i.e., internal, dynamic SQL) means to order and limit the size (depth) of EntityCollections from associations that EF would be a non-starter as an enterprise-scale O/RM tool (a.k.a., "ObjectSpaces Reloaded"). That's why I'm questioning my search terms.

Unless EF has an answer for serializing EntityReferencess to associations or regenerating EntityReferencess from custom EntityCollection lookup collections on the consumer, EF will join LINQ to SQL as two Microsoft O/RM tools that have no "out-of-the-box multi-tier story" and thus aren't considered enterprise-ready. However, if Danny Simmons is successful in implementing his "General-purpose Container Object" functionality over EF, which he describes in his December 19, 2007 So they're hard, but what if I need them... post ("them" refers to Web services), the story might not be "out-of-the-box" but it would be a story. (The way I read this post is that all associated entities are in memory before serialization.)

Update 12/23/2007: Incorrect search terms weren't the problem for documentation for filtering associations' EntityCollections. As mentioned in the LINQ and Entity Framework Posts for 12/20/2007+ post:

A search on 'CreateSourceQuery' returns valid hits to online help and Danny's post only, so there doesn't appear to be any sample code for the method, which also has an EntityReference counterpart.

I'm surprised that other users haven't raised questions by now about filtering sorted EntityCollections.

Note: I'm still waiting for even a demo version of Matt Warren's serializable "mini connectionless DataContext" for LINQ to SQL (see the 6/18/2007 update to the "Simplifying Data Tracking in n-Tier Projects" topic of my Changes Coming to LINQ for SQL post of May 15, 2007. In the meantime, Mike Taulty's approach that I discussed in the GetOriginalEntityState() Loses EntitySet/EntityRef Data post of October 15, 2007 almost works for LINQ to SQL.

Comments anyone?

Technorati Tags: Entity Framework, Entity Framework Beta 3, EntitySets, EntiyRefs, Serialization, DataContractSerializer, NHibernate, O/RM Tools, Object/Relational, LINQ, LINQ to SQL