LINQ and Entity Framework Posts for 9/7/2008+

Dinesh Kulkarni Posts Changes to LINQ to SQL RTM

I missed these two important posts because Dinesh posted them to the LINQ Project General forum instead of his Dinesh's Cyberstation blog:

LINQ to SQL Beta2 to RTM Key Changes (August 27, 2007)

• Table<TEntity>'s Add[All]()/Remove[All]() method names will change to Insert[All]OnSubmit()/Delete[All]OnSubmit(). In my opinion the new names apply relational metaphors to objects, which is a bad practice.
• The OnValidate() event gets a ChangeAction Enum parameter with None, Delete, Insert, and Update members. Relational terminology again applied to objects.
• Local method calls in LINQ to SQL projections from VB LINQ to XML queries are now supported. Dinesh provides a LINQ to XML query example.

Beta2 to RTM Changes in Attach() Behavior (September 11, 2007)

• Attempts to Attach() an object that is not truly detached from another DataContext by serialization/deserialization will throw an exception. An example is an object that has deferred loaders from another DataContext.
• Objects Attach()ed with SubmitChanges() don't have ChangeSet entities will default to "unchanged" status and no longer will be inserted to the underlying data store by default.

Problems Using Stored Procedures for LINQ to SQL Data Retrieval with ASP.NET

The final version of my first post about Problems Using Stored Procedures for LINQ to SQL Data Retrieval with ASP.NET 3.5's LinqDataSource and GridView controls is done. The primary issues with the LinqDataSoruce are:

• Lack of Persistence Ignorance (in general, not a result of using stored procs for data retrieval)
• Loss of server-side paging and consequent increased load on the database
• Inability to lazy-load EntityRefs without spewing dynamic SQL queries

Subsequent posts will cover issues with BindingSource components and DataGridView controls when using stored procedures to hydrate LINQ to SQL entities. (Update 9/12/2007.)

Frans Bouma Starts LINQ to LLBLGen Pro Series

Frans Bouma, the developer of the LLBLGen Pro commercial O/R mapping tool and principal antagonist of LINQ to SQL and the Entity Framework, has started a project to adapt LINQ to LLBLGen Pro (or vice versa). His first posts are here:

1. Developing Linq to LLBLGen Pro, Day 0 (September 11, 2007)
2. Developing Linq to LLBLGen Pro, Day 1 (September 12, 2007)

Thomas Wagner likens Frans' new series to A diary of LINQ pain.

Be sure to read the comments to the first post! (Update 9/13/2007.)

Jim Ng on Lambda Expressions and Expression Trees in VB

Jim Ng, the dev lead for the VB compiler team, promises a series of VB Team posts on lambda expressions and expression trees. The first post of the series is, appropriately enough, Lambda Expressions and Expression Trees of September 11, 2007. He's also written "Lambda Expressions," a "Basic Instincts" column for the September 2007 issue of MSDN Magazine. (Updated 9/12/2007.)

Julie Lerman Explains Why Entities Are Missing Foreign Key Values 

Julie's September 11, 2007 Entity Data Model Associations: Where's my Foreign Key? explains why entity properties don't include foreign keys in the Entity Data Model (EDM) unless they're also part of another entity's primary key. They've been sacrificed in favor of association and navigation properties. For example, the Order entity that's persisted to the Northwind Orders table has no CustomerID, EmployeeID, or ShipperID property; instead it has many:1 navigation properties to associations with Customer, Employee and Shipper entities. (I've singularized the entity names in Julie's EDM diagram.)

The Order_Detail entity's ProductID property, which is a member of the composite primary key, was missing in previous CTPs due to the ADO.NET team's over-zealous pursuit of object purity. Of course, it could have been worse: they could have hidden the OrderID property, too. As Julie points out, relationally-oriented folks like Jay Traband of IdeaBlade and me have complained about the missing foreign keys here and here. Lets hope that the ADO.NET team delivers on their intention to surface foreign key values as an option in the release version. (Updated 9/11/2007.)

Don Demsack (DonXML) Talks LINQ to XML on .NET Rocks Show #271

Don Demsack (a.k.a. DonXML) discusses LINQ to XML and the new XML language features in Visual Basic with Carl Franklin and Richard Campbell in this 66-minute .NET Rocks! audio clip posted September 11, 2007: Don Demsak on LINQ to XML.

ADO.NET Team Wants Entity Framework Docs Feedback

Glenn Gailey of the Entity Framework User Education (UE) team posted Please Give Feedback on Entity Framework Docs on September 11, 2007 requesting feedback :

• When a topic is particularly effective or ineffective.
• When the topic doesn't reflect the product behavior (i.e. there's a doc bug).
• When an important use case or scenario is not covered or more documentation or examples are needed.

Glenn includes instruction on how to submit feedback from the online docs at ADO.NET Entity Framework Beta 2 or the VS 2008 UI.

I'm planning to gripe about the lack of information on the topic of retrieving data by stored procedures. I'm sure Julie Lerman will make the same complaint.

Installing Project Jasper After Entity Framework Tools Clobbers Them

Running the installer that you download from the Jasper : MIX 2007 CTP page after installing the EF Tools August 2007 CTP causes VS 2008 to be unable to load the EDM Designer bits (Microsoft.Data.Entity.Design.Package.MicrosoftDataEntityDesignPackage, Microsoft.Data.Entity.Design.Package, Version 1.0.0.0 ... .)

To regain proper loading of the EDM Designer, uninstall the Microsoft Project Jasper May 2007 CTP, uninstall the ADO.NET Entity Framework Tools and reinstall the Tools.

Update 9/12/2007: Reinstalling the tools didn't solve the problem. I've posted a question about this problem in the ADO.NET Orcas forum.

Update 9/13/2007: Microsoft's Jeff Derstadt said in answer to my question:

The Jasper CTP is not supported for VS 2008 Beta 2. The installer included a patch for an earlier VS 2008 CTP where the EDM tools would not work properly. You will likely need to reinstall VS 2008 Beta 2 to remove this patch.

Andy Conrad in a reply to my comment asking if he had tried it said:

Just to be clear - the sample works with Jasper running over Beta 1 of Orcas.  The problem exists when installing Jasper over Entity Framework Beta 2.  We are currently investigating a solution for this.

I wonder how many folks are running Orcas Beta 1.

(I found this out the hard way when attempting to try the Cappuccino Project Jasper sample by Tok Wee Hyong of the .Net Singapore user group that Andrew Conrad recommended. I wasn't able to get it to run on either of my systems running VS 2008 Beta 2 due to reference version conflicts and lack of the multi-targeting selection button on the project Open dialog. Subsequently, I determined that installation of the Jasper CTP ate my VS 2008 Beta 2 installation.)

John Stallo Explains Building N-Tier Apps in VS 2008 with VB

This Channel 9 Interview - John Stallo on Building N-Tier Applications in VS 2008 (Beth Massi) post of September 10, 2007 from the VB Team also has a pointer to a walkthrough with a strongly typed DataSet noted my LINQ and Entity Framework Updates for 8/29/2007+ post. As mentioned in that post, the topic is apropos because you can substitute LINQ to SQL for the DataSet to evaluate if LINQ to SQL is right for you in a WCF-based service environment.

Mike Taulty Continues to Explore LINQ to XML

His first Monday post XML, LINQ, Databinding a ListBox to an Anonymous Type does exactly what it says, but you can expect to hear more on this topic. Northwind & Photos - How Many Times Will I Write This Code? is a scratch pad for code to write the Northwind.Employees.Photo values to a file instead of encoding them as base64 to a CDATA element. (Updated 9/10/2007.)

Pablo Castro Requests Feedback on Astoria Data Payload Formats

In Sunday's Astoria Design: payload formats Project Astoria Team post, Pablo Castro asks for input on the team's proposed three data payload formats:

• ATOM/APP (Atom Publishing Protocol) because it's popular: "The important characteristic about ATOM/APP is that everybody speaks it, and it’s increasingly becoming kind of a lingua franca of the web when it comes to data exchange between unrelated agents."
• JSON (JavaScript Object Notation) also because it's popular: "[JSON] has become popular lately mostly (my guess) because it makes it easy to consume results from services in JavaScript environments, particularly inside web browsers."
• Web3S (Web Structured, Schema’d & Searchable) as a replacement for POX (Plain Old XML) because "folks at the Live organization in Microsoft were working on a protocol for data exchange that would be used uniformly across many different services."

RDF (Resource Description Format), which was included in the May 2007 CTP has been dropped because "we didn’t see any early user actually using it and we haven’t seen a particular popular scenario where RDF was a must-have."

Without giving it a lot of thought so far, I think I prefer POX to Web3S. (Updated 9/10/2007.)

Erik Meijer Discusses LINQ in .NET Rocks Show #270

Erik Meijer, LINQ's chief spokesman on the conference circuit, joins Carl Franklin and Richard Campbell in Show #270, Erik Meijer on LINQ!!, to deliver 66 minutes of "detail about LINQ from soup to nuts. This is a very technical discussion, and not an overview." Erik starts at 06:30 and covers LINQ to XML, LINQ to SQL's DataContext object, SQL Server versus CLR data types. He discusses Standard Query Operator lambda expressions and their relationship to delegates and monads starting at 43:00, and LINQ's limitations at 52:00. At 53:45, Erik recommends using LINQ to Objects for queries across domain-specific implementations. He explains the SelectMany operator at 56:30, instance-based data updating at 58:45, LINQ to Entities at 1:00:30, and third-party LINQ providers at 1:01:30. (Updated 9/9/2008.)

Julie Lerman Hydrates EF Entities with Stored Procedures

If you think using stored procedures to hydrate (and cripple) LINQ to SQL with ISingleResultset<T> collections is a pain, wait until you try doing the same with Entity Framework entities. As Julie mentioned in her September 8, 2007 Finishing up READ Stored Procs brought in by the Entity Framework EDM Wizard post, selecting a stored procedure node in the EDM Wizard adds a function stub to the EDMX file's <edmx:StorageModels .../> section but doesn't add any elements to the <edmx:ConceptualModels .../> or <edmx:Mapping .../> sections. And there's no documentation for using stored procs to retrieve entity data except a brief answer by Zlatko Michailov on June 9, 2007 to Julie's Sprocs and views in June CTP forum question.

Julie uses Northwind's CustOrderHist stored proc as and example to provide the key to what sections you can generate from the designer and those you must write yourself. She promises that an article in her DevLife blog with step-by-step instructions will be posted shortly. She'll update her personal blog post and I'll update this item when the article's posted.

Mike Taulty Saves Memory by Streaming Documents with LINQ to XML

LINQ to XML's DOM programming model is somewhat more memory-efficient than conventional XML DOM processing, but public streaming input and output APIs would enable processing very large documents without consuming an excessive (usually about 4X the document size) amount of memory by a DOM tree.

Unfortunately, the LINQ to XML team decided in March 2007 to pull the streaming input API from the VS 2008 RTM. However, they left the XStreamingElement type in to enable implementing custom streaming input scenarios. XStreamingElement, to quote the XML Team in their March 5, 2007 Streaming with LINQ to XML - Part 1 post, "allows XML-like trees of IEnumerable<T> instances that can be lazily consumed while being saved as XML text."

Mike demonstrates in his his September 8, 2007 LINQ to XML and Larger Files post that pull-parsing the input document with an XmlReader in a "custom axis" function and a C# iterator reduces memory consumption from 190 MB (for the DOM) to 2 MB for a 100 MB XML file. With conventional output to a file, memory consumption increases to 7 MB. However, substituting XStreamingElement for XElement reduces memory usage to 428 KB.

For VB programmers, Mike provides an XElementEnumerable class to replace C#'s iterator, which VB 9.0 doesn't support.

Scott Guthrie Demos Custom Queries with the LinqDataSource

Scott's LINQ to SQL: Part 9 - Using a Custom LINQ Expression with the <asp:LinqDatasource> control post of late Friday night shows you how to substitute custom LINQ query expressions for the single-table queries that you create with the LinqDataSource's Configure Data Source wizard. The wizard limits customization to selecting all or specific properties of a LINQ to SQL entity to retrieve and specifying a Where clause filter and OrderBy sort of a single table. If you don't retrieve all fields, bound controls aren't updatable.

You can substitute a custom LINQ query expression for the default expression that the wizard generates in a handler for the the LinqDataSource's Selecting event. Just pass the query result's IEnumerable<T> to the e.Result property. Inline editing in DataGrids or other bound controls won't work if you pass a custom data shape or projection.

Update 8/12/2007: Be wary of returning List<T> objects from stored procedures (see Problems Using Stored Procedures for LINQ to SQL Data Retrieval, currently a work in progress.)

Andrew Conrad Posts Generic CopyToDataTable Extension Method

Andrew's Science Project post of September 7, 2007 includes the source code for a LINQ to DataSet extension method that removes the CopyToDataTable<T> method's restriction of the generic T type parameter to DataRow. The extension method enables loading instances of anonymous types into DataRows. Andrew provides the following usage notes:

• The initial schema of the DataTable is based on schema of the type T. All public property and fields are turned into DataColumns.
• If the source sequence contains a sub-type of T, the table is automatically expanded for any addition[al] public properties or fields.
• If you want to provide a existing table, that is fine as long as the schema is consistent with the schema of the type T. 

Andrew's sample source sequence consists of a base class Item, which has public Id, Price and Genre properties. Item has two subclasses: Book has a public Author property and Item has a public Director property. The DataTable generated from the anonymous type has five columns: Id, Price, Genre, Director and Author.

Tip: The sample code requires a reference to System.Data.DataSetExtensions.dll and using System.Data; using System.Data.DataSetExtensions; and using System.Reflection; declarations.

For a brief review of databinding with LINQ to DataSet, see Erick Thompson's LINQ to DataSet – Data Binding post of August 24, 2007. Daniel Moth has a July 27, 2007 LINQ to DataSet item with details of the classes in the System.Data.DataSetExtensions namespace, and here are links to three introductory LINQ to DataSet posts from February 2007: Part 1, Part 2 and Part 3.

Update 9/12/2007: Andrew says in a comment that he intends to update the code to solve these two issues:

1)  The code is not catching the error case when the type T of the source sequence does not match the schema of the provided datatable.  I suppose I could extend the table automatically in this case.

2)  The results of your query is just a sequence of scaler values.  The code wasn't really designed for this and I am not sure I see much value, but I suppose I could just make a table with a single column.

Upgrade List<T> Collections to BindingListView<T> for Sorting and Filtering

My September 7, 2007 Support Sorting and Filtering for List<T> Collections post shows you how to take advantage of Andrew Davey's BindingViewList class to bring advanced sorting and filtering features to controls and components bound to List<T> collections created by invoking LINQ Query's IEnumerable<T>.ToList() method.

Andrew Davey Generates Assertions with LINQ Expression Trees

Andrew Davey's September 7, 2007 Assertions via Linq Expressions post suggests minimizing the boredom of duplicating the code you're testing in a text message string, such as Debug.Assert(input != null, "input != null");, when writing Debug or NUnit assertions. His approach is to compile and invoke LINQ-style expression trees to specify the assertions and then invoke ToString() to write the string.

For example, calling this function

void Assert<T>(T obj, Expression<Func<T, bool>> test)
{
  System.Diagnostics.Debug.Assert(test.Compile().Invoke(obj),
      test.ToString());
}

with

Assert(input, i => i != null);

is the equivalent of writing

Debug.Assert(input != null, "input != null");

Andrew also suggests extension methods to provide alternative syntaxes.

Note: Andrew is the Bristol (UK) developer who wrote the BindingListView wrapper to make List<T> collections sortable and filterable (see Support Sorting and Filtering for List<T> Collections.)

Technorati Tags: LINQ to XML, LINQ to SQL, LinqDataSource, LINQ to Entities, Entity Framework, LINQ, Lambda Expressions, Expression Trees, Expressions, Extension Methods, XStreamingElement, VB 9.0, Unit Testing, NUnit