LINQ and Entity Framework Posts for 9/14/2007+

Changes in Store by RTM for LINQ to SQL's WinForm Databinding with Stored Procedures

My Upgrade to LINQ to SQL's WinForm Databinding with Stored Procedures Scheduled for RTM of September 19, 2007 describes the change from the ISingleResult<T> to the DataQuery<TEntity> type returned to the DataContext object by stored procedures that retrieve data. This means that populating DataContexts with stored procs will be closer to on par with dynamic SQL when Microsoft releases Visual Studio 2008. You'll probably hear more about this change from Dinesh Kulkarni in the near future. In the meantime, I'm anxious to hear from Scott Guthrie how the change will affect the LinqDataSource.

You can expect updates to (or a rewrite of) this and my earlier Problems Using Stored Procedures for LINQ to SQL Data Retrieval post shortly. The first update to this post, an added "DataQuery<T> and Table<TEntity> vs. ISingleResult<T> and List<T> Collections" section, is ready now. (Update 9/19/2007 17:00 PDT.)

Beth Massi and Charlie Calvert Talk VB 9.0 and C# 3.0 on .NET Rocks!

Carl Franklin and Richard Campbell interview Beth Massi, The Visual Basic Team's evangelist extradinoire, and Charlie Calvert of the C# Community in Charlie Calvert and Beth Massi on VB.NET, Orcas, and the FoxPro Knitting Guild, show #274. (Update #1 9/20/2007.)

Scott Guthrie on Using C#'s (New?) null Coalescing (??) Operator with LINQ to XML

Scott Guthrie's 9/19/2007 The C# ?? null coalescing operator (and using it with LINQ) post describes how to prevent exceptions in apps that use LINQ to XML when they encounter a null value type in an explicitly-typed XML element.

The original title of the post was "The New C# ?? null coalescing operator (and using it with LINQ)" and Scott originally started began this post with this sentence:

One of the subtle (but cool) new language features in the VS 2008 version of C# is the ?? "null coalescing" operator. [Emphasis added]

By the time I got around to adding this item, someone appeared to have let Scott know that the ?? operator arrived with VS 2005 in C# 2.0 and the underlined words were excised. Here's feed-squirrel.com's copy of the original post.

VBA has the Nz (null-to-zero) function (Nz(variant, [valueifnull]) to return 0 or a string when a Variant variable contained Nothing. Nz was very useful to replace holes in crosstab queries caused by lack of data for a cell. As I recall, an early version wasn't safe for scripting but was subsequently fixed. VB 6 didn't support this function and it's missing from VB.NET. (Update #1 9/20/2007.)

Tim Anderson posted on 9/19/2007 an interview with Scott, Scott Guthrie on .NET futures, conducted at the MIX07 UK conference held in London on September 11-12, 2007. Tim's September 11, 2007 Silverlight at Mix07 UK post mentions in his "Web service support and LINQ" section:

Silverlight will support JSON, WCF and SOAP. It will also include LINQ, with the possibility of creating custom LINQ data providers - Guthrie mentioned possibilities like a LINQ provider for Amazon’s S3 service.

Mike Tulty Repeats Disconnected Concurrency Tests with the Entity Framework and WCF

Following up on his Disconnected LINQ to SQL post of September 14, 2007, which tested optimistic concurrency management with LINQ to SQL in a Windows Communication Framework (WCF environment), his September 19, 2007 Disconnected LINQ to Entities post repeats the process with the Entity Framework (EF).

Mike finds that the EF complicates optimistic concurrency management (OCM) by:

1. Requiring invocation of the SetModifiedProperty("PropertyName") method on each property when the service doesn't save original values supplied by the client. (Julie Lerman reported this issue previously in her Exploring EntityKeys, Web Services and Serialization a little further post of September 2, 2007.)

2. Looping "around the properties of the entity, use reflection to compare the original and current value [using reflection]. If the values differ we copy the property value from the 'new' entity to the 'original' entity" the service saves original values supplied by the client.

It seems to me that, unlike LINQ to SQL, EF isn't very OCM-friendly. No wonder EF disables optimistic concurrency by default and it must be turned on explicitly, as Mike also mentions in his Entity Framework - Object Services Level. Concurrency. post of August 27, 2007.(Update 9/19/2007.)

Sort the Whole Resultset, Not Just the Page, When Using the LinqDataSource's Server-Side Paging Feature

After I got my ASP.NET/LinqDataSource/GridView test harness paging correctly with stored procedures for the primary LINQ to SQL resultset, EntityRefs and EntitySets—with some assistance from Matt Warren—it became clear that I had more work to do to make sorting work properly. Dynamic ORDER BY Clauses for ASP.NET Server-Side Paging with Stored Procedures of 9/18/2007 is a T-SQL-filled, illustrated chronicle of what's necessary to emulate with a stored procedure the server-side paging and sorting features of dynamic T-SQL. It's not pretty, but the test harness is finished. Now I can get back to finishing the WinForm stored procedure test harness. (Update 9/18/2007.)

Dare Obasanjo Analyzes Proposed Changes to Astoria Data Formats

Dare's Microsoft REST APIs: Astoria = Web3S + Atom - RDF of 9/18/2007 analyzes how Microsoft's Web3S (Web Structured, Schema’d & Searchable) specification fits into the RESTful protocol picture in general and the Astoria Web service in particular. The article's title derives from Pablo Castro's Astoria Design: payload formats post of 9/7/2007 , which announced support for a POX (plain old XML) format would be dropped in favor of Web3S, the APP (Atom Publishing Protocol) and JSON (JavaScript Object Notation) formats would remain, and RDF would be abandoned.

Dare includes links to his two prior posts about Astoria—Google Base Data API vs. Astoria: Two Approaches to SQL-like Queries in a RESTful Protocol—and Web3S—Web3S: A RESTful Protocol for Accessing Windows Live Services. If you're interested in an early, independent review of Astoria, check out Leonard Richardson's June 1, 2007 How's My Driving? #1: Microsoft Astoria post.

Dare concludes his post with this question:

For some reason, I'm not surprised about this decision. I do wonder if dropping RDF will actually bring to light some closet RDF supporters who'd love to see supported in Astoria?

A Google blog search on "Microsoft Astoria RDF" for the past month returned 288 hits but few complained about the demise of RDF as an astoria data format. Mike Bergman's Astoria is Whistling Past the Graveyard to Irrelevance post is one of the few. (Update 9/20/2007.)

Astoria Bits Updated for VS 2008 and Entity Framework Beta 2

Pablo Castro's Astoria CTP Refresh for Visual Studio 2008 Beta 2! post of September 17, 2007 announces updated (not upgraded) release that's compatible with Visual Studio 2008 Beta 2, Entity Framework Beta 2 and the EDM Tools August CTP. According to Pablo Castro, the Microsoft Codename "Astoria" - Sept 2007 Community Technology Preview offers no new features other than those required for compatibility with current beta and CTP bits.

Notice the the "Project" prefix has been dropped from the download headline as the result of Astoria emerging from the incubation project cocoon. Strangely, there have been only 59 posts in the Project Codename: Astoria forum since it opened on May 3, 2007.

So far, the Project Jasper team hasn't announced plans for a release compatible with VS 2008 Beta 2 and EF/EDM Tools Beta 2/August CTP. Andrew Conrad said in a September 13, 2007 Jasper using IronPython and VS 2008 August SDK post to the Project Codename: Jasper forum:

It is actually a bit more work to get Jasper working on Beta2 because of a number of breaking changes to the underlying ADO.Net Entity Framework from Beta 1 to Beta 2. As soon as I have a update with some solid plans, I will make it public.

We actually would love to get the community involved more with Project Jasper. Right now we are very interested in any/ all feedback folks might have.  We are in the middle of the planning process trying to figure out what to do in this space going forward – so feedback is invaluable.

Andy would be more likely "to get the community involved more with Project Jasper" if the team updated their bits to Beta 2. The forum has received a total of only 4 posts since it opened on May 2, 2007. (Update 9/18/2007.)

LINQ and WCF MSDN "Masterclass" Stealth Webcasts

Mike Benkovich, a Microsoft developer evangelist stationed in Minneapolis is 75% of the way through a series of four "Framework Masterclass" presentations on LINQ: 

• Framework Masterclass: LINQ to Entities (Level 200) - 9/24/2007
• Framework Masterclass: LINQ to XML (Level 200) - 9/17/2007
• Framework Masterclass: LINQ to SQL (Level 200) - 9/10/2007
• Framework Masterclass: Working with LINQ (Level 200) - 9/4/2007

He also has four Windows Communication Foundation presentations scheduled: Framework Masterclass:

• Framework Masterclass; Working with WCF - 10/1/2007
• Framework Masterclass: WCF Management & Diagnostics - 10/15/2007
• Framework Masterclass: WCF Custom Channels - 10/22/2007
• Framework Masterclass: WCF Performance & Deployment - 10/29/2007

I'm surprised that there hasn't been more publicity for his LiveMeeting presentations, all of which are about an hour in length. Mike posts the code from his presentations on his MSDN BenkoBlog.

Julie Lerman on Heterogeneous Joins with LINQ

Using LINQ to query Outlook emails joined with SQL data of September 16, 2007 matches simulated email messages from fictitious AdventureWorks customers and assigns them to the appropriate sales representative. In an excellent example of query composability, she defines Selects emails and contacts from Outlook folders and custSalesPerson from the AWCustomers object and performs a three-way Join to generate a "SenderName from CompanyName: Subject [for] SalesPerson" message.

An interesting alternative to the more common LINQ to XML joined with LINQ to SQL objects demo. (Update 9/17/2007.)

Danny Simmons Simulates a Multi-User Dungeon with Entity Framework

Danny's Concepts Part I: Getting an entity model up and running of September 15, 2007 describes an operational Multi-User Dungeon (MUD) game that he plans to use in "a series of blog posts I'd like to write explaining various Entity Framework (EF) concepts, particularly those related directly to writing code using the framework." The first episode includes a simplified Entity Data Model (EDM) diagram and offers links to its CSDL, MSL, and SSDL files. The CSDL file describes the scalar properties of Actor, Item, Room and Exit entities.

Danny discusses the following three EF concepts in depth:

1. Storing your metadata in resource files. I've always considered CSDL, MSL, and SSDL files to be free-standing so they can be edited without recompiling the application, similar to app.config files for connection strings. However, I suppose there is benefit to being able to deploy your client app as an Application.exe and app.config file pair.
2. Connection strings. If you accept the default connection string name in the EDM Designer, its codegen creates a parameterless constructor (ctor) for you. (LINQ to SQL creates a parameterless ctor, too, but it seems to disappear if you edit the construction string. The LinqDataSource needs a parameterless ctor. I'm checking further on this.)
3. Context creation and lifetimes. Danny's client is a WinForms app and assumes long-lived ObjectContexts. He also plans to explore Web services and Web site apps. Based on the problems I've encountered to date with stored procedures for LINQ to SQL data retrieval (see below), I'm sure he'll use dynamic SQL.

He concludes with sample code for console projects to:

Consolidate connection string building, but also creating a common, static entry point to the context, and automatically maintaining a separate context for each thread by putting the context into thread-local storage [and] prints out a list of all the rooms and items in the DPMud database but does so from two threads running concurrently.

Sounds to me like an interesting series. (Update 9/17/2007.)

Update 9/20/2007: Danny notes that storing metadata in resource files causes problems with Visual Studio's unit testing feature in his 9/19/2007 More about embedding metadata in resources... post. The problem arises because VS's unit testing feature uses unmanaged code!

Update: 1-1/2 Problems with Stored Procedures for LINQ to SQL Data Retrieval Now Solved

Many DBAs will require developers to use (DBA-approved) stored procedures to retrieve persisted entities from—as well as to persist them to—relational tables. There's been little in the way of Microsoft documentation or blog posts on how to use stored procedures for entity retrieval other than Scott Guthrie's LINQ to SQL (Part 6 - Retrieving Data Using Stored Procedures) post of August 16, 2007. Using the ASP.NET LinqDataSource control complicates the data retrieval issue with stored procedures for server-side paging of databound GridView and other grid-like controls. Stored procedures also interfere with populating entity associations.

My Problems Using Stored Procedures for LINQ to SQL Data Retrieval (initial of September 8, 2007, updated 9/14/2007 and 9/15/2007) pointed out the problems with server-side paging by handling the LinqDataSource's Selecting event and lazy-loading or eager-loading entity associations. The updated post shows how to solve the problem with inability to display page 2 or higher with a paging stored procedure (set the AutoPage property value to False.)

Input from Matt Warren showed how to override EntityNameLoad methods to substitute stored procedures for dynamic T-SQL for populating 1:many (1:n) associations, which is half the problem. However, you must force eager loading by using LoadWith expressions to invoke the overridden method. There doesn't appear to be a way to lazy-load 1:n associations, so this approach solves only 1/4 of the problem.

Matt posted this comment on 9/17/2007:

Using stored procedures to load associations does actually work with lazy loading. You don't need to use LoadWith.

Update 9/17/2007: Matt's right, as usual, so I changed the heading from 1-1/4 to 1-1/2 of the problems solved. There must have been cockpit error with my ASP.NET test harness that indicated my overriding function wasn't being invoked when performing a count of total Order_Details for the 10 displayed rows. The EntityNameLoad methods for populating 1:many (1:n) associations do work as expected.

There are still no docs or sample code for eager-loading or lazy-loading many:1 (m:1) associations, which is the remaining half of the problem.

Update 9/15/2007: Julie Lerman comments in her LINQ to SQL and stored procedures post about my "obsessive persistence" in tracking down these issues.

Enable LINQ to Query Non-Generic Collections

Fabrice Marguerie's Querying non-generic collections with LINQ to Objects post of 9/15/2007 shows you how to use the LINQ Cast<T> operator to enable use of other Standard Query Operators with ArrayList or other non-generic collections.

Installing the Project Jasper May 2007 CTP Permanently Clobbers the EDM Designer

Running the installer that you download from the Jasper : MIX 2007 CTP page after installing the EF Tools August 2007 CTP causes VS 2008 to be unable to load the EDM Designer bits (Microsoft.Data.Entity.Design.Package.MicrosoftDataEntityDesignPackage, Microsoft.Data.Entity.Design.Package, Version 1.0.0.0 ... .)

Uninstalling Jasper CTP and removing and reinstalling the tools doesn't solve the problem. I posted a question about this problem in the ADO.NET Orcas forum.

Microsoft's Jeff Derstadt said in answer to my question:

The Jasper CTP is not supported for VS 2008 Beta 2. The installer included a patch for an earlier VS 2008 CTP where the EDM tools would not work properly. You will likely need to reinstall VS 2008 Beta 2 to remove this patch.

Andy Conrad in a reply to my comment asking if he had tried it said:

Just to be clear - the sample works with Jasper running over Beta 1 of Orcas.  The problem exists when installing Jasper over Entity Framework Beta 2.  We are currently investigating a solution for this.

Uninstalling and reinstalling VS 2008 Beta 2 solved the problem.

I wonder how many folks are running Orcas Beta 1.

[Repeated from the last update to LINQ and Entity Framework Posts for 9/7/2008+ because of its importance.]

Mapping Stored Procedures to the Entity Data Model

Mike Dodaro, the ADO.NET team's programmer-writer, posted How to Map Stored Procedures using the ADO.NET Entity Framework on September 14, 2007. This article shows how to manually modify the SSDL, CSDL and MSL sections of the *.edmx file to accommodate stored procedures for entity data retrieval. His example uses the SalesOrderDetail table of the AdventureWorks Sales Model, which is an Entity Data Model (EDM) implementation based on the AdventureWorks sample database.

Hopefully, Mike will provide more insight into code for populating association/navigation entity properties with stored procedures.

SqlDependency Notifications from LINQ to SQL-Generated Queries

Ryan Dunn, co-author of the The .NET Developers Guide to Directory Services Programming, provides sample code to enable LINQ to SQL to participate with SQL Notification Services in his September 14, 2007 Using SQLDependency objects with LINQ post. His code creates a SQL dependency context, which Ryan says is very similar to a transaction context. Mike Pizzo, who pointed out the approach to Ryan, can probably provide more implementation details.

Update 9/15/2007: Julie Lerman provides more implementation information about the CallContext class in her Query Notifications and LINQ to SQL - Well I'll be, you *can* do it (with caveats) post.

Frans Bouma Continues LINQ to LLBLGen Pro Series

Frans Bouma, the developer of the LLBLGen Pro commercial O/R mapping tool and principal antagonist of LINQ to SQL, has started a project to adapt LINQ to LLBLGen Pro (or vice versa). His first posts are here:

1. Developing Linq to LLBLGen Pro, Day 0 (September 11, 2007)
2. Developing Linq to LLBLGen Pro, Day 1 (September 12, 2007)
3. Developing Linq to LLBLGen Pro, Day 2 (September 14, 2007)
4. Developing Linq to LLBLGen Pro, day 3 (September 19, 2007)

Thomas Wagner likens Frans' new series to A diary of LINQ pain.

Be sure to read the comments to the first post.

[Continued from LINQ and Entity Framework Posts for 9/7/2008+ ]

Julie Lerman and Entities in Web Pages

Julie's Entities, Viewstate and postback post of September 14, 2007 analyzes binary serialization and deserialization of entities to and from the ViewState object. She notes that managing optimistic concurrency is up to you because, as Mike notes in his LINQ to SQL analysis below, the ...Context object (DataContext or ObjectContext) holds the original values.

She also points to her Serializing entities for updates across web services thread in the ADO.NET Orcas form. The thread contains detailed responses from Danny Simmons that sound remarkably like those from Matt Warren about the corresponding issues with LINQ to SQL.

Mike Taulty Tackles Concurrency Issues with LINQ to SQL and WCF

Mike's Disconnected LINQ to SQL post of September 14, 2007 dissects managing optimistic concurrency with LINQ to SQL. His two code examples assume that LINQ to SQL's DataContext resides in a Windows Communication Foundation (WCF) physical tier that processes create, retrieve, update and delete (CRUD) operations with BasicHttpBinding (SOAP 1.1 transport) to a proxy client. One example does CUD operations with no concurrency checks; the other implements optimistic concurrency with new and original values stored on the server. He provides full SQL Server Profiler traces on the service side.

The LINQ to SQL team promised a sample implementation similar to this some time ago but haven't delivered so far.

Mike cites Rick Strahl's LINQ to SQL and Disconnected Entities Follow-up post of August 15, 2007 as a source of information on managing concurrency with detached entities, that is entities with no associated DataContext. I prefer the term detached, because LINQ to SQL's is a disconnected technology; the DataContext maintains data state between successive connections to SQL Server.

XML Schema Inference Add-In for VB 9.0

The VB Team announced on September 14, 2007 a new schema inference tool add-in for Visual Basic 9.0. The tool is especially important for users of VB's LINQ to XML literal XML feature, because you can add XML schema sets to a project in three ways:

1. Specify an XML file location
2. Specify an URL on the web that returns XML
3. Paste or type an XML document directly

Adding a schema to your XML document enables IntelliSense.

Scott Hanselman gives the add-in a rave review in his same-day Clever: XML To Schema InferenceWizard For Visual Studio 2008 post. For an early look as Microsoft's Schema Inference Engine, see my November 11, 2002 "Generate XSD Schemas by Inference" article for the long-departed XML & Web Services magazine.

Note: I'm still waiting for Ralf Lammel's successor to update LINQ to XSD to the Beta 2 LINQ implementation. Ralf has accepted a full professorship at the University of Koblenz and I wish him the best of luck in his new position. Ralf's last post with a LINQ tie-in was his More than a revision on “MapReduce revisited” of April 11, 2007, which include a C# 3.0, LINQ-style abstraction for MapReduce computations. 

A Compendium of Julie Lerman's Entity Framework Beta 2 Posts

Mike Taulty's Entity Framework Posts item of September 13, 2007 has links to all 13 of Julie Lerman's posts about Entity Framework Beta 2 and EDM Designer August 2007 CTP. Links to these articles also appear in previous OakLeaf posts, but having all links in one place is handy. Mike says her posts are "great stuff" and I agree.

First Appearance of LINQ for Java Clone

Anders Norås introduced Quaere, a.k.a language-integrated queries for Java, on September 10, 2007 at the JavaZone conference in Oslo, Norway, according to his Introducing Quaere - Language integrated queryies for Java post the next day. A snapshot of the source code for Quaere's first implementation, Quaere for Objects, is available as Quaere Snapshot.zip at http://andersnoras.com/files/folders/code/entry281.aspx.

Later posts explain the derivation of Quarere (9/12/2007, the Latin root of the English word query, imperative form of quaerere - to question or ask), Answering questions about Quaere (9/13/2007) and Solving Puzzles with Quaere (9/14/2007). The last post uses Quaere to solve the same puzzle Microsoft's Luke Hoban solved with LINQ in his Using LINQ to solve puzzles post.

Technorati Tags: LINQ to Entities, LINQ to SQL, SqlDependency, LINQ to XML, LINQ to XSD, LINQ, Entity Framework, Entity Data Model, EDM Designer, O/R Mapping, Object/Relational Mapping, Project Jasper, Project Astoria, WCF, Windows Communication Framework, VB 9.0, Quaere

Problems Using Stored Procedures for LINQ to SQL Data Retrieval

 

Scott Guthrie posted LINQ to SQL (Part 6 - Retrieving Data Using Stored Procedures) on August 16, 2007 and I dutifully added it as an update to my LINQ Updates for 8/15/2007 compendium. Scott says in his post's "To SPROC or not to SPROC? That is the Question" section:

As you'll learn in this post, you can also optionally map SPROCs in the database to your LINQ to SQL DataContext class, which allows you to alternatively retrieve the same Product objects by calling a stored procedure instead. ... This ability to use both dynamic SQL and SPROCs with a clean data model layer is pretty powerful, and provides a great deal of flexibility when working on projects. [Emphasis added.]

My tests show that you don't "retrieve the same Product objects by calling a stored procedures instead." LINQ to SQL's dynamic SQL statements retrieve a DataContext.Table<TEntity> collection, while stored procedures retrieve a DataContext.ISingleResult<T> collection or DataContext.IMultipleResults collection for stored procs that return more than one resultset. Differences between the DataContext.Table<TEntity> and DataContext.ISingleResult<T> types are more significant with smart-client (i.e., WinForm) than Web projects.

Substitution of stored procedures for dynamic SQL statements for retrieving data with the LinqDataSource in simple ASP.NET Web sites with self-contained LINQ to SQL classes works as expected and requires only adding three lines of code to the PageName.aspx.cs or .vb file. However, changing to stored procedures disables one of the LinqDataSource's most important features, server-side paging (see David Hayden's LinqDataSource - LINQ to SQL ASP.NET Website Rapid Application Development post.)

Update 9/18/2007: Problems encountered when substituting stored procedures for dynamic T-SQL when populating EntitySet and EntityRef associations of DataContexts generated by LinqDataSources have been resolved.

Update 9/17/2007: There are serious issues with the GridView's ad hoc sorting when emulating the LinqDataSource's server-side sorting feature with stored procedures (see below and Dynamic ORDER BY Clauses for ASP.NET Server-Side Paging with Stored Procedures of 9/18/2007.)

Update 9/14/2007: The problem with server-side paging is fixed (see below.) 

Update 9/12/2007: This post went through a major edit on 9/12/2007. Content on the WinForms test harness has moved to another post in progress due to the length of this post.

Stored Procedures versus Dynamic SQL for LINQ to SQL Data Layers

I ordinarily use stored procedures (SPs) for updating tables that persist LINQ to SQL objects, but I rely on dynamic T-SQL parameterized prepared statements (PPSs) to retrieve data for persisted objects. Since the arrival of SQL Server 7.0, SPs have not been compiled. SPs and PPSs now deliver identical performance because their execution plans are cached in an identical manner. Adding parameters to prepared statements eliminates the SQL injection threat.

However, I'm acquainted with several DBAs for Fortune-500 firms who insist on the use of SPs for all access to "their" databases (except by themselves, of course.) It's not a good consulting practice to tell a DBA to "take a hike" when objections to using PPSs for SELECT queries arise. So I decided to look into issues that might arise when substituting SPs for T-SQL PPSs used to hydrate LINQ to SQL objects.

I also make it a practice to create data access layers (DALs) with LINQ to SQL classes compiled to class libraries. This approach delivers "plain old CLR objects" (POCOs) to business logic and higher layers with an acceptable level of persistence ignorance, which I achieve by setting the DataContext class's Access property value to Internal.(For more about persistence ignorance, see Ian Cooper Takes on DDD, TDD and PI with LINQ to SQL and Linq To Sql POCO Support.) I also wrap List<T> types with BindingListView to deliver IBindingListView collections that support filtering and advanced sorting by DataGridView controls (see Support Sorting and Filtering for List<T> Collections.)

Here's my shortlist of problems and issues that arose when I tried retrieving persisted objects to the DataContext with SPs in several trial projects, with or without the LINQ to SQL class in its own DAL assembly:

• SPs don't populate EntityRef property values or EntitySet collections by default. If you plan to navigate associations, be ready to write code to populate them and update their contents when adding or deleting a member.
• DataContext objects with ISingleResult<T> objects returned by SPs don't offer the default lazy (delayed) loading option for EntityRefs or EntitySets.
• All associated entities must be eager-loaded by custom-written code.
• List<T> objects created from ISingleResult<T> objects returned by SPs don't support change-tracking directly*. Workarounds are kluges, at best, as noted below.
• The DataContext's IdentityManager doesn't recognize List<T> members generated from ISingleResult<T> objects until they're modified, which places them under the auspices of the ChangeTracker.
• The DataContext issues a dynamic T-SQL query for each LINQ query that uses a Table<TEntity> as its data source. Without Select permission on the targeted database table(s), SQL Server spews a stream of error messages. If there's a property setting to prevent this problem, I haven't been able to find it.
• The LinqDataSource control supports only DataContext objects as data sources in the first LinqDataSource Configuration wizard dialog, Choose a Context Object. If you set the DataContext object's access to Internal in your LINQ to SQL DAL, you can't use the LinqDataSource (or create partial DataContext classes.) If you change the DataContext object's access to Public, you lose persistence ignorance.
• The LinqDataSource control supports server-side paging with ad-hoc sort column sequence and direction when you retrieve data with dynamic SQL. Substituting SPs requires writing complex CASE statements to create dynamic ORDER BY clauses. Maintaining the sort sequence requires storing sort expressions in cookies or session state. Sorting by EntityRef values leads to exceedingly complicated logic in the SP. Server-side paging requires SQL Server 2005 or later. (Added 9/17/2007.) See Dynamic ORDER BY Clauses for ASP.NET Server-Side Paging with Stored Procedures of 9/18/2007.
• The ObjectDataSource control doesn't support ISingleResult<T> objects as its data source. You can select an ISingleResult<T> type as a business object in the first configuration wizard dialog, but the related SPs are methods of the DataContext object, not the BusinessObject, so they don't appear in the second dialog.
• The ObjectDataSource control requires adding custom SELECT, INSERT, UPDATE and DELETE methods to its BusinessObject but doesn't require making the DataContext object Public, assuming that you don't need to add code to the DataContext in a partial class file.
• DataGridView columns aren't sortable when bound to a List<T> object. A DataGridView bound though a BindindingSource component or directly to an IBindingList collection created by applying the DataTable<T>.GetNewBindingList() method has sortable columns if the source of the DataTable<T> is an SQL Server table or view object. However, you can't filter a BindindingSource populated by an IBindingList**.

* The Table<TEntity> of the designated SP object type will track edits to members of a List<T> object created by invoking the the ISingleResult<T> object's ToList<T> method. However, tracking object additions to and removals from List<T> objects with corresponding Table<TEntity> objects requires a substantial amount of hand-written code.

Note: Changing the sequence's type from ISingleResult<T> to List<T> by invoking the ToList() method is required to enable adding or removing rows of bound (directly or through a BindingSource component) DataGridView controls in WinForms but not GridView controls of WebForms. ASP.NET examples don't need to invoke the ToList() method unless you encounter "The query results cannot be enumerated more than once" exceptions or otherwise need to persist the ISingleResult<T> as a collection.

** You can enable advanced sorting and filtering of BindingSource components populated with List<T> or IBindingList collections by wrapping them with a BindingListView layer, as described in the Support Sorting and Filtering for List<T> Collections post.

Note: Hopefully, Entity Framework users won't encounter these problems with stored procedures for entity retrieval. It appears from Zlatko Michailov's posts in the ADO.NET Orcas forum's Sprocs and views in June CTP thread that the <Function ...> element in the Storage (SSDL) layer, <FunctionImportMapping ...> in the Mapping (MSL) layer and <FunctionImport ...> in the Conceptual (CSDL) layer makes basic entity retrieval operations transparent to stored procedures. However it's not clear how you can specify query paths with, for example ObjectContext.Order.Include("Order.Order_Detail"), or explicitly load related objects using Order.Order_Detail.Load() with stored procedures.

LINQ to SQL Stored Procedure ASP.NET Test Harness Background

In mid-August 2007, I created a simple ASP.NET Web Site test harness with a LinqDataSource control populated from a LINQ to SQL DataContext for the Northwind Customers table and bound a GridView to it. The project includes a reference to the LINQ to SQL class library that provides the DataContext. I added a similar page with a GridView bound to an ObjectDataSource connected to the DataContext.Customers table also. Both GridViews have server-side paging and sorting, selection, edits and inserts enabled. The two pages include buttons to add and remove fictitious BOGUS Customer, Order and OrderDetail objects programmatically. The BOGUS customer lets you edit data without changing the original Northwind database content.

Here's a screen capture of the ASP.NET test harness with a LINQ to SQL class library as a DAL:

Click the image to open a full-size version.

I changed the data retrieval source for the Customer class from an autogenerated SELECT prepared statement to a SELECT * FROM Customers stored procedure, usp_GetCustomers. I was unable to connect the LinqDataSource to the DataContext.usp_GetCustomers stored procedure to create editable DataGridViews. The second dialog, Configure Data Selection, doesn't show ISingleResult<T> objects in its Table list; it only lets you select Table<TEntity> objects. Here's the code for the LinqDataSource:

<asp:LinqDataSource ID="ldsCustomers" runat="server"
    ContextTypeName="NwindLinqSProcsVB.NorthwindDataContext"
    EnableDelete="True" EnableInsert="True" EnableUpdate="True"
    TableName="Customers">
</asp:LinqDataSource>

which invokes the DataContext.GetTable(Customer) method that generates this dynamic SQL:

SELECT COUNT(*) AS [value] FROM [dbo].[Customers] AS [t0]

SELECT TOP 10 [t0].[CustomerID], [t0].[CompanyName], [t0].[ContactName],
    [t0].[ContactTitle], [t0].[Address], [t0].[City], [t0].[Region],
    [t0].[PostalCode], [t0].[Country], [t0].[Phone], [t0].[Fax]
FROM [dbo].[Customers] AS [t0]

Based on the issues I'd seen so far with SPs for LINQ to SQL data retrieval and the LinqDataSource control, I posted the following comment to Scott's post:

Thursday, August 16, 2007 7:47 PM by Roger Jennings

Hi, Scott,

Great post (as usual). I'm running into a wall trying to substitute stored procedures for dynamic T-SQL in a DataContext that serves as the data source for a LinqDataSource control, which would enable editing of a bound GridView or DetailsView. ISingleResult<EntityClass> implements IEnumerable<EntityClass> but not Table<EntityClass>, which is required for the EntityClass to be visible in the Data Source Wizard's second dialog. Is there a solution (other than moving to the ObjectDataSource) that isn't a hack?

See the bottom of oakleafblog.blogspot.com/.../linq-updates-for-8152007.html for more detail.

Thanks in advance,

--rj

Scott replied early the next morning:

Friday, August 17, 2007 2:05 AM by ScottGu [Emphasis added.]

Hi Roger,

...

Any chance you could send me an email (scottgu@microsoft.com) with a quick sample that demonstrates what you are trying to do here? If the object you are trying to edit is a Table on the DataContext, then I think you can handle this scenario by handling the Selecting event on the LinqDataSource. Alternatively, the cleanest way might be to use the ObjectDataSource.

Hope this helps,

Scott

It did help, but I wanted to do more testing before sending the sample code.

Specific Problems with ASP.NET Server-Side Paging and EntityRefs

Update 9/14/2007: The server-side paging problem was solved by a very minor fix (see below.) However, I'm still having problems with populating EntityRefs with stored procs (see end of post). 

The most vexing problems I encountered with the LinqDataSource control (other than its by-design lack of persistence ignorance) are what appears to be a bug in the way the control attempts to handle server-side paging and lazy loading with stored procedures.

Creating the DataContext in the UI Layer and Losing Server-Side Paging

Scott's suggestion to handle the LinqDataSource.Selecting event worked as expected. The following two lines manipulate the DataContext object to return an ISingleResult<Customer> object:

Protected Sub ldsCustomers_Selecting(ByVal sender As Object, _
        ByVal e As System.Web.UI.WebControls.LinqDataSourceSelectEventArgs) _
        Handles ldsOrders.Selecting
    Dim dcNwind = New NorthwindDataContext()
    e.Result = dcNwind.usp_GetCustomers
End Sub

which generates this T-SQL:

declare @p3 int
set @p3=NULL
exec sp_executesql N'EXEC @RETURN_VALUE = [dbo].[usp_GetCustomers] ',
N'@RETURN_VALUE int output', @RETURN_VALUE=@p3 output
select @p3

Note: All T-SQL PPSs and stored procedure calls are from SQL Server Profiler, not the DataContext.Log property. Server-side paging is available only with SQL Server 2005+ as the back-end.

Unfortunately, the DataContext object exposes critical properties that should be kept private from upper layers, such as the database connection string.

Notice that there's no SELECT COUNT(*) AS [value] FROM [dbo].[Customers] AS [t0] statement in the preceding batch. Without the TotalRowCount value, paging reverts to the traditional (for ASP.NET) client-side paging that requires all Customer objects to be retrieved. Moving from server-side to client-side paging greatly increases the load on the database server for long lists.

Attempts to Fix the Server-Side Paging Problem

It should be possible to regain server-side paging by executing a stored procedure that emulates the LinqDataSource's PPS. Here's the stored procedure equivalent of the dynamic SQL statement for server-side paging:

CREATE PROCEDURE [dbo].[usp_GetCustomersPaged] (
@maximumRows int = 10,
@startRowIndex int = 20)
-- Created from query generated by LinqDataSource
AS
SET NOCOUNT ON
SELECT TOP (@maximumRows) [t1].[CustomerID], [t1].[CompanyName],
    [t1].[ContactName], [t1].[ContactTitle], [t1].[Address],
    [t1].[City], [t1].[Region], [t1].[PostalCode], [t1].[Country],
    [t1].[Phone], [t1].[Fax]
FROM (
    SELECT ROW_NUMBER() OVER (ORDER BY [t0].[CustomerID],
       [t0].[CompanyName], [t0].[ContactName], [t0].[ContactTitle],
       [t0].[Address], [t0].[City], [t0].[Region], [t0].[PostalCode],
       [t0].[Country], [t0].[Phone], [t0].[Fax]) AS [ROW_NUMBER],
       [t0].[CustomerID], [t0].[CompanyName], [t0].[ContactName],
       [t0].[ContactTitle], [t0].[Address], [t0].[City], [t0].[Region],
       [t0].[PostalCode], [t0].[Country], [t0].[Phone], [t0].[Fax]
    FROM [dbo].[Customers] AS [t0]
    ) AS [t1]
WHERE [t1].[ROW_NUMBER] > @startRowIndex
SET NOCOUNT OFF;

You execute the preceding stored proc with the following Selecting event handler:

Protected Sub ldsCustomers_Selecting(ByVal sender As Object, _
      ByVal e As System.Web.UI.WebControls.LinqDataSourceSelectEventArgs) _
      Handles ldsOrders.Selecting
   Dim dcNwind = New NorthwindDataContext()
   e.Arguments.RetrieveTotalRowCount = False
   e.Arguments.TotalRowCount = 91
   Dim intMaxRows As Integer = e.Arguments.MaximumRows
   Dim intStartRow As Integer = e.Arguments.StartRowIndex
   Dim intRowCount As Integer = 0
   e.Result = dcNwind.usp_GetCustomersPaged(intMaxRows, intStartRow)
End Sub

Note: If you don't set e.Arguments.RetrieveTotalRowCount = False, paging is disabled. If you don't supply a value for e.Arguments.TotalRowCount, only a header with no rows or paging elements appears. The fixed value 91 is present for testing only. The Beta 2 version of LINQ to SQL doesn't codegen output parameter code in Public Function StoredProcName(Parameters) blocks, so you must execute a SELECT COUNT(*) FROM TableName stored proc with a DataContext.ExecuteQuery(Of Integer)("EXEC usp_GetCustomersCount").First() or similar expression.

Unfortunately, the LinqDataSource control only correctly executes queries with 0 as the StartRowIndex value. Other values result in an empty GridView control, despite the fact that the SELECT stored procedure returns the proper rowset. This appears to be a bug.

Update 9/14-15/2007: D'oh! All that was needed to make stored procedure paging work past page 1 was to set the LinqDataSource.AutoPage property value to False. If you download the LinqDataSourcePagingTestHarness.zip sample, just make the change and you're good to go with the stored procedure provided.

You must customize your query to add custom WHERE clause constraints, such as the customer's Country field value in the Order subselect query below:

ALTER PROCEDURE usp_GetOrdersByCountryPaged (
    @Country nvarchar(15) = 'USA',
    @MaximumRows int = 18,
    @StartRowIndex int = 0)
AS
SET NOCOUNT ON
SELECT TOP (@MaximumRows) t1.OrderID, t1.CustomerID, t1.EmployeeID,
    t1.OrderDate, t1.RequiredDate, t1.ShippedDate, t1.ShipVia,
    t1.Freight, t1.ShipName, t1.ShipAddress, t1.ShipCity,
    t1.ShipRegion, t1.ShipPostalCode, t1.ShipCountry
FROM (
    SELECT ROW_NUMBER() OVER (ORDER BY t0.OrderID DESC) AS ROW_NUMBER,
        t0.OrderID, t0.CustomerID, t0.EmployeeID, t0.OrderDate,
        t0.RequiredDate, t0.ShippedDate, t0.ShipVia, t0.Freight,
        t0.ShipName, t0.ShipAddress, t0.ShipCity, t0.ShipRegion,
        t0.ShipPostalCode, t0.ShipCountry
        FROM dbo.Orders AS t0
WHERE t1.ROW_NUMBER > @StartRowIndex AND
    CustomerID IN (SELECT CustomerID FROM Customers WHERE Country = @Country)
ORDER BY t1.OrderID DESC
SET NOCOUNT OFF;

Update 9/13/2007: The preceding issue is reported as Bug ID 297959. The simplified Web site test harness included with the bug report is available from SkyDrive as LinqDataSourcePagingTestHarness.zip. The connection string expects to find the Northwind database permanently attached to a local .\SQLEXPRESS instance. The T-SQL script to create the stored procedure is included with the source code.

Forcing Dynamic SQL by Replacing Foreign Key with EntityRef Values

Scott Guthrie demonstrated replacement of a Product object's SupplierID and CategoryID values with Supplier.CompanyName and Category.CategoryName EntityRef values in his July 16, 2007 LINQ to SQL (Part 5 - Binding UI using the ASP:LinqDataSource Control) post. Performing a similar change by altering the definition for an Order object's EmployeeID column to a Template Field as shown here:

<asp:templatefield HeaderText="EmployeeID">
    <edititemtemplate>
        <asp:TextBox ID="TextBox1" runat="server"
             Text='<%# Bind("EmployeeID") %>'></asp:TextBox>
    </edititemtemplate>
    <itemtemplate>
        <asp:Label ID="Label1" runat="server"
             Text='<%# Eval("Employee.LastName") %>'></asp:Label>
    </itemtemplate>
</asp:templatefield>

and substituting the code to display Employee.LastName causes the LinqDataSource control to issue the correct stored procedure call:

declare @p4 int
set @p4=NULL
exec sp_executesql N'EXEC @RETURN_VALUE = [dbo].[usp_GetOrdersByCountry] @Country = @p0',N'@p0 nvarchar(3),@RETURN_VALUE int output',@p0=N'USA',@RETURN_VALUE=@p4 output
select @p4

followed by this dynamic T-SQL statement to retrieve the employee object for each row until all nine unique employee objects have been retrieved:

exec sp_executesql N'SELECT [t0].[EmployeeID], [t0].[LastName],
    [t0].[FirstName], [t0].[Title], [t0].[TitleOfCourtesy],
    [t0].[BirthDate], [t0].[HireDate], [t0].[Address], [t0].[City],
    [t0].[Region], [t0].[PostalCode], [t0].[Country],
    [t0].[HomePhone], [t0].[Extension], [t0].[Notes], [t0].[ReportsTo]
FROM [dbo].[Employees] AS [t0]
WHERE [t0].[EmployeeID] = @p0',N'@p0 int',@p0=7

which would result in a stream of error messages due to lack of client select privileges.

Update 9/14-18/2007: Matt Warren demonstrated how to populate EntitySet associations with stored procedures in these two posts: Re: Populate association with SP (9/7/2007) and Re: SPROCS, multiple results, and associations (9/12/2007). As noted in my reply at the bottom of the second post, Matt's solution populates EntitySet associations, but not EntityRef associations, with stored procs. I haven't found any other source of information on this topic, so I had to guess at the EntityRef syntax. On 9/18/2007 Matt provided the answer to what was wrong with my original code in a reply to my Problem Populating a LinqDataSource's EntityRef Association with a Stored Proc post in the Linq Project General forum. These later changes are underlined.

Here's the VB 9.0 code required Orders page to use a stored procedure to populate eager-load the Order.OrderDetails EntitySet and three EntityRef associations from stored procedures when using a LinqDataSource populated by a stored procedure:

Protected Sub ldsOrders_Selecting(ByVal sender As Object, _
        ByVal e As System.Web.UI.WebControls.LinqDataSourceSelectEventArgs) _
        Handles ldsOrders.Selecting
    Dim dlOrder As New DataLoadOptions()
    dlOrder.LoadWith(Of Order)(Function(o) o.Customer)
    dlOrder.LoadWith(Of Order)(Function(o) o.Employee)
    dlOrder.LoadWith(Of Order)(Function(o) o.Shipper)
    dlOrder.LoadWith(Of Order)(Function(o) o.Order_Details)
    dcNwind.LoadOptions = dlOrder
    e.Arguments.RetrieveTotalRowCount = False
    e.Arguments.TotalRowCount = dcNwind.ExecuteQuery(Of Integer) _
        ("EXEC usp_GetOrdersCountByCountry 'USA'").First()
    e.Result = dcNwind.usp_GetOrdersByCountryPaged("USA", _
        e.Arguments.MaximumRows, e.Arguments.StartRowIndex).ToList()
End Sub

Note: The three commented EntityRef instructions throw exceptions when calling the e.Result assignment instruction. This appears to be a bug (see . However, they don't throw exceptions if added to the ldsOrders_ContextCreating event handler. These instructions don't throw exceptions when the correct override code is provided. (Updated 9/18/2007.)

Here's the correct method override code for executing the stored procedures to add to the Partial Public Class NorthwindDataContext class:

Private Function LoadOrder_Details(ByVal Ord As Order) _
        As IEnumerable(Of Order_Detail)
    Return usp_GetOrder_DetailsByOrderID(Ord.OrderID)
End Function

Private Function LoadCustomer(ByVal Ord As Order) As Customer
    Return usp_GetCustomerByID(Ord.CustomerID).FirstOrDefault()
End Function

Private Function LoadEmployee(ByVal Ord As Order) As Employee
    Return usp_GetEmployeeByID(Ord.EmployeeID).FirstOrDefault()
End Function

Private Function LoadShipper(ByVal Ord As Order) As Shipper
    Return usp_GetShipperByID(Ord.ShipVia).FirstOrDefault()
End Function

The method override code for many:1 associations hasn't been confirmed yet. I've started a new thread named Problem Populating a LinqDataSource's EntityRef Association with a Stored Proc in the Linq Project General forum. (Updated 9/18/2007.)

Remaining Issues with the LinqDataSource Not Related to Stored Procedures

1. Specifying eager loading with LoadWith expressions doesn't appear to save SQL Server 2005 round-trips when populating EntitySet and EntityRef associations of DataContexts created by LinqDataSource controls.

2. There doesn't appear to be an obvious property accessible from an event handler that exposes DataContexts created by LinqDataSource controls. This is required to enable local logging of T-SQL sent to the server if one doesn't have SQL Server Profiler handy.

(Updated 9/18/2007.)

Technorati Tags: LINQ to SQL, LinqDataSource, LINQ, ASP.NET, Persistence Ignorance, GridView, Stored Procedures, Paging, EntityRefs, VS 2008 Beta 2

LINQ and Entity Framework Posts for 9/7/2008+

Dinesh Kulkarni Posts Changes to LINQ to SQL RTM

I missed these two important posts because Dinesh posted them to the LINQ Project General forum instead of his Dinesh's Cyberstation blog:

LINQ to SQL Beta2 to RTM Key Changes (August 27, 2007)

• Table<TEntity>'s Add[All]()/Remove[All]() method names will change to Insert[All]OnSubmit()/Delete[All]OnSubmit(). In my opinion the new names apply relational metaphors to objects, which is a bad practice.
• The OnValidate() event gets a ChangeAction Enum parameter with None, Delete, Insert, and Update members. Relational terminology again applied to objects.
• Local method calls in LINQ to SQL projections from VB LINQ to XML queries are now supported. Dinesh provides a LINQ to XML query example.

Beta2 to RTM Changes in Attach() Behavior (September 11, 2007)

• Attempts to Attach() an object that is not truly detached from another DataContext by serialization/deserialization will throw an exception. An example is an object that has deferred loaders from another DataContext.
• Objects Attach()ed with SubmitChanges() don't have ChangeSet entities will default to "unchanged" status and no longer will be inserted to the underlying data store by default.

Problems Using Stored Procedures for LINQ to SQL Data Retrieval with ASP.NET

The final version of my first post about Problems Using Stored Procedures for LINQ to SQL Data Retrieval with ASP.NET 3.5's LinqDataSource and GridView controls is done. The primary issues with the LinqDataSoruce are:

• Lack of Persistence Ignorance (in general, not a result of using stored procs for data retrieval)
• Loss of server-side paging and consequent increased load on the database
• Inability to lazy-load EntityRefs without spewing dynamic SQL queries

Subsequent posts will cover issues with BindingSource components and DataGridView controls when using stored procedures to hydrate LINQ to SQL entities. (Update 9/12/2007.)

Frans Bouma Starts LINQ to LLBLGen Pro Series

Frans Bouma, the developer of the LLBLGen Pro commercial O/R mapping tool and principal antagonist of LINQ to SQL and the Entity Framework, has started a project to adapt LINQ to LLBLGen Pro (or vice versa). His first posts are here:

1. Developing Linq to LLBLGen Pro, Day 0 (September 11, 2007)
2. Developing Linq to LLBLGen Pro, Day 1 (September 12, 2007)

Thomas Wagner likens Frans' new series to A diary of LINQ pain.

Be sure to read the comments to the first post! (Update 9/13/2007.)

Jim Ng on Lambda Expressions and Expression Trees in VB

Jim Ng, the dev lead for the VB compiler team, promises a series of VB Team posts on lambda expressions and expression trees. The first post of the series is, appropriately enough, Lambda Expressions and Expression Trees of September 11, 2007. He's also written "Lambda Expressions," a "Basic Instincts" column for the September 2007 issue of MSDN Magazine. (Updated 9/12/2007.)

Julie Lerman Explains Why Entities Are Missing Foreign Key Values 

Julie's September 11, 2007 Entity Data Model Associations: Where's my Foreign Key? explains why entity properties don't include foreign keys in the Entity Data Model (EDM) unless they're also part of another entity's primary key. They've been sacrificed in favor of association and navigation properties. For example, the Order entity that's persisted to the Northwind Orders table has no CustomerID, EmployeeID, or ShipperID property; instead it has many:1 navigation properties to associations with Customer, Employee and Shipper entities. (I've singularized the entity names in Julie's EDM diagram.)

The Order_Detail entity's ProductID property, which is a member of the composite primary key, was missing in previous CTPs due to the ADO.NET team's over-zealous pursuit of object purity. Of course, it could have been worse: they could have hidden the OrderID property, too. As Julie points out, relationally-oriented folks like Jay Traband of IdeaBlade and me have complained about the missing foreign keys here and here. Lets hope that the ADO.NET team delivers on their intention to surface foreign key values as an option in the release version. (Updated 9/11/2007.)

Don Demsack (DonXML) Talks LINQ to XML on .NET Rocks Show #271

Don Demsack (a.k.a. DonXML) discusses LINQ to XML and the new XML language features in Visual Basic with Carl Franklin and Richard Campbell in this 66-minute .NET Rocks! audio clip posted September 11, 2007: Don Demsak on LINQ to XML.

ADO.NET Team Wants Entity Framework Docs Feedback

Glenn Gailey of the Entity Framework User Education (UE) team posted Please Give Feedback on Entity Framework Docs on September 11, 2007 requesting feedback :

• When a topic is particularly effective or ineffective.
• When the topic doesn't reflect the product behavior (i.e. there's a doc bug).
• When an important use case or scenario is not covered or more documentation or examples are needed.

Glenn includes instruction on how to submit feedback from the online docs at ADO.NET Entity Framework Beta 2 or the VS 2008 UI.

I'm planning to gripe about the lack of information on the topic of retrieving data by stored procedures. I'm sure Julie Lerman will make the same complaint.

Installing Project Jasper After Entity Framework Tools Clobbers Them

Running the installer that you download from the Jasper : MIX 2007 CTP page after installing the EF Tools August 2007 CTP causes VS 2008 to be unable to load the EDM Designer bits (Microsoft.Data.Entity.Design.Package.MicrosoftDataEntityDesignPackage, Microsoft.Data.Entity.Design.Package, Version 1.0.0.0 ... .)

To regain proper loading of the EDM Designer, uninstall the Microsoft Project Jasper May 2007 CTP, uninstall the ADO.NET Entity Framework Tools and reinstall the Tools.

Update 9/12/2007: Reinstalling the tools didn't solve the problem. I've posted a question about this problem in the ADO.NET Orcas forum.

Update 9/13/2007: Microsoft's Jeff Derstadt said in answer to my question:

The Jasper CTP is not supported for VS 2008 Beta 2. The installer included a patch for an earlier VS 2008 CTP where the EDM tools would not work properly. You will likely need to reinstall VS 2008 Beta 2 to remove this patch.

Andy Conrad in a reply to my comment asking if he had tried it said:

Just to be clear - the sample works with Jasper running over Beta 1 of Orcas.  The problem exists when installing Jasper over Entity Framework Beta 2.  We are currently investigating a solution for this.

I wonder how many folks are running Orcas Beta 1.

(I found this out the hard way when attempting to try the Cappuccino Project Jasper sample by Tok Wee Hyong of the .Net Singapore user group that Andrew Conrad recommended. I wasn't able to get it to run on either of my systems running VS 2008 Beta 2 due to reference version conflicts and lack of the multi-targeting selection button on the project Open dialog. Subsequently, I determined that installation of the Jasper CTP ate my VS 2008 Beta 2 installation.)

John Stallo Explains Building N-Tier Apps in VS 2008 with VB

This Channel 9 Interview - John Stallo on Building N-Tier Applications in VS 2008 (Beth Massi) post of September 10, 2007 from the VB Team also has a pointer to a walkthrough with a strongly typed DataSet noted my LINQ and Entity Framework Updates for 8/29/2007+ post. As mentioned in that post, the topic is apropos because you can substitute LINQ to SQL for the DataSet to evaluate if LINQ to SQL is right for you in a WCF-based service environment.

Mike Taulty Continues to Explore LINQ to XML

His first Monday post XML, LINQ, Databinding a ListBox to an Anonymous Type does exactly what it says, but you can expect to hear more on this topic. Northwind & Photos - How Many Times Will I Write This Code? is a scratch pad for code to write the Northwind.Employees.Photo values to a file instead of encoding them as base64 to a CDATA element. (Updated 9/10/2007.)

Pablo Castro Requests Feedback on Astoria Data Payload Formats

In Sunday's Astoria Design: payload formats Project Astoria Team post, Pablo Castro asks for input on the team's proposed three data payload formats:

• ATOM/APP (Atom Publishing Protocol) because it's popular: "The important characteristic about ATOM/APP is that everybody speaks it, and it’s increasingly becoming kind of a lingua franca of the web when it comes to data exchange between unrelated agents."
• JSON (JavaScript Object Notation) also because it's popular: "[JSON] has become popular lately mostly (my guess) because it makes it easy to consume results from services in JavaScript environments, particularly inside web browsers."
• Web3S (Web Structured, Schema’d & Searchable) as a replacement for POX (Plain Old XML) because "folks at the Live organization in Microsoft were working on a protocol for data exchange that would be used uniformly across many different services."

RDF (Resource Description Format), which was included in the May 2007 CTP has been dropped because "we didn’t see any early user actually using it and we haven’t seen a particular popular scenario where RDF was a must-have."

Without giving it a lot of thought so far, I think I prefer POX to Web3S. (Updated 9/10/2007.)

Erik Meijer Discusses LINQ in .NET Rocks Show #270

Erik Meijer, LINQ's chief spokesman on the conference circuit, joins Carl Franklin and Richard Campbell in Show #270, Erik Meijer on LINQ!!, to deliver 66 minutes of "detail about LINQ from soup to nuts. This is a very technical discussion, and not an overview." Erik starts at 06:30 and covers LINQ to XML, LINQ to SQL's DataContext object, SQL Server versus CLR data types. He discusses Standard Query Operator lambda expressions and their relationship to delegates and monads starting at 43:00, and LINQ's limitations at 52:00. At 53:45, Erik recommends using LINQ to Objects for queries across domain-specific implementations. He explains the SelectMany operator at 56:30, instance-based data updating at 58:45, LINQ to Entities at 1:00:30, and third-party LINQ providers at 1:01:30. (Updated 9/9/2008.)

Julie Lerman Hydrates EF Entities with Stored Procedures

If you think using stored procedures to hydrate (and cripple) LINQ to SQL with ISingleResultset<T> collections is a pain, wait until you try doing the same with Entity Framework entities. As Julie mentioned in her September 8, 2007 Finishing up READ Stored Procs brought in by the Entity Framework EDM Wizard post, selecting a stored procedure node in the EDM Wizard adds a function stub to the EDMX file's <edmx:StorageModels .../> section but doesn't add any elements to the <edmx:ConceptualModels .../> or <edmx:Mapping .../> sections. And there's no documentation for using stored procs to retrieve entity data except a brief answer by Zlatko Michailov on June 9, 2007 to Julie's Sprocs and views in June CTP forum question.

Julie uses Northwind's CustOrderHist stored proc as and example to provide the key to what sections you can generate from the designer and those you must write yourself. She promises that an article in her DevLife blog with step-by-step instructions will be posted shortly. She'll update her personal blog post and I'll update this item when the article's posted.

Mike Taulty Saves Memory by Streaming Documents with LINQ to XML

LINQ to XML's DOM programming model is somewhat more memory-efficient than conventional XML DOM processing, but public streaming input and output APIs would enable processing very large documents without consuming an excessive (usually about 4X the document size) amount of memory by a DOM tree.

Unfortunately, the LINQ to XML team decided in March 2007 to pull the streaming input API from the VS 2008 RTM. However, they left the XStreamingElement type in to enable implementing custom streaming input scenarios. XStreamingElement, to quote the XML Team in their March 5, 2007 Streaming with LINQ to XML - Part 1 post, "allows XML-like trees of IEnumerable<T> instances that can be lazily consumed while being saved as XML text."

Mike demonstrates in his his September 8, 2007 LINQ to XML and Larger Files post that pull-parsing the input document with an XmlReader in a "custom axis" function and a C# iterator reduces memory consumption from 190 MB (for the DOM) to 2 MB for a 100 MB XML file. With conventional output to a file, memory consumption increases to 7 MB. However, substituting XStreamingElement for XElement reduces memory usage to 428 KB.

For VB programmers, Mike provides an XElementEnumerable class to replace C#'s iterator, which VB 9.0 doesn't support.

Scott Guthrie Demos Custom Queries with the LinqDataSource

Scott's LINQ to SQL: Part 9 - Using a Custom LINQ Expression with the <asp:LinqDatasource> control post of late Friday night shows you how to substitute custom LINQ query expressions for the single-table queries that you create with the LinqDataSource's Configure Data Source wizard. The wizard limits customization to selecting all or specific properties of a LINQ to SQL entity to retrieve and specifying a Where clause filter and OrderBy sort of a single table. If you don't retrieve all fields, bound controls aren't updatable.

You can substitute a custom LINQ query expression for the default expression that the wizard generates in a handler for the the LinqDataSource's Selecting event. Just pass the query result's IEnumerable<T> to the e.Result property. Inline editing in DataGrids or other bound controls won't work if you pass a custom data shape or projection.

Update 8/12/2007: Be wary of returning List<T> objects from stored procedures (see Problems Using Stored Procedures for LINQ to SQL Data Retrieval, currently a work in progress.)

Andrew Conrad Posts Generic CopyToDataTable Extension Method

Andrew's Science Project post of September 7, 2007 includes the source code for a LINQ to DataSet extension method that removes the CopyToDataTable<T> method's restriction of the generic T type parameter to DataRow. The extension method enables loading instances of anonymous types into DataRows. Andrew provides the following usage notes:

• The initial schema of the DataTable is based on schema of the type T. All public property and fields are turned into DataColumns.
• If the source sequence contains a sub-type of T, the table is automatically expanded for any addition[al] public properties or fields.
• If you want to provide a existing table, that is fine as long as the schema is consistent with the schema of the type T. 

Andrew's sample source sequence consists of a base class Item, which has public Id, Price and Genre properties. Item has two subclasses: Book has a public Author property and Item has a public Director property. The DataTable generated from the anonymous type has five columns: Id, Price, Genre, Director and Author.

Tip: The sample code requires a reference to System.Data.DataSetExtensions.dll and using System.Data; using System.Data.DataSetExtensions; and using System.Reflection; declarations.

For a brief review of databinding with LINQ to DataSet, see Erick Thompson's LINQ to DataSet – Data Binding post of August 24, 2007. Daniel Moth has a July 27, 2007 LINQ to DataSet item with details of the classes in the System.Data.DataSetExtensions namespace, and here are links to three introductory LINQ to DataSet posts from February 2007: Part 1, Part 2 and Part 3.

Update 9/12/2007: Andrew says in a comment that he intends to update the code to solve these two issues:

1)  The code is not catching the error case when the type T of the source sequence does not match the schema of the provided datatable.  I suppose I could extend the table automatically in this case.

2)  The results of your query is just a sequence of scaler values.  The code wasn't really designed for this and I am not sure I see much value, but I suppose I could just make a table with a single column.

Upgrade List<T> Collections to BindingListView<T> for Sorting and Filtering

My September 7, 2007 Support Sorting and Filtering for List<T> Collections post shows you how to take advantage of Andrew Davey's BindingViewList class to bring advanced sorting and filtering features to controls and components bound to List<T> collections created by invoking LINQ Query's IEnumerable<T>.ToList() method.

Andrew Davey Generates Assertions with LINQ Expression Trees

Andrew Davey's September 7, 2007 Assertions via Linq Expressions post suggests minimizing the boredom of duplicating the code you're testing in a text message string, such as Debug.Assert(input != null, "input != null");, when writing Debug or NUnit assertions. His approach is to compile and invoke LINQ-style expression trees to specify the assertions and then invoke ToString() to write the string.

For example, calling this function

void Assert<T>(T obj, Expression<Func<T, bool>> test)
{
  System.Diagnostics.Debug.Assert(test.Compile().Invoke(obj),
      test.ToString());
}

with

Assert(input, i => i != null);

is the equivalent of writing

Debug.Assert(input != null, "input != null");

Andrew also suggests extension methods to provide alternative syntaxes.

Note: Andrew is the Bristol (UK) developer who wrote the BindingListView wrapper to make List<T> collections sortable and filterable (see Support Sorting and Filtering for List<T> Collections.)

Technorati Tags: LINQ to XML, LINQ to SQL, LinqDataSource, LINQ to Entities, Entity Framework, LINQ, Lambda Expressions, Expression Trees, Expressions, Extension Methods, XStreamingElement, VB 9.0, Unit Testing, NUnit