Thursday, January 31, 2008

SD Times' "Does .NET With LINQ Beat Java?" Article Raises More Questions Than It Answers

David Worthington's Does .NET With LINQ Beat Java? Software Development Times article of January 30, 2008 has a snappy head but the article's deck, "Framework's data query capabilities give it an edge, experts claim," appears to be disputed by at least one "expert," namely Jonathan Bruce of DataDirect Technologies (see below.)

Worthington quotes Jonathan as saying:

DataDirect’s Bruce acknowledged that from a productivity point of view, LINQ combined with its tooling gives .NET shops a productivity advantage that the Java community cannot match. He credits Microsoft’s ability to “package [productivity patterns and tooling] all up into something useful."

That said, he noted that LINQ is an unproven technology that is new to the market and said that he could not imagine anyone making corporate bets on LINQ just yet. “On the Java side, data access is rounded and stable. As a technology officer, it is an easier bet to make [on] what will reduce risks from a data perspective,” he explained. [Emphasis added.]

If Bruce "could not imagine anyone making corporate bets on LINQ just yet," why has DataDirect announced their intention to provide EntityClient-enabled ADO.NET data providers for the Entity Framework? Java's "rounded and stable" data access sounds to me like an apologia for the status quo (a.k.a. JDBC).

Apparently, Bruce isn't aware of LINQ to Objects, LINQ to XML, and the widening variety of current third-party LINQ implemetations and upcoming Microsoft versions:

Another example is XQuery, a language designed to query XML data. XQuery, he said, does a better job with data from many venues because of XML’s flexibility. By contrast, he claimed, the current implementation of LINQ is very targeted to what the developer is querying and the ability to “mash up data sources” has not been delivered yet, Bruce explained.

One of LINQ's strong points is it's "ability to 'mash up data sources'." Jon Udell's LINQ 101 post of September 28, 2005 demonstrated a three-way join between an XML data source and two CLR objects with the PDC 2005's LINQ preview. You can now join generic sequences from sets of in-memory objects (LINQ to Objects), XML documents, relational databases (LINQ to SQL and LINQ to Databases), and other domains with LINQ to SharePoint, LINQ to ActiveDirectory, LINQ to Amazon, and LINQ to Streams. If this capability for joining IEnumerable<T> sequences doesn't enable mashups, I don't know what does.

I find LINQ to XML to be a much more approachable and practical method for querying and composing XML Infosets than XQuery and/or XSLT. It's interesting that Microsoft is working on LINQ to Stored XML to supplement or replace its pre-XQuery 1.0 implementation for the SQL Server 2005+ xml data type.

Note: I have some questions about Bruce's prognostications in the "Jonathan Bruce's Data Services and LINQ Crystal Ball Appears a Bit Cloudy" topic of LINQ and Entity Framework Posts for 1/28/2008+.

Tuesday, January 29, 2008

Logging EntityBag's SOAP Request and Response Messages

Windows Communication Framework's WCF Service Configuration Editor lets you log SOAP request and response messages, as well as specify WCF service and client configuration declaratively. Newly created WCF services that use the HTTP transport specify the wsHttp binding, which applies SOAP message-level security with data encryption, digital signatures, and enables other WS-* security features.

WCF's service configuration is specified by the <system.serviceModel> section of the service project's Web.config (or App.config) file. Following are default service configuration values of Danny Sullivan's sample C# EntityBagSample.sln project from the NorthwindService.csproj project's Web.config file:

<system.serviceModel>
  <services>
    <service name="NorthwindService.Service1"
     behaviorConfiguration="NorthwindService.Service1Behavior">
      <!-- Service Endpoints -->
      <endpoint address="" binding="wsHttpBinding" contract="NorthwindService.IService1"
       bindingConfiguration="wsHttp">
        <!-- 
            Upon deployment, the following identity element should be removed or replaced to reflect
            the identity under which the deployed service runs.  If removed, WCF will infer an 
            appropriate identity automatically.
        -->
        <identity>
          <dns value="localhost"/>
        </identity>
      </endpoint>
      <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/>
    </service>
  </services>

  <bindings>
    <wsHttpBinding>
      <binding name="wsHttp" maxReceivedMessageSize ="50000000" maxBufferPoolSize="50000000" >
      </binding>
    </wsHttpBinding>
  </bindings>

  <behaviors>
    <serviceBehaviors>
      <behavior name="NorthwindService.Service1Behavior">
        <!-- To avoid disclosing metadata information, set the value below to false and remove 
             the metadata endpoint above before deployment -->
        <serviceMetadata httpGetEnabled="true"/>
        <!-- To receive exception details in faults for debugging purposes, set the value below 
             to true.  Set to false before deployment to avoid disclosing exception information -->
        <serviceDebug includeExceptionDetailInFaults="true"/>
      </behavior>
    </serviceBehaviors>
  </behaviors>
</system.serviceModel>

The basicHttp binding provides no message- or transport-level security by default, which enables comparing performance and message transport size of insecure and secure services.

Note: If you don't want to change the security model, go to step 5 to enable message logging.

It's a relatively simple process to change the transport by the following steps:

1. Open EntityBagSample.sln and choose Tools, WCF Configuration Editor, choose File, Open and navigate to and open the ...\entitybag\NorthwindService\Web.config file, and expand Services and Bindings nodes.

2. Click the Bindings node and the New Binding Configuration link to open the binding page. Type basicHttp as the Name and change the MaxBufferPoolSize, MaxBufferSize, and MaxReceivedMessageSize values to very large number, such as 50000000.

3. Click the top Endpoints node to open its Service EndPoint page, change its Name to NwBasicHttp or the like and select basicHttpBinding from the Binding dropdown list.

4. Open the BindingConfiguration list and choose the basicHttp configuration you created in step 2.

5. Click the Diagnostics node and click the Message Logging section's Enable Message Logging Link to turn Message Logging on and create an  ...\entitybag\NorthwindService\Web_messages.svclog file. Optionally, click the Enable Log Auto Flush link to allow reading the log file while the service is running.

6. Click the Diagnostics node's Message Logging item to display the Message Logging page, set the LogEntireMessage and LogMessagesAtServiceLevel properties to True and increase the value of MaxSizeOfMessageToLog to a very large number.

 

7. Choose File, Save to save your changes. NorthwindService's Web.config file's <system.serviceModel> section now appears as shown here (changes are set bold):

<system.serviceModel>
  <diagnostics>
    <messageLogging logEntireMessage="true" logMalformedMessages="true"
      logMessagesAtServiceLevel="true" logMessagesAtTransportLevel="true"
      maxSizeOfMessageToLog="50000000" />
  </diagnostics>
  <services>
    <service behaviorConfiguration="NorthwindService.Service1Behavior"
      name="NorthwindService.Service1">
      <endpoint address="" binding="basicHttpBinding" bindingConfiguration="basicHttp"
        name="NwBasicHttpEP" contract="NorthwindService.IService1">
        <identity>
          <dns value="localhost" />
        </identity>
      </endpoint>
      <endpoint address="mex" binding="mexHttpBinding" name="NwMexEP"
        contract="IMetadataExchange" />
    </service>
  </services>

  <bindings>
    <basicHttpBinding>
      <binding name="basicHttp" maxBufferSize="50000000" maxBufferPoolSize="50000000"
        maxReceivedMessageSize="50000000" />
    </basicHttpBinding>
    <wsHttpBinding>
      <binding name="wsHttp" maxBufferPoolSize="50000000" maxReceivedMessageSize="50000000" />
    </wsHttpBinding>
  </bindings>

  <behaviors>
    <serviceBehaviors>
      <behavior name="NorthwindService.Service1Behavior">
        <!-- To avoid disclosing metadata information, set the value below to false 
             and remove the metadata endpoint above before deployment -->
        <serviceMetadata httpGetEnabled="true"/>
        <!-- To receive exception details in faults for debugging purposes, set the 
             value below to true.  Set to false before deployment to avoid 
             disclosing exception information -->
        <serviceDebug includeExceptionDetailInFaults="true"/>
      </behavior>
    </serviceBehaviors>
  </behaviors>
</system.serviceModel>

8. You must match the TestApp.csproj client project binding with that of the service, so choose File, Open and navigate to and open ...\entitybag\TestApp\App.config. Repeat step 2 for the Client node and step 3, except change the Endpoint name to basicHttpBinding_IService1.

 

9. Choose File, Save to save your changes. The TestApp's App.Config file's <system.serviceModel> section now appears as shown here (changes are set bold):

<configuration>
  <system.serviceModel>
    <bindings>
      <basicHttpBinding>
        <binding name="basicHttp" maxBufferSize="50000" maxBufferPoolSize="50000"
          maxReceivedMessageSize="50000" />
      </basicHttpBinding>
      <wsHttpBinding>
        <binding name="WSHttpBinding_IService1" closeTimeout="00:01:00"
          openTimeout="00:01:00" receiveTimeout="00:10:00" sendTimeout="00:01:00"
          bypassProxyOnLocal="false" transactionFlow="false" hostNameComparisonMode="StrongWildcard"
          maxBufferPoolSize="5000000" maxReceivedMessageSize="5000000"
          messageEncoding="Text" textEncoding="utf-8" useDefaultWebProxy="true"
          allowCookies="false">
          <readerQuotas maxDepth="32" maxStringContentLength="8192" maxArrayLength="50000000"
            maxBytesPerRead="4096" maxNameTableCharCount="16384" />
          <reliableSession ordered="true" inactivityTimeout="00:10:00"
            enabled="false" />
          <security mode="Message">
            <transport clientCredentialType="Windows" proxyCredentialType="None"
              realm="" />
            <message clientCredentialType="Windows" negotiateServiceCredential="true"
              algorithmSuite="Default" establishSecurityContext="true" />
          </security>
        </binding>
      </wsHttpBinding>
    </bindings>
    <client>
      <endpoint address="http://localhost:56475/Service1.svc" binding="basicHttpBinding"
        bindingConfiguration="basicHttp" contract="ServiceReference1.IService1"
        name="basicHttpBinding_IService1">
        <identity>
          <dns value="localhost" />
        </identity>
      </endpoint>
    </client>
  </system.serviceModel>

10. You must associate files with the *.svclog extension with ServiceModel Trace Viewer tool (SvcTraceViewer.exe), so navigate to and right-click ...\entitybag\NorthwindService\Web_messages.svclog, choose Open With, click Browse, navigate to and double-click \Program Files\Microsoft SDKs\Windows\v6.0A\Bin\SvcTraceViewer.exe, and set it as the default program for opening .svclog files.

11. Run EntityBagSample.sln a couple of times to generate trace files, and then double click Web_messages.svclog to open the Trace Viewer. Click the Activity item to load the main pane, and then select a message. Here's the screen capture of a GetCustomerAndOrders SOAP request message:

Click screen capture to open a full-size image.

Click here to open the full request message in IE (1.2 kB).

12. Click the next message after a request message to review the SOAP response message to the GetCustomerAndOrders request, which includes the ContextSnapshot, shown here:

Click screen capture to open a full-size image.

Notice the 4-second delay between completion of the request and response messages. This delay is the reason that I downgraded the message-level security to basicHttp. I wanted to determine if encryption, signing, and other security overhead was responsible for what I considered to be a slow response time. It wasn't. Stay tuned for future posts on EntityBag performance.

Click here to open the full response message in IE (20 kB).

Updated January 30, 2008: Fixed response message's full-size image link and added comment on response time.

LINQ and Entity Framework Posts for 1/28/2008+

Note: This post is updated daily or more often, depending on the availability of new articles.

Designing LINQ to Entities Queries that Deliver Entities, EntitySets and/or EntityReferences

The "correct" method for eager-loading entities' EntitySets and/or EntityReferences is to add an Include("EntitySetOrRefName") to the query, as in

var query = ctxNwind.Orders
            .Include("Customers")
            .Where(o => o.Customers.CustomerID == customerIDTextBox.Text)
            .OrderByDescending(o => o.OrderDate);

to return a Customer EntityReference for each Order in the sequence. The obvious issue with the Include() method is that you lose compile-time checking of the LINQ to Entities expression by late-binding the related entity name.

However, the Entity Framework Sample Query Explorer project that installs with the Entity Framework Beta 3/Entity Framework Tools CTP 2 doesn't use the Include() method in the six sample queries (LinqToEntities70 to LinqToEntities76) of the Entity Framework Sample Query Explorer | Linq To Entities | Relationship and Navigation node.

Relationship Collection 1 uses the following query:

public void LinqToEntities70()
{
    var query = context.Customers
                .Where(cust => cust.CustomerID == "ALFKI")
                .Select(c => c.Orders
                .Select(o => o));

    foreach (var order in query)
    {
        ObjectDumper.Write(order);
    }
}

to "Select a sequence of all the orders for a customer using Select." This query shouldn't work because it doesn't invoke the .Include("Orders") method for implicitly eager-loading each Customer entity's Orders EntitySet. (According to Julie Lerman, Danny Simmons considers the working query to be a bug.) The "correct" form of the query is:

var query = ctxNwind.Orders
            .Include("Customers")
            .Where(o => o.Customers.CustomerID == customerIDTextBox.Text)
            .OrderByDescending(o => o.OrderDate);

My February 1, 2008 Entity SQL Queries for Retrieving EntityReferences and EntitySets post (updated February 2) has more on this issue, including performance measurements.

Added: February 2, 2008

Keyvan Nayyeri Builds a Blog Engine with ASP.NET MVC and LINQ

Iranian developer Keyvan Nayyeri has started a multi-part series with these two parts:

Building a Simple Blog Engine with ASP.NET MVC and LINQ - Part 1 of January 30, 2008:

Microsoft released the first CTP of ASP.NET 3.5 Extensions and it includes ASP.NET MVC Framework as one of the main extensions for ASP.NET 3.5. In the first part of this article series about building a simple blog engine with ASP.NET MVC and LINQ, Keyvan introduces the MVC pattern, ASP.NET MVC Framework, and the fundamentals of a simple blogging engine.

Part 1 covers;

Building a Simple Blog Engine with ASP.NET MVC and LINQ - Part 2 of the same date:

In the second part of the article series about ASP.NET MVC Framework, Keyvan adds controllers to his blogging engine in order to describe how to use controllers in ASP.NET MVC and discusses some details related to controllers. He first discusses the concept of URL routing patterns and then explores the anatomy of a controller class. Finally, he examines how to implement the controllers in his sample blog application.

Part 2 includes the following topics:

Note: Kevin was born in Kazakhstan but now lives in Tehran.

Added: February 1, 2008

Stephen Toub Posts Essay on Recursion and Concurrency with PFx and PLINQ

The Parallel Programming with .NET blog's Recursion and Concurrency post by Stephen Toub of January 31, 2008 discusses scenarios for parallel processing of tree data structure traversal. Processing trees with multiple threads creates a number of issues, especially with recursive tree walking. The TaskParallel library simplifies walking the tree in parallel with a method such as this:

public static void Process<T>(Tree<T> tree, Action<T> action)
{
    if (tree == null) return;
    Parallel.Do(
        () => action(tree.Data),
        () => Process(tree.Left, action),
        () => Process(tree.Right, action));
}

To minimize system overhead, you can specify sequential process after reaching a specified depth in the tree:

private static void Process<T>(
    Tree<T> tree, Action<T> action, int depth)
{
    if (tree == null) return;
    if (depth > 5)
    {
        action(tree.Data);
        Process(tree.Left, action, depth + 1);
        Process(tree.Right, action, depth + 1);
    }
    else
    {
        Parallel.Do(
            () => action(tree.Data),
            () => Process(tree.Left, action, depth + 1),
            () => Process(tree.Right, action, depth + 1));
    }
}

The author concludes:

All in all, parallelism is a very interesting problem when it comes to recursion, and we hope the Parallel Extensions to the .NET Framework go a long way towards making the merging of these two concepts significantly easier.

Added: February 1, 2008

Bill Horst Tackles Left and Right Outer Joins with LINQ to Objects and LINQ to SQL

The VB team's Converting SQL to LINQ, Part 8: Left/Right Outer Join (Bill Horst) post of January 31, 2008 shows you how to emulate Left and Right Outer Joins with Group Join statements and Select clauses with nested IIf() function calls for LINQ to Objects. Substituting LINQ to SQL as the data source requires modification of the IIf() functions and an explicit cast to Nullable(Of OrderDate) for his Customer/Orders example.

Added: January 31, 2008

Julie Lerman Morphs Brad Abrams' and Lance Olson's MVC with EF Demo to AdventureWorks, VB, and Streamlined LINQ Queries

It didn't take Julie long to rework Brad and Lance's Mode-View-Controller C# 3.0 walkthrough with Entity Framework as the data source to run with AdventureWorks and VB 9.0. She posted her MVC with Entity Framework ... a twist on Brad Abram's example example at 3:04 PM EST on January 31, 2008, about 50 hours from the original ASP.NET MVC Example Application over Northwind with the Entity Framework post on January 29 at 10:14 AM PST (1:14 PM EST). [See the "Brad Abrams Posts Step-by-Step Details for Using EF with ASP.NET's MVC" topic below.]

Julie takes advantage of her LINQ to Entities experience to use a little-known syntax for returning child sequences:

var _prod = Northwind.Categories.
            Where(c => c.CategoryName == id).
            OrderBy(c => c.CategoryName).
            Select(c => c.Products).
            First().ToList(); 

The preceding query differs from the one I used with Northwind in the NwindEdmCS.sln sample code for my "Model Domain Objects with the Entity Framework" cover story for Visual Basic Magazine's March 2008 issue to populate the updatable orderDataGridView:

var query = ctxNwind.Customers
    .Where(cust => cust.CustomerID == customerIDTextBox.Text)
    .Select(c => c.Orders
    .Select(o => o))
    .FirstOrDefault()
    .OrderByDescending(o => o.OrderDate);
orderBindingSource.DataSource = query.ToList();

I discovered this syntax from the Entity Framework Sample Query Explorer’s LINQ to Entities | Relationship Navigation | Relationship Collection 1 (LinqToEntities70) query. It appears to me that the second Select() is necessary to sort the Orders. Here's the project's vanilla UI:

 

Click above capture for full-size image.

Added: January 31, 2008

Charlie Calvert Delivers a LINQ Expression Tree Tutorial

C# community evangelist Charlie Calvert's Expression Tree Basics post of January 31, 2008 demonstrates how to create an expression tree for the lambda function (a,b) => a + b, which adds two integers, a and b and returns the sum. Charlie explains that expression trees converts code into data, such as LINQ to SQL expression to T-SQL commands, and details the steps required to write and analyze a simple C# expression tree.

Added: January 31, 2008

SD Times' "Does .NET With LINQ Beat Java?" Article Raises More Questions Than It Answers

David Worthington's Does .NET With LINQ Beat Java? article of January 30, 2008 has a snappy headline but the article's deck, "Framework's data query capabilities give it an edge, experts claim," appears to be disputed by at least one "expert," namely Jonathan Bruce of DataDirect Technologies.

I attempt to avoid ad hominem posts but I couldn't let Jonathan's innuendo go unanswered. My response is here.

Added: January 31, 2008

Jonathan Bruce's Data Services and LINQ Crystal Ball Appears a Bit Cloudy

Julie Lerman's Jonathan Bruce of DataDirect's 2008 Predictions of 1/31/2008 listed his following predictions on the data access front in 2008:

  1. Data Services changes becomes universal programming model.
  2. IBM files a draws together a set of established JDBC experts to establish LINQ for Java, Java Specification Request (JSR)
  3. Dynamic LINQ bridges gap between compile-time type checking available in more static LINQ constructs.
  4. LINQ to SQL will be forced to open up their lighter weight model to third-party provider-writers.

According to Software Development Times magazine's Does .NET With LINQ Beat Java? article of January 30, 2008 by David Worthington [see above], Jonathan's "the program manager for .NET technology group at DataDirect Technologies, and formerly JDBC specification lead and architect for the Java platform at Sun."

So I thought something might have been lost in translation between Jonathan's Predictions for 2008? post of January 29, 2008 and Julie's copy. No such luck; Julie's quotes were identical, even to the missing the period in #2.

I can't fathom the first three prognostications, so here's my request for clarification:

1. What "Data Services changes will becomes [sic] the universal programming model?" ADO.NET Data Services? If not, what or whose "Data Services changes?" What's "the universal programming model." Is it (to paraphrase Ted Neward) the "One Programming Model God Intended" to silence the Tower of Programming Model Babel?

2. Will "IBM file ... a set of established JDBC experts" (sounds impractical) or "draw [them] together? The latter sounds like herding cats to me. Will LINQ to Java have any relationship to IBM's pureQuery (formerly JLINQ)? JLINQ appeared to me (and others) to be a lightweight code generator for Java classes that's similar to LINQ to SQL without LINQ.

3. "Dynamic LINQ bridges [what] gap between compile-time type checking" and something else? In my view, you either have compile-time checking or you late-bind SQL strings, which is what LINQ eliminates with LINQ to SQL and LINQ to Entities. I'm confused.

4. "LINQ to SQL will be forced to open up their lighter weight model to third-party provider-writers" when hell freezes over. Microsoft is placing its database-agnostic bets on the Entity Framework and isn't about to devote the resources needed to write expression trees for PL/SQL or DB2 SQL dialects. Data Direct has announced plans to offer EntityClient-enabled ADO.NET data providers for the Entity Framework.

Here's hoping for a bit of defogging of Jonathan's scrying.

Added: January 31, 2008

Elisa Flasko Publishes "Introducing LINQ to Relational Data" to the MSDN Library

Elisa answers the question I posed last week about LINQ to Relational Data becoming LINQ to Entities' new name with her "Introducing LINQ to Relational Data" white paper of January 30, 2008. LINQ to Relational Data is a euphemism for two implementations of LINQ: LINQ to SQL and LINQ to Entities. Elisa provides a basic tutorial for creating a LINQ to SQL DataContext and Entity Data Model (EDM) with a LINQ to Entities query. She provides feature checklists to suggest when to use LINQ to SQL or the Entity Framework with LINQ to Entities.

Added: January 31, 2008

Marcelo Lopez Ruiz Explains Exception Handling for ADO.NET Data Services

The current ADO.NET Data Services drop returns exception messages and a stack trace to the client the service throws an exception. As Marcel says, this level of detail represents a potential security problem, so this will be changed in a future CTP.

Marcel mentions that you can throw the WebDataServiceException to return a custom error message with a 400 - Bad Request response. The "Using ADO.NET Data Services (“Project Astoria”)" white paper provides an example of its use.

Marcelo promises more information about exceptions in his next post.

Added: January 30, 2008

Brad Abrams Posts Step-by-Step Details for Using EF with ASP.NET's MVC

Brad's ASP.NET MVC Example Application over Northwind with the Entity Framework post of January 29, 2008 describes in detail how to create a ASP.NET Model-View-Controller project with Entity Framework, rather than LINQ to SQL as the back end. In addition, Brad and Lance Olson include a few sample unit tests at the end of the walkthrough.

You can download the complete source code here.

Thanks to Guy Burstein for the heads-up.

Added: January 30, 2008

Logging EntityBag's SOAP Request and Response Messages

If you're working with Danny Simmons' EntityBagSample.sln project, you might want to change the security model and enable message logging to test the effect of applying WS-* message-level security to your test project. My Logging EntityBag's SOAP Request and Response Messages post of January 29, 2008 shows you how to do this and how to associate Web_message.svclog log files with the ServiceModel Trace Viewer utility (SvcTraceViewer.Exe.)

Frans Bouma Reaches the Twelfth Step in His Quest for LINQ to LLBLGen Pro

In his Developing LINQ to LLBLGen Pro, part 12 post of January 29, 2008 Frans tackles Cast, OfType, Except, Intersect, and Single Standard Query Operators, examines the 'as' and 'let' keywords, and discusses LINQ to SQL issues with the T-SQL DISTINCT operator. Finally Frans described the work remaining in his LINQ to LLBLGen Pro implementation.

Frans is further down the home stretch.

Julie Lerman Test-Drives Xceed's DataGrid for WPF with  Entity Framework

Julie provides the step-by-step details for hooking up the XCeed Data Grid for WPF to a LINQ to Entities IQueryable<SalesOrderHeader> query result in her Binding EF data to the (free) Xceed WPF Data Grid post of January 28, 2008. She wasn't able to verify the grid's sorting features but does demonstrate it's grouping capabilities.

WPF in .NET 3.x doesn't have a built-in data-bindable grid control. If you haven't checked out third-party data grids or other data-bound controls for WPF, give Xceed's DataGrid for WPF Live Explorer demo a try.

Note: Xceed offers a free license for the DataGrid Express Edition, which requires free registration. The Professional Edition license is $495.

Lang.NET Symposium v2 Free of LINQ Presentations(!)

The second Lang.NET Syposium kicked off on January 28, 2008 in Redmond. Surprisingly, there were no presentations on LINQ, but Ted Neward's Highlights of the Lang.NET Symposium, Day One post reviews Anders Heljsberg's C# presentation:

Anders walks us through the various C# 3.0 features and how they combine to create the subtle power that is LINQ (it's for a lot more than just relational databases, folks), but if you've seen his presentation on C# 3 at TechEd or PDC or any of the other conferences he's been to, you know how that story goes. The most interesting part of his presentation was a statement he made that I think has some interesting ramifications for the industry:

"I think that the taxonomies of programming languages are breaking down. I think that languages are fast becoming [an] amalgam. ... I think that in 10 years, there won't be any way to categorize languages as dynamic, static, procedural, object, and so on [paraphrased]."

Despite the lack of LINQ or Entity Framework content, Ted's literate and insightful review of the symposium's first day is a great read, especially if you're interested in dynamic languages.

Erik Meijer will present a Tuesday morning session on his Volta incubator project, but Volta's off-topic for this blog.

Update January 30, 2007: Ted's Highlights of the Lang.NET Symposium Day Two post describes Erik's presentation:

It's obvious why Erik is doing his talk at 9AM, because the man has far more energy than any human being has a right to have at this hour of the morning. Think of your hyperactive five-year-old nephew. On Christmas morning. And he's getting a G.I. Joe Super Bazooka With Real Bayonet Action(TM). Then you amp him up on caffeine and sugar. And speed.

Start with Erik's natural energy, throw in his excitement about Volta, compound in the fact that he's got the mic cranked up to 11 and I'm sitting in the front row and... well, this talk would wake the dead.

Erik demonstrates the same enthusiasm when speaking about LINQ or Haskell.

Ted also provides detailed coverage of Paul Vick's presentation: "He's talking on 'Bringing Scripting (Back) to Visual Basic', something that I can definitely agree with" and goes on to deplore the unjust denigration of VB by adherents of the "One Language God Intended":

Whatever you did, VB, your punishment couldn't have fit the crime. Hopefully your highly-publicized personal hell is almost over.

Ted says that videos of the presentations should be available on Channel9 in about a week.

Update January 31, 2007: Ted's Highlights of the Lang.NET Symposium Day Three post attests to his almost-photographic memory, because his Mac froze when he hooked it up to a projector for a quick Scala presentation and his notes went bibi. (Serves you right for using a Mac, Ted.)

Danny Sullivan Posts Complete EntityBag Test Project with Bug Fix

Danny supplemented his earlier Perseus class library, which you need to implement his EntityBag class for service-enabling Entity Framework projects, with an EntityBagSample.sln project that you can download as Perseus-1.1.zip from his new Perseus: Entity Framework Entity Bag site on the MSDN Code Gallery. The new code, updated on January 27, 2008, fixes a minor bug "with connection strings that use Name to look up values from the config file."

Stefan Cruysberghs Starts Introductory Entity Framework and LINQ to Entities Series

Remember when LINQ to SQL was newly released and almost every technical blogger with a penchant for .NET was competing with Scott Guthrie's LINQ to SQL tutorials?

The cycle's starting again with the Entity Framework and LINQ to Entities.

Stefan Cruysberghs, who previously published a four-part .NET - LINQ to SQL tutorial, posted a detailed .NET - ADO.NET Entity Framework & LINQ to Entities - Part 1 episode on January 27, 2008.

Mike Pizzo to Present Programming LINQ and the Entity Framework Webcast on January 30, 2008

ADO.NET principal architect Mike Pizzo will present a one-hour live webcast, Programming LINQ and the Entity Framework, at 11:00 AM PST on Wednesday January 30, 2008. According to the webcast registration page:

Language Integrated Query (LINQ) introduces an exciting new way for applications to build strongly typed queries that are deeply integrated into the programming language. The ADO.NET Entity Framework allows applications and services to work in terms of an application-oriented Entity Data Model, decoupling the application's data model from the storage considerations of the relational schema. Join this webcast to see how these two technologies work together to change the way applications work with data.

Jim Wooley Updates His ThinqLinq Site

Jim spent Sunday, January 27, 2008, adding new features and posts to his LINQ-Powered ThinkLinq site. Check out:

New MSDN Code Gallery Offers LINQ and Entity Framework Content

The newly established MSDN Code Gallery has opened with several recycled LINQ examples and copies of earlier LINQ-related blog posts. The Entity Framework content is described in last week's LINQ and Entity Framework Posts for 1/21/2008+ post.

Charlie Calvert describes the differences between MSDN Code Gallery, CodePlex, and MSDN Dowloads in his Code Gallery goes Live: New Site for Samples of January 28, 2008.

I've avoided the temptation (so far) to upload old articles from the OakLeaf blog, but I might try uploading recent ones.

Suggestion: The home page and individual items have RSS feeds; individual Popular Links don't but should.

Latest Third-Party LINQ Implementation: LINQ to Named Pipes

An exception to the redundancy noted above in the MSDN Gallery's LINQ topics is an example of a LINQ to Named Pipes implementation posted by Charlie Calvert.

Saturday, January 26, 2008

LINQ and Entity Framework Posts for 1/21/2008+

Mike Tulty Produces Three More ADO.NET Data Services Video Clips

His ADO.NET Data Services - Screecasts of January 25, 2008 has links to

which supplement these

produced in mid-January.

Added: January 25, 2008

Julie Lerman Implements EntityBag in a Visual Basic Service and Client

Julie's Creating and consuming a WCF service to test the EntityBag class post of January 25, 2007 includes details for employing Danny Simmons' EntityBag code in a VB WCF service and client. She warns that it's necessary to decorate your interface's ServiceOperation functions with <ServiceKnownType(GetType(EntityType))> for serialization to work correctly.

By default WCF implements WS-* security with wsHttpBinding, which greatly increases SOAP message overhead, as Julie notes in her Testing out the EntityBag post of January 24, 2008. You can use the WCF Service Configuration Editor to change the service to basicHttpBinding, which defaults to clear text messages.

Suggestion: I believe it would behoove the ADO.NET Team to provide a simple reference project with a self-hosted service based on Northwind or AdventureWorksLT using the basicHttpBinding. This would save everyone that downloads the Perseus classes a substantial amount of time and guesswork on implementation details.

Added: January 25, 2008

Diego Vega: ADO.NET Team Starts Entity Framework Portal on the MSDN Code Gallery Site

Diego's Welcome to the Entity Framework Toolkits & Extensions post of January 25, 2007 announces the creation of the ADO.NET Entity Framework & LINQ to Relational Data Portal on the MSDN Code Gallery. The gallery is a CodePlex implementation that's customized for contributions from Microsoft teams. Content is distributed under the Microsoft Public License (Ms-PL) unless otherwise noted.

The first toolkit and extension are:

  • eSqlBlast is Zlatko Michailov's interactive Entity SQL tool and supporting libraries.
  • Perseus is Danny Simmons' set of EntityBag and related classes for WCF services (see EntityBag topics below).

The first sample is:

Update 1 January 26, 2008: Sanjay's SampleEdmxCodeGenerator sources post provides instructions for using the SampleEdmxCodeGenerator.

Questions: Will EFx become the official abbreviation for Entity Framework and LINQ to Entities be renamed LINQ to Relational Data? I doubt it.

Update 2 January 26, 2008: Diego says in two comments that Asad Khan didn't write the EDMX code generator (but didn't say who wrote it) and that LINQ to Relational Data is a generic term for LINQ implementations for relational data (i.e., LINQ to SQL and LINQ to Entities at the moment).

Added: January 25, 2008

Sanjay Nagamangalam Describes How the ADO.NET Entity Designer Handles CodeGen

Sanjay's How does the ADO.NET Entity Designer generate code? of January 24, 2008 describes how the EntityModelCodeGenerator, which uses VS 2008's SingleFileGenerator extension mechanism, extracts the CSDL content from the EDMX file and generates code with the EntityClassGenerator.

His How to Extract CSDL from EDMX post of the same date shows you how to extract CSDL, MSL, or SSDL content from the EDMX file with LINQ to XML.

Update January 26, 2008: Sanjay's SampleEdmxCodeGenerator sources post provides instructions for using the SampleEdmxCodeGenerator.

Added: January 25, 2008

Danny Simmons Wraps Up His EntityBag Sextet with the RelationshipEntry Story

EntityBag Part VI – RelationshipEntry of January 24, 2008 is the final installment of Danny's six-part EntyBag Extravaganza. RelationshipEntry is a DataContract-serializable class with eight DataMember properties:

  • RelationshipName
  • State
  • Role1
  • Key1
  • AddedEntityIndex1
  • AddedEntityIndex2

and two non-serializable properties

  • Entity1
  • Entity2

Constructor, ResolveEntityAndKeys, and Methods topics explain how the class is used.

The three class files (EntityBag.cs, ContextSnapshot.cs, and RelationshipEntry.cs) and required extension methods (UtilityExtensionMethods.cs) are in the Perseus  Entity Framework Extension file that's described in the preceding "Diego Vega: ADO.NET Team Starts Entity Framework Portal on the MSDN Code Gallery Site" topic.

Added: January 25, 2008

Mike Taulty Minimizes LINQ to XML Code by Using the ReplaceWith() Method

Mike moves two nodes within a child node to its parent with three lines of LINQ to XML code in his Reparenting nodes in LINQ to XML of January 24, 2008.

The more I work with LINQ to XML, the better I like it (despite some reservations about namespace issues I discussed in Namespace Strangenesses in XML Infosets Transformed by LINQ to XML.)

The Pfx Team: Parallelizing LINQ Aggregate Queries with PLINQ

The Parallel Programming with .NET blog's Parallel Aggregations in PLINQ post of January 24, 2008 begins by describing how the Aggregate Standard Query Operator works, then discusses the general topic of parallel LINQ aggregation semantics, and identity issues that can cause the results of parallel and sequential aggregation to differ.

More EntityBag Details Exposed: The ContextSnapshot Object

Danny Simmons' EntityBag Part IV – ContextSnapshot Fields and Properties post of January 22, 2008 and EntityBag Part V – ContexSnapshot Constructing and Applying of January 23 cover the ContextSnapshot that's serialized across the wire. Topics include:

  • The DataMembers and Other Fields: unchangedEntities, modifiedOriginalEntities, modifiedEntities, deletedEntities, addedEntities, addedRelationships, deletedRelationships and unchangedManyToManyRelationships.
  • Public Properties and Related Methods: ConnectionString, Dictionary<EntityKey, int> AddedEntityKeyToIndex and IEntityWithKey GetAddedEntity(int addedEntityIndex)
  • Constructors: ContextSnapshot(ObjectContext context) for the service and ContextSnapshot(ObjectContext context, string connectionString) for the client with a cleared connection string
  • ApplyToContext(ObjectContext context) method: Attaches unchangedEntities, modifiedOriginalEntities and modifiedEntities, adds addedEntities and addedRelationships, deletes deletedEntities and deletedRelationships, and finally attaches unchangedManyToManyRelationships

The ContextSnapshot corresponds to the ephemeral "mini connectionless DataContext" for LINQ to SQL's erstwhile "n-tier story."

Danny's three posts that precede this are:

Following are links to a few of Danny's other related posts (in reverse chronological order):

Added: 1/23/2008

Namespace Strangenesses in XML Infosets Transformed by LINQ to XML

I was working with an Atom 1.0 document returned from an ADO.NET Data Service based on Northwind (what else?) when I ran across two anomalies in how LINQ to XML handles namespaces when composing XML Infosets with VB's literal XML or C#'s functional construction syntax.

VB MVP Bill McCarthy led me to an (apparently undocumented) workaround for a spurious xmlns="" namespace declaration, but I still haven't found the answer to preventing namespace declaration duplication with literal XML composition.

Read more about the issues here.

Added: 1/23/2008

Julie Lerman: Those Who Forget the LinqDataSource Are Bound To Repeat Its Problems in the EntityDataSource

Julie Lerman recounts the problems that she's encountered with the LinqDataSource for LINQ to SQL, especially with lack of updatability when you specify a projection and return an anonymous type, and hopes they won't be repeated in the forthcoming EntityDataSource.

She also points out an anomalous exception when editing with Entity.Property/EntityReference pairs in combo boxes with StoreOriginalValuesInViewState set to True.

I'm not sanguine that the ADO.NET team can overcome these EntityDataSource hurdles by RTM.

Julie observes that the EntityDataSource "has been mentioned in the forums." Danny Simmons said:

While the LINQDataSource will not work with LINQ to Entities, we are working on an Entity Data Source which we expect to ship in v1 of the EF.

in answer to Blinky Bill's Informal List of Limitations entry of 1/3/2008 that spelled out some of the more serious issues with Entity Framework Beta 3. His complaints—together with Danny's replies—are well worth reading.

Here's Danny's same-day response regarding the ObjectContext's lack of serializability:

I would argue that making the ObjectStateManager serializable introduces significant issues long-term for n-tiered stacks.  If it were serializable certain things would be easier, but it's somewhat like shooting yourself in the foot if you are building any kind of large-scale SOA system.  You might take a look at my blog post: http://blogs.msdn.com/dsimmons/archive/2007/12/20/why-are-data-centric-web-services-so-hard-anyway.aspx That said, making n-tiered stacks easier to create is certainly an important topic we are giving a lot of thought to.

It's an interesting response in the light of Danny's current work on the EntityBag, no?

(My apologies to philosopher George Santayana,whose "Those who cannot remember the past are condemned to repeat it" warning comes from Reason in Common Sense, the first volume of his The Life of Reason quintet.)

Alexander Nowak Solves Missing Entity Framework Files Problem with Unit Testing

Belgian developer Alexander Nowak's Unit testing ADO Entity Framework post of January 22, 2008 describes how he encountered "The specified metadata path is not valid" exceptions when attempting to unit test a sample Entity Framework Beta 3 project. The problem is related to issues with the design-time .edmx file and missing .ssdl, .msl, and .csdl from the specified MSTest folder that holds copies of the assembly and app.config or web.config files.

Alexander shows you how use the VS test environment's Deployment feature of the Rosario November 2007 CTP to add the missing metadata files to the appropriate MSTest folder.

Embedding the three metadata files in the assembly by the process Julie Lerman describes in her Embedding EDM schema files in Entity Framework Beta 3 post of December 14, 2007, should also solve the problem. Here's a brief excerpt from her detailed article on the topic.

Open the EDMX file in the design window and then you will see a new property in the property window called Metadata Artifact Processing. The default of this is "Copy to Output".

To get around a current known bug, build the project with the property set to "Copy to Output". Then change the Metadata Artifact Processing property to "Embed in Output Assembly". Eventually you won't need to build with Copy to Output first.

EntityBag Details Exposed: Modes, Constructor, Public Surface and Serialization

Danny Simmons' EntityBag Part II – Modes and Constructor and EntityBag Part III – Public Surface and Serialization posts of January 21, 2008 describe the EntityBag class's enum, objects, factory class and methods:

  1. Mode Enum: Constructed -> Deserialized -> LocalContext -> Unwrapped
  2. Root Object: A single public property over which to create a private ObjectContext
  3. CreateEntityBag<T>(context, root) factory class for data retrieval
  4. OnSerializing(streamingContext) method: if Constructed mode, create ContextSnapshot from persisted entities; if LocalContext mode, create ContextSnapshot from changes to original ContextSnapshot
  5. UnwrapInto<T>(context) method: for create, update, delete operations

In answer to a comment about EntityBag Part I – Goals of January 20, 2008, Danny says:

The EF won't have something like this built-in for v1, but we are looking hard at the topic for future releases.  I'm not 100% certain we'll do this, since there are some serious issues when it comes to interoperability, etc. (as I've noted).  I believe we made some major mistakes with the DataSet in this regard, and I don't want to repeat them.

Marcelo Lopez Ruiz Explains How to Use Service Operations with ADO.NET Data Services

Marcelo's Service Operations in ADO.NET Data Services post of January 21, 2008 explains how to set up ADO.NET Data Services with the RESTful equivalent of Web methods—Windows Communication Foundation (WCF) service operations that expose custom methods decorated with the [WebGet] attribute. The functions return an IQueryable<Model.Entity> or IEnumerable<Model.Entity> type, where Model is the name of the underlying Entity Data Model and Entity is the entity name, such as Customer[s].

Setting up service operations with custom WebGet methods lets you easily restrict particular users or groups to specific retrieval operations against your data services.

The following WebDataService's CustomersInLondon() method returns an IQueryable<Model.Customer> to enumerate all Northwind Customers in London from this URL: http://localhost/WebDataService1.svc/CustomersInLondon.

public class WebDataService1 : WebDataService< Model.Entities >
{
    public static void InitializeService(IWebDataServiceConfiguration config)
    {
        config.SetResourceContainerAccessRule("Customers", ResourceContainerRights.AllRead);
        config.SetServiceOperationAccessRule("CustomersInLondon", ServiceOperationRights.All);
    }
    
    [WebGet]
    public IQueryable<Model.Customers> CustomersInLondon()
    {
        return from c in this.CurrentDataSource.Customers
               where c.City == "London"
               select c;
    }  
}

The IQueryable<T> is composable, so you can filter the result with a URL, such as: http://localhost/WebDataService1.svc/CustomersInLondon('AROUT')/Orders. If you want to prevent users from modifying the query, change IQueryable<Model.Customers> to IEnumerable<Model.Customers>.

Marcelo also explains how substituting [WebInvoke] for [WebGet] lets you pass parameters in the request body instead of the URL, [SingleResult] indicates that only a single instance is returned, and [MimeTypeAttribute] applies a particular MIME type, such as text/html, to the returned element(s).

Danny Simmons Describes a Serializable EntityBag for n-Tier Entity Framework Deployment

I'm glad to see the ADO.NET Entity Framework team taking the pragmatic approach to Web service-enabling v1.0 with the serializable EntiyBag that Danny describes in his EntityBag Part I – Goals post of January 20, 2008.

For my take on Danny's answer to the LINQ to SQL Team's apparently-abandoned serializable "mini-connectionless DataContext" read Coming to the Entity Framework: A Serializable EntityBag for n-Tier Deployment, which ran too long for inclusion in this post.

[Repeated from the previous week's post.]

Jim Wooley's ThinqLinq Site is Live and LINQ In Action Has Gone to the Printer

Over the weekend Jim's new LINQ-powered ThinqLinq blog and site added new content and Manning Publications Co. sent LINQ in Action by Fabrice Marguerie, Steve Eichert and Jim Wooley to the printer. Copies should be in bookstores and Amazon.com by mid-February.

Congratulations to Fabrice, Steve and Jim and best wishes for the book's success!

I have all the LINQ books currently published, including the eBook version of LINQ in Action and I agree with Sam Gentile comment in his post of January 22, 2008:

My best of friends, Steve Eichert, along with  Fabrice, Jim, has shipped their LINQ in Action. I can tell you that I have the LINQ books, and this is by far the best.

Take Advantage of the LinqDataSource's GroupBy and OrderGrouopBy Features with Nested ListView Controls

Matt Berseth's Building a Grouping Grid with the ASP.NET 3.5 LinqDataSource and ListView Controls post of January 10, 2008 shows you how to create an expandable master/child ListView presentation that doesn't require a postback for expanding or collapsing. The preceding link is to a live demo and the source code is available to download.

Matt's earlier Creating a FrequencyDecoder.com Style Grid with the ListView, DataPager and LinqDataSource Controls post shows you how to create a soothingly attractive, customized pastel grid with a single ListView control.

Added 1/23/2008: Missed in previous linkblog post.