VB 9.0 Won't Support Collection Initializers (or Array Initializers!)

Paul Vick answered my Will Visual Basic 9.0 Have Collection Initializers? question in his June 27, 2007 What's in VB 2008? What's out? post. The answer, as I anticipated, is no. However, I was surprised to learn that VB 9.0 won't have array initializers, either. My limited tests indicated that array initializers were behaving as expected (i.e., like C# array initializers) in Beta 1. Paul says:

For VB 2008, we will only support initializing read-write fields of non-collection objects.

And explains the disappearance of array and collection initializers as follows:

Our original plans, going back to PDC05, included several more features for object initializers, such as being able to write to read-only properties, as well as collection and array initializers. In the end, the schedule for VS 2008 was not sufficient to implement these features with a high degree of confidence. Which unfortunately means that they will have to wait to a release beyond VS 2008.

Array and collection initializers deliver a major productivity boost, especially for unit testing where you don't want to hit the database because of greatly increased test times.

On the LINQ front, VB 9.0's lambda expressions won't support statement blocks, but that's been known for some time and probably isn't a major issue.

I was surprised to learn that the VB 9.0 compiler will deliver expression trees. Neither the September 2005 "Overview of Visual Basic 9.0" .doc file or the current HTML Overview of Visual Basic 9.0 version of February 2007 mentions expression trees, so I assumed

Technorati tags: VS 2008, VB 9.0, Orcas, LINQ, LINQ to SQL, Collection Initializers, Array Initializers

that support for writing them with VB was gone, too.

I doubt if you'll see many expression trees written in VB. Folks who write custom LINQ for Whatever implementations for specialized data domains are certain to do so with C# 3.0.

On the whole, I think most—if not all—developers who prefer writing VB code would rather have array and collection initializers than the capability to write expression trees.

Slightly off-topic: The more C# 3.0 LINQ code I write, the better I like C#'s "fat arrow" lambda (=>) operator (argument-list => expression-or-statement-block) rather than VB's forthcoming inline Function(argument-list) expression syntax.

Will Visual Basic 9.0 Have Collection Initializers?

For reasons unknown, all mention of collection initializers for Visual Basic 9.0 has disappeared. The disappearance seems to have occurred without comment from the VB team or VB users.

The September 2005 "Overview of Visual Basic 9.0" .doc file covered the topic briefly, but the current HTML Overview of Visual Basic 9.0 version of February 2007 discusses only object initializers and array initializers. VS 2008 Beta 1 online help has a "connection initializers (C#)" topic but no corresponding VB topic.

The 2005 Overview's section 1.3 "Object and Collection Initializers" says this about collection initializers:

As we have seen, object initializers are also convenient for creating collections of complex objects. Any collection that supports an Add method can be initialized using a collection initializer expression. For instance, given the declaration for cities as the partial class,

Partial Class City
  Public Property Name As String
  Public Property Country As String
  Public Property Longitude As Float 
  Public Property Latitude As Float
End Class

we can create a List(Of City) of capital cities of our example countries as follows:

Dim Capitals = New List(Of City){ _
  { .Name = "Antanarivo", _
    .Country = "Madagascar", _
    .Longitude = 47.4, _
    .Lattitude = -18.6 }, _
  { .Name = "Belmopan", _
    .Country = "Belize", _
    .Longitude = -88.5, _
    .Latitude = 17.1 }, _
  { .Name = "Monaco", _
    .Country = "Monaco", _
    .Longtitude = 7.2, _
    .Latitude = 43.7 }, _
  { .Country = "Palau",
    .Name = "Koror", _
    .Longitude = 135, _
    .Latitude = 8 } _
}

This example also uses nested object initial[iz]ers, where the constructors of the nested initializers are inferred from the context. In this case, each nested initializer is precisely equivalent to the full form New City{…}.

Note: The preceding excerpt also appears in the session of the same name made to the XML 2005 Conference & Exhibition held in Atlanta November 14-18, 2005. I doubt if that code worked in any VB 9.0 preview.

Neither the class nor the sample collection initializer will compile. It's one thing to change the syntax for a new feature, but another to use a defective class declaration in sample code.

The authors of the original version, Erick Meijer, Amanda Silver, and Paul Vick, imported and modified the class code and modified the List(Of City) code to utilize the current New Type With syntax in "Object and Array Initializers" of the February 2007 version:

As we have seen, object initializers are also convenient for creating collections of complex objects. Arrays can be initialized and the element types inferred by using an array initializer expression. For instance, given the declaration for cities as the class,

Partial Class City
  Public Property Name As String
  Public Property Country As String
  Public Property Longitude As Long 
  Public Property Latitude As Long
End Class

we can create an array of capital cities for our example countries as follows:

Dim Capitals = { _
  New City With { _
    .Name = "Antanarivo", _
    .Country = "Madagascar", _
    .Longitude = 47.4, _
    .Lattitude = -18.6 }, _
  New City With { _
    .Name = "Belmopan", _
    .Country = "Belize", _
    .Longitude = -88.5, _
    .Latitude = 17.1 }, _
  New City With { _
    .Name = "Monaco", _
    .Country = "Monaco", _
    .Longtitude = 7.2, _
    .Latitude = 43.7 }, _
  New City With { _
    .Country = "Palau",
    .Name = "Koror", _
    .Longitude = 135, _
    .Latitude = 8 } _
}

After fixing the class declaration by removing Property and changing Long to Float, the array initializer expression won't compile because there's a missing New City() type declaration in the first line, which should read Dim Capitals = New City(){ _. It's clear that no one tested the code before publishing the update.

Note: It's a shame that VB didn't adopt the Dim corollary to C#'s var arrayName = New[]{ ... } syntax, as Ralf Ehlert mentions in his "Problems with New Features of VB9" post in the Visual Studio VB Express Orcas forum.

The authors made no mention of VB 9.0 collection initializers, which were also missing from Jean-Marie Pirelli's "C# 3.0 and VB 9.0" session at Microsoft TechDays 07 March 27-28, 2007 in Geneva. Uncharacteristically, Jean-Marie provided both C# and VB examples, but he dispensed with VB examples when he reached the Collection Initializers slide. (Anders Hejlsberg and Jay Schmelzer received credit for the slides.)

It's interesting that Jeff Bramwell's Collection Initializer post in the Visual Basic Orcas forum didn't receive a reply from a Microsoft representative. The suggestion he received from Klaus Even Enevoldsen wasn't apropos collection initializers.

Note: My March 26, 2007 Updated "Overview of Visual Basic 9.0" Stealth Post lists other problems with the Overview reported by Jim Wooley. These errors relate to unimplemented VB 9.0 features scheduled for Beta 2.

There also were errors in the LINQ to SQL documentation that I reported in my April 26, 2007 More Issues with VB 9.0 Samples in the Updated LINQ to SQL Documentation post.

Update 6/24/2007: Added comment about data type error in Cities class declaration and corrected minor typos.

Update 6/25/2007: Bill McCarthy takes issue with my "It's clear that no one tested the code before publishing the update" statement about the second sample that uses the New Type With syntax in his VB9 and collection initializers post.

Regardless of syntax issues with the array initializer code, the class declaration has both syntax (Property) and data type (Long) errors.

Regarding the failure of the initializer code to compile, Bill says:

[I]t is clear the document is forward looking, and talking about how things will/should be.

So, given that, if we stop and analyze the syntax for array initializers it should be basically as presented. Perhaps one could argue the syntax should be Dim Capitals() = {.    It should also support the syntax you can use today in VS 2005, e.g: Dim ints() As Int32 = {1,2,3} , or as per the code Roger suggests Dim ints() As Int32 = New Int32(){1,2,3}

Okay so given the minimum it needs to support (current syntax), in VB9 we add anonymous types and inferred typing.  Inferred typing means we remove the "As XXX" part.  So this means the syntax Dim Capitals As City() becomes Dim Capitals and the Dim Capitals() As City syntax becomes  Dim Capitals().  Anonymous types means we can't define the type name. So New City() { would become New() { _ , and given that the existing syntax doesn't even require the New Int32(), the New() becomes superfluous because the set brackets { } define the array data.

IOW: the syntax as shown is as it should be. (IMO)

Explicitly typing the array members by adding New City With { ... syntax precludes the array being of an anonymous type or use of inferred typing and the Dim Capitals = New City(){ ... statement is required. I didn't propose Dim Capitals() As City = {...} or Dim Capitals As City() = New City(){...}  as inferred by Bill's "Dim ints() As Int32 = {1,2,3} , or as per the code Roger suggests Dim ints() As Int32 = New Int32(){1,2,3}" sentence.

You can create anonymously-typed array elements by substituting New With for New City With, but Dim Capitals() = {...} returns an array of type Object with Option Strict Off and won't compile with Option Strict On. If VB 9.0 finally supports the New() construct for anonymously typed arrays, the the preceding code should compile with Dim Capitals = New(){ _, but I don't believe that New(){ _ would be superfluous. Here are two C# array initializer examples that use new[]:

/* Array initializer with object initializers (array is inferred type LineItem) */

var LineItems = new[]
{
    new LineItem {OrderID = 11000, ProductID = 11, Quantity = 10, QuantityPerUnit = "24  500-g bottles", UnitPrice = 15.55M, Discount = 0.0F},
    new LineItem {OrderID = 11000, ProductID = 21, Quantity = 20, QuantityPerUnit = "12 1-kg cartons", UnitPrice = 20.2M, Discount = 0.1F},
    new LineItem {OrderID = 11000, ProductID = 31, Quantity = 30, QuantityPerUnit = "24 1-kg bags", UnitPrice = 25.45M, Discount = 0.15F}
};

/* Array initializer with object initializers (array and elements are anonymous types) */

var LineItems2 = new[]
{
    new {OrderID = 11000, ProductID = 11, Quantity = 10, QuantityPerUnit = "24 500-g bottles", UnitPrice = 15.55M, Discount = 0.0F},
    new {OrderID = 11000, ProductID = 21, Quantity = 20, QuantityPerUnit = "12 1-kg cartons", UnitPrice = 20.2M, Discount = 0.1F},
    new {OrderID = 11000, ProductID = 31, Quantity = 30, QuantityPerUnit = "24 1-kg bags", UnitPrice = 25.45M, Discount = 0.15F}
};

However, I haven't heard anything about VB 9.0 implementing New(). On the other hand, I'm not sure why I would want to create an anonymously typed array or what its use would be.

Technorati tags: Orcas Beta 1, Orcas, VS 2008, VB 9.0, Object Initializers, Collection Initializers, Array Initializers, LINQ

LINQ to DataSets Rehabilitates VB's Bang Operator

Visual Basic's bang (!) operator (a.k.a. pling and officially the dictionary lookup operator) seldom appears in VB.NET code. The purpose of the bang operator (alias the bang separator) is to specify members of string-keyed collections, such as fields of an ADO.COM Recordset, as in:

rsCusts!CustomerID = "BOGUS"
rsCusts!CompanyName = "Bogus Software, Inc."
rsCusts!ContactName = "Joe Bogus"

The preceding shaves a few keystrokes from "parens-and-quote" syntax, which relies on Fields being the default property of a Recordset:

rsCusts("CustomerID") = "BOGUS"
rsCusts("CompanyName") = "Bogus Software, Inc."
rsCusts("ContactName") = "Joe Bogus"

which, in turn, is shorthand for the fully-qualified syntax:

rsCusts.Fields("CustomerID") = "BOGUS"
rsCusts.Fields("CompanyName") = "Bogus Software, Inc."
rsCusts.Fields("ContactName") = "Joe Bogus"

One reason for bang's disappearance is that ! got a bum rap from VB.COM and VBA developers because amateur coders (especially Access rookies) used bang in place of dot (.) to specify properties of other objects, such as forms and controls. Doing this kills IntelliSense and compile-time checking. In fact, ! fell into such disrepute that many developers, such as this "VB Expert," believed that VB.NET didn't support the bang operator. Some folks claimed that performance suffered when ! was tokenized to the paren-quote syntax, although I never found anyone who reported testing the performance hit.

Paul Vick's July 15, 2003 Dictionary Lookup Operator post explains how the bang operator works and, in a comment, it's equivalent in C#—x!y : x["y"].

C# Joins and Associations in Typed and Untyped DataSets

LINQ to DataSets is most likely to be used in place of DataViews where joins between related tables are required. (DataViews don't support joins or projections (SELECT lists), so you need LINQ to DataSets or a custom DataView class, such as the JoinView from MSDN, to accommodate them.) I needed to verify the differences in the syntax for LINQ to DataSet queries over typed or untyped multi-table DataSets that use joins or navigate associations and include value types (specifically DateTime) with nulls. Erick Thompson wrote a Nulls - LINQ to DataSets Part 3 treatise in his LINQ to DataSet Documentation Series but didn't mention nullable datatypes in conjunction with the added DataRow.Field<T> collection.

Here's the C# test query for the typed DataSet with joins, which returns nine rows with values from each of four related Northwind tables, including one null ShippedDate value to test null handling:

var query1 = from c in dsNwindT.Customers
             join o in dsNwindT.Orders on c.CustomerID equals o.CustomerID
             join d in dsNwindT.Order_Details on o.OrderID equals d.OrderID
             join p in dsNwindT.Products on d.ProductID equals p.ProductID
             where p.ProductID == 2 && c.Country == "USA"
             orderby o.OrderID descending
             select new { c.CustomerID, c.CompanyName, o.OrderID,
                          ShippedDate = o.Field<DateTime?>("ShippedDate"),
                          p.ProductName, d.Quantity };

With the exception of the code to supply the ShippedDate value (emphasized), the preceding query statement is identical to the corresponding LINQ to SQL query (just replace the dsNwindT DataSet with a DataContext.

You lose strong typing of related DataRows when you navigate associations (ChildRows or ParentRows) between related DataTables instead of joining the DataTables. This requires FieldName = DataTable.Field<DataType>("FieldName") expressions for non-nullable and FieldName = DataTable.Field<NullableType>("FieldName") expressions for nullable fields for all but the topmost table of the hierarchy:

var query2 = from c in dsNwindT.Customers
             from o in c.GetChildRows("FK_Orders_Customers")
             from d in o.GetChildRows("FK_Order_Details_Orders")
             from p in d.GetParentRows("FK_Order_Details_Products")
             where p.Field<int>("ProductID") == 2 && c.Country == "USA"
             orderby o.Field<int>("OrderID") descending
             select new {
                 c.CustomerID, c.CompanyName,
                 OrderID = o.Field<int>("OrderID"),
                 ShippedDate = o.Field<DateTime?>("ShippedDate"),
                 ProductName = p.Field<string>("ProductName"),
                 Quantity = d.Field<short>("Quantity") };

Join syntax for untyped DataSets becomes quite cumbersome because of the need to apply the AsEnumerable() method to each DataTable and DataTable.Field<DataType>("FieldName") to the joined fields:

var query3 = from c in dsNwindU.Tables["Customers"].AsEnumerable()
             join o in dsNwindU.Tables["Orders"].AsEnumerable() on
                 c.Field<string>("CustomerID") equals
                 o.Field<string>("CustomerID")
             join d in dsNwindU.Tables["Order_Details"].AsEnumerable() on
                 o.Field<int>("OrderID") equals d.Field<int>("OrderID")
             join p in dsNwindU.Tables["Products"].AsEnumerable() on
                 d.Field<int>("ProductID") equals p.Field<int>("ProductID")
             where p.Field<int>("ProductID") == 2 &&
                 c.Field<string>("Country") == "USA"
             orderby o.Field<int>("OrderID") descending
             select new {
                 OrderID = o.Field<int>("OrderID"),
                 ShippedDate = o.Field<DateTime?>("ShippedDate"),
                 ProductName = p.Field<string>("ProductName"),
                 Quantity = d.Field<short>("Quantity") };

The added typing for untyped DataSets with associations instead of joins (emphasized) isn't that great:

var query4 = from c in dsNwindU.Tables["Customers"].AsEnumerable()
             from o in c.GetChildRows("FK_Orders_Customers")
             from d in o.GetChildRows("FK_Order_Details_Orders")
             from p in d.GetParentRows("FK_Order_Details_Products")
             where p.Field<int>("ProductID") == 2 &&
                c.Field<string>("Country") == "USA"
             orderby o.Field<int>("OrderID") descending
             select new {
                 CustomerID = c.Field<string>("CustomerID"),
                 CompanyName = c.Field<string>("CompanyName"),
                 OrderID = o.Field<int>("OrderID"),
                 ShippedDate = o.Field<DateTime?>("ShippedDate"),
                 ProductName = p.Field<string>("ProductName"),
                 Quantity = d.Field<short>("Quantity") };

But the added typing probably would trouble C# purists who substitute lambda functions for enhanced query syntax because doing so saves a few characters.

VB Associations in Typed and Untyped DataSets

When skimming the sample queries in the "LINQ over DataSet for VB Developers" whitepaper for the May 2006 LINQ preview, I noticed extensive use of the bang operator in DataTable!FieldName expressions taking the place of FieldName = DataTable.Field<DataType>("FieldName") and FieldName = DataTable.Field<NullableType>("FieldName") expressions. Strangely, there's no mention of "bang" or "dictionary" in the nine-page document.

Unfortunately, Orcas Beta 1's VB compiler doesn't implement the Join keyword and joins using SQL-89 equi-join syntax are limited to two tables. Thus, my tests with VB are limited to navigating associations.

Here's the VB version of C# query2, which returns objects rather than designated data types for fields other than CustomerID and CompanyName:

Dim Query2 = From c In dsNwindT.Customers, _
             o In c.GetChildRows("FK_Orders_Customers"), _
             d In o.GetChildRows("FK_Order_Details_Orders"), _
             p In d.GetParentRows("FK_Order_Details_Products") _
             Where CInt(p!ProductID) = 2 AndAlso c!Country.ToString = "USA" _
             Order By o!OrderID Descending _
             Select c.CustomerID, c.CompanyName, o!OrderID, _
                 o!ShippedDate, p!ProductName, d!Quantity

Notice that no special treatment of the ShippedDate field is required to handle null values. However, if you attempt to strongly type ShippedDate with a CType(o!ShippedDate, Nullable(Of DateTime)) expression, a runtime error occurs when iterating the row with the null value.

And here's the VB version of C# query4, which returns objects for all fields:

Dim Query4 = From c In dsNwindU.Tables("Customers"), _
             o In c.GetChildRows("FK_Orders_Customers"), _
             d In o.GetChildRows("FK_Order_Details_Orders"), _
             p In d.GetParentRows("FK_Order_Details_Products") _
             Where CInt(p!ProductID) = 2 AndAlso c!Country.ToString = "USA" _
             Order By o!OrderID Descending _
             Select c!CustomerID, c!CompanyName, o!OrderID, _
                 o!ShippedDate, p!ProductName, d!Quantity

Notice that the AsEnumerable() method is missing from the DataTable identifier in the first line. The C# query won't compile without it but the VB compiler doesn't care if it's present or not.

I believe it's a good bet that VB's Join syntax for untyped DataSets in Beta 2 will look like this, which is considerably more concise than that of the C# version:

Dim Query4 = From c In dsNwindT.Customers
             Join o In dsNwindT.Orders On c!CustomerID Equals o!CustomerID
             Join d In dsNwindT.Order_Details On o!OrderID Equals d!OrderID
             Join p in dsNwindT.Products On d!ProductID Equals p!ProductID
             Where CInt(p!ProductID) = 2 AndAlso c!Country.ToString = "USA"
             Order By o!OrderID descending
             Select c!CustomerID, c!CompanyName, o!OrderID, o!ShippedDate,
                 p!ProductName, d!Quantity

Update 5/18/2007: According to VB Team member Amanda Silver, VB's Join syntax uses the Equals operator (equals in C#) instead of the = operator to make explicit that the expression uses Object.Equals as the comparison operator.

Conclusion: C# purists castigate VB for its verbosity and some condemn VB code as prolix. VB's syntax for LINQ queries against joined or associated tables is equally or more concise than that of C#.

Update 5/24/2007: Paul Vick points out in his Javascript in VB ... post that "the Silverlight-based Javascript compiler that we released in Silverlight 1.1" as well as the VBx compiler for VB 10.0 is written in VB. Miguel de Caza mentions in his Compiler Lab: Second Day post:

Their new DLR-based Javascript compiler (that implements the current ECMA spec) was written by two developers in four months. Nandan Prabhu was at the conference and he explained that if they were to rewrite it today it would probably take three months for a programmer to implement as such a person would not have to keep up with the daily changes in the DLR [Dynamic Language Runtime].

It will be interesting to see how the final implementation of VB's Join keyword handles null value types in strongly typed projections.

Update 5/22/2007: You'll need to use the FieldName = DataTable.Field<DataType>("FieldName", DataRowVersion.Version) overload of the Field method in both VB and C# if you want to filter by a member of the DataRowVersion enumeration: Current, Default, Original or Proposed. (The "LINQ to DataSet for C# Developers" whitepaper describes the DataRowVersion overloads, but doesn't include a sample of ther use. "LINQ to DataSet for VB Developers" doesn't even mention these overloads.)

Technorati tags: LINQ to DataSets, LINQ, VB 9.0, C# 3.0, Orcas Beta 1, Orcas, Bang Operator

New Series on Closures in Visual Basic 9.0

If the term "lifting the variable x" leaves you scratching your head, read Jared Parsons' multi-part Closures in VB blog series. Jared is a developer on the VB team working with the VB compiler and debugger.

Jared describes lexical closures (usually just closures) as:

[T]he underpinnings for several new features in Visual Basic 9.0.  The[y] are part of the guts of Lambda and Query expressions. ... A closure is a feature which allows users to se[a]mlessly access an environment (locals, parameters and methods) from more than one function. ... Closures are responsible for making the single variable "x" available to [multiple] functions in a process that is referred to as "lifting the variable".

Update: 6/15/2007: Following are links to the first two four members of the series:

1. Closures in VB Part 1: The Basics (May 2, 2007)
2. Closures in VB Part 2: Method Calls (May 3, 2007)
3. Closures in VB Part 3: Scope (May25, 2007)
4. Closures in VB Part 4: Variable Lifetime (June 15, 2007)

Jared says:

This will be a several part series on Closures in VB 9.0; how they work, their limitations, pitfalls surrounding their use. 

So I'll add links to later series members as Jared posts them.

Update: 5/11/2007: If you'd really like to deep-dive into closures, check out these VB 9.0-based posts from Brian Beckmann:

• Lambdas, Closures, Currying, and All That. Behind the scenes, I’m working on the lambda-execution mode for the IQ97. [IQ97 is a simulator of the HP97 programmable printing calculator that Brian wrote in a 2006 CTP of VB9.] I’m hoping for a few tweaks to VB and LINQ before I can do this satisfactorily, but I can take the opportunity now—with the current CTP —to illustrate the general technique of defining with lambda and what I hope it might look like in VB some time down the road.
• Closures without Closures; Currying without Currying. Last installment, we found we could write a factorial function with a function call in the recursive branch, but not a recursive call. This ‘nearly recursive’ style allows us to write factorial without using its name.
• Rebirth of the Y [Combinator]. Let’s review the bookends of our so-far successful foray into recursion elimination. The general theme has been replacement of recursive calls by self-application of functions adhering to recursive types .
• A Hint and a Challenge about the search for a non-recursive implementation of the Y Combinator in VB 9.0.

Mike Champion's Brian Beckmann on LINQ underpinnings - Bringing functional programming to "Mort" post links to a Channel9 interview with Brian, a former cosmologist: Brian Beckman: Monads, Monoids, and Mort.

Update 5/13/2007: Mads Torgersen, a Microsoft program manager who's responsible for the design of the C# language, plows the ground in Brian's neighborhood to prove that you can write write a recursive lambda expression, using factorial as an example, and can substitute a named generic recursive method for the Y combinator. Here's the result:

i => new Func<Func<int,int>,Func<int,int>>(fac => x => x == 0 ? 1 : x * fac(x - 1))(new SelfApplicable<Func<Func<Func<int,int>,Func<int,int>>,Func<int,int>>>(y => f => x => f(y(y)(f))(x))(y => f => x => f(y(y)(f))(x))(fac => x => x == 0 ? 1 : x * fac(x - 1)))(i)

The preceding looks to me like a write-only function because it's impossible (for most folks) to read. Brian's VB examples are certainly more readable. (After reviewing this post, Sahil Malik said C# has become too complicated.) Mads has a few other interesting posts relating to C# 3.0 and LINQ in his "Language Designers Workshop" blog. (He isn't a prolific blogger.)

Update 5/11/2007: Amanda also says that VB 9.0 will get the Let keyword in Beta 2 and Beta 2 will deliver full suport of lambdas by VB 9.0.

Update 5/15/2007: Paul Vick also covers closures for VB programmers in his two-part series on lambda expressions and an earlier post:

• Lambda expressions, Part I: Syntax
• Lambda expressions, Part II: Closures
• Local variables: scope vs. lifetime (plus closures)

Technorati tags: Closures, Lexical Closures, Visual Basic, VB, Visual Basic 9.0, VB 9.0, Orcas, LINQ, Lambda Expressions, Query Expressions

PS: Amanda Silver has posted the slide deck from her VSLive! Orlando The .NET Language Integrated Query (LINQ) Framework presentation. She promises to update this post with links to sample code in a few days and answer questions related to her session.

Déjà Vu All Over Again: Entity Framework Cut from Orcas

Microsoft software architect Mike Pizzo announced in the ADO.NET Entity Framework Update post of 10:09 PM, Saturday, April 28, 2007:

[W]e have decided to ship the ADO.NET Entity Framework and Tools during the first half of 2008 as an update to the Orcas release of the .NET Framework and Visual Studio.

Mike does his best to spin the bad news by starting the announcement with:

Microsoft is deepening its investment in the ADO.NET Entity Framework as a critical piece of Microsoft’s Data Platform vision.

Many Microsoft observers will read that sentence as:

Microsoft is deep-sixing its investment in the ADO.NET Entity Framework as a critical piece of Microsoft’s Data Platform vision.

It's too soon to make that judgment, but there's no question that Mike's announcement delivers yet another blow to those developers (like me) who followed Microsoft's primrose path into its previous object/relational mapping (O/RM) tool void.

Microsoft MVP Scott Bellware avoided the most recent O/RM trap by abandoning a planned short-term implementation of the Entity Framework (EF), as described in his April 24, 2007 Falling Back to NHibernate - A Phone Call with the Entity Framework Folks:

We made a decision today to put an early end to our exploration of the Entity Framework for our current product work, and to fall back on NHibernate. ...

When the EDM designer started to drop out of the CTP's (post-August 2006), we had hoped that it would reappear around beta 1, and would be fairly solid around beta 2. We're scheduled to ship our current work product in coincidence with Orcas. We expect to start customer previews when a go-live license is offered. Without an editor for the metadata, we really can't justify the use of the EF to our customers.

The phone call [to the ADO.NET EF team] took place in two parts. The second part of the call got into some NDA-covered material that Kevin wasn't able to stick around for. Tim and Dan shared some info that gave me with a definitive and conclusive basis for making a sound call for our product. I'm very grateful to the EF team to have been enabled to make such a clear cut decision.

I have the feeling that Scott might have received advance notice (under NDA) that EF and its components were going back to the drawing board. Regardless, he said in an April 22, 2007 comment to my Persistence Ignorance Is Bliss, but Is It Missing from the Entity Framework? post:

I won't remain committed to the Entity Framework if the EDM Designer doesn't arrive by Orcas RTM.

I see the EDM designer as one of the most compelling features of EF in its current incarnation. It's the piece that translates well to how we expect our customers will expect to work with the entities in our product.

If the EDM designer is not in place by RTM, then I don't expect that we can ship a product to our customers if part of the product's customization story is based on the presence of the EDM.

We would need to fall back on NHibernate, and we have had this contingency plan in mind since we began the foray into EF.

Scott implemented his contingency plan two days later. (He insists he's not a member of the "NHibernate Mafia".)

Note: There still might be hope for EF with the Domain Driven Development (DDD) crowd. Interknowlegy's Tim McCarthy is writing .NET Domain-Driven Design with C#: Problem-Design-Solution to be published by Wiley. You can download the slides and code for Tim's April 19, 2007 "Domain-Driven Design using the ADO.NET Entity Framework" presentation to the International Association of Software Architects' Southern California chapter (IASA SoCal). The code is based on Paul Gielens original code for "Organizing Domain Logic" which compares Microsoft's Three-Layered Services Architecture (MS-TLSA) approach against DDD. (The three layers are the traditional data, business and presentation, not the EDM's storage, mapping, and coneptual layers.)

Does Microsoft Have a Data Access Strategy?

Mike posted Microsoft’s Data Access Strategy to the Data blog three minutes after the preceding bombshell. He answers the question with:

Yes, it turns out we do. Microsoft envisions an Entity Data Platform that enables customers to define a common Entity Data Model across data services and applications. The Entity Data Platform is a multi-release vision, with future versions of reporting tools, replication, data definition, security, etc. all being built around a common Entity Data Model.

Mike disclosed another part of the data access strategy in the first post:

Microsoft will be leveraging the Entity Data Model in future versions of Microsoft products such as SQL Server. This Data Platform vision enables customers to leverage their investment in a common conceptual entity model across product lines. [Emphasis added.]

Frans Bouma, the developer of LLBGen Pro, a commercial O/RM tool, has a different (and sinister) take on Microsoft's data access strategy:

The announcement speaks about Microsoft building applications on top of Entity Framework and also that the Entity Framework is moving towards SqlServer. Now, if you can add 1 and 1 together, you of course realize that if you have an application, say Sharepoint vNext, build on top of Entity Framework and you have one of your major cash cows SqlServer enabled with Entity Framework, you have a combination to earn more money: people who want the next version of your application also want your new database system.

The Entity Framework in ADO.NET vNext was designed to be database independent, and it might still be database independent when it eventually ships (I personally don't believe their H1 2008 mark). If you have one of your other major cash cows, say Sharepoint vNext, build on top of ADO.NET vNext, you then open up the road to host Sharepoint on say Oracle, DB2 or an open source database. This then would alienate one of your other cash cow products, SqlServer which will have the Entity Framework build-in.

Personally, I don't buy Frans' theory at present. Microsoft's "major cash cows" are Windows operating systems, the Office suite, and well-entrenched server-based applications, such as Exchange Server and SQL Server. By default, Windows SharePoint Services (WSS), which is a component of Windows Server 2003, uses freely distributable SQL Server 2005 Express (SSX) as its data store not as its host operating system.

Paul Wilson, the developer of the commercial WilsonORMapper (a simpler competitor to LLBGen Pro) says "Great News: Only One O/RM Shipping in Orcas." Wilson is a fan of LINQ to SQL because it's simpler but "good enough for the vast majority of cases."

Mike contrasts LINQ to SQL and LINQ to Entities:

LINQ to SQL supports rapid development of applications that query Microsoft SQL Server databases using objects that map directly to SQL Server schemas. LINQ to Entities supports more flexible mapping of objects to Microsoft SQL Server and other relational databases through extended ADO.NET Data Providers.

Despite the comments regarding support by LINQ to SQL for databases other than SQL Server by Dinesh Kulkarni, Scott, Guthrie, Matt Warren and Keith Farmer quoted in my Future LINQ to SQL Support for Multiple Databases? post of April 19, 2007, Mike has put that issue to bed. It's clear as part of Microsoft data access strategy that LINQ to SQL is LINQ to SQL Server (only).

Shades of WinFS, ObjectSpaces, Project Green, and the Microsoft Business Framework

The previous indication of trouble in the Entity Framework camp was the announcement at VSLive! San Francisco that the Entity Data Model (EDM) Designer had been cut from the Orcas release.

Earlier, a major controversy about EF's lack of "Persistence Ignorance" and its need to support Plain Old CLR Objects (POCOs), test-driven development (TDD), and agile programming techniques erupted at Microsoft's MVP Summit conference in Redmond on March 12 - 15, 2007.

But the real problem is that .NET developers might equate the result of EF's integration into the next version of SQL Server (code-named "Katmai") with the demise of ObjectSpaces, EF's ill-fated predecessor OR/M tool, when it was integrated into WinFS. My June 26, 2006 Microsoft Bites the Bullet: WinFS Bites the Dust post delivers the grisly details of the deaths of WinFS, ObjectSpaces, Project Green, and the MBF.

Here's Andrew Conrad's statement of the official status of ObjectSpaces in Visual Studio 2005 (then codenamed "Whidbey") as of April 6, 2004:

ObjectSpaces is not participating in Whidbey beta 1, however, it remains part of the Whidbey/Yukon wave and will be made available as a downloadable add-on pack for the .NET Framework Whidbey shortly after Whidbey ships. This additional development and stabilization period will be focused on improving the overall ObjectSpaces programming experience and providing tighter integration with WinFS, the next generation file system in Longhorn. The schedule for the ObjectSpaces mapping tool is also being adjusted accordingly.

His post concludes with:

Bottom line - this will mean some change in the release schedule, but should not adversely effect anyone's plans to deploy the ObjectSpaces framework. In fact, I believe the tighter integration with WinFS will significantly justify the delay.

Does that language sound familiar? Compare the comments with those to Mike's initial post.

Even more ominous is Barbara Darrow's April 26, 2007 speculation in CRN that "Katmai, Orcas Link Could Delay Longhorn Release." If true, which I doubt, Katmai's EF dependency could delay it and Longhorn Server until probably late 2008.

Mike replies with a Sunday-afternoon comment regarding spin, ObjectSpaces, and WinFS:

I apologize for the “PR-spin” feel people have associated with this post. Even a short delay of the Entity Framework is a difficult message, especially to convey through the imprecise nature of written communication. How do we convey that we believe this to be the right decision, without people losing confidence in the technology, particularly in light of Microsoft’s less-than-perfect record in shipping ORM solutions, and in the wake of WinFS? I struggled with the wording. With the approach. With the message. Yes, marketing was involved in reviewing the content (of course). I regret now accepting some of their input, but the message is the same. The Entity Framework will not ship in Orcas. It will ship shortly there-after. The slip isn’t about Microsoft’s lack of commitment to the technology. It does give us more time to address some key feedback we’ve received to date from internal and external customers, including providing at least CTP-quality tools for defining flexible mapping between the store and the objects. There’s no way I can convince you that this isn’t another ObjectSpaces, or another WinFS. You’ll have to look at how we continue to deliver over the next few months. You’ll have to look at the quality of the bits. You’ll have to look at the other investments Microsoft is making around the Entity Data Platform, including announcements and demos this week at Mix.

This year I celebrate my 20th year at Microsoft. In that time I’ve seen Microsoft invests heavily in a number of technologies that, for one reason or another, never come to market. This isn’t one of them.

I'm inclined to believe Mike's comment. NetDocs also comes to mind as an example of a well-funded Microsoft technology that never came to market.

"Those who cannot remember the past are condemned to repeat it."

George Santayana (1863 - 1952, The Life of Reason)

Mike says in his initial post, "Stay tuned for announcements starting next week at Mix around new products building on the ADO.NET Entity Framework." I plan to do this, but with trepidation.

Update 5/3/2007: Redmond Developer News senior editor Kate Richards quotes me in her April 30, 2007 ".NET Entity Framework Slips Beyond Orcas" article:

Roger Jennings, a developer and technology writer for OakLeaf Systems, doesn't think the later ship date is going to be a big deal. "They decided that they couldn't release the Entity Framework without the Entity Data Model Designer, which was my contention to start with," he said. "And they've also just announced a couple of incubation projects that depend on it that also might have an influence on the delivery date." ...

"Almost everybody is considering [Entity Framework] to be ObjectSpaces revisited, but I don't think that's the case," says Jennings.

The "incubation projects" are the ADO.NET Team's "Project Jasper" (a.k.a., Dynamic ADO.NET) and "Project Astoria" (a.k.a., "RESTful Data in the Cloud.)

Technorati tags: Entity Framework, Entity Data Model, EDM Designer, LINQ to Entities, LINQ, Orcas, Orcas Beta 1, SQL Server, Katmai, MIX07

Note: This item was linked from the Discussions section of the Techmeme entry for the original ADO.NET post. There's also a Discussions link to a Channel9 thread.

More Issues with VB 9.0 Samples in the Updated LINQ to SQL Documentation

It's evident that no one tech-edited or double-checked the VB 9.0 code examples in the "LINQ to SQL: .NET Language-Integrated Query for Relational Data" whitepaper Dinesh Kulkarni, Luca Bolognese, Matt Warren, Anders Hejlsberg, Kit George that was updated in March 2007.

I encountered the most egregious example of the authors' disdain for VB in the following code snippet from the "Stored Procedures Invocation" topic:

<StoredProcedure(Name:= "VariableResultShapes")> _
  <ResultType(typeof(Customer))> _
  <ResultType(typeof(Order))> _
public VariableResultShapes(shape As Integer?) As IMultipleResults
  Dim result As IExecuteResult = ExecuteMethodCallWithMultipleResults(Me, _
    CType(MethodInfo.GetCurrentMethod(), MethodInfo), shape)
    return CType(result.ReturnValue, IMultipleResults)
End Function

The example uses C#'s typeof() instead of VB's GetType() function in lines 2 and 3, is missing the Function keyword on line 4, and uses the unsupported ? symbol for Nullable(Of T) on line 4. There are obvious errors on 50% of the lines of the example. Further, the following required Imports statements are not to found anywhere in the document:

Imports System.Data.Linq
Imports System.Data.Linq.Provider
Imports System.Reflection

It would behoove authors of whitepapers with a penchant for C# to at least check to see if their VB sample code compiles.

By the way, Kit George is a program manager on the VB team and presented the LINQ Overview Webcast on April 25, 2007.

Updated 4/27/2007: Dropped initial (off-topic) paragraphs.

Technorati tags: VB 9.0, C# 3.0, LINQ, LINQ to SQL, Orcas March 2007 CTP, Orcas Beta 1

LINQ and LINQ to SQL Webcast

Marc Schweigert of Microsoft's Public Sector DPE (Developer and Platform Evangelism) Team has produced a 90-minute Webcast entitled Introduction to LINQ + LINQ to SQL. The presentation aired at 2:00 PM EST on 4/24/2007, immediately after the VB team's Orcas Overview Webcast and the day before its LINQ Overview presentation. Registration is required to view the archived Webcast.

Following is the abstract:

Modern applications operate on data in several different forms: Relational tables, XML documents, and in-memory objects. Each of these domains can have profound differences in semantics, data types, and capabilities, and much of the complexity in today’s applications is the result of these mismatches. In this talk, we will explain how the Orcas release of Visual Studio aims to unify the programming models through LINQ capabilities in C#, a strongly typed data access framework, and an innovative Application Programming Interface (API) for manipulating and querying XML.

Database-centric applications have traditionally had to rely on two distinct programming languages: one for the database and one for the application. In the second part of this talk we will introduce LINQ to SQL, a component of the LINQ project designed to help integrate relational data and queries with C# and Visual Basic. LINQ to SQL enables developers to express queries and updates in terms of their local programming language without sacrificing the server-side execution model of today’s high-performance SQL-based approaches. Using these advances, database queries that previously were stored as opaque strings now benefit from static type checking, CLR metadata, design-time type inference, and of course IntelliSense. LINQ to SQL also supports a rich update capability that lets you save changes to an object graph back to the database using optimistic concurrency or transactions.

LINQ to SQL receives about the last 45 minutes of the presentation, most of which uses C#.

Technorati tags: LINQ, LINQ to Objects, LINQ to XML, LINQ to SQL, Orcas, Orcas Beta 1

Danny Simmons Featured on .NET Rocks

Carl Franklin's and Richard Campbell's .NET Rocks interview with Danny Simmons for Show #226, "Daniel Simmons on ADO.NET Entity Framework" appeared today. The show runs 1:08:00 and the interview with Danny starts at 0:08:20.

Update: 5/16/2007: The transcript for this segment is here. Danny's interview starts at the bottom of page 3.

Danny says at the start of his interview that he's been at the center of the Entity Framework's Object Services layer from its beginning.

Persistence Ignorance Missing in the Entity Framework

Here's an early excerpt that relates to Danny's participation in the "Persistence Ignorance" flare-up during the recent MVP Summit at Redmond:

11:45 Richard Campbell mentioned the "O/RM Smackdown" and that he sat at the back of the Entity Framework session at the recent MVP Summit in Redmond. He described "usual suspects," who were in the center of the room on the right, as the "NHibernate Mafia" (James Kovacs, Scott Bellware, Jeffrey Palermo, Jean-Paul Boodhoo, all NHIbernate believers) in a little cluster hammering Microsoft's ADO.NET folks. Danny was sitting at the left in the front, and popped up a couple of times to answer questions. When the session broke, it took him an hour to get out of the room.

14:30 Danny Simmons: Next day we scheduled a lunch [with the NHibernate Mafia] and just went at it. These folks have the impression that we just don't get their style of working. We don't get what they do within NHibernate and are trying to build "this other product." We gave Jeffrey Palermo our vision for a few releases out, this is what it could be like. Jeffrey said, "Yeah! Perfect! You've got it!" I said, "Now let me tell you why it's not coming out that way in this release."

Comment: I would have enjoyed hearing the "vision" and "why it's not coming out" in the Orcas release.

Jimmy Nilsson and the Disappearing Entity Framework Docs

15:50 Carl Franklin: Jimmy Nilsson blogged about the Entity Framework and all traces of the Microsoft documentation for it suddenly disappeared from the Internet. People kid Jimmy that he "killed the product."

Comment: Go to Jimmy's blog and search for the "Infrastructure and EDM" item of 5/12/2006 and its update to find the alleged culprit. (As far as I can determine, Jimmy's attributes the term "Persistence Ignorance" to Martin Fowler, but I haven't found any evidence of Fowler's use of the term or that Fowler has added it to his Catalog of Patterns of Enterprise Application Architecture or Development of Further Patterns of Enterprise Application Architecture page.)

For more details on the missing EF and EDM documents see my 3/15/2006 ADO.NET 3.0 Entity Framework Ephemera and, for their restoration, ADO.NET 3.0 Entity Framework Docs Redux of 6/19/2006.

Defining the Entity Framework and Distinguishing It from an O/RM Tool

The ADO.NET team avoids categorizing the Entity Framework as an Object/Relational Mapping tool, such as NHibernate and commercial O/RM products, although the Object Services layer maps entities to partial CLR classes and relationships. Danny defines the Entity Framework:

16:45 The idea of the Entity Framework is to create a new data model that's a higher abstraction than the relational model. The goal is to give the business model the same theoretical underpinnings--to give it the long-term legs--of the relational model. It's based on Peter Chen's Entity-Relationship model.

17:50 The Entity Framework makes a real runtime representation of the structure--the model of your data, and the constraints on that, in terms that are much closer to the way your application runs.

18:20 In general, it falls into the O/RM model. But there's a key distinction. We're trying to capture the structure and the relationships and constraints on the data as a separate thing before you get to the point of designing your business objects that have behaviors.

Comment: Don't miss Channel9's Dr. Peter Chen: Entity Relationship Model - Past, Present and Future segment video segment, in which Peter Chen and ADO.NET architect Jose Blakeley participate.

EDM Designer[s] and Their Delivery

The EDM Designer won't be in the Orcas RTM, but Danny says:

26:10 We will ship out of band on the Web more visual tools [for] designing the schemas, and maybe even getting to the point of visually specifying the mapping. Today you have to do that in a declarative format and long term I believe we're going to need a whole variety of [designers].

Comment: It's doubtful if many developers will adopt Orcas's Entity Framework for production apps until an "out of band" drop of the forthcoming EDM Designer proves it can handle modifications to XML schema files that cover 95% or more of developers' requirements.

LINQ to Entities vs. Entity SQL

It's still not clear to most developers when to use LINQ to SQL, the EDM with LINQ to Entities, or the EDM with Entity SQL (eSQL). Danny says the following about EDM with LINQ to Entities or eSQL:

29:30 It's a key concept in LINQ [to Entities] to define an interface to IQueryable and get a handle on that specification instead of the compiler actually compiling it. We do a runtime translation of that query specified into the same query trees and then it runs through all the rest of our stack the same as if you had specified the query any other way.

30:00 EntitySQL is very close to SQL, but it has extensions that understand inheritance and navigating relationships. It is a full-representation query language, so you can compose your queries, you can do joins, you can do very, very rich queries.

30:30 You can either buy into LINQ, which for some solutions makes very great sense, or use the text-based [eSQL] query option.

Comment: Danny didn't compare the Entity Framework and LINQ to SQL. LINQ to SQL connects SQL Server (only) databases directly to table-based objects with no intervening mapping layer. LINQ to SQL supports limited limited inheritance; the rows for base class and subclasses must be contained in a single database table. Unlike the Entity Framework, you can generate an SQL Server database from an entity model you create from scratch. Julie Lerman's LINQ to SQL vs. Entity Framework post provides more detailed distinctions, and check out Kevin Hoffman's early (August 2006) LINQ to Entities vs. LINQ to SQL - What should I use and when? post, which contains a feature-by-feature comparison.

Code Generation in Orcas and Future Versions

The present Entity Framework implementation, including the EDM Wizard, starts with the database schema and works up to the EDM and its Object Services.

43:10 The kind of generation we have now goes up from the database to the model. But a lot of people want to go from the object down and generate the database. If I add a property to an object, I want a script generated to add the field to the table and another script to remove it if I change my mind.

44:45 An interesting designer story is "I'm not trying to generate my code from my model, but I'm writing my code and putting hints in it, so we can generate the model out of the code."

45:25 In Orcas, we're getting some of this direction. Like many of the Visual Studio Designers, "Hey, there's this way you can design the model and then we'll do code generation of partial classes." But you also have the option of writing your classes from scratch, implementing a few interfaces that allow us to be aware of what's going on, and then writing the model by hand.

In a later release, we want to go to the next step, which is to write your classes and you tell us to build the model for you. Either one time or in an incremental fashion because "the truth is in the classes." But we have some customers who say "the truth is in the model," so we want to support them, too.

48:00 If you take the Entity Framework vision farther and farther forward, you eventually say, "Well, why do I need to specify the relational model at all? Why don't I hand my entity model directly to SQL [Server] and say 'Store this and do it fast!'"

Comment: The Orcas March 2007 CTP's EDM Wizard does a reasonable job of generating the XML mapping files and class code from the database up but, unlike LINQ to SQL, won't generate a database (or write CREATE DATABASE/TABLE or ALTER TABLE scripts) from the model.

Entity Framework in Orcas March 2007 CTP and Beta 1

50:40 From the standpoint of the Entity Framework, the March CTP and Beta 1 are almost identical. There's some last bit of features that we're busily working on now that will appear in Beta 2 and the Orcas Release. Search Technorati for "Entity Framework" and you'll find some interesting blog entries.

Comment: Be sure to use the double-quotes when you search for "Entity Framework" or you'll receive 4,000+ hits (90% off-topic) instead of 400+ that I found today.

Update 4/6/2007: Major restructuring and additions. 4/7/2007: Added links, made minor corrections, and fixed typos.

Technorati tags: Entity Framework, Entity Data Model, EDM, EDM Designer, LINQ to Entities, LINQ, Orcas March 2007 CTP, Orcas

Orcas EDM Wizard and Designer Previewed at VSLive! SFO

Last night the ADO.NET Team posted EDM Wizard and Designer Featured in VSLive! San Francisco Keynote, which briefly describes Britt Johnson's "Data Explosion: The Last and Next Decade in Data Management with the Microsoft Data Platform" keynote. Near the end of the presentation he demonstrated the Entity Data Model (EDM) Wizard and EDM Designer, neither of which made the Orcas March 2007 CTP cut, with a pair of screencasts.

See the 3/29/2007 and 5/21/2007 updates below.

A careful reading of this paragraph from the post:

Britt also featured 5 short videos or screencasts that demonstrated some of the work, specifically around Tools, that the Data Programmability team (including the ADO.NET) has been doing. These videos provide some great information and a preview of the new EDM Wizard coming in Orcas, as well as a sneak preview of a new EDM Designer that we can expect to see released after the upcoming Orcas release [Emphasis added].

indicates that the Orcas RTM bits won't include the EDM Designer. Frans Bouma and I have added questions about this to the post's comments. I also asked the same question in the ADO.NET Orcas forum.

A similar post in the Data Programmability Team blog has a slightly expanded list of new features and screencasts:

These videos provide some great information and a preview of the new XML Editor, XSLT Debugger, and EDM Wizard coming in Orcas, as well as a sneak preview of a new XSD Designer and EDM Designer that we can expect to see released after the upcoming Orcas release.

Elisa Johnson's VSLive Keynote - San Francisco post of the same date says:

During his presentation Britt talked a lot about Conceptual Data Programming and where Microsoft plans to focus on for future innovation in the Data Access space. He also gave a sneak peak at two tools that hadn't previously been seen... the much anticipated EDM Designer and the XSD Designer (you can expect to see more on these sometime after the upcoming Orcas release).

A working preview of the EDM Designer debuted in September 2006; my October 4, 2006 New Entity Data Model Graphic Designer Prototype post provided a walkthrough with the Northwind database.

Sanjay Nagamangalam, the ADO.NET Program Manger who presented the EDM Designer screencast, posted on March 3, 2007 the following message in the ADO.NET Orcas forum:

An EDM designer is foremost on our minds and we are looking to provide a tool for Beta 1. We don’t have a date yet though.

Of course, the message was accompanied by the usual "This posting is provided 'AS IS' with no warranties, and confers no rights" disclaimer.

Shades of ObjectSpaces. Déjà vu all over again. Please say it isn't so.

Update 3/28/2007: Visual Studio Magazine writer Lee Thé writes in his "Microsoft Moves DBMS into the VS Developer Mainstream" article that covered Britt Johnson's keynote:

The final demonstration featured an entity data models (EDM) Wizard that generates classes from the conceptual model. But the portion of this demo that roduced spontaneous applause from the audience was an EDM designer functionality that will not be available in the Orcas beta release. This hotly anticipated functionality is a database designer and an entity modeler. These features let you map a database to a model, keeping the model in sync. The project manager leading this demonstration created a user entity type from an users' table with drag-and-drop functionality, where glyphs showed mapping of entity type to table. Sophisticated graphical representations of entities and relationships enabled the user to work at the level of the business relationships rather than programming abstractions. [Emphasis added]

It remains to be seen if Lee's report is correct. Stay tuned for Microsoft's official confirmation or denial of the EDM Designer's disappearance from Orcas.

Note: Lee's report on VSLive!'s Monday keynote, "Windows Vista, the 2007 Office system, and ASP.NET AJAX" by Prashant Sridharan is here, and his coverage of K. D. Hallman's general session address, "Visual Studio Everywhere: Tools for Office, Office Business Applications, and Custom Application Extensibility," for Redmond Developer News is here. Click here to read the March 28, 2007 VSLive! Show Daily.

InfoWorld's Paul Krill mentions the EDM Designer in his March 27, 2007 "Microsoft maps data management plans" article without discussing the Designer's future availablity. He notes that the EDM Wizard, XSD Designer, and XSLT Debutter are slated for the Orcas Beta release in May. Elisa Johnson says that The XSD Designer will arrive post-Orcas.

Kevin Hoffman complains bitterly in his Microsoft finally shows off their EDM designer... but it won't ship with Orcas?!? post about the apparent demise of the EDM Designer.

Update 5/21/2007: The 1:04:00 video of Britt Johnson's keynote is available here. Click here for Prashant Sridharan's keynote video.

Update 3/29,2007: Julie Lerman—2,500 miles from San Francisco at the DevConnections conference in Florida—reports in What to expect in next (and future) Orcas bits for Entity Framework :

Britt Johnston did a keynote and showed [a video of] the latest prototype of the EDM Modeler and also let us know that it won't be ready for Orcas but they plan to release it shortly after Orcas. This is really frustrating, but it is just the reality and as developers we know the difficulties of designing tools... so it is what it is and until we have it, I will learn a LOT with the XML and personally hold off on doing any seriously complex modeling. [Emphasis added.]

Pending official word from Microsoft, I'm inclined to believe Julie's rendition, although telepathy might be involved. I'm still waiting for someone from the ADO.NET team to post an official response, either as a blog comment or an answer to my question in the ADO.NET Orcas forum.

Note: There's more from Julie's report on Brian Dawson's two Entity Framework presentations at DevCon in her post.

Technorati tags: Entity Framework, Entity Data Model, EDM, EDM Wizard, EDM Designer, Orcas, ObjectSpaces, Object/Relational Mapping

Third-Party LINQ Providers

Following is a short list of the third-party LINQ providers I've found to date in more-or-less chronological order:

• LINQ to WebQueries by Hartmut Maennel handles searches in the SiteSeer and MSDN Web sites. (This provider predates Fabrice's LINQ to Amazon provider by a few days.)
• LINQ to Amazon by Fabrice Marguerie, a co-author of the forthcoming LINQ in Action book, was the first third-party LINQ provider that I know of. LINQ to Amazon returns lists of books meeting specific criteria.
• LINQ to RDF Files by Hartmut Maennel handles queries against Resource Description Format files' triples. Part I of the two-part post is here.
• LINQ to MySQL by George Moudry, based on the LINQ May 2006 CTP, was in the development stage as of January 2007, but George says it's "capable of simplest queries and updates" and "now has support for most primitive joins."
• LINQ to NHibernate by Ayende Rahien (a.k.a. Oren Eini) translates LINQ queries to NHibernate Criteria Queries and is based on the Orcas March 2007 CTP. The documentation that describes development of the provider presently is at the Part 1 stage.
• LINQ to LDAP by Bart de Smet is a "query provider for LINQ that's capable of talking to Active Directory (and other LDAP data sources potentially) over LDAP." As of 4/11/2007, Bart's "IQueryable Tales - LINQ to LDAP" consisted of Part 0: Introduction, Part 1: Key Concepts, Part 2: Getting Started with IQueryable, Part 3: Why do we need entities?, Part 4: Parsing and executing queries, and Part 5: Supporting Updates.
• LINQ to Flickr by Mohammed Hossam El-Din (Bashmohandes) uses the open-source FlickrNet C# library as its infrastructure.
• LINQ to Google Desktop by Costa Rican programming language enthusiast Luis Diego Fallas supports GDFileResult and GDEmail types. A subsequent Adding support for projections to Linq to Google Desktop implements the LINQ Select expression.
• LINQ to SharePoint by Bart de Smet supports writing LINQ queries for SharePoint lists in both C# 3.0 and Visual Basic 9.0 and communicates with SharePoint via Web services or though the SharePoint Object Model. The SpMetal command-line utility automates C# or VB class generation.
• LINQ to Streams (SLinq, Streaming LINQ) by Oren Novotny processes continuous data streams, such as stock tickers or sensor data. The project's home page on CodePlex includes an animated GIF simulation of a stock ticker displayed in a DataGridView. The current version supports Select, Where, Order By, and Descending only.
• LINQ to Expressions (MetaLinq) by Aaron Erickson (the developer of Indexes for Objects a.k.a i4o*) lets you query over and edit expression trees with LINQ. Like .NET strings, LINQ expression trees are immutable; the only way you can change a LINQ expression tree is to make a copy, modify the copy, and then replace the original. MetaLinq's ExpressionBuilder lets you create an "Editable Shadow of an expression tree, modify it in place, and then by calling ToExpression on the shadow tree, generate a new, normal, immutable tree." ExpressionBuilder is an analog of the .NET StringBuilder.

* i4o isn't a LINQ provider, per se, but a helper class that can increase the speed of LINQ queries against large collections by a factor of 1,000 or more. InfoQ published on June 22, 2007, Aaron Erickson on LINQ and i4o, an interview of Aaron Erickson by Jonathan Allen about i4o's purpose and background.

Ayende observes:

There is an appalling lack of documentation about how to implement LINQ providers. ... I decided to document what I found out while building LINQ for NHibernate.

However, the "appalling lack of documentation" hasn't thwarted the work of third-party LINQ providers for specialty data domains. A search of the CodePlex site on LINQ returned 15 projects as of 6/3/2007.

If you know of other third-party LINQ providers in development, please leave a comment.

Thanks in advance.

Update 3/29/2007: .NET Rocks features "Oren Eini On NHibernate and RhinoMocks!" as its 3/29/2007 broadcast. LINQ to NHibernate might become a much more popular LINQ provider if the ADO.NET Team doesn't finish the EDM Designer by Orcas RTM. The interview with Oren starts at 11:00. (Thanks to Danny Simmons for the heads-up.) .NET Rocks TV (dnrTV) taped an instructional video interview with Oren on January 25, 2007.

Update 4/7/2007: Added Bart de Smet's LINQ to LDAP project, which includes extensive IL code inspection of LINQ queries.

Bobby Diaz has extended Ayende's initial Part 1 From ... In ... Where ... Select implementation with Part 2: Ordering and Paging (adds Order By, Then By, OrderByDescending, ThenByDescending, Take, Skip, and Distinct) and Part 3: Aggregate and Element Operators (adds First, FirstOrDefault, Single, SingleOrDefault, Average, Count, LongCount, Max, Min, and Sum.) The momentum is building behind LINQ to NHibernate.

Update 4/11/2007: Added Mohammed Hossam El-Din's LINQ to Flickr provider and updated LINQ to LDAP with parts 4 and 5.

Update 5/6/2007: Added Bart de Smet's LINQ to SharePoint project, version 0.1.2.0 Alpha release.

Update 5/12/2007: Added Helmut Maennel's early LINQ to WebSearch and LINQ to RDF Files providers. I haven't tested these providers with later LINQ implementations.

Update 6/3/2007: Added Oren Novotny's LINQ to Streams (SLinq) and Aaron Erickson's LINQ to Expressions (MetaLinq) and Indexes for Objects (i4o).

Update 6/11/2007: Added Luis Diego Fallas' LINQ to Google Desktop of May 11 and 12, 2007, which I had missed.

Update 7/22/2007: Added link to inteview with Aaron Erickson about i4o.

Update 8/7/2007: Aaaron Erickson has written "Indexed LINQ" for .NET Developer's Journal. Subtitiled "Optimizing the performance of LINQ queries using in-memory indexes," the article covers the theory behind creating indexes on in-memory collections and applying an extension method to the Where() standard query operator to enable speeding queries by up to a factor of 100 or so with the indexes. You can download the source code, runtime binary or both for i4o at http://www.codeplex.com/i4o.

Technorati tags: LINQ, Orcas March 2007 CTP, Orcas, NHibernate