The Language Integrated Query (LINQ) Project

Two features once promised for Visual Studio 2005 are conspicuous by their absence in VS 2005 Beta 2 Community Technical Preview drops:

• An object-relational mapping API and toolkit for ADO.NET 2.0 called ObjectSpaces
• An XQuery 1.0 implementation for the .NET Framework 2.0

Searching descriptions of PDC 2005 sessions for the keyword "query" indicates that the forthcoming Languate Integrated Query (LINQ) for .NET Project (nee Integrated Query Framework (IQF)) will resurrect in the VS Orcas release at least some of the promised features of the original .NET 2.0 implementations of ObjectSpaces and the XQuery 1.0 processor. (Copies of session descriptions are at the end of this post.) Recent MSDN articles and Microsoft patent applications add evidence of major changes in store for VB 9.0 and C# 3.0 in the next VS release.

Note: Technorati category links in the preceding text are experimental

ObjectSpaces Support in VS 2005 Beta 1 Orca (apparently an acronym for Object-Relational Component Architecture) was a technical preview of ObjectSpaces that first appeared in a PDC 2001 ADO.NET presentation and was promised as a Web release for VS .NET 1.0. ObjectSpaces introduced OPath, a proprietary object query language. I spent a substantial amount of time testing the PDC's technical preview bits. I don't believe there were subsequent Web release(s). Luca Bolognese's "DAT410 NET Framework: Developing Applications Using ADO.Net ObjectSpaces" presentation at PDC 2003 set the stage for ObjectSpaces' inclusion in the VS Whidbey release. Luca demonstrated a VS add-in tool to simplify the O-R mapping process, which required a set of three XML configuration (schema description) files for the object (Osd.xml), relational (Rsd.xml), and mapping (Msd.xml) layers.

Luca promised future support for generics with syntax similar to: Dim osQuery As New _ ObjectQuery(Of Customer)("Region='WA'","Orders") Dim osReader As ObjectReader(Of Customer) = _ ObjectSpace.GetObjectReader(osQuery). The final bulletpoint on Luca's "Conclusions" slide was: "It is included in the next version of Visual Studio." Peter Provost blogged the presentation and Paul Wilson's "ADO.NET v2.0: ObjectSpaces Delivers an O/R Mapper" article has sample code from the PDC alpha version. Dino Esposito's February 2004 MSDN article, "A First Look at ObjectSpaces in Visual Studio 2005," provides additional architectural details and sample code. The Whidbey alpha included a System.Data.ObjectSpaces namespace. Luca reappeared at Tech*Ed 2004 with the "DAT 301: ObjectSpaces" presentation, which included more advanced examples of OPath query syntax. Microsoft originally planned to use XPath as the query language but found that virgules (/) turned off developers, who preferred periods (.) as separators. Thus typical OPath query expressions against Northwind-based Orders and Details objects might have been: Orders.Details.Quantity > 50

Orders[ShippedDate > RequiredDate]

Orders[Freight > 1000].Details[Quantity > 30]

At the end of the presentation, Luca announced that ObjectSpaces had acquired a dependency on WinFS and thus was postponed to the Longhorn/Orcas timeframe. (He appeared close to tears as he made the announcement.) The dependence of the long-delayed Microsoft Business Framework (MBF) API on ObjectSpaces as its object persistence platform probably was a contributing factor to the demise of ObjectSpaces in VS 2005. Subsequently, Microsoft announced that WinFS had been excised from the planned Longhorn release version, now renamed Vista.

Here are three published Microsoft patent applications that include Luca Bolognese as an inventor and relate to Object Spaces:

• Mapping architecture for arbitrary data models (20050050068)
• System and method for presenting a query expressed in terms of an object model (20040078359)
• System and method providing API interface between XML and SQL while interacting with a managed object environment (20030236859)

Matt Warren's "ObjectSpaces:There and Back Again" and Dinesh Kulkarni's "ObjectSpaces -> DLinq" blog entries throw additional light on ObjectSpace's transition from mapping files to LINQ and DLinq.

XQuery 1.0 Support in .NET Framework 2.0 Beta 1

About three years ago, Microsoft sponsored an XQuery demonstration site that included an XQuery engine (Microsoft.Xml.XQuery) for .NET 1.0. This early XQuery implementation didn't support all XQuery keywords, constructs, or use cases, but it provided a simple route for .NET developers to become familiar with XQuery 1.0 and XPath 2.0 syntax.

Beta 1 of the .NET Framework 2.0 included a System.Xml.Query namespace, which provided an XQueryCommand class (the XQuery processor) to implement a subset of XQuery 1.0 for XML documents. Mark Fussell's March 2004 MSDN article, "What's New in System.Xml for Visual Studio 2005 and the .NET Framework 2.0 Release," has a section named "The XQuery Language," which includes a simple C# example of an XQuery FLWOR (for, let, where, order by, and return) expression:

using (XmlWriter writer = XmlWriter.Create("output.xml ")) { XQueryCommand xq = new XQueryCommand(); string query = "" + "{ for $s in /bookstore/book " + "where $s/@genre='autobiography' " + "return $s/title }" + ""; xq.Compile(query); xq.Execute("books.xml", new XmlUrlResolver(), writer); }

Notice that the XQuery processor compiles the FLWOR query and then executes the compiled version against the books.xml file to write the output.xml document. Output.xml contains the title subelements of book elements whose genre attribute value is autobiography. (After XQuery appeared in the Whidbey bits, Microsoft turned off the XQuery demo site.)

Mark's article also included a section titled "Universal Type Support and Conversion," which described changes to the the XmlReader, XmlWriter, and XPathNavigator classes required to support the XQuery 1.0 and XPath 2.0 Data Model with XML schemas and CLR types. Dare Obasanjo's January 2005 MSDN article, "An Overview of Cω: Integrating XML into Popular Programming Languages," describes constructs in Microsoft Research's Cω (C-Omega) programming language that "...bridge the gap between Relational, Object, and XML data access by creating a type system that is a combination of all three data models." Although Dare's article discusses Cω features in a C# context, it's reasonable to expect that VB in Orcas will gain similar type system additions and programming constructs.

Whidbey Beta 2 removed the XQueryCommand class for the reasons cited in Microsoft's brief "Status of XQuery in the .NET Framework 2.0" white paper. The Microsoft XML Team's Mike Champion and Arpan Desai weigh in on what appears to have been a traumatic decision. Michael Rys, program manager for SQL Server’s XML Technologies, explains why SQL Server's xml data type uses XQuery 1.0 despite the XML team's arguments against it's inclusion in the .NET Framework 2.0.

The XML Team's white paper suggests that VS 2005 "... customers continue to use XSLT and XPath to address their XML filtering and transformation scenarios." The System.Xml.Xsl.XslCompiledTransform object represents an XSLT processor that supports XSLT 1.0 syntax. Following is C# sample code from VS 2005's online help for XslCompiledTransform:

// Create the XslCompiledTransform object. XslCompiledTransform xslt = new XslCompiledTransform(); // Compile the style sheet. xslt.Load("output.xsl"); // Execute the transform and output the results to a file. xslt.Transform("books.xml", "books.html");

The XslCompiledTransform.Load method, which compiles the specified stylesheet, corresponds to the XQueryCommand.Compile method and the XslCompiledTransform.Transform method approximates the XQueryCommand.Execute method.

References to LINQ and XLinq Predecessors

Arpan Desai, program manager for XSLT and XQuery in Whidbey, says "...we came up with a new query architecture to support both XSLT and XQuery." Michael Brundage, author of XQuery: The XML Query Language, is an inventor of the Common Query Runtime (CQR). CQR "...features compilers for each XML query language (such as XSLT and XQuery), a common intermediate format (QIL [Query Intermediate Language]), and multiple backend execution engines (over XML and SQL).

Following are Microsoft's U.S. and international patent applications related to IQF (from Michael Brundage, all filed in 2003):

• 304064.1 Common Query Runtime System and API (20040260691)
• 303845.1 Query Optimizer System and Method (20050004892)
• 301638.1 Query Intermediate Language Method and System (20040267760)

Numbers in parenthesis are U.S. Patent Office published application document numbers with links to the applications' full text and drawings. The Patent Office published the applications in 2004 and 2005. The patent applications reference XQuery and OPath as XML query languages and provide examples of relational queries using the Northwind database's tables.

The ".NET Language" prefix indicates to me (and others, such as Wesner Moise) that implementations will be specific to VB or C#. As an example, VB and C# appear to share anonymous types, but only Paul Vick's PDC 2005 session mentions query comprehensions and object initializers.

Soumitra Sengupta's "Changing the Game" entry to the XML Team's Weblog provides additional insight on the effect of LINQ and XLinq on the future of Microsoft's XML technologies.

PDC 2005 Sessions on LINQ and Related Topics

Links to individual sessions at http://commnet.microsoftpdc.com/content/sessions.aspx don't appear to be possible, so following are copied descriptions of LINQ-related sessions at PDC 2005. InfoWorld published on August 25, 2005 an article by Paul Krill that refers to the LINQ sessions. A similar article by Darryl K. Taft in the August 26 issue of eWeek.com describes a purported FoxPro-LINQ connection.

The .NET Language Integrated Query Framework: An Overview Speaker(s): Anders Hejlsberg Modern applications operate on data in several different forms: Relational tables, XML documents, and in-memory objects. Each of these domains can have profound differences in semantics, data types, and capabilities, and much of the complexity in today's applications is the result of these mismatches. The future "Orcas" release of Visual Studio aims to unify the programming models through integrated query capabilities in C# and Visual Basic, a strongly typed data access framework, and an innovative API for manipulating and querying XML. This session introduces each of these areas and walks through how they are related.

Using the .NET Language Integrated Query Framework with Relational Data Speaker(s): Luca Bolognese Database-centric applications have traditionally had to rely on two distinct programming languages: one for the database and one for the application. This session introduces future advances Microsoft is making for the "Orcas" release of Visual Studio in programming languages and frameworks to help integrate relational data and queries with C# and Visual Basic. These advances enable developers to express queries and updates in terms of their local programming language without sacrificing the server-side execution model of today's high-performance SQL-based approaches. Using these advances, database queries that previously were stored as opaque strings now benefit from static type checking, CLR metadata, design-time type inference, and of course IntelliSense.

Using the .NET Language Integrated Query Framework with XML Data Speaker(s): Dave Remy One of the key challenges to working with XML data has been the impedance mismatch between XML and programming languages. This session introduces future advances Microsoft is making for the "Orcas" release of Visual Studio in programming languages and frameworks to help integrate XML and queries with C# and Visual Basic. The advances include a framework for navigating, querying, and transforming XML that is both easier to use and more efficient than current XML programming techniques. This framework marries the capabilities of XPath, XQuery, and the DOM with the language integrated query framework planned for C# and Visual Basic.

Visual Basic: Future Directions in Language Innovation Speaker(s): Amanda Silver, Paul Vick Visual Basic 9.0 will offer radical improvements in its ability to work with data in all its forms: as objects, as XML, as relational data. Join the language architects for a detailed discussion of features such as query comprehensions, object initializers and anonymous types that enable querying data in a more flexible, natural way than ever before. Also, get a glimpse into the future of dynamic programming in VB with coverage of new features intended to radically simplify working with dynamically typed data on the .NET platform.

C#: Future Directions in Language Innovation from Anders Hejlsberg Speaker(s): Anders Hejlsberg Join Anders Hejlsberg, Distinguished Engineer and chief architect of the C# language, for an in-depth walkthrough of the new language features in C# 3.0. Understand how features like extension methods, lambda expressions, type inference, and anonymous types make it possible to create powerful APIs for expressing queries and interacting with objects, XML, and databases in a strongly typed, natural way. It is suggested that you attend "The .NET Language Integrated Query Framework: An Overview" before attending this session.

--rj

Technorati tags: LINQ, ADO.NET vNext, ObjectSpaces