Blog Search Engine Comparisons - Google, MSN, Yahoo!, Technorati, Feedster and Ice Rocket

You might have noticed that I've been adding Technorati subject tags to the end of my posts since mid-September. The most common tags are LINQ, XLinq, and DLinq because I've been testing and writing about Microsoft's new Language Integrated Query, Visual Basic 9.0, and C# 3.0 enhancements since their September 13 public release as technology previews at the Professional Developers Conference (PDC) 2005 in Los Angeles. Update 10/5/2005: Technorati responded to my technical service request of 10/3/2005 on 10/4/2005 and respidered the OakLeaf blog, which corrected the problems with the missing tagged posts, as you can see from clicking the following two links and DLink. The blog-wide topic tags are operational; give XLINQ a try, expand the Tagged link, and click a topic tag, such as SQL Server 2005. Original 10/3/2005: Posts I tagged with LINQ and XLink from September 13 to 26 appear on Technorati Tags pages, as you can see by clicking the LINQ and XLinq links in the preceding paragraph. However, the five posts after September 26 don't appear on the Tags pages, despite having manually pinged the Technorati site repeatedly. DLinq tags I added after September 26 don't register in the tag pages, either. I also added 16 topic tags to Technorati's Blog Finder on September 30. Although the topic tags appear in my Account Profile, searches on any of the added tags don't return links to OakLeaf pages. The effect is as if Technorati embargoed my blog site on September 27. This problem piqued my curiousity regarding the efficacy of other search services with blog posts, so here are search results as of October 3, in descending order of the number of valid hits: Google Blog Search and Blogger Search for LINQ blogurl:http://oakleafblog.blogspot.com/ returned all 17 posts through October 1 and 18 items after this October 2 post. Feedster Search for LINQ and RSS URL for the Atom feed returned 15 posts. (The first page gave 28 and the second page gave 16 as the result count. 15 is a manual count.) Technorati Search for LINQ and rj originally returned 12 hits of which 11 were my posts on September 26 or before. Update 10/5/2005: This search now returns 19 posts in the last 67 days. Thus Technorati moves to number 1 position in the search results. Ice Rocket Search for LINQ and rj returned 10 hits with September 29 as the most recent date on the first try, then expanded to 42 posts. Searching for LINQ and Author=rj returned 43 posts starting with the latest entry (October 3). Tag=LINQ AND rj returns 39 hits. Most of the 30+ results were duplicates. The duplicates might be a problem with detecting posting dates versus dates of minor post edits or updates. MSN Search with URL=oakleafblog.blogspot.com and LINQ returned 9 posts. Note that you must open the Settings page, clear the 'Group results from the same site' checkbox, and click Save to obtain more than two posts per site. Google Search for oakleafblog and LINQ returned 8 non-similar hits. Google Reader (beta, requires registration) finds 10 sources for LINQ, but none from this blog. A search on the OakLeaf blog address returns all current posts; strangely, adding LINQ returns only one. Yahoo! Search with domain=oakleafblog.blogspot.com and LINQ OR DLinq OR XLinq returned 1 post on September 18. As of 10/7/2005, it returns 9 or 10 rather random results; some of the links don't include LINQ, XLink, or DLink in the post. Repeated searches appear to increase the number of results. (Your results might differ due to posts added after the preceding tests.) Obviously Google Blog Search is the champ in this case, but that's understandable because Google owns Blogger. Business Week's Stephen Baker calls Google Blog Search "lackluster," but it works for me. (Blogger uses Google Blog Search for Blogger Search). Technorati Text Search doesn't return OakLeaf posts after September 26. (Update: All OakLeaf posts through 10/3/2005 are now returned.) 50% fewer links from MSN Search than Google Blog Search and only a single hit from Yahoo! Search surprised me. Maybe Yahoo is minimizing blog responses in preparation for their future blog and RSS search tool. So far, I haven't found an explanation for the Technorati problems. I'm waiting for Technorati's tech service team to respond to a "My site is not properly indexed" problem report. (Update: Technorati responded to my tech service request and respidered the site on 10/4/2005.) --rj Technorati: Search Engines Search Weblogs Blogging Google MSN Yahoo Technorati Feedster Ice Rocket Updated 10/3/2005: Added Feedster and Ice Rocket Search results. Ice Rocket's Blogs Trend Tool provides a time-based graphical display of the percentage of all blogs that mention up to three search terms, such as LINQ, XLinq, and/or DLinq. I would have added Bloglines,if this Ask Jeeves offshoot had returned results from a LINQ search dated later than December 8, 2004. Yahoo!'s Russell Beattie asks "What's up with Bloglines," but doesn't answer the question. Inexplicably, Barry Diller intends to drop the Ask Jeeves trademark, which seems to me to be the operation's primary asset. Updated 10/4/2005: See inline updates for fix by Technorati technical service. Updated 10/8/2005: The Wall Street Journal posted a "New Search Engines Help Users Find Blogs" story on 10/7/2005 (free), which contains the following controversial quotation: "David Sifry, chief executive of Technorati, says his company gets an edge from exclusive deals in which some blog-hosting companies ping Technorati before anyone else. After receiving a heads-up, Technorati visits the blog and updates its database." Business Week's Steven Baker comments on the quote in "Is Technorati making deals to get data first?" Jon Udell's "What is the future of open blog infrastructure?" item comments on Baker's story. Personally, I doubt if "exclusive dealers" would add the complexity required to delay pings to Technorati's "competitors." Also added results for new beta version of Google Reader. Differences in results from Google Blog Search are surprising. Looks to me as if Google Reader needs some work.

Compose XML Content with XLinq Expressions and VB 9.0 XML Literals

My XLinqTestProjectVB project expands on the VB 9.0 examples included in Erik Meijer's "XLinq: XML Programming Refactored (The Return Of The Monoids)" presentation at the XML 2005 Conference to be held in Atlanta on November 14 - 18, 2005. The project demonstrates composing well-formed XML documents from Northwind sample data by the use of:

1. VB 9.0 expressions that use the XLinq API's XDocument, XDeclaration, XDocumentType, XComment and XElement classes.
2. VB 9.0 expressions that use the XLinq API's XElement class and add elements with calculated values in a specified position in the document.
3. VB-only XML Literals created by copying and pasting XML content as VB 9.0 code.

You'll be able to download the project's code from the Fawcette site in the near future. (I'll add a link when the code becomes available.) In the meantime, I've provided some VB 9.0 code examples from the project to illustrate how to compose XML documents with XLink expressions as well as VB 9.0 XML literals.

A Simple Sales Order XDocument

The following code composes and displays a static XML document in a text box on the project's Windows form. The document adds instances of the XDocument (container), XDeclaration, XDocumentType, XComment, and XProcessingInstruction classes to the composed XElement instances. (XDeclaration and XDocumentType weren't finished when Microsoft released the initial Technology Preview.)

(To test the code, create a LINQ Windows form project, add a txtXML multiline text box to the form, paste the code for the SalesOrderDocument method to the form, and execute it with a button or from the Form1_Load event handler.)

Private Sub SalesOrderDocument()
   'Create a static SalesOrders XDocument
   'from XLinq expressions

   'Define a static BillTo group
   Dim expBillTo = _
   New XElement("BillTo", _
   New XAttribute("CustomerID", "RATTC"), _
   New XElement("CompanyName", _
    "Rattlesnake Canyon Grocery"), _
   New XElement("ContactName", _
    "Paula Wilson"), _
   New XElement("ContactTitle", _
    "Assistant Sales Representative"), _
   New XElement("Address", "2817 Milton Dr."), _
   New XElement("City", "Albuquerque"), _
   New XElement("Region", "NM"), _
   New XElement("PostalCode", "87110"), _
   New XElement("Country", "USA"), _
   New XElement("Phone", "(505) 555-5939"), _
   New XElement("Fax", "(505) 555-3620"))
   '
   'Define a static ShipTo group
   Dim expShipTo = _
   New XElement("ShipTo", _
   New XElement("ShipName", _
     "Rattlesnake Canyon Grocery"), _
   New XElement("ShipAddress", _
     "2821 Milton Dr."), _
   New XElement("ShipCity", "Albuquerque"), _
   New XElement("ShipRegion", "NM"), _
   New XElement("ShipPostalCode", "87110"), _
   New XElement("ShipCountry", "USA"))
   '
   'Define three static LineItem groups
   Dim expLineItem1 = _
   New XElement("LineItem", _
   New XAttribute("OrderID", 10262), _
   New XElement("ProductID", 5), _
   New XElement("UnitPrice", 17.0), _
   New XElement("Quantity", 12), _
   New XElement("Discount", 0.2))
   '
   Dim expLineItem2 = _
   New XElement("LineItem", _
   New XAttribute("OrderID", 10262), _
   New XElement("ProductID", 7), _
   New XElement("UnitPrice", 24.0), _
   New XElement("Quantity", 15), _
   New XElement("Discount", 0.0))
   '
   Dim expLineItem3 = _
   New XElement("LineItem", _
   New XElement("Quantity", 2), _
   New XAttribute("OrderID", 10262), _
   New XElement("ProductID", 56), _
   New XElement("UnitPrice", 30.4), _
   New XElement("Discount", 0.0))
   '
   'Create a SalesOrder document
   Dim expOrder = _
   New XDocument( _
   New XDeclaration("1.0", "UTF-8", "yes"), _
   New XDocumentType(), _
   New XComment("XElement Demo"), _
   New XProcessingInstruction("XLinkProject", _
     "ProcessingInstructions"), _
   New XElement("SalesOrder", _
   New XAttribute("OrderID", 10262), _
   New XAttribute("EmployeeID", 8), _
   New XAttribute("ShipVia", 3), _
   expBillTo, _
   New XElement("OrderDate", "1996-7-22"), _
   New XElement("RequiredDate", "1996-8-19"), _
   New XElement("ShipDate", "1996-7-25"), _
   New XElement("Freight", 48.29), _
   expShipTo, _
   New XElement("LineItems", _
   expLineItem1, expLineItem2, expLineItem3)))
   '
   'Show formatted document in the text box
   txtXML.Text = expOrder.ToString
End Sub

This figure shows the project's form displaying the first part of the XML document composed by executing the preceding method.

XLinq overloads the Object.ToString method to display the XML document formatted with 1-character indents and CR/LF pairs.

A Sales Order XElement with Added XElement Groups

The following two methods create a simple Sales Order document from XElements and then add after the LineItems group a Summary group with calculated order totals.

(If you copy the following code to a project, don't forget to copy and paste the expressions that define the expBillTo, expShipTo, and three expLineItem# groups.)

Private Sub SalesOrderExpression()
   'Create a static SalesOrders XElement
   'from XLinq expressions
   '
   'Add expBillTo, expShipTo,
   'and expLineItem1...3 from above
   '
   'Create a SalesOrder parent XElement
   Dim expOrder = _
   New XElement("SalesOrder", _
   New XAttribute("OrderID", 10262), _
   New XAttribute("EmployeeID", 8), _
   New XAttribute("ShipVia", 3), _
     expBillTo, _
   New XElement("OrderDate", "1996-7-22"), _
   New XElement("RequiredDate", "1996-8-19"), _
   New XElement("ShipDate", "1996-7-25"), _
   New XElement("Freight", 48.29), _
     expShipTo, _
   New XElement("LineItems", _
     expLineItem1, expLineItem2, expLineItem3))
   '
   'Add calculated items and order total after
   'the element
   Dim decFreight As Decimal = 48.29D
   Dim xeLineItems As XElement = _
     expOrder.Element("LineItems")
   xeLineItems.AddAfterThis(New _
     XElement("Summary", _
   New XElement("Items", _
     SalesOrderTotal(expOrder)), _
   New XElement("Freight", decFreight), _
   New XElement("OrderTotal", _
     SalesOrderTotal(expOrder) + decFreight)))
   '
   txtXML.Text = expOrder.ToString
End Sub
'
Function SalesOrderTotal(ByVal xeSO _
  As XElement) As Decimal
   'Calculate LineItems discounted total
   Dim decTotal As Decimal
   For Each xeItem As XElement In _
     xeSO.Descendants("LineItem")
      Dim decPrice As Decimal = _
        CDec(xeItem.Element("UnitPrice"))
      Dim decQuantity As Integer = _
        CInt(xeItem.Element("Quantity"))
      Dim decDiscount As Decimal = _
        CDec(xeItem.Element("Discount"))
      decTotal += decQuantity * decPrice * _
        (1 - decDiscount)
   Next
   Return Math.Round(decTotal, 2)
End Function

The following figure shows the last part of the XML document composed by the preceding two methods. The added Summary group with calculated element values is highlighted.

Composing XML Documents with VB 9.0 XML Literals

VB 9.0 lets you compose XML documents with XML Literals, which you can copy and paste from a source document to a VB 9.0 method. The following code creates a document similar to the preceding, but your code formatting controls formatting of the resulting XML document. Notice the ASP.NET-like syntax for inserting named XML literals.

    Private Sub SalesOrderLiteral()
        'Copy and paste XML content to create
        'Visual Basic XML literals
        '
        Dim litBillTo = _
  <BillTo CustomerID="RATTC">
    <CompanyName>
      Rattlesnake Canyon Grocery
    </CompanyName>
    <ContactName>Paula Wilson</ContactName>
    <ContactTitle>
      Assistant Sales Representative
    </ContactTitle>
    <Address>2817 Milton Dr.</Address>
    <City>Albuquerque</City>
    <Region>NM</Region>
    <PostalCode>87110</PostalCode>
    <Country>USA</Country>
    <Phone>(505) 555-5939</Phone>
    <Fax>(505) 555-3620</Fax>
  </BillTo>
        '
        Dim litShipTo = _
  <ShipTo>
    <ShipName>Rattlesnake Canyon Grocery</ShipName>
    <ShipAddress>2821 Milton Dr.</ShipAddress>
    <ShipCity>Albuquerque</ShipCity>
    <ShipRegion>NM</ShipRegion>
    <ShipPostalCode>87110</ShipPostalCode>
    <ShipCountry>USA</ShipCountry>
  </ShipTo>
        '
        Dim litLineItem1 = _
    <LineItem OrderID="10262">
      <ProductID>5</ProductID>
      <UnitPrice>17</UnitPrice>
      <Quantity>12</Quantity>
      <Discount>0.2</Discount>
    </LineItem>
        Dim litLineItem2 = _
    <LineItem OrderID="10262">
      <ProductID>7</ProductID>
      <UnitPrice>24</UnitPrice>
      <Quantity>15</Quantity>
      <Discount>0</Discount>
    </LineItem>
        Dim litLineItem3 = _
    <LineItem OrderID="10262">
      <Quantity>2</Quantity>
      <ProductID>56</ProductID>
      <UnitPrice>30.4</UnitPrice>
      <Discount>0</Discount>
    </LineItem>
        '
        Dim litOrder = _
<SalesOrder OrderID="10262" EmployeeID="8"
  ShipVia="3">
  <%= litBillTo %>
  <OrderDate>1996-7-22</OrderDate>
  <RequiredDate>1996-8-19</RequiredDate>
  <ShipDate>1996-7-25</ShipDate>
  <Freight>48.29</Freight>
  <%= litShipTo %>
  <LineItems>
    <%= litLineItem1 %>
    <%= litLineItem2 %>
    <%= litLineItem3 %>
  </LineItems>
</SalesOrder>
  '
  'Add calculated items after </LineItems>
  Dim xeLineItems As XElement = _
    litOrder.Element("LineItems")
  xeLineItems.AddAfterThis(New _
    XElement("Summary", _
  New XElement("Items", _
    SalesOrderTotal(litOrder)), _
  New XElement("Freight", 48.29), _
  New XElement("OrderTotal", _
  SalesOrderTotal(litOrder) + 48.29)))
  '
  'Display the formatted document
  '(with fix for Lf -> CrLf)
  txtXML.Text = Replace(litOrder.ToString, _
    vbLf, vbCrLf)
  txtXML.Text = Replace(txtXML.Text, "><", _
    ">" + vbCrLf + "<")
    End Sub

The need for a fixup to convert vbLF (newline) characters to conventional vbCrLf (carriage return/line feed) pairs undoubtedly is an oversight in the VB 9.0 technology preview release.

The following figure shows the document composed by the following method, with assistance from the previously added SaleOrderTotal method.

Notice that indents are two characters because the individual element groups have two-character indents.

Update 10/11/2005: Microsoft MVP Don Demsak (a.k.a. DonXML) draws an analogy between VB XLinq literals and "classic ASP" with his XAML + XLinq + VB.Net's XML Literals Equals Classic ASP For WinForms? post.

--rj

Technorati: LINQ XLinq VB 9.0

Jon Udell: LINQ 101

InfoWorld's Jon Udell continues his investigation of LINQ and XLinq features with the C# implementation by demonstrating "a three-way join across an XML data source and two CLR objects." He concludes:

When I said that LINQ turns query inside out, I wasn't kidding. There is a lot here to think about and explore.

--rj Technorati: LINQ XLinq

More on Object-XML and Object-Relational Impedance Mismatch

In a very early (late 1950's) avatar, I was a studio and transmitter engineer for Berkeley's listener-sponsored FM station, KPFA and later majored in electrical engineering at UC Berkeley. Thus my initial reaction to the term "impedance mismatch" was from the audio and RF environments where mismatches cause serious problems, such as standing waves in antenna transmission lines. The advent of object-oriented programming has added a new meaning to the term. Wikipedia's Object-SQL Impedance Mismatch topic and Scott Ambler's The Object-Relational Impedance Mismatch essay provide an introduction to the usage in object/relational mapping (O/RM) scenarios. Dare Obasanjo's The Impedence Mismatch between W3C XML Schema and the CLR post discusses XML serialization of object state and conversion of XML to objects. The latter topic includes a reference to Programming with Circles, Triangles and Rectangles by Erik Meijer, et al. Following are additional links to other posts about impedance mismatch between objects, XML documents, and relational data from the CORBA, Java and LINQ perspective: Middleware Matters: Obsessed with the impedance mismatch—IONA Technologies' Steve Vinoski discusses mismatch issues between objects and XML, with a reference to COmega:

Basically, unless I'm real busy doing something else, my thoughts invariably turn to the impedance mismatch problem. I spend a fair bit of time searching for what others are doing to solve it, and I know about Comega and some of the domain-specific language ideas, but finding relevant work is hard because AFAIK there's no crisply defined category for such work.

Oracle Developer JAVA MONKEY: Advances in Java's Relationship to Data—Rene Bonvanie discusses "Understanding new and existing features of J2EE and EJB for working with data" and quantifies the programming effort to overcome impedance mismatch between objects and relational data:

With the ease of use associated with JDBC and SQLJ, however, comes what is frequently called impedance mismatch. Both cater directly to the relational database—working in rows and columns. Java, on the other hand, is naturally object-oriented and is designed to help developers think in classes and business objects. Additional work, estimated as high as up to 40 percent of the initial development effort, may be spent resolving this mismatch. [Emphasis added.]

Abstraction Impedance: Hidden Barriers and Explicit Boundaries—Dion Hinchcliffe (Sphere of Influence) qualifies impedance with "abstraction" and distinguishes "structural" and "behavioral" abstraction impedance:

The premise of the LINQ project, and many others like it, is the following: Now that the object-oriented paradigm has become established as the predominate model of abstraction in software development, developers have been incurring a large amount of overhead in mapping it to other abstractions, specifically XML and relational databases. ... Abstraction impedance isn't just a developer productivity tax. It leads to subtle and pernicious defects in software, contortions of design, and severely degraded performance. All of these lead to low quality software that doesn't deliver the expected value to the customer. ... So, I find the LINQ effort to be admirable and extremely interesting and I'm eager to see how effective it is. Solving this problem well may end up being one of the bigger advances in software development this decade.

Stay tuned for additional links about the LINQ Project, O/R or O/X mapping, and impedance mismatch.

--rj

Technorati: LINQ DLinq XLinq COmega

Delphi to Adopt LINQ Technology

eWeek's Darryl K. Taft writes in "Borland Plans to Support MS LINQ in Delphi Platform," published September 17, 2005:

Microsoft announced its Language Integrated Query Project earlier this month, and already third parties are talking about extending their technology to support the software giant's heralded new technology. In a Weblog post, Borland Software Corp.'s chief scientist Danny Thorpe talked about how Borland's Delphi programming environment can take advantage of LINQ (Language Integrated Query) to enable developers to more easily integrate and manipulate data in building Delphi-based applications.

ISV support for and comment on LINQ is growing. Danny's post relates primarily to LINQ's SQL-like syntax for querying data sources exposed by DLinq. (Note: Anders Hejlsberg held Danny's position at Borland before moving to Microsoft.) --rj Technorati: LINQ DLinq

CyrusN's C# 3.0 and LINQ Tutorials

Cyrus Najmabadi, a developer on Microsoft's C# team has started a series of tutorials on C# 3.0 and LINQ:

• So what's the deal with this whole C# 3.0 / LINQ thingy?—Background on the reasons for introducing LINQ and simple syntax examples.
• So what's the deal with this whole C# 3.0 / LINQ thingy? (Part 2)—LINQ query syntax and sequence.
• LINQ specifiqs - var—Implicitly Typed Local Variables (a.k.a. var).

These posts, which expand on earlier Microsoft white papers, are likely to be of equal interest to C# and VB programmers. I'll update the list as Cyrus adds new posts. --rj Technorati: LINQ

Comparing C# 3.0 and VB 9.0 LINQ Syntax

Prior to LINQ's introduction, differences between the C# and VB (.NET) languages were almost exclusively syntatical. The .NET Framework's classes and the CLR's MSIL unified the capabilities of programs written with C# and VB. If the release version of Visual Studio "Orcas" includes LINQ and its DLinq and XLinq APIs, much of the everyday data-management and XML-processing syntax of C# 3.0 and VB 9.0 source code will diverge dramatically. .NET developers will need to evaluate the language's query operator set and query effectiveness before deciding between C# 3.0 or VB 9.0 for writing data-intensive .NET 3.0 applications or components.

.NET developers usually base their choice of language on their programming background. As a general rule C/C++/Java programmers, who are likely to have learned these languages while attending a college or university, choose C#. VB (COM) developers, most of whom learned the language by reading books and attending conferences or training classes, adopt VB .NET. The VS team has identified the following three Windows programmer personae:

• Mort, the opportunistic developer, likes to create quick-working solutions for immediate problems, focuses on productivity, and learns new language skills as needed. Morts commonly create ad hoc, line-of-business applications and components with VB.
• Elvis, the pragmatic programmer, likes to create long-lasting solutions that address the problem domain, and learns while working on the solution. Elvises commonly use C# to develop reusable components and enjoy implementing advanced .NET features, such as remoting and the Windows Communication Framework (WCF or Indigo).
• Einstein, the paranoid programmer, likes to create the most efficient solution to a given problem, and typically learn in advance before working on the solution. Einsteins usually have a C/C++ background and a master's degree or higher, and some write Windows device drivers.

The preceding descriptions were adapted from Nikhil Kothari's blog entry. Wesner Moise offers similar descriptions for the three personae, and Rockford Lhotka provides a detailed analysis of Mort as a business programmer.

Microsoft's VB and C# teams diverge on LINQ syntax. VB 9.0 query comprehensions support SQL-like operators, while C# keywords have a decided XQuery FLWOR (for, let, where, order by, return) flavor. (Wes's "LINQ Syntax" item offers a sample FLWOR expression and its C# query counterpart.)

Compare this preliminary list of VB 9.0 query keywords from Paul Vick's PDC 2005 presentation:

• Project: Select <expr> From <expr> In <expr>
• Filter: Where <expr>
• Test: Any(<expr>), All(<expr>)
• Join: <expr> [[Left Right] Outer] Join <expr> On <expr>
• Group: Group By <expr>
• Aggregate: Count(<expr>), Sum(<expr>), Min(<expr>), Max(<expr>), Avg(<expr>)
• Partition: Top(<expr>);
• Set: Distinct, Union, Intersect, Except
• Order: Order By <expr> [[Asc] Desc]

with this much longer list of C# 3.0 query operators from the LINQ Project overview document, which is missing the From and In operators:

• OfType: Filter based on type affiliation
• Select/SelectMany: Project based on transform function
• Where: Filter based on predicate function
• Count: Count based on optional predicate function
• All/Any: Universal/Existential quantification based on predicate function
• First/FirstOrDefault: Access initial member based on optional predicate function
• ElementAt: Access member at specified position
• Take/Skip: Access members before/after specified position (paging)
• TakeWhile/SkipUntil: Access members before/after predicate function is satisfied
• GroupBy: Partition based on key extraction function
• ToDictionary: Create key/value dictionary based on key extraction function
• OrderBy/ThenBy: Sort in ascending order based on key extraction function and optional comparison function
• OrderByDescending/ThenByDescending: Sort in descending order based on key extraction function and optional comparison function
• Reverse: Reverse the order of a sequence
• Fold: Aggregate value over multiple values based on aggregation function
• Min/Max/Sum/Average: Numeric aggregation functions
• Distinct: Filter duplicate members
• Except: Filter elements that are members of specified set
• Intersect: Filter elements that are not members of specified set
• Union: Combine distinct members from two sets
• Concat: Concatenate the values of two sequences
• ToArray/ToList: Buffer results of query in array or List
• Range: Create a sequence of numbers in a range
• Repeat: Create a sequence of multiple copies of a given value

In addition to the query comprehension operator (keyword) differences between VB and C#, C# reverses the sequence of SQL's SELECT and FROM clauses. The following VB expression:

expr = Select s.ToUpper() From s In names Where s.Length = 5 Order By s

translates to this C# expression:

expr = from s in names where s.Length == 5 orderby s select s.ToUpper() ;

Paul Vick addresses the query comprehension operator sequence issue from the VB Team's standpoint—following the SQL convention simplifies usage because VB programmers (Morts) are likely to be conversant with SQL. However, substitution of In for As departs from SQL convention. I would have no problem with either

expr = Select s.ToUpper() From names As s Where s.Length = 5 Order By s

or

expr = From s In names Where s.Length = 5 Order By s Select s.ToUpper()

but I'd prefer the latter construction if it improved performance, reduced resource consumption, or is the only sequence that would support IntelliSense fully. Implementing IntelliSense for the previous example's Select list appears to me to be impossible in advance of defining the data source with the From ... In clause.

Wesner Moise makes the following observations in his VB 9 item of September 16, 2005:

The difference in focus between the C# and VB team is becoming much more evident: The VB team is opting for greater relevance and convenience in the short term, while the C# team is trying to avoid creating long-term baggage. The end result is that Visual Basic is definitely going to be much more fun to program. [Emphasis added.]

While the C# [team] has decided to include a few choice SQL keywords (select, from, where, orderby and groupby) and no XML literals, the VB team has opted to include the full range of keywords available in a SQL query as well as support for embedded XML with schema validation.

Paul Vick confirms Wes's conclusion from the XLinq perspective in his "Integrating XML: Playing with Fire?" item. However, DLinq users undoubtedly will want to see Having, the In() predicate and other commonly-used ANSI SQL reserved words added to VB's query comprehension operators list.

The upshot: VB 9.0 will be firmly in the Mort camp and C# 3.0 will continue to appeal primarily to the Elvis and Einstein crowd. Whether the two sets of query comprehension operators will have identical capabilities with both the DLinq and XLinq APIs remains to be seen.

--rj

Update: 10/11/2005: Interact Software, Ltd's Ian Griffiths explains the significance of and demonstrates C# 3.0 expression trees in his "C# 3.0 and LINQ - Expression Trees" post.

Technorati: LINQ XLinq VB 9.0 C# 3.0

Amanda Silver, Paul Vick, and Erik Meijer VB 2005 Interview

Carl Franklin interviews the VBTeam's Amanda Silver and Paul Vick, plus the SQL Server group's Erik Meijer (a.k.a., VBTV's "Head in a Box") for the second half of the September 26, 2005 .NET Rocks show. The interview doesn't deal with LINQ in depth—consider it a sales pitch for VB 2005 with a low-level introduction to forthcoming LINQ features. The interview contains only about 10 minutes of LINQ-related conversation. Following is a log for the VB-related topics of the full WMV file: 00:24:27 - New VB 2005 language features 00:32:20 - LINQ introduction by Amanada and Paul 00:39:20 - Commercial break 00:41:45 - More LINQ and DLinq features 00:44:45 - Erik Meijer (on XLinq) 00:49:20 - More VB 2005 language features 01:04:11 - New features deferred to VB 9.0 from VB 8.0 (400 to 500 in the queue) 01:09:00 - Bill Gates' influence on VB 2005 and successors 01:11:00 - VB generics 01:14:40 - VB Express, programmer education, and hobbyist programmers 01:17:25 - Links to resources 01:20:00 - Coaching Anders Hejlsberg for a VB demonstration 01:24:00 - End Carl recorded the interview on September 14, 2005 at PDC 2005. --rj Technorati: LINQ DLinq XLinq

XLinq Presentation at the XML 2005 Conference

Erik Meijer will present "XLinq: XML Programming Refactored (The Return of the Monoids)" at Atlanta's XML 2005 Conference on Wednesday, November 16, 2005. Note that the conference program lists the presentation title as "XMP [sic] Programming Refactored (The Return of the Monoids)" because the XLinq name was under NDA until PDC 2005; "XMP" is an obvious typo. The paper, by Erik and Brian Beckman*, provides a concise description of the objectives and implementation of the XLinq API for LINQ. XLinq replaces the document-oriented XML DOM with an element-oriented API that enabled constructing elements and attributes independently of a containing document. The paper is unique because all but one of its examples use VB 9.0 rather than C# 3.0. VB 9.0 is required to implement XML literals and axis members. XML literals enable definining elements and attribute with VB 9.0 code similar to the following:

Dim BillTo =
<billTo country="US" >
  <name>Robert Smith</name>
  <street>8 Oak Avenue</street>
  <city>Old Town</city>
  <state>PA</state>
  <zip>95819</zip>
</billTo>

Notice the lack of VB's line-continuation characters, which aren't required or permitted in XML-literal statements. Whitespace formatting is preserved and XLinq overloads the ToString method to simplify display of formatted documents. (Oleg Tkachenko has some reservations about functional construction of XML trees and VB literal XML.) Axis members substitute dot syntax for XPath axis specifiers. For example, BillTo.street returns all street child elements from the BillTo XElement object. Read Erik's paper to learn more about VB 9.0's XLinq implementation. --rj Technorati: LINQ XLinq * Brian Beckman was one of the original developers of Microsoft's XLANG—a notation for the specification of message exchange behavior among participating web services—and is the author of the "Physics of Racing" series.

Jon Udell: Microsoft driving toward .Net unity

The InfoWorld Test Center's lead analyst, columnist, and blogger-in-chief says: "LINQ is an extraordinary innovation that turns traditional query inside out. But the bigger story from PDC 2005 is that the .Net vision of unifying many balkanized disciplines within the Microsoft ecosystem is finally becoming a reality." Read Jon's September 21, 2005 "Strategic Developer" column. Not to be outdone by his colleague, Tom Yeager—InfoWorld Test Center's chief technologist—devotes this week's "Ahead of the Curve" column to LINQ. --rj Technorati: LINQ

LINQ Interview with Anders Hejlsberg and Paul Vick

InfoWorld columnists Jon Udell and Tom Yeager interviewed Anders Hejlsberg and Paul Vick at PDC 2005. Here's a link to the 27:41 Podcast, "A conversation with with Anders Hejlsberg and Paul Vick about LINQ," which also includes transcripts of selected comments by Anders. Jon's item also includes a link to "The LINQ Project: .NET Language Integrated Query" MSDN white paper by Anders Hejlsberg and Don Box. The white paper emphasizes C# 3.0, but includes a few VB 9.0 examples. Paul Vick at 13:40 says, "One of the great advantages [of VB 9.0 is that] you learn one language. It can be C# or Visual Basic, but you learn one language; you don't have to go out and learn SQL. A lot of our developers do know SQL because they've been forced to, but now you don't actually have to do that." Microsoft initially hyped CLR integration with SQL Server 2005 as a means to eliminate Visual Studio developers' need to be proficient in [T-]SQL, as mentioned in my recent "When To Use SQLCLR, And When Not To" article. Substituting a knowledge of VB 9.0 query comprehensions for [T-]SQL proficiency is equally impractical in the real world. No matter what programming language or platform you choose, a working knowledge of SQL is essential for developers of data-intensive applications or components. --rj Technorati: LINQ

Cure Data Type "Impedance Mismatch" With LINQ

This article from Fawcette Technical Publications' .NETInsight newsletter describes how to install and test the C# 2.0 and VB 8.0 implementations of the LINQ Project, DLink and XLink. Although this post from the VBTeam's blog states that the initial PDC 2005 Technology Preview (TP) bits work with the current VS 2005 release candidate, which is available for download from MSDN, I've found the following issues on various VS 2005 versions:

• Neither the VB or C# preview run correctly on the Beta 2 version (build 8.0.50215.44) mentioned as required in the Readme document for the C# version. This build is running here under Windows 2003 Server SP-1.
• The SQLMetal.exe command-line utility to generate C# entity classes for SQL Server databases and DLinq throws an unhandled exception—Invalid object name 'Dbo.customers'—when run under Beta 2 (build 8.0.50215.44).
• The C# preview doesn't add LINQ templates or integrate correctly under the VS 2005 RC IDE after running Install Visual Studio IDE Support.cmd. Attempts to edit any C# LINQ source code result in an instantaneous VS 2005 shutdown and restart. This build is also running here under Windows 2003 Server SP-1.
• The only VS build that appears to me to work correctly with the Technology Preview is Beta 2 build 8.0.50727.7 (a.k.a., the July CTP), which I'm running under Windows XP SP-2.
• The C# SampleQueries project throws 60 compilation errors, regardless of the VS 2005 build under which it runs.

I've also determined that DLinq support for VB projects in the initial Technology Preview is non-existent; installing the VB TP bits doesn't install DLinq.dll to provide the System.Data.DLinq classes. My article quotes the VBTeam's Amanda Silver: "For example, we've been working on XML integration pretty fiercely, meanwhile, we're [sic] don't have support in this initial version for DLinq." You can, however, emulate some VB DLinq features by executing simple LINQ Select ... From ... In ... Where ... Order By queries against in-memory instances of hand-coded entity classes that you populate with the ADO.NET 2.0 SqlDataReader.GetObject(objRow) method. --rj Technorati: LINQ Update: Here's a lightly edited question and answer regarding future C# 3.0 LINQ compatibility with the VS 2005 RC and RTM versions from the September 22, 2005 C# 3.0 chat: Q: May we expect a release of LINQ that works with the RC and/or RTM of .NET 2.0 in the near term? A: Yes. We are very actively working on it right now and would like to get it out as soon as we can.

Michael Rys' Personal Take on XLinq and XQuery

Michael Rys has posted his view of the relationship between XLinq's XPath-based query language and XQuery here. He's also posted links to 11 new XQuery Working Drafts dated September 15, 2005. Nine of the 11 documents are intended to evoke response to the Last Call comments. Candidate Recommendations might arrive the the near future (at last!) --rj Techorati: LINQ XQuery

Erik Meijer Selects VB as His "Language of Choice"

Erik Meijer recently posted a "Visual Basic and LINQ" guest entry to the Lambda the Ultimate programming-languages Weblog in which he stated that Visual Basic "has become his programming language of choice." This is no faint praise from a "language wonk" who's one of the major contributors to the LINQ Project and the Haskell99 functional programming language. Here's Erik's brief description of what VB 9.0 will offer to LINQ programmers that isn't likely to be available with C# 3.0:

Visual Basic adds some further enhancements to leverage the fact that it allows static typing where possible and dynamic typing where necessary in the form of relaxed delegates, improved nullable support, dynamic identifiers (makes writing meta-circular interpreters a breeze) and last but not least dynamic interfaces, or as I like to refer to them [as] strong duck typing (compare to simplified qualified types/type classes).

These enhancements enable VB 9.0 to support "XML Literals" and "Late-Bound XML." Watch for code examples in future OakLeaf blog items. Dare Obasanjo also weighs in on XLinq and Visual Basic 9 with a quote from the Microsoft XML Team's Mike Champion about Erik's contributions to X#, Xen, and COmega (Cω). On the software patent front, Erik is the primary inventor of two pre-grant publications—both dated August 18, 2005—of U.S. patent applications assigned to Microsoft Corp. that relate to Cω:

• 20050183071 Interaction with nested and non-nested streams
• 20050183058 Fast flattening of nested streams

Cω returns data rows from SQL-style queries as flattened streams, which “represent ordered homogeneous collections of zero or more values.” Streams are related to the .NET Framework 2.0's generic IEnumerable<T> interface. Each row of the stream is an anonymous struct (a tuple), which is defined by the Cω enhancements to C# 1.1. If a row is hierarchical (i.e., contains other rows), the runtime flattens the stream as its read from the data source, which can be a database or XML document.

--rj

Technorati: LINQ COmega

Update 9/19/2005: According to Microsoft's Matt Warren, Erik won Universal Studio’s Fear Factor—Live! award, which is no small achievement.

Update 9/28/2005: Erik presented "Why VB is my Favorite Language" at the JAOO 2005 Aarhus (Denmark) conferrence on September 27. Here's the abstract:

Some people have compared type systems with a dial; however in most languages that dial is either completely off (dynamically typed languages) or completely on (statically typed languages). Visual Basic is unique in that it allows the dial to cover the whole spectrum; it supports static typing where possible and allows dynamic typing where necessary. In this talk we will show some of the present and future support of mixed static and dynamic typing in Visual Basic.

--rj

Bill Gates Podcast on the LINQ Project

Infoworld's Jon Udell recorded a half-hour podcast interview with Bill Gates on September 13, 2005. The interview includes a serveral-minute discussion on the LINQ Project that starts at 3:33. You can read the transcript of the interview here. --rj

Technorati: LINQ

Erik Meijer on LINQ as a 6GL Language Feature

Erik Meijer is one of the primary contributors to Microsoft Research's Cω (C-Omega) language and the forthcoming C# 3.0 and VB 9.0 versions for .NET 3.0 in "Orcas". He's a co-author of several papers about extensions to a general-purpose programming language (C#) that reduce the "impedance mismatch" of relational-to-object and XML-to-object mapping and traditional query languates, such as SQL, XSLT, and XQuery. Here are links to four Microsoft Research papers that will interest developers of data-intensive .NET applications:

• "Unifying Objects, Tables and Documents” by Erik Meijer, Wolfram Schulte and Gavin Bierman
• "Programming with Rectangles, Triangles, and Circles" by Erik Meijer, Wolfram Schulte and Gavin Bierman presented at the XML 2003 Conference
• "The Essence of Data Access in Cω" by Erik Meijer, Wolfram Schulte and Gavin Bierman (rejected by the ACM's Symposium on Priniciples of Programming Languages - POPL 2005)
• "XML Types for C#" by Erik Meijer and Wolfram Schulte (the original Bill Gates ThinkWeek presentation on what ultimately became Cω.)

You can also watch (and listen to) Erik's entertaining 1H:22M-long "Data Integration in Programming Languages" Producer for PowerPoint 2002 presentation and check out Erik's c.v. on his Microsoft Research home page. Erik will present three papers that relate to Cω, C# and VB topics at the September 2005 JAOO Conference in Aarhus, Denmark. JAOO claims to be "the Premier European Conference on Software Engineering, Methods and Best Practices." Here's Erik's latest bio from the conference's site:

Erik Meijer is an architect in the Microsoft SQL [S]erver division where he currently works together with the Microsoft Visual C# and the Microsoft Visual Basic language design teams on data integration in programming languages. Prior to joining Microsoft he was an associate professor at Utrecht University and adjunct professor at the Oregon Graduate Institute. Erik is one of the designers of the standard functional programming language Haskell98 and more recently the C-Omega language.

Following is the abstract of Erik's "6GL features languages to come" JAOO 2005 presentation. The JAOO 2005 folks' definition of 6th Generation Languages is: "6GLs seek to go beyond scripting to full blown application languages which are easier to use and more productive than Java/C# languages or generators."

Moving forward, the dominant costs for IT projects are people costs more than hardware costs. Meeting business needs revolves around improving developer productivity. This provides great opportunities for language designers and compiler writers to concentrate on providing more powerful high-level programming abstractions rather than than on low-level optimizations. This desire for more abstraction has lead to a plethora of new domain specific programming languages for manipulating data including XML with XSLT and XQuery, objects with OQL, and relational data with SQL. We argue that instead of creating such specialized languages, general purpose languages such as C# and Visual Basic should be extended with query capabilities. This allows programmers to query any form of data, using a broad notion of "collection".

We will discuss various advancements in Visual Basic towards simplifying the development of data intensive applications. In particular we will concentrate on ways to bridge the impedance mismatches between objects, relation data, and XML and the importance of dynamism in this trend. [Emphasis added.]

It's especially interesting that Erik will emphasize VB extensions in this presentation, because all his preceding papers have been devoted to the C# extensions added by Cω. However, VB certainly qualifies as a "full blown application [language] which [is] easier to use and more productive than Java/C# languages or generators." [Emphasis added.]

It surprises me that Anders Hejlsberg receives all—or at least the majority—of the credit for LINQ development. As Wesner Moise notes in his September 14, 2005 LINQ post,"Visual Basic seems to have introduced more extensive changes to the language than C#. Many of the COmega-inspired features seem to have migrated into Visual Basic instead of C#; one wonders whether this was due in no small part to Erik Meijer’s close relationship to the VB team." --rj

P.S.: Channel9 has a 12-minute video of Gavin Bierman, Erik's co-author on several papers, discussing data query languages in Cω. Gavin is a member of the Programming Principals and Tools Group of Microsoft Research in Cambridge, U.K. Gavin has prepared a MSDN Webcast about Cω that also describes other research projects going on at MSR Cambridge.

--rj

Technorati: LINQ COmega

WinFS and the Language Integrated Query (LINQ) Project

Microsoft's pre-PDC release to MSDN subscribers of WinFS Beta 1 bits was a surprise to the trade press, as well as most .NET developers. InfoWorld's "Surprise! Microsoft's WinFS beta arrives" article by Eric Knorr appeared on August 31, 2005 and briefly describes the first WinFS tools for developers. Paul Thurrott's "Windows Storage Foundation (WinFS) Preview" provides a historical context for WinFS—begining with the Object File System (OFS, codenamed Cairo) that Microsoft once promised for Windows NT. Quentin Clark, Microsoft's director of program management for WinFS, briefly describes reasons for early release of the WinFS Beta and the relationship between WinFS and IQF in his initial post—Shipping WinFS Beta 1—on the WinFS team's blog. Here's the early version of the WinFS/LINQ story from Quentin's post:

Another piece of feedback was concerns over the WinFS api’s being a different data access pattern than our existing managed code data acess APIs. Further, that our APIs were not aligned with broader data platform needs like OR mapping. This was a big one, and as you explore the SDK and the APIs, you will see the beginnings of how we will be addressing this. We are in the process of building-out the next version of ADO.NET to have new features that provide a data model, object-relational mapping, and flexible query infrastructure. The new data model is about entities, and the WinFS data model of Item types is built on that model. Looking through our SDK and code samples you will see how Items are composed of underlying entities. OR mapping is a big requirement - WinFS is a very prescribed mapping (defining a type in WinFS generates both the underlying storage schema and the partial class to program to that type). But the real-world has lots of requirements for flexible mapping – to existing data, to existing objects, etc. On query, many of you have heard about Anders Hejlsberg’s work on Language Integrated Query – and the new ADO.NET functionality will plug directly underneath so that you can use the new query patterns on any entity data, including of course now WinFS Items.

Thus, it's likely that you'll be able to integrate WinFS and LINQ code in the .NET projects you create with Visual Studio "Orcas." PDC 2005 has four WinFS breakout sessions. Here's the description of DAT312—"WinFS" and ADO.NET: Future Directions for Data Access Scenarios (Samuel Druker)—that contains references to ObjectSpaces and "integrated query functionality:"

The next generation of ADO.NET incorporates innovations that integrate Entities, Relationships, Associations, a seamless CLR object programming surface, and flexible storage mapping for programming to data at all tiers. These technologies address the needs for "WinFS" data modeling and programming scenarios on the desktop and server, as well as the requirements from the ObjectSpaces project. This session discusses how ADO.NET entities will form the core for data integration, enabling the interoperability of applications and modules by providing a high-level framework for programming with data that includes integrated query functionality.

Following are links to the other three WinFS-specific sessions—DAT200 Future Directions for Data-Driven Applications: Storage, Applications, API’s (David Campbell); DAT209 "WinFS" Future Directions: An Overview (Shishir Mehrotra); and DAT310 "WinFS" Future Directions: Building Data Centric Applications Using Windows Presentation Foundation ("Avalon") and Windows Forms (Ramesh Nagarajan). Robert Scoble's one-hour Channel 9 video with members of the WinFS team includes a demonstration of some WinFS features and applications. The demo starts at about 11:00:00 into the program. As to the ultimate release of WinFS, Paul Thurrott bases the following timeline on "internal Microsoft documentation:"

WinFS Beta 1 will be followed by at least one Community Technology Preview (CTP) release, which is currently due on February 15, 2006. Then, on May 1, 2006, Microsoft is scheduled to release WinFS Beta 2. Beta 3 is currently scheduled for November 15, 2006, with a Beta 3 Refresh release expected in April 2007. WinFS is currently scheduled for RTM in Q3 2007, well after Longhorn Server is released.

--rj Technorati: LINQ, COmega, ADO.NET vNext, WinFS

The Language Integrated Query (LINQ) Project

Two features once promised for Visual Studio 2005 are conspicuous by their absence in VS 2005 Beta 2 Community Technical Preview drops:

• An object-relational mapping API and toolkit for ADO.NET 2.0 called ObjectSpaces
• An XQuery 1.0 implementation for the .NET Framework 2.0

Searching descriptions of PDC 2005 sessions for the keyword "query" indicates that the forthcoming Languate Integrated Query (LINQ) for .NET Project (nee Integrated Query Framework (IQF)) will resurrect in the VS Orcas release at least some of the promised features of the original .NET 2.0 implementations of ObjectSpaces and the XQuery 1.0 processor. (Copies of session descriptions are at the end of this post.) Recent MSDN articles and Microsoft patent applications add evidence of major changes in store for VB 9.0 and C# 3.0 in the next VS release.

Note: Technorati category links in the preceding text are experimental

ObjectSpaces Support in VS 2005 Beta 1 Orca (apparently an acronym for Object-Relational Component Architecture) was a technical preview of ObjectSpaces that first appeared in a PDC 2001 ADO.NET presentation and was promised as a Web release for VS .NET 1.0. ObjectSpaces introduced OPath, a proprietary object query language. I spent a substantial amount of time testing the PDC's technical preview bits. I don't believe there were subsequent Web release(s). Luca Bolognese's "DAT410 NET Framework: Developing Applications Using ADO.Net ObjectSpaces" presentation at PDC 2003 set the stage for ObjectSpaces' inclusion in the VS Whidbey release. Luca demonstrated a VS add-in tool to simplify the O-R mapping process, which required a set of three XML configuration (schema description) files for the object (Osd.xml), relational (Rsd.xml), and mapping (Msd.xml) layers.

Luca promised future support for generics with syntax similar to: Dim osQuery As New _ ObjectQuery(Of Customer)("Region='WA'","Orders") Dim osReader As ObjectReader(Of Customer) = _ ObjectSpace.GetObjectReader(osQuery). The final bulletpoint on Luca's "Conclusions" slide was: "It is included in the next version of Visual Studio." Peter Provost blogged the presentation and Paul Wilson's "ADO.NET v2.0: ObjectSpaces Delivers an O/R Mapper" article has sample code from the PDC alpha version. Dino Esposito's February 2004 MSDN article, "A First Look at ObjectSpaces in Visual Studio 2005," provides additional architectural details and sample code. The Whidbey alpha included a System.Data.ObjectSpaces namespace. Luca reappeared at Tech*Ed 2004 with the "DAT 301: ObjectSpaces" presentation, which included more advanced examples of OPath query syntax. Microsoft originally planned to use XPath as the query language but found that virgules (/) turned off developers, who preferred periods (.) as separators. Thus typical OPath query expressions against Northwind-based Orders and Details objects might have been: Orders.Details.Quantity > 50

Orders[ShippedDate > RequiredDate]

Orders[Freight > 1000].Details[Quantity > 30]

At the end of the presentation, Luca announced that ObjectSpaces had acquired a dependency on WinFS and thus was postponed to the Longhorn/Orcas timeframe. (He appeared close to tears as he made the announcement.) The dependence of the long-delayed Microsoft Business Framework (MBF) API on ObjectSpaces as its object persistence platform probably was a contributing factor to the demise of ObjectSpaces in VS 2005. Subsequently, Microsoft announced that WinFS had been excised from the planned Longhorn release version, now renamed Vista.

Here are three published Microsoft patent applications that include Luca Bolognese as an inventor and relate to Object Spaces:

• Mapping architecture for arbitrary data models (20050050068)
• System and method for presenting a query expressed in terms of an object model (20040078359)
• System and method providing API interface between XML and SQL while interacting with a managed object environment (20030236859)

Matt Warren's "ObjectSpaces:There and Back Again" and Dinesh Kulkarni's "ObjectSpaces -> DLinq" blog entries throw additional light on ObjectSpace's transition from mapping files to LINQ and DLinq.

XQuery 1.0 Support in .NET Framework 2.0 Beta 1

About three years ago, Microsoft sponsored an XQuery demonstration site that included an XQuery engine (Microsoft.Xml.XQuery) for .NET 1.0. This early XQuery implementation didn't support all XQuery keywords, constructs, or use cases, but it provided a simple route for .NET developers to become familiar with XQuery 1.0 and XPath 2.0 syntax.

Beta 1 of the .NET Framework 2.0 included a System.Xml.Query namespace, which provided an XQueryCommand class (the XQuery processor) to implement a subset of XQuery 1.0 for XML documents. Mark Fussell's March 2004 MSDN article, "What's New in System.Xml for Visual Studio 2005 and the .NET Framework 2.0 Release," has a section named "The XQuery Language," which includes a simple C# example of an XQuery FLWOR (for, let, where, order by, and return) expression:

using (XmlWriter writer = XmlWriter.Create("output.xml ")) { XQueryCommand xq = new XQueryCommand(); string query = "" + "{ for $s in /bookstore/book " + "where $s/@genre='autobiography' " + "return $s/title }" + ""; xq.Compile(query); xq.Execute("books.xml", new XmlUrlResolver(), writer); }

Notice that the XQuery processor compiles the FLWOR query and then executes the compiled version against the books.xml file to write the output.xml document. Output.xml contains the title subelements of book elements whose genre attribute value is autobiography. (After XQuery appeared in the Whidbey bits, Microsoft turned off the XQuery demo site.)

Mark's article also included a section titled "Universal Type Support and Conversion," which described changes to the the XmlReader, XmlWriter, and XPathNavigator classes required to support the XQuery 1.0 and XPath 2.0 Data Model with XML schemas and CLR types. Dare Obasanjo's January 2005 MSDN article, "An Overview of Cω: Integrating XML into Popular Programming Languages," describes constructs in Microsoft Research's Cω (C-Omega) programming language that "...bridge the gap between Relational, Object, and XML data access by creating a type system that is a combination of all three data models." Although Dare's article discusses Cω features in a C# context, it's reasonable to expect that VB in Orcas will gain similar type system additions and programming constructs.

Whidbey Beta 2 removed the XQueryCommand class for the reasons cited in Microsoft's brief "Status of XQuery in the .NET Framework 2.0" white paper. The Microsoft XML Team's Mike Champion and Arpan Desai weigh in on what appears to have been a traumatic decision. Michael Rys, program manager for SQL Server’s XML Technologies, explains why SQL Server's xml data type uses XQuery 1.0 despite the XML team's arguments against it's inclusion in the .NET Framework 2.0.

The XML Team's white paper suggests that VS 2005 "... customers continue to use XSLT and XPath to address their XML filtering and transformation scenarios." The System.Xml.Xsl.XslCompiledTransform object represents an XSLT processor that supports XSLT 1.0 syntax. Following is C# sample code from VS 2005's online help for XslCompiledTransform:

// Create the XslCompiledTransform object. XslCompiledTransform xslt = new XslCompiledTransform(); // Compile the style sheet. xslt.Load("output.xsl"); // Execute the transform and output the results to a file. xslt.Transform("books.xml", "books.html");

The XslCompiledTransform.Load method, which compiles the specified stylesheet, corresponds to the XQueryCommand.Compile method and the XslCompiledTransform.Transform method approximates the XQueryCommand.Execute method.

References to LINQ and XLinq Predecessors

Arpan Desai, program manager for XSLT and XQuery in Whidbey, says "...we came up with a new query architecture to support both XSLT and XQuery." Michael Brundage, author of XQuery: The XML Query Language, is an inventor of the Common Query Runtime (CQR). CQR "...features compilers for each XML query language (such as XSLT and XQuery), a common intermediate format (QIL [Query Intermediate Language]), and multiple backend execution engines (over XML and SQL).

Following are Microsoft's U.S. and international patent applications related to IQF (from Michael Brundage, all filed in 2003):

• 304064.1 Common Query Runtime System and API (20040260691)
• 303845.1 Query Optimizer System and Method (20050004892)
• 301638.1 Query Intermediate Language Method and System (20040267760)

Numbers in parenthesis are U.S. Patent Office published application document numbers with links to the applications' full text and drawings. The Patent Office published the applications in 2004 and 2005. The patent applications reference XQuery and OPath as XML query languages and provide examples of relational queries using the Northwind database's tables.

The ".NET Language" prefix indicates to me (and others, such as Wesner Moise) that implementations will be specific to VB or C#. As an example, VB and C# appear to share anonymous types, but only Paul Vick's PDC 2005 session mentions query comprehensions and object initializers.

Soumitra Sengupta's "Changing the Game" entry to the XML Team's Weblog provides additional insight on the effect of LINQ and XLinq on the future of Microsoft's XML technologies.

PDC 2005 Sessions on LINQ and Related Topics

Links to individual sessions at http://commnet.microsoftpdc.com/content/sessions.aspx don't appear to be possible, so following are copied descriptions of LINQ-related sessions at PDC 2005. InfoWorld published on August 25, 2005 an article by Paul Krill that refers to the LINQ sessions. A similar article by Darryl K. Taft in the August 26 issue of eWeek.com describes a purported FoxPro-LINQ connection.

The .NET Language Integrated Query Framework: An Overview Speaker(s): Anders Hejlsberg Modern applications operate on data in several different forms: Relational tables, XML documents, and in-memory objects. Each of these domains can have profound differences in semantics, data types, and capabilities, and much of the complexity in today's applications is the result of these mismatches. The future "Orcas" release of Visual Studio aims to unify the programming models through integrated query capabilities in C# and Visual Basic, a strongly typed data access framework, and an innovative API for manipulating and querying XML. This session introduces each of these areas and walks through how they are related.

Using the .NET Language Integrated Query Framework with Relational Data Speaker(s): Luca Bolognese Database-centric applications have traditionally had to rely on two distinct programming languages: one for the database and one for the application. This session introduces future advances Microsoft is making for the "Orcas" release of Visual Studio in programming languages and frameworks to help integrate relational data and queries with C# and Visual Basic. These advances enable developers to express queries and updates in terms of their local programming language without sacrificing the server-side execution model of today's high-performance SQL-based approaches. Using these advances, database queries that previously were stored as opaque strings now benefit from static type checking, CLR metadata, design-time type inference, and of course IntelliSense.

Using the .NET Language Integrated Query Framework with XML Data Speaker(s): Dave Remy One of the key challenges to working with XML data has been the impedance mismatch between XML and programming languages. This session introduces future advances Microsoft is making for the "Orcas" release of Visual Studio in programming languages and frameworks to help integrate XML and queries with C# and Visual Basic. The advances include a framework for navigating, querying, and transforming XML that is both easier to use and more efficient than current XML programming techniques. This framework marries the capabilities of XPath, XQuery, and the DOM with the language integrated query framework planned for C# and Visual Basic.

Visual Basic: Future Directions in Language Innovation Speaker(s): Amanda Silver, Paul Vick Visual Basic 9.0 will offer radical improvements in its ability to work with data in all its forms: as objects, as XML, as relational data. Join the language architects for a detailed discussion of features such as query comprehensions, object initializers and anonymous types that enable querying data in a more flexible, natural way than ever before. Also, get a glimpse into the future of dynamic programming in VB with coverage of new features intended to radically simplify working with dynamically typed data on the .NET platform.

C#: Future Directions in Language Innovation from Anders Hejlsberg Speaker(s): Anders Hejlsberg Join Anders Hejlsberg, Distinguished Engineer and chief architect of the C# language, for an in-depth walkthrough of the new language features in C# 3.0. Understand how features like extension methods, lambda expressions, type inference, and anonymous types make it possible to create powerful APIs for expressing queries and interacting with objects, XML, and databases in a strongly typed, natural way. It is suggested that you attend "The .NET Language Integrated Query Framework: An Overview" before attending this session.

--rj

Technorati tags: LINQ, ADO.NET vNext, ObjectSpaces