Wednesday, June 06, 2007

Tech●Ed 2007 Coverage from FTPOnline

Redmond Media Group's FTPOnline columnists (including me) are covering Tech●Ed 2007—the 15th Anniversary of Microsoft's yearly technical conference—for the .NET Insight newsletter's "Tech●Ed Show Daily" issues. (Updated June 8, 2007.)


From the June 5, 2007 .NET Insight "Tech●Ed Show Daily" issue:

Muglia Maps Microsoft's 'Journey Towards Dynamic IT'

The traditional "vision" keynote gives way to optimization, virtualization and standardization of existing IT infrastructure along with new agile business processes and modeling to reverse the 70/30 ratio of IT maintenance to new solution spending at Tech●Ed's 15th anniversary.

Hot at Tech●Ed 2007: SQL Server 2008

The next version of SQL Server will deliver native data types for documents, filestreams and geocoding, provide policy-based management, support the Entity Data Model (EDM) and Language Integrated Query (LINQ), and integrate with Microsoft's enhanced business intelligence (BI) offerings.

Click here for a table of Tech●Ed sessions about SQL Server 2008 and implementing business intelligence projects.


From the June 7, 2007 .NET Insight "Tech●Ed Show Daily" issue:

Hot at Tech●Ed 2007: Introducing the Entity Framework and LINQ

The Entity Framework (EF) and Language Integrated Query (LINQ) are two new technologies in Visual Studio 2008 that form one of the Pillars of SQL Server 2008 (formerly "Katmai"): Dynamic Development. Both play nicely with SQL Server 2005 and 2000, too.

Click here for a table of Tech●Ed sessions about the Entity Framework and LINQ.

Exploit SQL Server Compact Edition and Occasionally Connected Systems

SQL Server Compact Edition v3.1 and v3.5's free license covers deployment on Windows PC clients as well as devices, so you can take advantage of this lightweight, in-process relational database as a local data cache and automatically synchronize its data with a back-end SQL Server 200x instance.

Click here for a table of Tech●Ed sessions about SQL Server Compact Edition, data synchronization, and occasionally connected systems.

Video: Preview SQL Server 2008's Database Engine (01:04:00)

Christian Kleinerman, group program manager of Microsoft's Relational Engine Team, offers a first look at SQL Server 2008, formerly code named "Katmai." Specifically, his presentation at VS Live! Orlando delivers a high-level overview of enhancements to the relational database engine. (Taped May 8, 2007.)


Note: Visual Studio Magazine has become a controlled-circulation publication. Sign up now for your free subscription.

Click here to sign up for any or all of Redmond Media Group's Insight Newsletters:

  • .NET Insight—Takes subscribers through the maze of Visual Studio and .NET technologies, from development tools and platforms to hosted Web services and migration strategies. Published weekly.
  • Java Insight—Delivers the latest news, trends and techniques for creating enterprise Java applications, including technologies like J2EE, Web services, JSP, wireless systems and much more. Published weekly.
  • Web Design & Development Insight—Delivers proven tactics for creating intuitive and reliable sites. Published monthly.
  • Enterprise Architect Insight—Provides enterprise architects with information on technologies and techniques that are shaping the design and integration of enterprise systems today. Published twice monthly.
  • Upside Business Insight—Takes you behind the news and technologies that are shaping the high-tech industry. It will help you gain insights on who is shaping technology and where the industry is going, so you can better plan your IT efforts. Published monthly.
  • SQL Pro Insight—Provides essential content on SQL Server to help with your database administration and programming needs. Get tips on performance, management, security, and more. Published twice monthly.
  • Eclipse Insight—Provides information on the Eclipse platform, including new projects and technologies that shape this exciting trend. Published monthly.

Tuesday, June 05, 2007

LINQ to XSD Preview Alpha 0.2 for Orcas Beta 1 Available

Ralf Lämmel has posted an updated LINQ to XSD Preview Alpha 0.2 version that's compatible with LINQ to XML in Orcas Beta 1. The download date is 5/24/2007, but Ralf's LINQ to XSD Preview Alpha 0.2 to go with Orcas Beta 1 post didn't appear on the XML Team's blog until today. This drop updates the Alpha 0.1 version for the LINQ May 2006 CTP.

Ralf says the functionality is the same as version 0.1 for the March 2007 CTP and suggests that users read the updated “Revealing the X/O impedance mismatch” whitepaper. Ralf equates LINQ to XSD with the Philosopher's Stone, an analogy with which I agree (at least for LINQ to XML), by the paper's title to "Revealing the X/O impedance mismatch: Changing lead into gold."

According the to the ReadMe.htm file, version 0.2 offers the following features:

  • Comprehensive and canonical XML-schema-to object-model mapping.
  • The generated classes model typed views on XML trees.
  • Typed descendant and ancestor axes on XML object types.
  • Discoverable object models including tool tips for XML schema constraints.
  • Visual Studio integration through project templates with build tasks.
  • Command-line interface for mapping tool.

Alpha 0.2 offers very limited support for VB 9.0, to wit:

  • Code generation emits only C# classes.
  • To use LINQ to XSD with VB, you must generate a C# library and add it to your VB project.

My initial tests with relatively simple XML files and queries, functional construction, and literal XML file generation indicate that the Alpha 0.2 bits are stable.

Here are the two previous OakLeaf posts about LINQ to XSD:

The December post has a substantial bibliography for LINQ to XSD and related topics.

Here's hoping that LINQ to XSD will move from the incubator to full-fledged Beta status with VB support in the near future. 

Saturday, June 02, 2007

Entity Framework Updates

It's been a busy week for the Entity Framework (EF) and its components. If you're on your way to or at Tech*Ed 2007, you might find the information below useful when attending any of these LINQ and EF-related sessions:

Session Description
DAT04-TLC ADO.NET Entity Framework: Provider Model and Integration with Third-Party Databases
DAT318 Applied ADO.NET Entities: How to Leverage the Entity Framework in Your Application
DAT319 ADO.NET Entity Framework 1: Introduction
WEB308 Building a Complete Web Application Using ASP.NET "Orcas" and Microsoft Visual Studio Code Name "Orcas" (Part 2 of 2)
DEV319 LINQ and XML for the Microsoft Visual Basic Developer
DEV324 The .NET Language Integrated Query (LINQ) Framework
DEV332 XAML As a Better Microsoft Visual C# (Includes LINQ content)
DEV348 Using the .NET Language Integrated Query (LINQ) Framework with Relational Data
  Hands-On Labs
DAT06-HOL Understanding the ADO.NET Entity Framework
DEV19-HOL Introduction to LINQ
DEV20-HOL Mapping Your Objects to Database Tables with LINQ to SQL

6/3/2007: LINQ to Streams (SLinq) and LINQ to Expressions (MetaLinq) added to Third-Party LINQ Providers Post

An "appalling lack of documentation," as Oren Eini calls it, hasn't thwarted the work of third-party LINQ providers for specialized data domains. There now are 10 third-party LINQ providers and one helper (indexing) class listed on the Third-Party LINQ Providers page. A search of the CodePlex site on LINQ returned 15 projects as of 6/3/2007.

6/2/2007: The ADO.NET Team's Response to Defining the Direction of LINQ to Entities/EDM

ADO.NET architect Mike Pizzo responded in detail to my in absentia feedback for the Tech*Ed "focus group on LINQ/EDM ... to help define the direction of LINQ to Entities" in this long comment to my May 29, 2007 post. It's a great read for EF enthusiasts (and doubters)!

6/2/2007: Danny Simmons Now Groks Persistence Ignorance

One of my requests in my LINQ to Entities/EDM feedback was to "Respond formally to the concerns about lack of persistence ignorance and other issues voiced by the "O/RM Mafia" at the last MVP Summit in Redmond. Both Mike and Danny Simmons responded on the same day. Mike said:

The “O/RM Mafia” has been extremely effective in influencing our Object Services Dev Lead. I’m not sure what they have over him, but he’s totally bought into the idea of persistence ignorance (he’s said more than once that if only he’d “gotten” it a year ago, we would have gone a very different direction). Though I can’t say we’ll reach full persistence ignorance for V1, we are actively working on removing dependencies on domain classes in our remaining milestones.

Danny Simmons posted Persistence Ignorance: OK, I think I get it now early this morning (but not as early as Mike's comment), as promised in his March 28, 2007 A delay before I write more about persistence ignorance... post (shortly after the MVP summit ruckus.) Here's the takeaway:

So, to sum it all up, message received: persistence ignorance is needed, and we're now working to deliver it.  The first release of the entity framework likely will not have full support for it (there's just not enough time), but we will work to get into the first release the key changes which could not be made later so that the stage is set for the next release.

EF v1 might be a somewhat leaky abstraction but subsequent releases should deliver "'pure POCO', persistence ignorant domain models" to the agile-programming and domain-driven-design crowd, hopefully without a crippling performance hit.

Update 6/3/2007: Fixed Danny Simmons' name. Sorry about that, Danny.

6/1/2007: IBM, Oracle, and MySQL Will Support the Entity Framework

IBM's interest in LINQ and writing a EntityClient provider has been known since the last Entity Data Model "ProviderFest" but, in the words of Christian Shay, Oracle's Product Manager in the .NET and Windows group, "I think Oracle is keeping their level of support secret."

Oracle will be one of the presenters at Tech*Ed's DAT04-TLC: "ADO.NET Entity Framework: Provider Model and Integration with Third-Party Databases" chalk talk on Friday, June 8 from 1:00 PM to 2:15 PM in Blue Theater 12. Mike Pizzo says in reply to my point 13 on Oracle:

We have had two “ProviderFests” where vendors have come to Redmond for a week at a time to extend and test their providers with the latest Entity Framework bits and give us feedback. SQLite has already posted a version of their provider that works with the Entity Framework beta, and IBM, DataDirect, Oracle and MySQL will all join me for a chalk talk at TechEd on Friday to discuss their experiences extending their providers for the Entity Framework. [Emphasis added.]

Hopefully, the MySQL representative will demonstrate their EntityClient provider. David Sceppa's new blog contains updates to the status of third-party EF data providers.

Update 6/15/2007: David Sceppa reports in his TechEd: Provider Writers Talk About the Entity Framework post that IBM "demonstrated an early build of their provider working with the 101 LINQ to Entities samples." MySQL missed the session "due to scheduling problems but they are still very much involved."

Update 6/16/2007: Mike Pizzo expands on David Sceppa's report with brief details of DAT04-TLC presentations by DataDirect Technologies, Sybase, Oracle, and VistaDB in his Database Vendors Discuss the Entity Framework and LINQ at TechEd post of 6/15/2007 in the ADO.NET blog. As far as I can determine, this is the first mention of Sybase as participant in the EntityClient provider program. Mike also confirmed IBM's intention to add EntityClient and LINQ support to their Informix ADO.NET Data Provider.

5/31/2007: Matt Warren Writes "The Origin of LINQ to SQL" with a Dash of ObjectSpaces and WinFS Spice

The father of LINQ to SQL tells the inside story of how LINQ to SQL and LINQ itself came to be, describes how ObjectSpaces was drawn into the black hole of WinFS, and discloses that Microsoft's second attempt to productize an object/relational mapping (O/RM) tool started "the political nightmare that became [his] life for the next three years."

I'm sure all of us in the LINQ camp would enjoy more details on the three-year "political nightmare."

I had to make a few changes in my post about Matt's article as a result of Mike Pizzo's comment. 

Update 6/8/2007: Matt added a detailed LINQ to SQL: The Mapping Engine post on 6/6/2007 and the IQueryable’s Deep Dark Secret exposé on 6/1/2007.

5/31/2007: .NET Rocks Presents Frans Bouma on LLBLGen

Frans Bouma is the lead developer of LLBLGen Pro, a popular commercial object/relational mapping (O/RM) tool for .NET and one of about 40 O/RM apps listed in the Object-Relational-Mappers category of Fabrice Marguerie's SharpTools directory. Frans has been critical of the Entity Data Model and its Object Services layer's design since Microsoft first announced the EF. The interview with Frans starts at about 14:30. (Start there to learn what LLBL abbreviates.)

Frans was worried about "the effect of the EF on his business initially" (24:30), "If they [Microsoft] pull this off, I'm really in trouble." However, he concluded that EF "is a persistence solution, not an entity solution" (38:17) and decided "there's room for more than one [O/RM] tool" (39:10).

5/30/2007: An Entity SQL Primer

Microsoft promised an eSQL: An Entity SQL Language, ADO.NET Technical Preview document in the The ADO.NET Entity Framework Overview (June 2006) whitepapers, which appeared briefly in May 2006 and then vanished for about three weeks. As far as I've been able to determine, the ADO.NET team never published this document. 

There is an Entity SQL Quick Reference Guide (June 2006, a Word DOC in your C:\Program Files\Microsoft SDKs\ADO.NET vNext CTP\Docs folder after installing the Orcas March 2007 CTP), but it's not comprehensive and lacks syntax examples.

Mark Shields and Fabio Valbuena posted in the ADO.NET Team blog a much more detailed tutorial for Entity SQL (eSQL) on May 30, 2007. The post includes syntax examples using the Northwind sample database (what else?) and has the following sections

  • Types
  • Expressions
  • Query Expressions
  • SELECT VALUE and SELECT ROW
  • OFTYPE, ISOF, and TREAT
  • Paging
  • Relationship Navigation
  • Multi(set) Operations

If you're attempting to query the EDM with anything more complex than SELECT VALUE, you need to create a reference copy of this post.

5/29/2007: First Entity Framework CTP Scheduled for June

I reported that ADO.NET's Zlatko Michailov said on May 21, 2007, "The CTP is scheduled for June" in this ADO.NET Orcas forum thread in my Defining the Direction of LINQ to Entities/EDM post. However, you might not have seen this note if you didn't make it to the end of my list.

Mike Pizzo confirms in his comment that:

The first CTP of the Entity Framework outside of Orcas will be in June, and we will have CTPs/betas aligned with each of the Orcas Betas & RTM.

Good news. Let's hope the CTP has the new graphical EDM Designer.

Friday, June 01, 2007

Google Gears Piques New Interest in Data Synchronization

Google's announcement of their new Gears browser extension API for enabling offline capabilities in Web apps with these services:

  • LocalServer to cache and serve application resources locally.
  • Database (SQLite) to serve as a searchable local data store with online synchronization capability
  • WorkerPool to enable clients to run resource-intensive operations on a new thread

Google's Choosing an Offline Application Architecture page shows how the client application's UI can switch between the Web server and local database cache as its data source. Running SQLite or any other database in the browser requires write access to the client's file system. Synchronization can be manual, as in the Gears-enabled Google Reader (a button toggles between online and offline mode), or background, which can take advantage of the WorkerPool.

Update 6/2/2007: Tim Anderson's Why Google Gears? Thoughts from Google Developer Day provides an independent overview of Google Gears and notes the importance of synchronization. Tim also mentions full-text search in his SQLite will be everywhere post. According to Michael Cleverly's October 11, 2006 blog entry for D. Richard Hipp's keynote at the 13th Annual Tcl/Tk Conference:

As of SQLite 3.3.8 (released Monday!) full text search support in SQLite. Richard's authorized to announce help from engineers at Google. Later question elicited that roughly half of the FTS code was written by Richard and Dan, the other half by four engineers from Google. He isn't able to/can't comment on their motivations/plans/internal usage. [Minor edits.]

[The blog also notes that Microsoft uses SQLite in the Xbox.]

Google's Database API page states, "Google Gears includes SQLite's full-text search extension fts2."

Scott Hanselman says in his Google Gears - Maybe all Rich Internet Applications needed was Local Storage and an Offline Mode post:

Stunning move by Google today in the Rich Internet Application space. While most of us (myself included) are off debating Flash vs. Silverlight vs. Apollo vs. Whatever, Google introduces Google Gears ... a technology all of the above (or none of the above) can utilize ...

This is a huge move and is quite brilliant. In one seemingly innocuous move (and one tiny 700k (yes, 700K) download) Google is well positioned to get Google Docs, including Writely, Spreadsheet and Presentation, along with who knows what else, enabled for offline use. And the whole thing is Open Sourced via the New BSD License.

Dare Obasanjo quotes Robert Scoble's Google brings developers offline with “Gears”; new offline Reader post, and asks "[W]hat took them so long?", and closes with "Welcome to the future."

Robert says:

Gears supports using Adobe’s Apollo and Flash and should support other technologies including Microsoft’s Silverlight. [Emphasis added.]

But eWeek's Darryl K. Taft clarifies Gears' support for Silverlight in today's "Google Gears Aims to Bolster Adobe Apollo, Others" article:

Meanwhile, Google Gears will work with Microsoft's Silverlight cross-platform, cross-browser tool for building RIAs. Gears hopes to complement Microsoft's Silverlight and other technologies by offering an offline infrastructure that developers can incorporate into any Web application—even those that use plug-ins from other companies, a Google spokesperson said. [Emphasis added.]

However, Microsoft has its own local database cache: SQL Server Compact Edition (SSCE) and Synchronization Services (Sync Services). Orcas Beta 1 includes Sync Services Beta 1 and an early version of the graphical Sync Designer for SSCE 3.5. Sync Services is limited to synchronization between SSCE 3.1 or 3.5 on the client side and SQL Server 200x behind the Web server. Unfortunately, SSCE v3.x doesn't have full-text search capability.

Presumably, it would be possible to install the SSCE bits (1.8 MB) in conjunction with the Silverlight 1.1 runtime (4.2 MB) but you'd also need to install at least the Sync Services client pieces. The entire gears.dll is 852 kB for IE and 700 kB for Firefox. (The SQLite library itself varies in size from 224 to 513 kB, depending on options). However, a more appropriate comparison might include the 2.3 MB for the Flash player and its helper file. Note that the Gears library doesn't include a synchronization service. Developers would need to write their own service, which is no small chore (especially in JavaScript.)

Update 6/6/2007: Dare analyzes Google Gears in his June 5, 2007 Google Gears: Replacing One Problem with Another post and concludes that the missing synchronization component might be a showstopper:

It seems that without providing data synchronization out of the box, Google Gears leaves the most difficult and cumbersome aspect of building a disconnected Web app up to application developers. This may be OK for Google developers using Google Gears since the average Google coder is a Ph.D but the platform isn't terribly useful to Web application developers who want to use it for anything besides a super-sized HTTP cookie.

Collectively, the comments to Dare's post point out that general-purpose synchronization systems aren't easy to get right and that users of the Gears toolkit will need to write their own sync services. One commenter (Sam Sethi) points out that the Remember the Milk online to-do list has added Google Gears synchronization.

Update 6/7/2006: eWeek's Microsoft Watch blogger, Joe Wilcox, posted his Can Microsoft Be the Wrench in Google Gears? article on June 6, 2007. Joe quotes the same paragraph from Date's entry (above), and observes:

Quite possibly, synchronization is the killer app that will determine whether desktop software maintains its relevance or the Web becomes the more popular platform. ... The natural place for synchronization services is as part of the operating system.

Microsoft DevSource's Jeff Cogswell responded to Joe's post in his June 7, 2007 Synchronization: The future of the web? item:

Unfortunately, I think Joe's blog is missing something here, and I'm speaking as a developer. Synchronizing data is never a trivial nor generic task, because how it works and what it does depends on the particular application, and often requires substantial user input, and is heavily tied to file formats and how the data is organized inside the file.

ZDNet Blogger David Berlind posted an audio interview about Google Gears that he conducted on May 30, 2007 with Google director of engineering Linus Upson. Berlind quotes Adobe’s vice president of product management Michele Turner:

[W]e were developing identical technology to facilitate the offline component of the Apollo runtime. ... For example, they’re [Google] using SQLite and we were already incorporating SQLite into Apollo. So, now we’re aligning our efforts with Google on things like the synchronous and asynchronous calls that must be made to the SQLite database in order to enable the offline capability.

Google Gears piqued more interest in data synchronization than I expected.

Recent Microsoft Synchronization Services Updates

My "Update Local Data Caches with Sync Services" article in the May 2007 issue of Visual Studio Magazine carries this deck:

"The Microsoft Synchronization Services 1.0 API for SQL Server 2005 Compact Edition and the new Sync Designer in the Orcas March 2007 CTP team up to generate a local data cache that synchronizes incrementally and bidirectional with SQL Server [Express]."

Sync services and the Sync Designer is the technology behind Occasionally Connected Systems, which--together with the Entity Framework--comprise the Dynamic Development "Pillar of Katmai."

My earlier "Lighten Up Your Local Databases" in the March 2007 Visual Studio Magazine issue covers SQL Server 2005 Compact Edition v3.1 , which is the first version licensed for use in conventional PCs, and v3.5, which is included in Visual Studio codename "Orcas." Microsoft initially called V3.5 "SQL Server Everywhere Edition." V3.5 adds important features, such as timestamp fields to aid synchronization and simplify concurrency conflict detection. SQL Server Mobile Edition v3.0 and earlier were licensed for use by devices and Tablet PCs only.

Microsoft's Rafik Robeal released Demo VII: Tombstone Cleanup and Detecting Stale Offline Clients for the Sync Services runtime. Tombstone cleanup detects and prevents stale clients from synchronizing and tells them to reinitialize their local data cache. A stale offline client is a computer that hasn't synchronized in the last n days, so the client's sync request is rejected.

Rafik has updated the preceding six demos to VS Orcas Beta 1 in his Sync Services Demos Refresh post:

Update 6/6/2007: Rafik just added a Sync Events for Conflict Handling, Progress Reporting, Business Logic … post that describes Sync Services new set of events. Some of the events are implemented in Beta 1, but you'll have to wait for Beta 2 for the others. See my Synchronization Services Runtime Beta 1 for ADO.NET 2.0 Is Available and A Sync Services Bidirectional Test Harness posts for issues with the Beta 1 release.

IBM and Oracle to Support the ADO.NET Entity Framework

Microsoft's two three major relational database management system (RDBMS) competitors—IBM, Oracle, and MySQL—have announced plans to support the ADO.NET Entity Framework.

Update 6/15/2007: David Sceppa reports in his TechEd: Provider Writers Talk About the Entity Framework post that IBM "demonstrated an early build of their provider working with the 101 LINQ to Entities samples." MySQL missed the session "due to scheduling problems but they are still very much involved."

Update 6/16/2007: Mike Pizzo expands on David Sceppa's report with brief details of Tech*Ed's DAT04-TLC ADO.NET Entity Framework: Provider Model and Integration with Third-Party Databases presentations by DataDirect Technologies, Sybase, Oracle, and VistaDB in his Database Vendors Discuss the Entity Framework and LINQ at TechEd post of 6/15/2007 in the ADO.NET blog. As far as I can determine, this is the first mention of Sybase as participant in the EntityClient provider program. Mike also confirmed IBM's intention to add EntityClient and LINQ support to their Informix ADO.NET Data Provider.

IBM

From my Future LINQ to SQL Support for Multiple Databases? post: According to the April 2007 "Looking to LINQ" story in Redmond Developer News:

Microsoft is working with other database vendors to build interoperability into their repositories as well. Oracle Corp. officials would only say they're monitoring LINQ's progress, but IBM Corp. says it plans to support LINQ in both its DB2 database and the Informix IDS platform.

Curt Cotner, an IBM Fellow and CTO for database servers, says LINQ could very well be critical in eliminating the fragmentation that exists today among programmers and database developers. Still, Cotner expects LINQ to have its share of both ardent supporters and critical detractors. "It's going to be popular for a significant segment of customers, but there will be another segment that will see it as not down the path they're trying to follow," he says. ...

But Cotner doesn't see [the Java Persistence API,] JPA and LINQ as competing specifications per se. "It addresses a different segment of the population," he says. And frankly, he admits from a developer standpoint, JPA can't touch LINQ in terms of its ability to build native queries into code from the language and development environment.

Supporting LINQ in a third-party RDBMS implies writing a custom EntityClient provider, as mentioned by Elisa Johnson, Program Manager, Microsoft Data Programmability in an April 23, 2007 comment in the ADO.NET Team's ADO.NET Entity Framework - Orcas Beta 1 post:

We are working closely with many DB vendors, including IBM, to help them update their ADO.NET providers to support the Entity Framework. Unfortunately, I do not have any information as to when they may release a new provider.

David Sceppa, an ADO.NET program manager and author of Programming Microsoft ADO.NET 2.0 Core Reference, refers to Tech*Ed's "ADO.NET Entity Framework: Provider Model and Integration with Third-Party Databases" chalktalk in this May 25, 2007 Entity Framework Provider Talk at TechEd post:

I’m pleased to say that DataDirect, IBM and others will be in attendance to talk about the work they’re doing to add support for the Entity Framework to their ADO.NET providers.

  • DataDirect will discuss heterogeneous data sources in an Entity Framework environment, the importance of best of class ADO.NET providers as an essential component for the Entity Framework and present a use case on a how to ensure a successful Entity Framework development and deployment cycle.

  • IBM will talk about their investments to support the Entity Framework for their Data Servers and to demonstrate an early build of their provider working with the Entity Framework.

ORACLE

Christian Shay, Product Manager in the .NET and Windows group at Oracle left this comment to the Future LINQ to SQL Support for Multiple Databases? post on May 31, 2007:

I think Oracle is keeping their level of support secret. But, a clue has just been revealed and apparently, more will be known at TechEd. Oracle is co-presenting a talk called "ADO.NET Entity Framework: Provider Model and Integration with Third-Party Databases" Check it out: http://cshay.blogspot.com/2007_05_01_archive.html.

So it's now clear that Oracle is doing more than "monitoring LINQ's progress."

MySQL

Update 6/2/2007: According to Mike Pizzo, the ADO.NET architect who's leading the chalk talk:

We have had two “ProviderFests” where vendors have come to Redmond for a week at a time to extend and test their providers with the latest Entity Framework bits and give us feedback. SQLite has already posted a version of their provider that works with the Entity Framework beta, and IBM, DataDirect, Oracle and MySQL will all join me for a chalk talk at TechEd on Friday to discuss their experiences extending their providers for the Entity Framework.

Hopefully, Sybase and the PostgreSQL folks will join the EF bandwagon before long.

 

Embedded RDBMSes

David Sceppa added the ADO.NET Orcas: Sample Provider post to the ADO.NET Team blog on March 16, 2007. The entry begins:

Last month, the ADO.NET team hosted a number of ADO.NET Data Provider writers on campus to present information on how to enhance an existing provider to support the Entity Framework features in the upcoming Orcas release.

David's ADO.NET vNext Provider Writer Event Summary post of August 25, 2006, mentions that representatives from DataDirect, IBM, MySQL, SQLite, and Sybase attended the event.

SQLite

PHX Software's Robert Simpson announced on September 6, 2006 a preview release of an "ADO.NET vNext" provider for SQLite and wrote about the conference in this ADO.NET Entity Framework blog. (Google uses SQLite as the embedded database for Google Gears.)

VistaDB

VistaDB Software, Inc. announced the intent to support LINQ in a future version of VistaDB. VistaDB 3 is an embedded SQL database for .NET, the .NET Compact Framework, and Mono that supports datatypes compatible with SQL Server 2005 (except xml) and uses T-SQL syntax. Jason Short wrote in an April 7, 2007 "LINQ and VistaDB" blog post:

I feel quite strongly that LINQ support in the future will be a vital feature of VistaDB. Since we are already a fully managed database for Dot Net, the ability to make fully type safe queries will definitely set us apart from other database systems. And the fact that we will not have to drop to SQL code to perform that queries will mean a faster running database as well (and that is what you really wanted to hear, right?). ...

LINQ support is on the list for research right now. When will [w]e release it? I don’t know. I would really like to hear from you about your plans to support LINQ. I think it is quite obvious that LINQ is going to change the way a lot of applications are written to query their database.

SQL Server Compact Edition

SSCE has an EntityClient provider, but it doesn't appear to work in Orcas Beta 1.

(The preceding content, except the new Oracle and Tech*Ed session information, is repurposed from my Future LINQ to SQL Support for Multiple Databases? post.)

Thursday, May 31, 2007

Matt Warren's "The Origin of LINQ to SQL"

Read the inside story of how LINQ to SQL and LINQ itself came to be, learn how ObjectSpaces was drawn into the black hole of WinFS, and discover that Microsoft's second attempt to productize an object/relational mapping (O/RM) tool started "the political nightmare that became [Matt's] life for the next three years."

The first major event after the Microsoft Business Division (MBD) absorbed the Server and Tools Business (STB) unit was to deep-six this year's Professional Developer's Conference. What's next on MBD's chopping block?

  • LINQ to SQL?
  • LINQ to Entities, Entity Framework, and Entity Data Model?
  • LINQ itself?

Update: 6/2/2007: ADO.NET Architect Mike Pizzo states in a comment to my May 29, 2007 Defining the Direction of LINQ to Entities/EDM post's items #1 and #4:

The bottom line on Entity Framework and LINQ to SQL was that they are each designed around a different set of priorities; LINQ to SQL is designed around rapid development, with direct (1:1) mapping, while the Entity Framework is designed around exposing a rich conceptual model with flexible mappings to extended ADO.NET Data Providers. We considered not shipping one or the other of these to reduce confusion, but ultimately decided that neither stack fully addressed the customer commitments and expectations set by the other. So we will ship both; LINQ to SQL in Orcas and the Entity Framework in an update to .NET Framework 3.5 in first half 2008. Over time we will look to customers to help us decide whether it makes sense to continue to invest in enhancing both stacks or merging into a single stack over time.

It’s an undeniable fact that Microsoft has fumbled the ball repeatedly when it comes to delivering an O/R solution. The biggest reason we didn’t ship ObjectSpaces was not technical, but rather a lack of recognition at a management level of the importance of providing such a solution. The only way for ObjectSpaces to gain traction was by aligning with something bigger (i.e., MBF or WinFS). The biggest thing that’s changed (thanks largely to the success of Hibernate) is that management now gets the importance of a good O/R solution. The support for LINQ, LINQ to SQL, and the Entity Framework at the executive level is what ObjectSpaces never enjoyed, and what gives me confidence that we will finally shed the albatross that’s been hanging around our neck since .NET Framework 1.0.

I'd venture a guess from the tenor of his post that Matt's "political nightmare" of the last three years will be sweet dreams compared to the future under MBD's aegis.

Don't miss it.

Note: Dare Obasanjo ends his Surviving WinFS: How LINQ to SQL Came to Be post on this positive slant:

This takes me back. I had friends who worked on ObjectSpaces and it was quite heart breaking to see what internal politics can do to passionate folks who once believed that technologies stand based on merit and not who you know at Microsoft. At least this story had a happy ending. Passionate people figured out how to navigate the internal waters at Microsoft and are on track to shipping a really compelling addition to the developer landscape.

I'm a strong proponent of LINQ to SQL as an alternative to the delayed and overweight Entity Framework, so I hope Dare's right. I'm concerned, however, that LINQ to SQL, whose origins are in the C# team, might be sacrificed on the alter of the ADO.NET team's Entity Framework.

Backstories from the OakLeaf Blog

A Ribbon UI for Visual Studio Code Name Orcas?

SQL Server "Katmai" to Deliver Entity Data Platform and Support LINQ

Déjà Vu All Over Again: Entity Framework Cut from Orcas

Orcas December 2006 CTP Coming with New LINQ/EDM Bits

Wednesday, May 30, 2007

SqlMethods Stealth Class Enables LIKE and DATEDIFF for LINQ to SQL

Anders Hejlsberg took the wraps off LINQ to SQL's SqlMethods class hidden in the System.Linq.Data.SqlClient namespace with this ADO.NET Orcas forum post. Google and Windows Live searches for linq sqlmethods return only one unique hit. Yahoo! Search returns 3 hits, one of which is the result of a typo and the other two duplicate the primary Google hit with omitted results included.

Orcas Beta 1's online help documents the SqlMethods class and its Like and DateDiff methods but doesn't provide sample use cases.

The Like Method and T-SQL LIKE Operator

The SqlMethods class includes two overloads of the Like method that the Canonical Query Tree (CQT) sends as T-SQL directly as the LIKE operator:

Dim matchExpression As String
Dim pattern As String
Dim returnValue As Boolean

returnValue = SqlMethods.Like(matchExpression, pattern)

And

Dim matchExpression As String
Dim pattern As String
Dim escapeCharacter As Char
Dim returnValue As Boolean

returnValue = SqlMethods.Like(matchExpression, pattern, escapeCharacter)

The .NET String methods provide Contains(chars), StartsWith(chars), and EndsWith(chars), which the CQT translates to LIKE '%chars%', LIKE 'chars%', and LIKE '%chars' operators, respectively. Note that Contains(chars)isn't the same as the Contains(Of T) standard query operator that determines whether an ICollection(Of T) contains a specified value.

However Contains(chars) doesn't support multiple character sets separated by wildcards, such as LIKE '%abcd%efgh%', which are valid in T-SQL. Here's a LINQ to SQL VB query that uses the SqlMethods.Like method:

Dim Query = From c In dcNwind.Customers, o In c.Orders, _ 
d In o.Order_Details _
Where SqlMethods.Like(c.CompanyName, "%fred%kist%") _
Order By o.OrderID Descending _
Select c.CustomerID, c.CompanyName, o.OrderID, _
o.OrderDate, d.Product.ProductName, d.Quantity 

The preceding query sends the following parameterized T-SQL query to an updated version of the the Northwind sample database:

SELECT [t0].[CompanyName], [t0].[CustomerID], [t1].[OrderDate], [t1].[OrderID], [t3].[ProductName], [t2].[Quantity]
FROM [dbo].[Customers] AS [t0], [dbo].[Orders] AS [t1], [dbo].[Order Details] AS [t2], [dbo].[Products] AS [t3]
WHERE ([t3].[ProductID] = [t2].[ProductID]) AND ([t0].[CompanyName] LIKE @p0) AND ([t1].[CustomerID] = [t0].[CustomerID]) AND ([t2].[OrderID] = [t1].[OrderID])
ORDER BY [t1].[OrderID] DESC
-- @p0: Input NVarChar (Size = 11; Prec = 0; Scale = 0) NOT NULL [%fred%kist%]

 which returns the following rows (formatting added):

CustID	OrderID	OrderDate	Quan	ProductName
ALFKI 11011 04/09/2007 40 Escargots de Bourgogne
ALFKI 11011 04/09/2007 20 Flotemysost
ALFKI 10952 03/16/2007 16 Grandma's Boysenberry Spread
ALFKI 10952 03/16/2007 2 Rössle Sauerkraut
ALFKI 10835 01/15/2007 15 Raclette Courdavault
ALFKI 10835 01/15/2007 2 Original Frankfurter grüne Soße
ALFKI 10702 10/13/2006 6 Aniseed Syrup
ALFKI 10702 10/13/2006 15 Lakkalikööri
ALFKI 10692 10/03/2006 20 Vegie-spread
ALFKI 10643 08/25/2006 15 Rössle Sauerkraut
ALFKI 10643 08/25/2006 21 Chartreuse verte
ALFKI 10643 08/25/2006 2 Spegesild

The DateDiff Methods and T-SQL DATEDIFF Function

The SqlMethods class includes seven DateDiffInterval methods, where Interval is Year, Month, Day, Hour, Minute, Second, and Millisecond. (Surprisingly, Quarter and Week intervals are missing.) Each interval method has two overloads, one that accepts a pair of DateTime and one that accepts a pair of Nullable(Of DateTime) datatypes, as shown here:

Dim startDate As DateTime
Dim endDate As DateTime
Dim returnValue As Integer

returnValue = SqlMethods.DateDiffYear(startDate, endDate)

And

Dim startDate As Nullable(Of DateTime)
Dim endDate As Nullable(Of DateTime)
Dim returnValue As Nullable(Of Integer)

returnValue = SqlMethods.DateDiffYear(startDate, endDate)

The query to return rows for all products shipped in the past month to Brazil is:

Dim Query = From c In dcNwind.Customers, o In c.Orders, _ 
d In o.Order_Details _
Where SqlMethods.DateDiffMonth(o.OrderDate.Value, Today) = 1 And _
c.Country = "Brazil" _
Order By o.OrderID Descending _
Select c.CustomerID, c.CompanyName, o.OrderID, _
o.OrderDate, d.Product.ProductName, d.Quantity

which sends the following T-SQL statement:

SELECT [t0].[CompanyName], [t0].[CustomerID], [t1].[OrderDate], [t1].[OrderID], [t3].[ProductName], [t2].[Quantity]
FROM [dbo].[Customers] AS [t0], [dbo].[Orders] AS [t1], [dbo].[Order Details] AS [t2], [dbo].[Products] AS [t3]
WHERE ([t3].[ProductID] = [t2].[ProductID]) AND (DATEDIFF(Month, [t1].[OrderDate], @p0) = @p1) AND ([t0].[Country] = @p2) AND ([t1].[CustomerID] = [t0].[CustomerID]) AND ([t2].[OrderID] = [t1].[OrderID])
ORDER BY [t1].[OrderID] DESC
-- @p0: Input DateTime (Size = 0; Prec = 0; Scale = 0) NOT NULL [5/30/2007 12:00:00 AM]
-- @p1: Input Int (Size = 0; Prec = 0; Scale = 0) NOT NULL [1]
-- @p2: Input NVarChar (Size = 6; Prec = 0; Scale = 0) NOT NULL [Brazil]

and returns the following rows:

CustID	OrderID	OrderDate	Quan	SkuName
RICAR 11059 04/29/2007 30 Konbu
RICAR 11059 04/29/2007 12 Alice Mutton
RICAR 11059 04/29/2007 35 Camembert Pierrot
HANAR 11052 04/27/2007 30 Ipoh Coffee
HANAR 11052 04/27/2007 10 Sirop d'érable
GOURL 11049 04/24/2007 10 Chang
GOURL 11049 04/24/2007 4 Queso Manchego La Pastora
COMMI 11042 04/22/2007 15 Gula Malacca
COMMI 11042 04/22/2007 4 Sirop d'érable
HANAR 11022 04/14/2007 35 Teatime Chocolate Biscuits
HANAR 11022 04/14/2007 30 Gudbrandsdalsost

There undoubtedly are additional T-SQL-specific operators and functions that would be useful if the ADO.NET team decides to expand the methods exposed by the SqlMethods class. Any suggestions?

Technorati tags: , , ,

Tuesday, May 29, 2007

Defining the Direction of LINQ to Entities/EDM

The ADO.NET team today posted an invitation targeted at "experienced IT developers" among Tech*Ed 2007 attendees to "attend a focus group on LINQ/EDM ... to help define the direction of LINQ to Entities." I'm not attending Tech*Ed because Microsoft moved it from New Orleans, a city I love almost as much as San Francisco, to Orlando, which I detest. (I'd rather travel to Flint, Michigan, than Orlando, Florida.)

Update 6/7/2007: LLBLGen Pro's Frans Bouma posted SqlServer 2008: Does it or doesn't it have the Entity Framework?, which supports my request for read-only foreign key values (#9) and a comparison of the Entity Framework's current and short-term future feature set with other popular object/relational mapping (O/RM) tools (#2). Ayende (Oren Eini) replied with Phsycic Debugging from Blogs, which contains an analysis of why it might be difficult for the ADO.NET team to deliver foreign key values without retrieving the corresponding object.

Update 6/2/2007: ADO.NET architect, Mike Pizzo responded in detail to my feedback in this comment.

I won't be able to join the focus group, so I'll add here the two cents I would have provided there. Following are what I consider the more important actions required to make the Entity Framework, Entity Design Model, Entity Client, and LINQ to Entities a competitive O/RM tool:

  1. Come clean with positioning of EF/EDM versus LINQ to SQL by releasing the retracted white paper that compared features of these technologies. (Julie Lerman mentioned an ADO.NET Team post that "addresses one of the biggest questions about querying against the Entity Framework - what are the pros & cons of Entity SQL vs. LINQ" that's disappeared. She says, "In discussing this, the post also lays out a lot of great background info.")
  2. Provide a similar white paper comparing the projected feature set of EF v1.0 and, perhaps, EF vNext with that of NHibernate, NPersist, and commercial OR/M tools such as LLBLGen Pro, Wilson ORMapper for .NET, et al.
  3. Recommit to Mike Pizzo's fervent promise of RTM with a graphical EDM Designer in the first half of 2008, not "in the Katmai timeframe."
  4. Deal openly with Mats Helander's contention that Microsoft is crying "O/R wolf" and won't release the EF/EDM because of the "bad press" ensuing from tools that "don't work or underperform."
  5. Respond formally to the concerns about lack of persistence ignorance and other issues voiced by the "O/RM Mafia" at the last MVP Summit in Redmond.
  6. Provide a detailed development path to and timeline for the version that enables generating the database schema from a domain model, rather than developing the domain model from the database schema.
  7. Eliminate the inability to navigate associations having composite primary keys. (LINQ to SQL doesn't have a problem with composite primary keys).
  8. Implement new object addition and all other advertised features that throw "Not yet implemented" exceptions.
  9. Provide read-only access to foreign key values.
  10. Define the support for n-tier architectures and provide non-trivial, loosely-coupled sample projects with WCF.
  11. Deliver a log feature similar to LINQ to SQL's that captures the [not just T-]SQL sent to the server.
  12. Provide a more friendly method of responding to concurrency exceptions, such as that provided by LINQ to SQL.
  13. Commit to writing an Oracle managed provider if Oracle is reluctant to do so. Developers need proof that EF/EDM/LINQ to Entities is database-agnostic. [Oracle is now on board and participating in a Tech*Ed 2007 chalktalk about EntityClient providers.]
  14. Let developers know when they can expect the first CTP and the approximate schedule of CTPs/betas to follow.*
  15. Provide detailed change lists for each CTP.
  16. Accelerate work on adding DML constructs (INSERT, UPDATE, DELETE) to Entity SQL/Entity Client.
  17. Provide an analysis of EF/EDM in terms of Martin Fowler's Patterns of Enterprise Application Development and compliance with domain-driven design (DDD) and test-driven development practices as Ian Cooper did for LINQ to SQL in his Being Ignorant with LINQ to SQL essay.

Update 6/14/2007: Added item 17 as promised in Ian Cooper Takes on DDD, TDD and PI with LINQ to SQL.

I'll update this list as I think of additional issues or missing features. If you have an EF bug you'd like squashed or a feature you'd like to see in EF/EDM/LINQ to Entities, please leave a comment.

* Note: ADO.NET's Zlatko Michailov said on May 21, 2007, "The CTP is scheduled for June" in this ADO.NET Orcas forum thread. However, future CTP drop schedules should be posted in the ADO.NET Team blog for better visibility to interested parties via syndication.

P.S. The sign-up site includes this sentence about the focus group session: "This session will describe our current thinking about the LINQ/EDM APIs we will offer with the Orcas release of Visual Studio." [Emphasis added.]

Last I heard, EF/EDM/LINQ to Entities had been removed from the Orcas release.

Update 5/30/2007: Added five topics.

Sunday, May 20, 2007

LINQ to DataSets Rehabilitates VB's Bang Operator

Visual Basic's bang (!) operator (a.k.a. pling and officially the dictionary lookup operator) seldom appears in VB.NET code. The purpose of the bang operator (alias the bang separator) is to specify members of string-keyed collections, such as fields of an ADO.COM Recordset, as in:

rsCusts!CustomerID = "BOGUS"
rsCusts!CompanyName = "Bogus Software, Inc."
rsCusts!ContactName = "Joe Bogus"

The preceding shaves a few keystrokes from "parens-and-quote" syntax, which relies on Fields being the default property of a Recordset:

rsCusts("CustomerID") = "BOGUS"
rsCusts("CompanyName") = "Bogus Software, Inc."
rsCusts("ContactName") = "Joe Bogus"

which, in turn, is shorthand for the fully-qualified syntax:

rsCusts.Fields("CustomerID") = "BOGUS"
rsCusts.Fields("CompanyName") = "Bogus Software, Inc."
rsCusts.Fields("ContactName") = "Joe Bogus"

One reason for bang's disappearance is that ! got a bum rap from VB.COM and VBA developers because amateur coders (especially Access rookies) used bang in place of dot (.) to specify properties of other objects, such as forms and controls. Doing this kills IntelliSense and compile-time checking. In fact, ! fell into such disrepute that many developers, such as this "VB Expert," believed that VB.NET didn't support the bang operator. Some folks claimed that performance suffered when ! was tokenized to the paren-quote syntax, although I never found anyone who reported testing the performance hit.

Paul Vick's July 15, 2003 Dictionary Lookup Operator post explains how the bang operator works and, in a comment, it's equivalent in C#—x!y : x["y"].

C# Joins and Associations in Typed and Untyped DataSets

LINQ to DataSets is most likely to be used in place of DataViews where joins between related tables are required. (DataViews don't support joins or projections (SELECT lists), so you need LINQ to DataSets or a custom DataView class, such as the JoinView from MSDN, to accommodate them.) I needed to verify the differences in the syntax for LINQ to DataSet queries over typed or untyped multi-table DataSets that use joins or navigate associations and include value types (specifically DateTime) with nulls. Erick Thompson wrote a Nulls - LINQ to DataSets Part 3 treatise in his LINQ to DataSet Documentation Series but didn't mention nullable datatypes in conjunction with the added DataRow.Field<T> collection.

Here's the C# test query for the typed DataSet with joins, which returns nine rows with values from each of four related Northwind tables, including one null ShippedDate value to test null handling:

var query1 = from c in dsNwindT.Customers
             join o in dsNwindT.Orders on c.CustomerID equals o.CustomerID
             join d in dsNwindT.Order_Details on o.OrderID equals d.OrderID
             join p in dsNwindT.Products on d.ProductID equals p.ProductID
             where p.ProductID == 2 && c.Country == "USA"
             orderby o.OrderID descending
             select new { c.CustomerID, c.CompanyName, o.OrderID,
                          ShippedDate = o.Field<DateTime?>("ShippedDate"),
                          p.ProductName, d.Quantity };

With the exception of the code to supply the ShippedDate value (emphasized), the preceding query statement is identical to the corresponding LINQ to SQL query (just replace the dsNwindT DataSet with a DataContext.

You lose strong typing of related DataRows when you navigate associations (ChildRows or ParentRows) between related DataTables instead of joining the DataTables. This requires FieldName = DataTable.Field<DataType>("FieldName") expressions for non-nullable and FieldName = DataTable.Field<NullableType>("FieldName") expressions for nullable fields for all but the topmost table of the hierarchy:

var query2 = from c in dsNwindT.Customers
             from o in c.GetChildRows("FK_Orders_Customers")
             from d in o.GetChildRows("FK_Order_Details_Orders")
             from p in d.GetParentRows("FK_Order_Details_Products")
             where p.Field<int>("ProductID") == 2 && c.Country == "USA"
             orderby o.Field<int>("OrderID") descending
             select new {
                 c.CustomerID, c.CompanyName,
                 OrderID = o.Field<int>("OrderID"),
                 ShippedDate = o.Field<DateTime?>("ShippedDate"),
                 ProductName = p.Field<string>("ProductName"),
                 Quantity = d.Field<short>("Quantity") };

Join syntax for untyped DataSets becomes quite cumbersome because of the need to apply the AsEnumerable() method to each DataTable and DataTable.Field<DataType>("FieldName") to the joined fields:

var query3 = from c in dsNwindU.Tables["Customers"].AsEnumerable()
             join o in dsNwindU.Tables["Orders"].AsEnumerable() on
                 c.Field<string>("CustomerID") equals
                 o.Field<string>("CustomerID")
             join d in dsNwindU.Tables["Order_Details"].AsEnumerable() on
                 o.Field<int>("OrderID") equals d.Field<int>("OrderID")
             join p in dsNwindU.Tables["Products"].AsEnumerable() on
                 d.Field<int>("ProductID") equals p.Field<int>("ProductID")
             where p.Field<int>("ProductID") == 2 &&
                 c.Field<string>("Country") == "USA"
             orderby o.Field<int>("OrderID") descending
             select new {
                 OrderID = o.Field<int>("OrderID"),
                 ShippedDate = o.Field<DateTime?>("ShippedDate"),
                 ProductName = p.Field<string>("ProductName"),
                 Quantity = d.Field<short>("Quantity") };

The added typing for untyped DataSets with associations instead of joins (emphasized) isn't that great:

var query4 = from c in dsNwindU.Tables["Customers"].AsEnumerable()
             from o in c.GetChildRows("FK_Orders_Customers")
             from d in o.GetChildRows("FK_Order_Details_Orders")
             from p in d.GetParentRows("FK_Order_Details_Products")
             where p.Field<int>("ProductID") == 2 &&
                c.Field<string>("Country") == "USA"
             orderby o.Field<int>("OrderID") descending
             select new {
                 CustomerID = c.Field<string>("CustomerID"),
                 CompanyName = c.Field<string>("CompanyName"),
                 OrderID = o.Field<int>("OrderID"),
                 ShippedDate = o.Field<DateTime?>("ShippedDate"),
                 ProductName = p.Field<string>("ProductName"),
                 Quantity = d.Field<short>("Quantity") };

But the added typing probably would trouble C# purists who substitute lambda functions for enhanced query syntax because doing so saves a few characters.

VB Associations in Typed and Untyped DataSets

When skimming the sample queries in the "LINQ over DataSet for VB Developers" whitepaper for the May 2006 LINQ preview, I noticed extensive use of the bang operator in DataTable!FieldName expressions taking the place of FieldName = DataTable.Field<DataType>("FieldName") and FieldName = DataTable.Field<NullableType>("FieldName") expressions. Strangely, there's no mention of "bang" or "dictionary" in the nine-page document.

Unfortunately, Orcas Beta 1's VB compiler doesn't implement the Join keyword and joins using SQL-89 equi-join syntax are limited to two tables. Thus, my tests with VB are limited to navigating associations.

Here's the VB version of C# query2, which returns objects rather than designated data types for fields other than CustomerID and CompanyName:

Dim Query2 = From c In dsNwindT.Customers, _
             o In c.GetChildRows("FK_Orders_Customers"), _
             d In o.GetChildRows("FK_Order_Details_Orders"), _
             p In d.GetParentRows("FK_Order_Details_Products") _
             Where CInt(p!ProductID) = 2 AndAlso c!Country.ToString = "USA" _
             Order By o!OrderID Descending _
             Select c.CustomerID, c.CompanyName, o!OrderID, _
                 o!ShippedDate, p!ProductName, d!Quantity

Notice that no special treatment of the ShippedDate field is required to handle null values. However, if you attempt to strongly type ShippedDate with a CType(o!ShippedDate, Nullable(Of DateTime)) expression, a runtime error occurs when iterating the row with the null value.

And here's the VB version of C# query4, which returns objects for all fields:

Dim Query4 = From c In dsNwindU.Tables("Customers"), _
             o In c.GetChildRows("FK_Orders_Customers"), _
             d In o.GetChildRows("FK_Order_Details_Orders"), _
             p In d.GetParentRows("FK_Order_Details_Products") _
             Where CInt(p!ProductID) = 2 AndAlso c!Country.ToString = "USA" _
             Order By o!OrderID Descending _
             Select c!CustomerID, c!CompanyName, o!OrderID, _
                 o!ShippedDate, p!ProductName, d!Quantity

Notice that the AsEnumerable() method is missing from the DataTable identifier in the first line. The C# query won't compile without it but the VB compiler doesn't care if it's present or not.

I believe it's a good bet that VB's Join syntax for untyped DataSets in Beta 2 will look like this, which is considerably more concise than that of the C# version:

Dim Query4 = From c In dsNwindT.Customers
             Join o In dsNwindT.Orders On c!CustomerID Equals o!CustomerID
             Join d In dsNwindT.Order_Details On o!OrderID Equals d!OrderID
             Join p in dsNwindT.Products On d!ProductID Equals p!ProductID
             Where CInt(p!ProductID) = 2 AndAlso c!Country.ToString = "USA"
             Order By o!OrderID descending
             Select c!CustomerID, c!CompanyName, o!OrderID, o!ShippedDate,
                 p!ProductName, d!Quantity

Update 5/18/2007: According to VB Team member Amanda Silver, VB's Join syntax uses the Equals operator (equals in C#) instead of the = operator to make explicit that the expression uses Object.Equals as the comparison operator.

Conclusion: C# purists castigate VB for its verbosity and some condemn VB code as prolix. VB's syntax for LINQ queries against joined or associated tables is equally or more concise than that of C#.

Update 5/24/2007: Paul Vick points out in his Javascript in VB ... post that "the Silverlight-based Javascript compiler that we released in Silverlight 1.1" as well as the VBx compiler for VB 10.0 is written in VB. Miguel de Caza mentions in his Compiler Lab: Second Day post:

Their new DLR-based Javascript compiler (that implements the current ECMA spec) was written by two developers in four months. Nandan Prabhu was at the conference and he explained that if they were to rewrite it today it would probably take three months for a programmer to implement as such a person would not have to keep up with the daily changes in the DLR [Dynamic Language Runtime].

It will be interesting to see how the final implementation of VB's Join keyword handles null value types in strongly typed projections.

Update 5/22/2007: You'll need to use the FieldName = DataTable.Field<DataType>("FieldName", DataRowVersion.Version) overload of the Field method in both VB and C# if you want to filter by a member of the DataRowVersion enumeration: Current, Default, Original or Proposed. (The "LINQ to DataSet for C# Developers" whitepaper describes the DataRowVersion overloads, but doesn't include a sample of ther use. "LINQ to DataSet for VB Developers" doesn't even mention these overloads.)

Friday, May 18, 2007

Erik Meijer and LINQ 2.0 - Round 2

Microsoft's peripatetic LINQ theorist/evangelist Erik Meijer delivered the second edition of his "LINQ 2.0: Democratizing the Cloud" presentation to the XTech 2007 Conference in Paris on May 16, 2007. The session's overview demonstrates that LINQ 2.0 is more than Erik's vision for "simplifying the development of data-intensive distributed applications."

Erik observes:

One painful problem that is yet unsolved is the “last mile” of data programming namely the mapping between data models at the edges, in particular the mapping between relational data and objects.

My response? "Ain't that the truth!"

Jeni Tennison wrote a brief review of Erik's session to which Mike Champion replied.

Update 5/22/2007: Erik also presented XML and LINQ: What's new in Orcas and beyond at XTech 2007.

What's new in round 2 is that LINQ 2.0 will address an issue that plagues the current (delayed) version of the Entity Framework (EF) and Entity SQL (eSQL)—lack of update capability.

Object/Relational Mapping and Updatable Views

ADO.NET program manager Zlatko Michailov wrote in his Entity Client post of February 14, 2007:

Entity SQL will not support any DML constructs – INSERT, UPDATE, or DELETE in the Orcas release. Data modification could still be done through ObjectContext, or through the native SQL of the target store. It is our goal to implement DML support but for now it’s more important to verify we’ve built the right query language. [Emphasis added.]

You can read more about this issue in Bob Beauchemin's Re: UPDATE, INSERT, DELETE in eSQL? thread of September 14, 2006 in the ADO.NET Orcas forum. Pablo Castro replied on September 15, 2006:

We want to do DML for Entity SQL over the mapping layer, it's just that we'll not get to it in the upcoming release.

Just three weeks before Zlatko's belated admission, ADO.NET data architect Mike Pizzo had written in his January 23, 2007 Data Access API of the Day - Part IV (Programming to the Conceptual Model…) post:

These rich conceptual schemas are exposed and queried through an "Entity Client".  The Entity Client is an ADO.NET Data Provider that builds queries against storage-specific providers using client-side read/write "views". Queries and updates written against these conceptual views are expanded by the Entity Client and executed as queries against underlying storage-specific providers.  All the actual query execution is done in the store (not on the client), and the results are assembled into possibly hierarchical, polymorphic, results with nesting and composite members. [Emphasis added.]

Updating views isn't a walk in the park, as Microsoft Research's Sergei Melnik writes in his August 23, 2006 "Mapping-Driven Data Access" position paper:

[T]o date there is no elegant and widely accepted mechanism
for specifying view update behavior and processing updates
in mapping-driven data access scenarios. As a consequence, commercial
database systems offer poor support for updatable views.

Sergei goes on to say, about three weeks before Pablo's announcement that updates wouldn't make it into the first EF release:

The database architects and researchers at Microsoft have been working on an innovative mapping architecture which aims to address the challenges identified in the previous sections. It exploits the following ideas:

• Mappings are specified using a declarative language that has well-defined semantics and puts a wide range of mapping scenarios within reach of non-expert users.
• Mappings are compiled into bidirectional views, called query and update views, that drive query and update processing in the runtime engine.
• Update translation is done using a general mechanism that leverages view maintenance, a mature database technology [5, 13].

The new mapping architecture enables building a powerful stack of mapping-driven technologies in a principled, future-proof way. Moreover, it opens up interesting research directions of immediate practical relevance.

It has been implemented in the upcoming release of the ADO.NET Entity Framework, which provides a mapping-driven data access layer for .NET applications. [Emphasis added.]

It appears that view maintenance turned out to be a bit harder to implement than Sergei, Mike Pizzo and the ADO.NET team estimated. LINQ to SQL now supports updates with dynamic SQL and stored procedures, but it offers only one-to-one mapping of tables to entities and has limited inheritance support. (The Beta 1 version also has a few bugs in its stored procedure support).

Abandon eSQL and Move to LINQ 2.0 for Updates and Queries?

Here's Erik's grand plan for LINQ 2.0, O/R mapping, and updatable views:

Just as we provided deep support for XML in Visual Basic, in LINQ 2.0 we hope to directly support relationships and updatable views at the programming-language level. In that case, we only need a very thin layer of non-programmable default mapping at the edge between the relation and object world and allow programmers to express everything else in their own favourite LINQ-enabled programming language. The result is that just as LINQ 1.0 side-stepped the impedance mismatch “problem” with something better (monads and monad comprehensions), LINQ 2.0 will sidestep the mapping “problem” with something better (composable and programmable mapping).

If LINQ 2.0 can deliver "composable and programmable mapping," why add DML commands to a non-standard, proprietary SQL dialect—eSQL? If LINQ 2.0 is two years away, as Erik estimates, and EF's RTM has been delayed until 2007H1 (or later), why not just let EF [N]hibernate for another year, then release with LINQ 2.0 for Entities?

However, if "we only need a very thin layer of non-programmable default mapping", perhaps Erik doesn't intend LINQ 2.0 to supplement EF and LINQ to Entities, but instead to replace them. I don't consider three layers of complex XML mapping documents to be "a very thin layer."

The First Round

In his original "Democratizing the Cloud" session at QCon 2007 Conference (London) on March 15, 2007, Erik introduced the term LINQ 2.0 but concentrated on automatically refactoring data intensive apps to n-tier projects for the Web that support all client devices. None of the bloggers who reported on Erik's session covered it thoroughly, but my "LINQ 2.0" in Early Returns from QCon 2007, London post has links to as many as I could find at the time. Tim Anderson's Getting to grips with LINQ 2.0 review is the most comprehensive.

In round 2, Erik reiterated his dynamic refactoring proposal, and added data concurrency and synchronization to the mix. His overview doesn't mention how or whether Plinq and Synchronization Services for ADO.NET fit into his concurrency and synchronization vision. However, my take is that "composable and programmable mapping" has the potential to offer more immediate benefit to data-oriented .NET developers than tool-driven n-tier/device refactoring or solving concurrency/sync problems.

Round 3

The next round of "LINQ 2.0: Democratizing the Cloud" is scheduled for June 14, 2007 at DevDays Europe 2007 in Amsterdam's RAI convention center. In addition to Erik, DevDays features a panoply of .NET heavy hitters, such as Scott Guthrie, Francisco Ballena, Dino Esposito, Ingo Rammer, Gert Drapers, Jan Thielens, Daniel Moth, Tim Sneath, and Alex Thissen. Molly Holzschlag will be "Gazing into the Future of Web Development."

Here are the LINQ and Entity Framework-related sessions at DevDays Europe 2007:

Round 4

On June 15, 2007, Erik's giving the same presentation as an invited talk (keynote) at XMIME-P 2007, the 4th International Workshop on XQuery Implementation Experience and Perspectives, in Beijing. Erik's Amsterdam presentation is scheduled for June 14, 2007. (Hope he makes it to Beijing in time. Erik says his 9:20 PM flight from AMS to PEK will get him to the the airport at 8:55 AM. Hope he gets to the venue on time. (PEKing airport is about 10-12 miles from Beijing's center.) Update 6/16/2007: Erik says he "made it to his hotel before 9:00 AM, checked in, took a shower, and spent a nice day at the workshops. Both talks went very well." Apparently, his route provided a substantial tailwind.)

Here's the blurb:

The web is rocking the world of developers. Our customers love consistency. They want to have the same rich experience, anywhere, any time, on any device. Our sales people love market share.They want no platform that cannot leverage their web services. We as developers have embraced agile methods. We want to keep our options open as long as possible and create software incrementally by successive refactorings. This surely sounds like a contradiction, another impossible triangle just like infamous Relations-Objects-XML trio we tackled with LINQ 1.0. As the Dutch artist MC Escher once said "Only those who attempt the absurd will achieve the impossible". With LINQ 2.0, we are trying to stretch the .NET framework to cover the Cloud such that it will become possible to incrementally and seamlessly design, develop, and debug complex distributed applications using your favorite existing and unmodified .NET compiler and deploy these applications anywhere.

Thanks to Mike Champion (still writing for the the XML Team) for bringing Erik's round 2 presentation to my attention. I believe this is the first of his recent LINQ presentation I didn't catch before Erik gave it.