Thursday, December 14, 2006

Anders Hejlsberg's Recent LINQ/C# 3.0 Presentations

Anders Hejlsberg, Microsoft Technical Fellow and chief architect of the C# language, gave two LINQ-related presentations at Tech•Ed Developers 2006 (a.k.a. Tech•Ed Europe 2006) held in Barcelona on November 7 to 10. Microsoft EMEA recently posted videos and slides of the two sessions on their "ShOtime" site. There isn't much new in the presentation contents but—if you're new to LINQ and C# 3.0—you should find the topics interesting. Visual Studio: The .NET Language Integrated Query (LINQ) Framework Overview

Modern applications operate on data in several different forms: Relational tables, XML documents, and in-memory objects. Each of these domains can have profound differences in semantics, data types, and capabilities, and much of the complexity in today's applications is the result of these mis-matches. Anders Hejlsberg, Technical Fellow and chief architect of the C# language will explain how the Orcas release of Visual Studio aims to unify the programming models through LINQ capabilities in C# and Visual Basic, a strongly typed data access framework, and an innovative Application Programming Interface (API) for manipulating and querying XML.
Length: 1:05. Download slides here. Main session topics:
  • The LINQ Project
  • Demo: LINQ to Objects
  • C# 3.0 Language Innovations
  • LINQ to SQL and XML
  • Demo: LINQ to SQL and LINQ to XML
  • LINQ Architecture & Components
C# 3.0: Future Directions in Language Innovation

Join Anders Hejlsberg for an in-depth walkthrough of the new language features in C# 3.0. Understand how features like extension methods, lambda expressions, type inference, and anonymous types make it possible to create powerful Application Programme Interfaces (APIs) for expressing queries and interacting with objects, XML, and databases in a strongly typed, natural way.

Length: 1:06. Download slides here. Main session topics:

  • The Evolution of C#
  • Extension Methods and Lambda Expressions
  • Demo: Lambda Expressions and Extension Methods
  • Initializers and Query Expressions
  • Anonymous Types and Expression Trees
  • Demo: Expression Trees
  • Automatic Properties and LINQ Technical Preview

Technorati Tags: Orcas, LINQ, C# 3.0

Tuesday, December 12, 2006

Microsoft's Data Access APIs: ODBC to LINQ

Mike Pizzo, an Architect on the Data Programmability team at Microsoft, has started a four part series that captures the history of Microsoft's alphabet soup of data access APIs and the layout of acronyms to come. Here are the topics, with a link to the first two:

I'll update this post as new episodes hit the blog.

Commentary

It's back to the "early years" for Access 2007. The Access team now recommends abandoning Access Data Projects (ADPs) with SQL Server tables connected by OLE DB to DocObject front ends and ADODB in favor of old-timey ODBC-linked tables and DAO.

Here's Mike's take on ODBC and DAO in Access:

And then something happened. Visual Basic became popular as a scriptable "automation language". ODBC, being a C-style interface, was not directly consumable from VB. However, some of you clever folks figured out that Microsoft Access supported executing queries against ODBC Datasources, and that Access did support scriptable automation through its Data Access Object (DAO) API. Voila! Now you could write applications against ODBC sources using VB.

However, DAO went through Access's internal "Jet" (Joint Engine Technology) database engine, which defaulted to building local keysets for each result in order to do advanced query processing and cursoring against the remote data. This was fine if you needed that functionality, but significant performance overhead and additional round trips when you didn't.

Déjà vu all over again! DAO is the only API that supports the new Access Attachment data type, multi-select lookup fields, and "rich-text" and append-only memo fields of SharePoint lists.

So the Access team now suggests that developers abandon SQL Server 2005 Express back ends in favor of links to non-relational Windows SharePoint Services (WSS) 3.0 or Microsoft Office SharePoint Services (MOSS) 2007 lists.

Updated: 12/14/2006 for Part II; 12/24/2006 for Part III; 1/24/2007 for Part IV

Saturday, December 09, 2006

Jon Udell Moves from InfoWorld to Microsoft

Veteran technology writer, analyst and eclectic Web developer Jon Udell will cease writing his "Strategic Developer" column and "Jon's Radio" blog for IDG's InfoWorld magazine on December 15, 2006. After a month's hiatus, he'll start his new job as a Microsoft "evangelist for the Net." Jon will report to Jeff Sandquist who runs Microsoft's Channel9 and On10 sites. Jeff Sandquist is Robert Scobel's former boss. Note: Robert worked for Fawcette Technical Publications when I wrote my first article for Visual Basic Programmer's Journal, now Visual Studio Magazine. Jon described his future activities at Microsoft on December 8, 2006 in A conversation with Jon Udell about his new job with Microsoft:

That means blogging, podcasting, and screencasting on topics that I think are interesting and important; it means doing the kinds of lightweight and agile R&D that I've always done; and it means brokering connections among people, software, information, and ideas—again, as I've always done.
He went on to clarify his role and expectations in a new JonUdell WordPress blog. He'll continue working from his office in Keene, NH and spend about one week per month in Redmond. Jon as an XML Evangelist I worked with XML Web services in the early .NET 2002 days, wrote a book on the topic and ended up one of the winners of a Microsoft .NET Web Services contest. But it was Jon's early promotion of semi-structured XML for data interchange that got me interested in XML as a data-interchange format with Jean Paoli's InfoPath and resulted in my writing Introducing Microsoft Office InfoPath 2003 for Microsoft Press. Over the almost five-year period from 1/3/2002 to 11/15/2006, Jon wrote 148 columns, blog posts, analyses, and reviews that contained the term XML. Note: I ran this InfoWorld power search to return a table with rows for each article, imported the table into Excel and then into Access to improve searching and sorting. Jon's articles and posts on XQuery encouraged me to abandon Transact-SQL for a few days and try Microsoft's original XQuery implementation for .NET 1.0—an interactive preview of and downloadable code for the Microsoft.Xml.XQuery classes, which was at http://131.107.228.20 in mid-2002. The interactive demo—shown in the screen capture below—and its use cases worked well enough that I wrote a "Get Ready for XQuery" article and a downloadable .NET Web form app for Fawcette's XML & Web Services Magazine (free registration required for page 2 and beyond).

(Click screenshot above for full-size image.)

Jon's November 15, 2006 "Strategic Developer" column, XQuery and the power of learning by example, stresses the importance of use cases as XQuery tutorials and laments the lack of use cases in Extensible Business Reporting Language (XBRL). Notice the start of a list of W3C Use Cases in the left frame of the preceding Web page.

Microsoft's removal of XQuery support from .NET 2.0 was a major disappointment for me, but the SQL Server 2005 xml data type's XQuery support compensates—at least in part—for its 11th-hour excision from the .NET Framework. Jon's April 26, 2005 A conversation with Michael Rys about SQL Server 'Yukon' blog post provides a succinct overview of SQL Server 2005's XML and XQuery support and is a great companion piece to my slightly earlier "Exploit Yukon's XML Data Type" article. Jon's other XQuery articles greatly influenced Chapter 12, "Exploring the XML Data Type," of Expert One-on-One Visual Basic 2005 Database Programming.

Note: Now that XQuery 1.0 is a W3C Proposed Recommendation, hopefully Microsoft will implement the missing Let keyword and incorporate the XQuery classes as an Orcas add-on to the .NET Framework. Although LINQ to XML (formerly XLinq) and LINQ to XSD (a.k.a. LINQ to XML Objects) has a more elegant, SQL-like query syntax, XQuery is finally closing in on Recommendation status. Jon on Language Integrated Query (LINQ) Jon was one of the early proponents of Microsoft's LINQ technologies. He first wrote about LINQ in his September 13, 2005 An interview with Bill Gates from PDC 2005 blog/podcast, which includes a transcription. His initial take on LINQ:

The biggest round of applause, I think, went to Anders Hjelsberg, whose LINQ technoloogy was shown issuing SQL-like queries, as native C# syntax, to query in-memory CLR objects, to query SQL tables, and to perform joins between both. On the XML side, LINQ was used to construct XML, though not to query it—I'll want to ask Anders about that when we meet later this week.

He followed up with a September 21, 2005 Microsoft driving toward .Net unity "Strategic Developer" column based on PDC 2005 presentations:

LINQ is an extraordinary innovation that turns traditional query inside out. But the bigger story from PDC 2005 is that the .Net vision of unifying many balkanized disciplines within the Microsoft ecosystem is finally becoming a reality.

His LINQ 101 blog post a week later demonstrated the code for a three-way join across an XML data source and two CLR objects. His conclusion:

Could you the same thing in XQuery? Sure, given three sources of equivalent XML data. And in fact the preview includes implementations of the XQuery use cases. LINQ's home base, though, is the domain of CLR objects. From there it extends its generalized query operators into the realms of objects, XML, and relational data. All of the capabilities of the .NET Framework are available within -- and across -- these contexts. What's more, you can invent new query operators using C# and the Framework.

When I said that LINQ turns query inside out, I wasn't kidding. There is a lot here to think about and explore.

Jon got his chance to interview Anders Hjelsberg for his May 12, 2006 A conversation with Anders Hejlsberg about the May 06 preview of LINQ podcast (with transcript). Here's Jon's conclusion:

I'm not certain myself how much of LINQ is genuinely new, and how much is a creative synthesis of pre-existing ideas, but I do know that I'm fascinated by what it can do and where it might go. Its reach will ultimately be determined by the extent to which non-Microsoft databases of all flavors—relational, object, XML—are willing and able to plug into the LINQ framework. To that end, one of the most significant points in today's interiew is the announcement of IQueryable, an interface that formalizes how providers plug into LINQ. Watching the uptake of this mechanism by third parties over the next year or so will tell us a lot about whether LINQ will be a Microsoft-only phenomenon or something that more broadly influences the industry.

Well said. I'd like to hear Jon's thoughts on LINQ for XSD, despite it's Alpha 0.1 preview status, and PLINQ, which doesn't yet have a publicly available implementation.

So here's hoping that Jon won't expend all his Channel9 and On10 efforts on the Web, social networking, public libraries, and the like, but will continue evangelizing XML, XSD, XSLT, XQuery, and LINQ and analyzing trends in these technologies in his new role at Microsoft.

And here's wishing Jon best of luck at Microsoft. As I said in my comment to Jon's blockbuster post: "It’s by luring folks of your stature that Microsoft succeeds."

Technorati Tags: Jon Udell, XML, XSD, XML Schema, XQuery, LINQ to XSD, LINQ to XML Objects, LINQ to XML, XLinq, LINQ to SQL, DLinq, C# 3.0, VB 9.0, Orcas

Note: I've added a Jon Udell tag to all OakLeaf blog posts the mention Jon.

Orcas RTM to Slip from 2007 to 2008?

Will the proliferation of LINQ to Whatevers, Entity Framework/Data Model accoutrements, ASP.NET Ajax extensions, and other "hot new features" push Orcas's RTM date into 2008? Will Orcas become another Windows Vista? My May 15, 2006 Scott Guthrie on Orcas RTM Dates and LINQ with ASP.NET post quoted a May 14 ScottGu post:

We are looking to ship [Orcas] the second half of next year. We will start having full Orcas CTP drops (of all technologies) starting later this summer, and will also have a go-live license of Orcas before the final RTM date. So not too far off now when you can use the above [LINQ]techniques in production.

Scott gave Orcas another go for 2007 in his September 11, 2006 "Atlas" 1.0 Naming and Roadmap item:

Things will get even better next year with Visual Studio “Orcas” where we are adding rich JavaScript [I]ntellisense, debugging and WYSIWYG designer support for the ASP.NET AJAX Extensions within Visual Studio and many other great features to take advantage of.

But Mary Jo Foley contends in her recent "Foley on Microsoft" column for Redmond Magazine, "Microsoft Prognostications: What's Up in 2007?"...
Visual Studio "Orcas" will slip into 2008. While most developers and partners with whom I've spoken consider Visual Studio "Orcas" to be a 2007 deliverable, few Microsoft developer division officials have called the product "Visual Studio 2007." I'm hearing rumblings of an early 2008 product.

She calls the prognostication "On the 'relatively reasonable'" side in the Let the 2007 Microsoft prediction season officially begin post for her "All About Microsoft" ZDNet blog.

Following up in a December 7, 2006 in The Microsoft dating game post, Mary Jo wrote:

I Googled. I Live-Searched. I Asked.com. I couldn't find any proof that any Microsoft official actually said the words "Orcas will ship in 2007" — at least in an on-the-record form. (If anyone else can find such a statement, please do share!)

As I and other Microsoft watchers know quite well, Microsoft has been setting expectations since 2005 that it plans to ship the next version of Visual Studio in 2007. No one at Microsoft has sent me a nasty note when I've noted in countless articles and blog posts that Orcas is expected to be a 2007 deliverable. And if past behavior counts for anything, the Visual Studio team has been good about churning out new tool-suite releases every two years.

I'm not certain that ScottGu qualifies as a Microsoft official; he's not in the Microsoft Executives list. But he said at least twice that Orcas was scheduled for a 2007 RTM, and he certainly convinced me. On the other hand, Mary Jo has a great batting average for reportorial accuracy in my book. Type Foley in the text box and then click SEARCH BLOG to find my prior references to her articles.
Stay tuned.
Subsequent events (12/12/2006) It appears that the party line is the second half of 2007. Here's an outtake from a 12/12/2006 LINQ to SQL chat from Microsoft's Amanda Silver:

AmandaS [MSFT] (Expert): Q: [19] any estimated release date [for Orcas]? ... [I] am waiting for any offcial date...

A: We're aiming to ship in the second half of next year. We've already started having fully integrated CTPs (with features in all areas of the product.)

Amanda is Lead Product Manager for Visual Basic. Her statement and the earlier quote from Scott Guthrie are too similar to be coincidental. (The transcript for the C# LINQ to SQL chat is here.)

Technorati Tags: Orcas, LINQ, C# 3.0, VB 9.0, Atlas, Ajax

OakLeaf Blog Gets Categories and Atom 1.0

I've just switched the OakLeaf blog to the new Blogger Beta version that supports categories, Atom 1.0 (finally), and RSS 2.0. All posts on and after June 13, 2006 have detailed categories (Blogger calls them Labels) collections at the end of the post, most of which duplicate the Technorati links. Earlier items have major category labels, such as LINQ or SQL Server. Categories let some RSS/Atom feed readers, such as Internet Explorer 7.0, filter items by category. The following screen capture shows IE 7 Feeds view's scrolling listbox with the LINQ to XSD label selected.

Note: One of the things I like about IE 7's Feeds view is that its chronology is based on Last Updated date/time, not the original post's date/time. This means that updated posts percolate to the top of the list; many of my posts are updated two or three times.

Here's Dare Obasanjo's three-pane, open-source RSS Bandit aggregator (Jubilee Edition Beta 1) displaying the preceding post:

Windows Live Writer (WLW) beta 1.0.1(6) works with Blogger Beta, as shown by the WYSIWYG screen capture below: Note: You must upgrade WLS to version 1.0.1(6) or later to work with Blogger Beta after early November 2006. However, WLW 1.0.1(6) doesn't support adding Blogger Beta Labels as tags. For the moment you'll need to do a final edit in the Web-based Blogger editor, which hasn't changed significantly in the Beta version. Technorati Tags: , , ,

Friday, December 08, 2006

PLINQ Gathers Momentum

Parallel Language Integrated Query or Parallel LINQ (PLINQ) finally emerged from the shadows with Microsoft Program Manager Joe Duffy's December 1, 2006 "A new role for me at Microsoft" post:

I recently left the CLR team to join a new team focusing on parallelism on Microsoft's platform in the 3-10 year timeframe. I am leading design and development of the Parallel LINQ (PLINQ) project that I alluded to here.

We're looking for supersmart technical people to join the team and help change the face of programming for anybody writing code on the CLR or VC++. PLINQ isn't the only project. Solid CS skills are a must, but you don't necessarily have to be a concurrency guru (right away).

These [22] job postings have some detail. If something catches your eye, respond on the jobs site or send me your resume directly at joedu AT you-know-where DOT com (i.e. microsoft DOT com).

I'll of course still be blogging about everything concurrency related, so you won't notice much of a difference content-wise. And I'm still happy to answer any CLR related questions and help you find the right folks inside the team to listen to your feedback.
Joe is the author of an MSDN Magazine January 2006 "{END BRACKET}" column, "Transactions for Memory," and September 2006 "CLR Inside Out" column, "Using Concurrency for Scalability." There's no mention of LINQ in either article, but the handwriting was on the wall.

The first concrete information about PLINQ came from an August 28, 2006 eWeek article, "Microsoft's PLinq to Speed Program Execution," by Darryl K. Taft, who quotes Anders Heljsberg:

"With PLinq, effectively you write the code the same way, but we arrange for it to run on multiple CPUs," Hejlsberg said in an interview at Microsoft in early August. "So the queries get split up and run on multiple CPUs, and then you just wait for all the results to arrive. And lo and behold, without any changes your program just ran six times faster. It's instant gratification. ...

"It doesn't necessarily mean that every program is going to run six times faster, but there are classes of programs that we can make run faster without the user having to get into the nitty-gritty business of writing concurrent constructs."

Danny Thorpe, formerly Borland's Chief Scientist and now Microsoft Sr. Product Manager/Architect, noted in a September 2005 blog post announcing Borland's Delphi would support LINQ, that LINQ had parallel computing potential. Taft's September 27, 2005 eWeek article, "Borland Plans to Support MS LINQ in Delphi Platform" quotes Danny:

"LINQ presents an enormous opportunity to put parallel computing in a package that everyday coders can use safely and easily. Exploit those multicore processors! Even qrid computing is within reason."

I don't usually write about projects that don't have at least at least a public preview or alpha version available for testing. However, it appears that Microsoft is staffing to deliver PLINQ in the Orcas timeframe. So following the PLINQ story looks worthwhile.

Don Domsak's recent Parallel Language Integrated Query (PLINQ) post adds fuel to the PLINQ fire with this conclusion:

Overall, this stuff looks very cool. And it seems that LINQ is going to be a big part of how they plan to implement it, giving me even more of a reason to get going on LINQ before all the cool kids realize that WPF, WCF and WF are yesterday's news, and LINQ is where it is at ;).

Note: Don Domsak, better known as DonXML is a Microsoft MVP for XML and a VB .NET fan. Don recently gave presentations on LINQ and the Entity Framework at the Heartland Developer Conference (HDC) 2006.

Question of the day: Will the PLINQ moniker morph to initial-letter-capped Plinq (a la Blinq), PLinq (per the original DLinq and XLinq API names), or adopt the LINQ to Whatever style as LINQ to Cores?

Update: 7/30/2007: Mary Jo Foley posted What will next-generation multicore apps look like? today, which quotes the presentation that Craig Mundie, Microsoft's Chief Research and Strategy Officer, gave to last week's Financial Analysts Meeting. There's no mention of PLINQ in the transcript that Mary Jo cites or the PowerPoint slides. It's surprising that PLINQ wasn't part of the presentation because it's probably Microsoft's first concrete implementation of declarative languages for multi-core processing.

Update 1/23/2007: Joe Duffy updated his post about PLINQ talk at the Declarative Aspects of Multicore Programming (DAMP) workshop he gave at POPL 2007 earlier in January by adding a link to his slides.

Update 12/12/2006: Anders Hejlsberg discusses PLINQ, concurrent programming, and LINQ for Orcas in an 8:29 interview from Tech•Ed: Developers 2006 Barcelona. The page also includes links to Eric Rudder's 1:15:00 Windows Vista, 2007 Office System and AJAX keynote, and brief interviews with Scott Guthrie, Paul Andrew, and Keith Smith.

Technorati Tags: Orcas, PLINQ, LINQ, C# 3.0, VB 9.0

Thursday, December 07, 2006

Reduce Object/XML Impedance Mismatch with LINQ to XSD

LINQ to XSD from Microsoft's XML Team promises a similar reduction of the impedance mismatch between .NET objects and XML documents as LINQ to Entities and ADO.NET 3.0's Entiry Framework and Entity Data Model deliver for objects and relational data.

LINQ to XSD, which entered the public preview state on November 27, 2006 with the release of the Alpha 0.1 version, builds on LINQ to XML's generic XML trees to operate on typed XML trees, which are "instances of .NET types that model the XML types of a specific XML schema (XSD)." Thus LINQ to XSD carries the subtitle "LINQ to XML Objects."

Note: If you don't want to download the preview bits, which require the Microsoft .NET LINQ CTP (May 2006), read the "LINQ to XSD Overview: An Incubation Project on Typed XML Programming" white paper.

Updated: February 7, 2008

A First Look at LINQ to XSD Syntax

The XML Team's blog post and documentation for the LINQ to XML preview cites this prototypical LINQ for XML query that returns the total value of Item elements in in a purchaseOrder document:

(from item in purchaseOrder.Elements("Item") select (double)item.Element("Price") * (int)item.Element("Quantity")).Sum();

Here's the strongly typed LINQ to XSD version of the preceding query:

(from item in purchaseOrder.Item select item.Price * item.Quantity ).Sum();

Obviously, the type-safe version of the query's syntax is clearer. It also provides Intellisense for elements and corresponds more closely to a LINQ to SQL query against a purchaseOrder table with a related Item table.

Aside from the obvious syntatical benefits and compile-time vs. run-time syntax checking, LINQ to XSD supports contract-first rather than code-first development. Many Web services developers are contract-first devotees, but Dare Obasanjo—one of Microsoft's foremost XML/XSD gurus—recommends the code-first approach in his April 22, 2005 "Contract-First XML Web Service Design is No Panacea" post.

Note: Click here for a list of Dare's technical papers, articles, and other writings as of May 2005.

LINQ to XSD can handle code-first (or, more properly, document-first) design by inferring and refining XML schemas from existing XML documents with the Microsoft XSD Inference 1.0 tool. My "Generate XSD Schemas by Inference" article for the November 2002 issue of XML Magazine explains how to get the most out the tool. (Free registration is required to read page 2 through 4).

O/X Impedance Mismatch Background

My September 25, 2005 post, "More on Object-XML and Object-Relational Impedance Mismatch," cited Dare's "The Impedence Mismatch between W3C XML Schema and the CLR" post, which discusses XML serialization of object state and conversion of XML to objects. Dare suggests his earlier "XML Schema Design Patterns: Is Complex Type Derivation Unnecessary?" post and the "Programming with Circles, Triangles and Rectangles" paper by Gavin Bierman, Wolfram Schulte and Erik Meijer for more details on X/O impedance mismatch issues. This paper extends an earlier presentation to the Workshop on Declarative Programming in the Context of OO Languages (DP-COOL) 2003—Unifying Tables, Objects and Documents by Eric Meijer and Wolfram Schulte.

Note: The BSM trio also presented "LINQ: Reconciling Objects, Relations and XML in the .NET Framework" to SIGMOD 2006. Unfortunately, only the first page is available for download.

Apparently, Erik and his colleagues are sanguine that the longstanding issues with X/OM (XML-to-Object mapping) can be resolved. He and Data Progammability Team program manager Ralf Lämmel authored "Revealing the X/O Impedance Mismatch" (July 19, 2006 draft) for submission to and possible publication in the post-proceedings for Datatype Generic Processing (DPG '06), Lecture Notes in Computer Science (LNCS, Springer US). The paper begins with an excerpt from Dave Thomas's "The Impedance Imperative: Tuples + Objects + Infosets = Too Much Stuff!" article for the September-October 2003 issue of the Journal of Object Technology.

Another paper, "Mappings make data processing go ’round: An inter-paradigmatic mapping tutorial," by Erik and Ralf (2005) starts with the same Dave Thomas quotation. The abstract describes the tutorial's objective:

Each data-processing application depends on a horde of interrelated data models and artifacts that are derived from data models (such as data-access layers). Such conglomerations of data models are challenging due to paradigmatic impedance mismatches, performance requirements, loose-coupling requirements, and others.

This ubiquitous problem calls for a good understanding of techniques for mappings between data models, actual data, and operations on data. This tutorial lists and discusses mapping scenarios, mapping techniques, impedance mismatches and research challenges regarding mappings. [Emphasis added].

Note: Dave Thomas and Erik Meijer are speakers for the QCon Conference to be held March 12 - 16, 2007 in London. Erik and Martin Fowler are among the conference's keynoters.

Ralf presented two papers at XML 2006 on December 5, 2006: "Typed XML programming without abandoning XML" (a.k.a. "Functional OO Programming with Triangular Circles" with Dave Remy as co-author) and "XML streaming, XML trees and XML queries—Can they co-exist in harmony?" (by Michael Champion). Mike chaired the Enterprise XML Computing sessions and provides a blow-by-blow description of the December 5 and part of the December 6 sessions in his XML 2006 Observations post.

Note: Dave Remy said on June 16, 2006: "I am now working hands on a very cool XML developer tool (can't say specifically what quite yet) that uses WPF (also known as Avalon) and XLinq extensively. I look forward to doing some future blogs in these areas." (To my knowledge, no future blogs on the topic appeared as of early 2008.)

Charting the Course of LINQ to XSD in Orcas

The more general tutorial, "Mappings make data processing go 'round—An inter-paradigmatic mapping tutorial," "... lists and discusses mapping scenarios, mapping techniques, impedance mismatches and research challenges regarding mappings" with the emphasis on O/RM and X/OM. This paper is included in the Summer School Proceedings of the Generative and Transformational Techniques in Software Engineering (GTTSE) 2005, of which Ralf was an organizer.

Ralf offers the "XML Typed Programmer" quartet with the following posts, some of which are quite detailed (i.e., lengthy):

Note: The current LINQ to XSD implementation is C# 3.0 only, which is surprising when you consider Erik's penchant for Visual Basic and his recent VB IsNot C# seminar at Expo-C 2006.

Tech support for LINQ to XSD is from the XML and the .NET Framework newsgroup, not the expected LINQ Project General newsgroup.

LINQ to XSD is an "incubator" project and the preview comes with a "While the underlying LINQ to XML technology is expected for Orcas, no such commitment exists for LINQ to XSD" caveat. But it seems to me that LINQ to XSD is a necessary addition to mainstream LINQ to SQL (formerly DLinq) and LINQ to XML (formerly XLinq) APIs, and developing LINQ to Entities and LINQ to DataSets technologies. Without LINQ to XSD, the final Orcas LINQ implementation will be incomplete.

Updated: 12/8/2006 and 12/11/2006 Minor additions and corrections. Reformatted: 2/7/2008.

Technorati Tags: LINQ to XSD, LINQ to XML Objects, LINQ to XML, XLinq, C# 3.0, VB 9.0, XML, XSD, XML Schema, Orcas, Domain-Specific Languages, DSL, DSL Tools

Saturday, November 25, 2006

Orcas December 2006 CTP Coming with New LINQ/EDM Bits

Visual Studio Orcas CTPs will replace ADO.NET vNext CTP drops starting with the Orcas December 2006 CTP. Charlie Calvert, Microsoft's Community Program Manager for the C# group, reports in his November 21, 2006 Community Convergence XII post:

The Orcas December CTP will contain new LINQ bits. The team has been churning out daily builds filled with new LINQ content. Over the next weeks it will all come together in a CTP that will be released to the public. This will be the first major update to the LINQ bits since the May CTP. The first Orcas beta is not due until 2007.
Charlie's post follows ADO.NET Technical Lead Pablo Castro's November 7, 2006 ADO.NET Orcas and Samples item:

As we make progress on the Orcas release of Visual Studio, the various teams—including ourselves—are working hard at integrating everything in a single product, Visual Studio Orcas. For the ADO.NET Entity Framework this means that you won’t see component-specific CTPs any more for Orcas, and instead you’ll see Orcas CTPs that have all these technologies incorporated. We may still do separate CTPs of related technologies if we see that it would help at some point in time.

Also, since the ADO.NET Entity Framework is in the Orcas builds, we don’t need the generic “vNext” token for it; from now on we’ll refer to it simply as ADO.NET Orcas, which will include the ADO.NET Entity Framework and LINQ to ADO.NET in its various flavors (LINQ to DataSet, LINQ to SQL and LINQ to Entities).

The December CTP release undoubtedly will integrate updated versions of the the ADO.NET Entity Framework from the October 2006 Orcas CTP and its sample code plus the the August 2006 ADO.NET vNext CTP's Entity Data Model (EDM) designer prototype.


Here's the latest (December 8, 2006) update on the Orcas December CTP 2006 release from the Visual C# Community Center:
The December Orcas CTP will be released soon. This CTP will be a regular install, and will not ship inside a Virtual Machine. It will co-exist with Visual Studio 2005. The main feature in the CTP will be the ability to work with LINQ. These will be the first new LINQ bits we have seen since the May CTP. The December CTP is a pre-beta release. The Orcas Beta is not expected until sometime in 2007. The team is also near the release of Service Pack 1 for Visual Studio 2005. Right now, I'm still exploring this issue, but I've heard that the order in which we install Service Pack 1 and the LINQ May CTP may be important. The reports I'm hearing at this time suggest that we should install the service pack first, and then the May CTP. I should have more information on this issue by the end of the week.

I believe Charlie meant "December CTP" where he mentions "May CTP" in the preceding paragraph.


Charlie has posted the first four five installments of The LINQ Farm, a series of articles for beginning LINQ C# programmers:

  1. LINQ for Beginners (10/29/2006)
  2. Query Expressions (10/31/2006)
  3. Query Operators (11/11/2006)
  4. Using Distinct and Avoiding Lambdas (11/19/2006)
  5. Focus on Grouping (12/2/2006)

He avoids deep discussions of lambdas, which makes the articles useful for VB developers also. Charlie points to a video and some recent articles of interest to LINQ and ADO.NET EF developers:

Here are a few links to recent Channel9 videos that might interest LINQ developers:

Updated: 12/7/2006 with Charlie Calvert's fifth episode of the LINQ Farm: Focus on Grouping. 12/8/2006 for potential Orcas issues with VS 2005 SP1.

Technorati Tags: Orcas, ADO.NET vNext, Entity Framework, Entity Data Model, LINQ, LINQ to Entities, LINQ to DataSets, LINQ to SQL, Entity SQL, eSQL, DLinq, XLinq, C# 3.0, VB 9.0, ObjectSpaces, Domain-Specific Languages, DSL, DSL Tools

Wednesday, November 15, 2006

Microsoft Offers SQL Server 2005 SP2 CTP

Microsoft announced on November 7, 2006 the availability of a Communtity Technical Preview (CTP) of the forthcoming SQL Server 2005 Service Pack 2 for all editions except Express, and special SP2 CTPs for the Express edition (SQLX) and Express with Advanced Features. SP2 will be required to run any version of SQL Server under Windows Vista and "Longhorn" Server. The "What's New in SQL Server 2005 SP2" page summarizes upgrades to:

  • Analysis Services
  • Database Engine
  • Integration Services
  • Reporting Services and
  • SharedTools
for editions that incorporate the features. Download the appropriate SP2 CTP bits from the SQL Server Community Technology Preview Program page. Microsoft senior vice president of Data and Storage Platforms Paul Flessner expanded on the preceding list during his keynote speech at the 2006 Professional Association for SQL Server (PASS) Community Summit in Seattle on November 15, 2006. Added features include:
  • Data compression
  • Increased business intelligence functionality
  • Security updates relating to Common Criteria
  • Manageability enhancements
  • Support for Windows Vista and
  • Optimization for use in the 2007 Microsoft Office system environment
Microsoft is pursuing Common Criteria security certification for SQL Server 2005 Enterprise Edition SP1 and SP2. Many governments require Common Criteria certification and it's a preference or requirement for many industries. In the meantime, Microsoft has made available Enterprise Strategy Group (ESG) reports on SQL Server 2005 security—Microsoft SQL Server Runs the Security Table and The Security Development Lifecycle (SDL). Advantage, Microsoft. According to ESG, "Microsoft SQL Server 2005 is particularly worth examining as it is the first major product release that Microsoft has put through the SDL." Flessner also announced that Visual Studio Team Edition for Database Professionals (formerly code-named "DataDude") is set to RTM on November 30, 2006.

Updated: 11/16/2006 with additional security data.

Technorati tags: , , , , , , ,

Thursday, November 09, 2006

ADO.NET Sync Framework for Occasionally Connected Systems

WiFi Internet connectivity is becoming ubiquitous in first-world urban areas, but there are classes of data collection/management applications that must be operable during wired and/or wireless network outages—emergency and health services, for example—or exhibit dramatically decreased user productivity without a live network connection—field sales workers come to mind. Some occupations are inherently occasionally connected: governmental, academic, and corporate field workers in rural locations; land surveyors; forest rangers; environmental scientists; marine biologists and the like. Seldom, occasionally, and usually-connected users need the ability to cache data locally and synchronize local data with large back-end databases. The preferred caching and synchronization model is Outlook 2003/2007, where clients always work with their local data store and the server only propagates data changes to the client. Following are the most common methods of client-side caching and synchronization:

  • ADO.NET Datasets persisted as XML files
  • Remote Data Access (RDA) for SQL Mobile/SQLce databases
  • Merge replication to SQL Server 2005 (not Express)
  • Occasionally Connected Systems (OCS)Sync Framework (future)
ADO.NET Datasets Microsoft's developers designed ADO.NET Datasets to support data management applications for usually, occasionally, or seldom-connected mobile users. Datasets persist dynamic (read/write) and a subset of catalog (read-only, slowly changing) relational data in XML files. Datasets replace combinations of merge and snapshot replication with a lighter-weight, XML file-based system that uses DataAdapters to communicate with back-end RDBMSes. I described disk and RAM resource consumption issues with large ADO.NET 1.1 and 2.0 Datasets in my May 2004 "Store Large Lookup Tables in DataSets" article for Visual Studio Magazine (reading page 2 requires free registration.) The article's Table 1 shows the time to load Datasets of various sizes from XML files Although multi-MB Datasets are practical for today's laptop, tablet, and ultra-mobile (portable) PCs, their resource consumption precludes use on Windows CE, PocketPC, SmartPhone, and similar devices. Single-file relational databases, such as SQL Server 2005 Compact Edition (SQLce), are better suited than Datasets for portable PCs and devices when data storage requirements exceed a few MB. Remote Data Access RDA is the basic application for synchronizing data between SQL Server Mobile or, more recently, SQLce—formerly called SQL Server Everywhere (SQLEv). RDA pulls data from a remote SQL Server 2000 or 2005 table to a client's SQLce table. The SQLce database keeps track of changes changes to the pulled data and, on demand, pushes the changes to the back-end server. RDA also can execute CRUD (CREATE, RETRIEVE, UPDATE, and DELETE) operations on the remote server with Transact-SQL (T-SQL) statements. SQL Server doesn't push changes in its data to SQLce; thus, RDA is a one-way replication technology.

RDA connects to the back-end database with the HTTP or HTTPS protocols to Internet Information Services (IIS) on the server and pass compressed data through firewalls. Custom C#/VB code or the Connection Manager application handles the details of the connection. RDA is the only method of replicating data between SQL Server 2005 Express (SQLX) and SQLce, because SQLX (unlike MSDE 2000) isn't a merge replication publisher. RDA can detect but doesn't handle concurrency or referential integrity conflicts, and schema changes aren't allowed.

Merge Replication Replication is the traditional means of updating data between SQL Server instances, and merge replication is the most common type. Merge replication uses a different Connection Manager to handle communication with the back-end database using HTTP or HTTPS protocols to pass compressed data through firewalls. Merge replication is two-way, supports referential integrity, handles concurrency conflicts, and allows schema changes. Merge replication, including subscription and publication, is a heavyweight process compared to RDA push/pull operations, but is quite reliable. OCS Sync Framework The OCS Sync Framework involves a SyncAgent on the client that uses a SyncTable to manage a ClientSyncProvider for each type of local store, such as SQLce or SQLX. Server-side components include a ServerSyncProvider, SyncGroup, SyncAdapter, and SyncAdapter commands. The SyncAdapter and SyncAdapter commands (DBCommand objects) are ADO.NET database-agnostic objects so synchronization can occur with SQL Server, Oracle, IBM DB2 or any other RDBMSes that have an .NET native data provider. IncrementalInserts, IncrementalUpdates and IncrementalDeletes enable clients to request the server to update catalog data. The client and server can push changes directly with SQL Server TDS or another RDBMS's wire protocol when a physical network connection is available. Otherwise the client can push and pull changes with a proxy that connects to a service. Synchronization methods can be:
  • Pull only from server (unidirectional)
  • Push only to server (unidirectional)
  • Push and pull (bidirectional)
  • Snapshot (no synchronization)

The OCS Sync Framework appears to fall midway between the lightweight RDA and heavyweight merge replication method in operational complexity and resource consumption, but the current (very early) implementation requires a substantial amount of client code to specify synchronization type and SQL statements for DBCommands.

Live From Redmond: Smart Client: Offline Data Synchronization and Caching for Smart Clients is Steve Lasker's 94-minute Webcast that covers Dataset, RDA, merge replication, SQL Query Notifications, and a preview of the OCS Sync Framework (requires free registration). Steve is a Microsoft program manager/technical lead and informal evangelist for OCS, Windows forms, VB .NET, and ADO.NET Datasets. Steve also delivered a more general Live From Redmond: Visual Studio: Developing Local and Mobile Data Solutions with SQL Server Compact Edition Webcast on December 4. Steve also offers an Occasionally Connected Systems slide deck and a screencast that demonstrates how the OCS Sync Framework operates and VB WinForm code to synchronize a client with SQL Server. Most public OCS-related information presently comes from Steve's blog. His "Tech Ed 06 Slides and Demos on SQL Server Everywhere Edition" post has links to DEV351: Developing Local and Mobile Data Solutions with SQL Server Everywhere Edition and DEV343: Offline Data Synchronization and Caching with SQL Server Everywhere Edition.

Update: 12/9/2006: 12/4/2006 Webcast name updated.

Technorati tags: , , , , , , , , , , , , ,

Wednesday, November 08, 2006

SQL Server 2005 Compact Edition RC1 Available

Microsoft accounced on November 7, 2006 the availability of SQL Server [2005] Compact Edition (SQLce) 3.1 Release Candidate 1 (RC1). SQLce—formerly named SQL Server Everywhere Edition (SQLEv) 3.1—is a lightweight, in-process cousin of SQL Server 2005 and SQL Server 2005 Express. SQLce is the successor to earlier SQL Server Mobile 3.0 and SQL CE 2.0 and 1.0 editions. You can download the SQLce RC1 runtime and readme, as well as SQLce RC1 Books Online. SQLce is an embeddable, small-footprint, single-user database for mobile and desktop applications. As an example, you can substitute SQLce for SQL Server 2005 Express (SQLX) when you need local storage for usually, occasionally or seldom-connected systems. SQLce uses the ADO.NET programming model, supports most SQL Server 2005 data types and T-SQL reserved words, and is freely distributable.

According to the "Microsoft SQL Server 2005 Compact Edition Overview" white paper, SQLce:

  • Occupies only 1.8 MB of disk space
  • Supports single-file databases up to 4 GB in size
  • Uses 128-bit SSL for secure and reliable functionality, even through firewalls
  • Uses 128-bit RSA file-based encryption on devices for database file security
  • Provides an encrypted data format with password protection
  • Will use the ADO.NET Sync Framework in Visual Studio vNext ("Orcas")

A separate OakLeaf blog post, "ADO.NET Sync Framework for Occasionally Connected Systems" has the details on Microsoft's new approach to SQLce synchronization that's coming in Orcas.

I wrote about Mobile Edition's morph to Everywhere Edition in my May 1, 2006 "SQL Server 2005 Mobile Edition Goes Everywhere" article for Fawcette Technical Publications Online's "Special Report: SQL Server." "SQL Everywhere" had a "déjà vu all over again" ring to me. It appears that I wasn't the only one non-plussed about the "Everywhere" suffix, which appeared to me to encroach on Sybase's SQL Anywhere trademark for a similar database product. Sybase also claims here that SQL Everywhere is their trademark. Here's what Microsoft has to say about renaming SQLEv to SQLce:

SQL Server 2005 Compact Edition, was previously named SQL Server 2005 Everywhere Edition during the product development phase. Customers have communicated to Microsoft through the pre-release community technology preview (CTP) program and other channels that this product name may cause potential confusion with other database product names in the industry. Microsoft has taken this feedback seriously and has accordingly changed the official product name.
Of course, this isn't the first time that Microsoft has done an about-face on product and architecture names. Remember .NET Everything? Steve Lasker's "SQL Server Everywhere Gets a New Name" post has more to say about the name change and SQLce's features in the forthcoming RTM bits. Steve is a Microsoft program manager/technical lead for Occasionally Connected Systems (OCS) and the forthcoming ADO.NET Sync Framework. Steve's Live From Redmond: Visual Studio: Developing Local and Mobile Data Solutions with SQL Server Compact Edition December 4, 2006 Webcast provides a detailed overview of SQLce's capabilities. His presentation includes a substantial amount of VB (!) ADO.NET code. (Free registration required.) Several comments about the post complain about SQLce's 4 GB size limit, which is a significant problem for local document storage. In a reply to a comment by John, Steve says, "Again, the current 4gb limit is purely a device optimization that we will definitely remove now that we’re running on larger desktop scenarios. ... The 4gb limit isn’t a 'free' restriction as we’ve done with Express." Microsoft's product page for SQLce hadn't been updated from SQLEv to SQLce terminology when this post was written. If you're working with the SQLEv CTP on devices, you can download the CTP for SQLEE Access Database Synchronizer to synchronize data between Access 2000 - 2007 databases and SQLEv CTP or SQLce RC1. There's more information on all these topic, plus details for the Microsoft SQL Server 2005 Compact Edition Tools for Visual Studio SP1 Beta, which Microsoft says updates the VS 2005 SP1 Beta smart device development components with the design time environment for SQLce, at the SQL Server Everywhere Edition [sic] blog. The SQLce newsgroup is here.

Updated 11/9/2006 with additional details about and links to SQLce, SQLEv, OCS and Sync Framework topics. 12/9/2006: Added 12/4/2006 Webcast link.

Technorati tags: , , , , , , , , , , , , ,

Wednesday, November 01, 2006

Objectify Data with ADO.NET vNext

The "Objectify Data with ADO.NET vNext" article from the October 2006 issue of Visual Studio Magazine explains how the ADO.NET vNext August 2006 Community Technical Preview reincarnates ObjectSpaces’ object/relational mapping (O/RM) technology and OPath query language as the Entity Framework (EF), Entity Data Model (EDM), Entity SQL (eSQL), and LINQ to Entities for the Visual Studio “Orcas” release. Here's a screen capture of the article's VB 9.0 sample test harness: The article's AdventureWorks SalesPersons bonus project (AWv3TestApp.sln) invokes the AdventureWorksV3 EDM class library to populate a DataGridView with records for sales personnel who are eligible for bonuses—five years employment by AW for this example. Clicking the Add or Deduct buttons increcments or decrements bonuses, persists the changes, and refreshes the grid. If you're interested in learning more about the ADO.NET team's new approach to O/RM check out these previous OakLeaf posts:

Click here for a page of links to earlier posts about LINQ, LINQ to SQL (DLinq), and LINQ to XML (XLinq)

Technorati Tags: ADO.NET vNext, Entity Framework, Entity Data Model, LINQ, LINQ to Entities, LINQ to DataSets, LINQ to SQL, Entity SQL, eSQL, DLinq, XLinq, C# 3.0, VB 9.0, Orcas, ObjectSpaces, Domain-Specific Languages, DSL, DSL Tools