Saturday, February 23, 2008

LINQ and Entity Framework Posts for 2/18/2008+

Note: This post is updated daily or more often, depending on the availability of new articles.

Alex James Posts a Teaser About Entity Framework Support for Associations with Payloads

As Alex mentions in his Associations with Payloads post of 7/24/2008, Entity Framework (EF)automatically creates a many:many association instead of an entity for a table that consists entirely of two (or more) foreign key fields.

If you add payload field(s) to the table, such as Quantity [,UnitPrice, Discount, etc.], EF treats the table as an entity with one:many and many:one associations with its related tables.

Alex says:

It turns out however that with a little jigger pokery in the CSDL, MSL and SSDL it does support them, sort of.

However he doesn't provide the "jigger pokery," which I assume is Kiwi for "hack" or the like, but says it's coming in the next post. Since his last preceding post was February 11, you might be in suspense for a couple of weeks.

Added: 2/24/2008

Update 2/26/2007: See LINQ and Entity Framework Posts for 2/25/2008+ for the outcome.

Scott Hanselman Uses LINQ to XSD as an Example of How Microsoft Projects  Appear to be Abandoned

In LINQ to Everything - LINQ to XSD adds more LINQiness of February 22, 2008, Scott says he hears the following about many small Microsoft projects:

"Is _______ dead? I haven't heard anything in a month from _____ team's blog! It must be dead.

and then goes on to describe the benefits of augmenting LINQ to XML with LINQ to XSD using the arcane Open Financial Exchange (Ofx) XML schemas. (Microsoft, CheckFree, and Intuit initiated Ofx, which appears to have a semi-active blog.).

I suggested in a comment:

Another way to indicate a project isn't dead is to respond to questions about its status in the appropriate MSDN forum.

I requested the status of LINQ to XSD for .NET 3.5 RTM in a LINQ Project General forum Status of LINQ to XSD for RTM? on November 17, 2007. Two others seconded my request.

Not a word from the XML team.

I'm trying again with a status request for Shyam Pather's LINQ to Stored XML implementation, but I'm not optimistic about the result.

See the "LINQ to XSD Alpha 0.2 for VS 2008 RTM Is Available for Downloading" topic below for more details and background on the new release.

Added: 2/23/2008

Jim Wooley Uses LINQ to XML to Query All Shakespeare Plays Very Quickly

Jim's Querying the complete plays of Shakespeare using LINQ to XML post of February 22, 2008 describes his quest to find with LINQ to XML the fifty characters (SPEAKERS) in 37 plays having the most LINEs. The total size of the 37 files he concatenated into a single collection of XElements is about 7.75 MB.

Jim says:

The amazing thing with this process, running all three queries here, including the one which loads the full XML from the various files takes less than a second. I haven't had time to do a full performance test, including memory load, but the initial results are quite impressive! [Emphasis added.]

I agree.

Added: 2/23/2008

LINQ to Objects and LINQ to XML Videos from Tech*Ed 2007 Developers Barcelona Start with Disclaimer

Charlie Calvert's Video of Luke Hoban's In-Depth Look at C# 3.0 post of February 22, 2008 points to videos of these two presentations:

These are the first videos of presentations from Microsoft events that start with a disclaimer. Fortunately, you don't need to choose the "I have read the disclaimer and agree to be bound by it" option to watch the segment. I'm waiting for the first conference that requires agreement to a disclaimer as a condition of registration.

Added: 2/23/2008

Scott Guthrie: Silverlight 2.0 Beta 1 Will Support LINQ to XML

Scott's First Look at Silverlight 2 post of February 22, 2008 says that his team is "shortly going to release the first public beta of Silverlight 2" and notes that:

It also includes LINQ and LINQ to XML library support (enabling easy transformation and querying of data), as well as local data caching and storage support.  The .NET APIs in Silverlight are a compatible subset of the full .NET Framework.

It's amazing that the Silverlight team has been able to incorporate such an amazing array of new features in such a small package:

The Beta1 release of Silverlight 2 is 4.3MB in size, and takes 4-10 seconds to install on a machine that doesn't already have it.

Added: 2/22/2008

Jon Skeet Offers a Animated LINQ to Objects Visualizer

Jon's Visual LINQ: Watch query expressions as they happen! post describes an animated WPF project that displays the dynamics of LINQ to Objects query processing. You can download the source code, watch a screencast, or both.

Added: 2/22/2008

LINQ to XSD Alpha 0.2 for VS 2008 RTM Is Available for Downloading

Shyam Pather, Dev Lead for the XML Team, says in his LINQ to XSD Alpha 0.2 post of February 21, 2008:

I’m happy to announce that today we are re-releasing the LINQ to XSD Preview Alpha 0.2 for Visual Studio 2008. The previous preview release of LINQ to XSD targeted Beta 1 of Visual Studio 2008 but did not work on later builds. Many of the people who originally downloaded and tried the LINQ to XSD Preview Alpha requested an update for the final release of Visual Studio 2008 – this release is it. You can download it from here.

I'm one of the one of "the people who originally downloaded and tried the LINQ to XSD Preview Alpha" who "requested an update for the final release of Visual Studio 2008." See my LINQ to XSD Redux and LINQ to Stored XML Coming post of December 4, 2007 (Updated 1/14/2008) for more background on the original development of LINQ to XSD by Ralf Lammel.

Good news! Now what we need from the XML Team is a progress report on LINQ to Stored XML.

Added: 2/21/2008

ADO.NET Team: LINQ to SQL Support for New SQL Server 2008 Features "Coming Soon"

According to Faisal Mohamood, LINQ to SQL Program Manager, in his Coming soon to LINQ to SQL post of February 21, 2008:

Things aren’t so quiet over here on our end, even if it looks that way. The team has been busy adding features so that you will have better support for SQL Server 2008 in the future. Things we’ve been working on include support for new T-SQL data types that are being introduced in SQL Server 2008 – namely DATE, TIME, DATETIME2 and DATETIMEOFFSET, so that you can enjoy the same rich LINQ support and CUD support over data that uses many of the new types that are being introduced in SQL Server 2008.

Stay tuned and watch this blog for more on this as we get closer to releasing some of this and other new functionality!

Sounds to me like the LINQ to SQL upgrade is a candidate for VS 2008 SP1.

Added: 2/21/2008

VS 2008 SP1 Appears to be in the Works Before the Product's Official Release

The ReadMe file for SSCE 3.5 SP1 Beta for EF Beta 3 contains the following statement and admonition:

Microsoft SQL Server Compact 3.5 SP1 is the service pack release for the SQL Server Compact 3.5. SQL Server Compact 3.5 SP1 releases with SQL Server 2008 and Visual Studio 2008 SP1. [Emphasis added.]

Important:

All references to SQL Server 2008 and Visual Studio 2008 SP1 in this SQL Server Compact 3.5 documentation refer to a pre-release version of SQL Server 2008 and Visual Studio 2008 SP1. [Emphasis added.]

The SQL Server Compact Release Versions post of February 8, 2008 (updated for build 3.5.5626) contains a table with the following three latest entries for SSCE releases:

Product Name Release Vehicle Version

SQL Server Compact 3.5 SP1 Beta for ADO.NET Entity Framework Provider Beta

Download Center

3.5.5626

SQL Server Compact 3.5 SP1 Beta

SQL Server 2008 CTP6

3.5.5626

SQL Server Compact 3.5 SP1 Beta

Visual Studio 2008 SP1 Beta

 

These are the first references in official Microsoft documents about VS 2008 SP1 that I've seen. On 2/4/2008, Brian Keller said in a VSTS Forum Answer Re: When is Visual Studio 2008 SP1 coming out? post:

We haven't announced any dates around SP1. Once details become available we'll make them known but for now it's too early to say anything about the schedule.

The VS 2008 Web Development Hot-Fix Roll-Up Available that Scott Guthrie announced on February 8, 2008 isn't a service pack.

It's interesting that Microsoft is announcing a service pack for VS 2008 before the product is officially released.

Added: 2/21/2008

SQL Server Compact Edition 3.5 SP1 Beta for Entity Framework Beta 3 Is Available

Ambrish Mishra, Program Manager for SSCE, announced Download available for SQL Server Compact 3.5 SP1 Beta for ADO.Net Entity Framework Beta 3 on February 20. This is a special version (Build 3.5.5626) of SSCE SP1 Beta 1 specifically for EF Beta 3. This build is the same as that released with the current SQL Server 2008 CTP6. (See the SQL Server Compact Release Versions post of February 8, 2008 for build and release vehicles.)

Ambrish's post includes step by step instructions for creating a working Entity Data Model with the included Northwind.sdf database file.

You can download SSCE v3.5 SP1 Beta for EF Beta 3 here. Be sure to check out the ReadMe file before installing this SSCE Beta, which obviously is not fully cooked. There are 11 known issues, some of which seem to me to be serious deficiencies.

I remain curious as to why anyone would layer heavyweight EF components over a lightweight, single-user database like SSCE.

Added: 2/21/2008

Roger Alsing: LINQ to NPersist Is Progressing

Roger's More Linq support for NPersist post of February 20, 2008 describes his technique for adding LINQ support for Mats Helander's NPersist object/relational mapping tool by generating LINQ-like NPath instead of SQL queries. Mats welcomed Roger back to the NPersist project in his Roger Is Back post of January 9, 2008. NPersist is now at version 1.0.10; you can read more about it in this 200-page Inside NPersist manual (.doc format).

Here are links to Roger's earlier posts on the topic:

Download the current NPersist source code, binaries or both here.

Update 2/24/2008: Roger has LINQ to NPersist running with a Microsoft Access database because his development machine is dead, according to his Linq to NPersist to MS Access post.

Dave Longnecker Replaces Web Gallery 2.0's Custom n-Tier Data Layer with LINQ to SQL

The details are in Dave's WebGallery 2.0 #2 - HttpHandler and LINQ Data Model post of February 20, 2008. The post also includes some links to his earlier posts on LINQ:

Added: 2/21/2008

Richard Bushell: Use Timestamp Fields for Concurrency Conflict Management with the LinqDataSource Control

If you use LINQ to SQL's default value-based concurrency conflict management feature, the LinqDataSource control stores the original values in ViewState. In his The LinqDataSource and the Hidden Viewstate of 2/21/2008, Richard goes into detail about how value-based concurrency works and the value of replacing original values with a single timestamp column.

His earlier How to Update Data with LINQ-to-SQL post of February 18, delves into updating rows without bringing an entity instance into memory.

Added: 2/21/2008

Ruurd Boeke Continues His Postcompiling for Entity Framework Series

Ruurd's initial post about using PostSharp to add IPOCO interfaces to an Entity Data Model (see the "An Automatic IPOCO Implementation for Entity Framework" topic below) has turned into a Postcompiling for Entity Framework series. Here's his table of contents with current links:

Here's a link to Ruurd's Entity Framework Contrib (EFC) project on CodePlex.

Added: 2/21/2008

Julie Lerman to Present Two Entity Framework Sessions at Tech*Ed US Developer 2008

Julie's Entity Framework at TechEd post of February 19, 2008 announces that she'll present

  • Advanced Entity Framework: Entity Data Model in the Enterprise
  • .NET 3.5 Data Access Guidance

at TechEd 2008 US Developer in Orlando June 3 - 8, 2008.

Register before April 4, 2008 and save $200 off registration costs.

Don't miss her .NET Rocks! Entity Framework podcast next Tuesday, February 26.

Added: 2/20/2008

Beth Massi Demos LINQ to SQL Master/Child Forms with Stored Proc Updating

Beth's One-To-Many (Master-Detail) Forms with LINQ to SQL post of 2/19/2008 shows you how to create an Orders/OrderDetails master-detail form, save edits made with stored procedures to the entities, and eliminate associated entity and referential integrity conflict exceptions when deleting OrderDetails entities.

Beth doesn't use Northwind for the demo because she wants foreign key values to be NOT NULL and use timestamp columns for concurrency conflict management. Both good ideas, but easy to implement on a copy of Northwind.

Added: 2/19/2008

Frans Bouma Tackles Function Mapping and Local Methods in LINQ to LLBLGen Pro

Frans is closing in on his epic LINQ to LLBLGen Pro implementation in Developing Linq to LLBLGen Pro, part 13 of February 19, 2008. This episode deals with issues with invoking functions within LINQ queries that don't map to database functions.

All store-specific ADO.NET data providers must support a designated set of canonical functions defined for aggregate, bitwise, date and time, math, string and other operations. Functions invoked by members of a projection that aren't canonical functions will execute on the result set in memory but not if they're included in the query (for example, in the Where clause.)

Frans' post deals with the following specific issues:

  • Constants in the projection
  • In-memory methods/property calls in the projection
  • Mapping methods and properties as database constructs
  • Booleans in projections
  • Lambda-based function usage
  • Object creation inside projections
  • LINQ to SQL oddness

and is longer than most of my posts!

His next post, which he hopes will be the last will cover defining function mappings for RDBMSs other than SQL Server and adapting LINQ queries to use LLBLGen's prefetch paths feature.

Added: 2/19/2008

Jim Wooley Explores LINQ Queries Against Self-Joined Tables

Jim's Managing self-referencing tables with LINQ to SQL post of February 18, 2008, demonstrates how to query a table with a self-join that generates a data tree. He uses the Northwind Employees table whose ReportsTo column contains the EmployeeID of the person's boss.

Jim notes that you can't use LoadOptions to eager load the Employees set, because eager LoadOptions doesn't allow cycles (a.k.a., cyclic references.)

Added: 2/19/2008

Pablo Castro: Links and Link Expansion in ADO.NET Data Services

In his Related entries and feeds: links and link expansion post of February 18, 2008 Pablo requests input from the community on the Astoria team's approach to handling GET operations for associated entries in Atom graphs:

This write up briefly describes how we model associations between resources as Atom links and proposes a usage pattern of the atom:link element to support retrieving resource graphs in a single response. We're looking for feedback on the approach and also to get folks thinking about inlined content and whether it should be considered an extension to Atom.

Pablo refers to his earlier post, AtomPub support in the ADO.NET Data Services Framework of 2/13/2008 for background. I discussed this topic in my  Pablo Castro: ADO.NET Data Services and the Atom Publishing Protocol post of February 14, 2008, updated 2/16/2008 for Dare Obasanjo's posts on AtomPub extensions.

The two topics under consideration are:

1. Links for modeling associations between resources: Use the current link syntax:

<link rel="related" title="Customer" href="Orders(11077)/Customer"  type="application/atom+xml;type=entry" />
<link rel="related" title="Employee" href="Orders(11077)/Employee" type="application/atom+xml;type=entry" />
<link rel="related" title="Order_Details" href="Orders(11077)/Order_Details" type="application/atom+xml;type=feed" />
<link rel="related" title="Shipper" href="Orders(11077)/Shipper" type="application/atom+xml;type=entry" />

with the "starting" end of the association as the title. An alternative is a custom Atom attribute. I recommend minimizing the use of custom attributes. It would be helpful if developers using Entity Framework data sources would singularize EntityType names and End names of the 1 or 0...1 side of AssociationSets, as in the above example. Processing should retain the nouns' number.

2. Expanding links inline. Using the current $include constructs to indicate which associations to expand and include them as content within the <link> element seems appropriate to me.

Note: This may expand into an independent post after I do some additional investigation on the topic, as well as how ADO.NET Data Services fit into the Internet Service Bus that's implemented as a CTP by BizTalk Services.

Update 2/22/2008: Pablo's "Data Friction", spot-on post of February 20, 2008 cites and agrees with Jon Udell's Overcoming data friction article of the same date, which laments the need for Web screen scraping because the underlying databases aren't Web-accessible.

Ruurd Boeke: An Automatic IPOCO Implementation for Entity Framework

Ruurd's Introducing Entity Framework Contrib: Easy IPoco implementation V 0.1 of February 18, 2008 describes his Entity Framework Contrib (EFC) project on CodePlex. EFC implements the

  • IEntityWithChangeTracker
  • IEntityWithKey
  • IEntityWithRelationship

interfaces, which increase the Entity Framework's "persistence ignorance quotient" to IPOCO (Plain Old CLR Objects with Interfaces). Postsharp automatically generates and implements the three interfaces from a POCO class in IL when you build your project. However, the class must be decorated with a custom attribute.

The Postsharp folks ask:

Have you already implemented things like transaction management, logging, caching, or security? In each method, you had to write a same dozen of lines of code, right?

With PostSharp, you can encapsulate these aspects as custom attributes. You can make your own custom attributes that will really add new behaviors to your code! This is sometimes called aspect-oriented programming (AOP) or policy injection.

For more on persistence ignorance, see Danny Simmons' Persistence Ignorance: OK, I think I get it now post of June 2, 2007 and my Persistence Ignorance Is Bliss, but Is It Missing from the Entity Framework? of April 24, 2007.

LINQ to Opf3: An Implementation of LINQ for Chili Software's Opf3 O/RM

Christian Liensberger's Opf3 and LINQ support post of February 18, 2008 announces that Chili Software's Object Persistence Framework 3 (Opf3) now includes support for LINQ by adding the Chili.Opf3.Linq namespace.

Opf3 is a simple O/RM that offers a free Express Edition that maps up to eight tables to classes with a maximum of eight properties each. Opf3 supports most popular commercial and open-source databases that have ODBC or OLE DB drivers.

A single developer license with source code costs US$199; a site license is $699. A wizard plug-in that generates classes and enables customizing the autogenerated mapping is available for $99 and $349. Opf3/Wizard bundles are $249 (single) and $999 (site).

For comparison, LLBLGen Pro licenses range from €229 for a single license to €159 each for 20 or more, excluding VAT.

Charlie Calvert: LINQ Librarian

Charlie's LINQ Books post of February 17, 2008 lists published and forthcoming LINQ-related books in the following categories:

  • Books on LINQ (5 published)
  • Books that Feature LINQ (3 C#, 2 ASP)
  • Not Yet Published (8, including mine)
  • Books on the LINQ Beta (2)

The preceding lists don't include Programming the Microsoft® ADO.NET Entity Framework by David Sceppa (Microsoft Press, scheduled for 7/9/2008.

Saturday, February 16, 2008

LINQ and Entity Framework Posts for 2/11/2008+

Note: This post is updated daily or more often, depending on the availability of new articles.

Updated: 2/15/2008 for the Amazon S3 outage and 2/16/2008 for the reason for the outage. See the "Justin Etheredge Offers Preview of LINQ to [Amazon] SimpleDB" topic.

Danny Simmons Deals with the ObjectContext Lifetime Dilemma

Danny's Context Lifetimes -- Dispose or Reuse? post of 2/17/2008 (yes, a Sunday post) attempts to answer its title's question based on "understanding the overall pattern of your app." (Other than "too slow" I suppose.)

On the whole, Danny's criteria for reusing and disposing also can be applied to LINQ to SQL DataContext objects, which are lighter weight.

The question, which I've asked in the comments, is how will the (hopefully) imminent ASP.NET EntityDataSource server control affect the recommendations?

Added: 2/17/2008

David DeWinter Starts Project to Synchronize LINQ to SQL DBML Files with Database Schema Changes

David's LINQ to SQL and Database Schema Sync post of February 16, 2008 describes his objectives in starting a project to keep his "generated entity classes in sync with the current database schema" by a mechanism other than clearing the designer surface and starting over. He cites many types of manual adjustments to the entities in the designer that must be made to many tables each time you drop them on the designer surface. The goal is to maintain a record of systematic changes for repetition on both unchanged and altered or added tables.

His second The LINQ to SQL Metamodel post of the same day describes his use of Project NORMA to enable Visual Studio 2008 to render Object Role Modeling (ORM) diagrams of Database, Connection, Table, Type, Column, Association, Function and TableFunction elements. (Some of the diagrams are amazingly complex.)

Note: The Entity Data Model's Model Browser has a similar feature called Update Model from Database that opens the Entity Data Model Wizard that lets you select items to add. The Wizard automatically updates existing and deleted items. My tests so far with the Beta 3 version have resulted in non-displayable models.

Added and subscribed: 2/16/2008

Dare Obasanjo Seconds the ADO.NET Entity Data Team's Decision to Adopt AtomPub for Updates

In his ADO.NET Data Services (Astoria) Adopts AtomPub post of February 16, 2008, Dare says:

I'm glad to see Microsoft making a huge bet on standards based, RESTful protocols especially given our recent history where we foisted Snakes On APlane on the industry. [Emphasis Dare's.]

However since AtomPub is intended to be an extensible protocol, Astoria has added certain extensions to make the service work for their scenarios while staying within the letter and spirit of the spec.

Dare then goes on to analyze in his Thoughts on Google's Proposal for Granular Updates in AtomPub post of February 16, 2008 Joe Gregorio's How to do RESTful Partial Updates proposal of February 15, 2008 for updating specific properties of an entry without retrieving and returning the entire entry. Dare's primary complaints are:

The main problem is that it changes the expected semantics of HTTP PUT in a way that not only conflicts with how PUT is typically used in other HTTP-based protocols but also how it is used in AtomPub. It's also weird that the existence of xml:id in an Atom document is now used to imply special semantics (i.e. this field supports direct editing). I especially don't like that after all is said and done, the server controls which fields can be partially updated or not which seems to imply a tight coupling between clients and servers (e.g. some servers will support partial updates on all fields, some may only support partial updates on atom:title + atom:category while others will support partial updates on a different set of fields). So the code for editing a title or category changes depending on which AtomPub service you are talking to.

Dare's solution:

If a field is important enough that it needs to be identifiable and editable then it should be its own resource. If you want to make it part of another resource then use atom:link to link both resources.

Added: 2/16/2008

Julie Lerman Creates ADO.NET Data Services PopFly Mashup

Julie's presenting a Web Mashups with ADO.NET Data Services session at DevConnections, so she decided to create a PopFly mashup with the aid of the Popfly Explorer plugin for VS 2008 [Express].

She recounts her experiences creating a simple read-only client with the NewsReader block in her ADO.NET Data Services + PopFly post of February 15, 2008.

Added: 2/15/2008

Microsoft Appoints Scott Guthrie Corporate Vice President for the .NET Developer Platform

Scott received a well-deserved promotion in Redmond's latest management reshuffling, as noted in a February 14, 2008 press release, Microsoft Announces New, Expanded Roles for Key Executives. From the release:

Scott Guthrie, corporate vice president, .NET Developer Platform. Previously general manager, Guthrie will continue to oversee several development teams responsible for delivering Microsoft Visual Studio developer tools and Microsoft .NET Framework technologies for building client and Web applications.

Congratulations, Scott. Hopefully, the rise in grade won't stem the flow of your great tutorial and useful linkblog posts. As I said in a comment to Joe Wilcox's Microsoft Puts on Midlife Executive Bulge post of February 15:

Of all the execs whose titles Microsoft bumped up yesterday, not one deserves a promotion more than Scott Guthrie. Scott's enthusiasm for the .NET development platform, as evidenced in his blog and technical presentations, continue to attract programmers of all skill levels to Visual Studio 2008 and .NET 3.5.

Other developer-oriented groups would do well to clone Guthrie's approach to marketing their platforms.

Microsoft also boosted Scott's boss, S. "Soma" Somasegar, from Corporate to Senior Vice President:

S. Somasegar, senior vice president, Developer Division. Previously corporate vice president, Somasegar will continue to oversee all developer-related languages, tools and platforms within Microsoft. He also leads Microsoft India Development Center and Microsoft Canada Development Centre.

Added: 2/15/2008

Updated 2/16/2008: .NET Rocks! interviewed Scott for show #316 (2/14/2008) about IIS 7.0, Silverlight 2.0, MIX 08 and other hot topics. Scott's 34-minute segment starts at 04:00.

Justin Etheredge Offers Preview of LINQ to [Amazon] SimpleDB

Justin's been working on LINQ to SimpleDB since shortly after Amazon opened SimpleDB's limited beta. His LINQ To SimpleDB Issues post of December 31, 2007 (before he had access to the beta) details the issues with dealing with numbers when the database only supports strings. I've missed Justin's LINQ-related posts on his CodeThinked blog up until now because it doesn't have high visibility for "LINQ" in search engine results.

Justin released LINQ to SimpleDB Alpha 1 (source code and runtime binary) to CodePlex on January 23, 2008 under the Microsoft Public License (Ms-PL). His LinqToSimpleDB Preview post of January 19, 2008 offers operating instructions. The Releases page describes the included functionality as:

  • Query SimpleDB Items by Attributes
  • Retrieve Attributes for items
  • Query items by Name
  • Create and Delete Domains
  • Create and Delete Items
  • Create and Delete Attributes

Note: My Amazon Announces Beta of SimpleDB Web Services in the Cloud of Post of December 14, 2007 and Dare Obasanjo's Amazon SimpleDB: The Good, the Bad and the Ugly post of December 21, 2007 analyze SimpleDB's feature set.

I plan to give LINQ to SimpleDB a test drive as soon a SimpleDB beta test slot becomes available.

Added: 2/14/2008

Updated 2/15/2008: All Amazon Web services (including SimpleDB) appear to have suffered a massive outage at about 4:30 AM Pacific time according to Nicholas Carr's post of this morning. The problem was reported to have been corrected at 7:30 AM but some users were still reporting problems at 8:30 AM, according to Michael Krigman's Amazon S3 web services down. Bad, bad news for customers. blog, "Rearranging the Deck Chairs: IT Project Failures."

Amazon's Service Level Agreement for S3 (which presumably would apply to SimpleDB when it's out of beta) is 99.9% availability during a monthly billing period. Assuming 30.5 days/month, 0.1% downtime is 0.732 hours.

Updated 2/16/2008: Nick explains in today's Why S3 failed update that:

It was not a hardware failure. Rather, the service's authentication system, which verifies the identity of a user, became overloaded with user requests. As one person explained to me, it amounted to a kind of accidental DDoS (distributed denial of service) attack, and Amazon didn't have enough capacity in place in one of its data centers to handle the surge.

Nick's post includes the official reason in a message from Amazon.com.

Mike Taulty Tabulates Entity Framework's Mapping Capabilities

Mike's ADO.NET Entity Framework Mapping Table post of February 14, 2008 begins with:

One of the things that's been really bothering me for quite a while now is the aspect of mapping in the Entity Framework.

Technical things can trouble me. I like to get them into my head, understood and then I can move on. I have a very, very bad memory and the only way I can remember anything is by understanding how it works and then I find it relatively easy to piece back together at a later point.

Strange that Mike should mention this. I'm bothered by the same questions about EF's mapping capabilities, despite the fact that I use them almost every day. I also have the same memory problem. So Mike put together a list of these capabilities with details examples. Here's a list of what he covers:

  1. Renaming an Entity Type
  2. Renaming an Attribute (although I'd call it Renaming a Property)
  3. Removing an attribute (ditto)
  4. Specifying a default value for an attribute that does not have a default value
  5. Overriding a default value for an attribute that has a default value
  6. Changing the data type of an attribute
  7. Changing the nullability of an attribute
  8. Multiple store tables, single entity type
  9. Single store table, multiple entity types or entity sets

For all my sample projects that use EF/EDM, especially those based on Northwind, I use:

  1. To singularize Entity Types, leaving the Plural for Entity Collections/Sets
  2. To singularize the name of navigation properties for Many:1 associations
  3. To remove images (e.g., Category.Picture and Employee.Photo)

But I hadn't thought of using 5 to add Today as Order.OrderDate's default value or 7 to change the nullability of value-type properties, such as Order.OrderDate, Order.Freight, Product. UnitPrice, etc. The latter might solve some of my issues with EF projects in VB (e.g. performance problems with COALESCE and no support for the Coalesce operator by the VB version of ADO.NET Data Services clients -- see below). Minimizing Nullable<T> data types also minimizes Nullable`1[System.Decimal]" type declarations in ADO.NET Data Services' Atom-formatted payloads.

Update 2/14/2008: Default values are constants of the property's type, so specifying the DateAndTime.Today property doesn't work. (Nor does it throw an exception).

I'd say the preceding list would make a good start for a series of ADO.NET Team blog posts, as well as the final EF documentation.

Added: 2/14/2008

Pablo Castro: ADO.NET Data Services and the Atom Publishing Protocol

Pablo explains the Project Astoria team's approach to adopting the Atom Syndication Protocol (Atom, RFC 4287) and Atom Publishing Protocol (APP, RFC 5023) as data-exchange (payload) formats in his AtomPub support in the ADO.NET Data Services Framework post of February 13, 2008.

The Astoria Team plans to post its proposed extensions and application-level features for discussion on the atom-syntax and atom-protocol mailing lists. Read more about the pending issues here.

Added and updated: 2/14/2008

Tercer Planeta's Working on a LINQ-Enabled Application Framework

Jose Marcenario and Benjamin Eidelman, principals of Tercer Planeta (Third Planet, a .NET consulting firm in Argentina), have determined that "it would be great to have a LINQ 'Queryable' data access layer." So they've "started to analyze LINQ-to-SQL integration in Enterprise applications of different scale."

Benjamin's initial post, First Thoughts on Designing a LINQ-enabled Application Framework of February 13, 2008, deals with issues arising from LINQ to SQL's infamous lack of "an out-of-the-box n-tier story" because DataContext objects aren't serializable.

He points to Manuel Bauer's InterLINQ project on CodePlex that serializes an expression tree and returns the query result. (InterLINQ is destined to become a component of the poor man's Model Driven Architecture for .NET project from HSR Hochschule für Technik Rapperswil, Switzerland' .NET Competency department). However, it doesn't appear that Tercer Planeta will take the InterLINQ route.

At the end of the post, Benjamin says:

These days we're starting to see the light and the end of tunnel, with custom tools and code we started to write.

More on this on following posts...

Added: 2/14/2008

VB-Specific Bugs in ADO.NET Data Services CTP

Today's Fix for "Known Bug" when Creating ADO.NET Data Services with Visual Basic and a Web Application post updates an ASP.NET 3.5 Extensions Preview Released post of December 10, 2008 included a "Data Services (Project Astoria)" topic that itemizes some errors in the QuickStart instructions for creating an ADO.NET Data Service.

The need to do a fixup by removing the default namespace for a VB ADO.NET Data Services project that uses a ASP.NET Web Application (rather than a file-system Web site) turned out to be a known but unpublicized bug.

There's another VB-related bug that's described in my LINQ Query Expression Fails in VB with 'Coalesce' Not Supported Error, C# OK post in the ADO.NET Data Services (Pre-release) forum. I'm waiting for a reply indicating whether that problem is known.

While I'm griping, the numeric operators documentation shows lteq and gteq instead of the correct le and ge for <= and >=, respectively. There's also an issue with the use of $expand in LINQ to REST and WebDataQueries, which has known problems.

Errata for docs and lists of supported and unsupported operators, etc., would be very useful for ADO.NET Data Services pilgrims (like me.)

Added: 2/13/2008

Charlie Calvert: Using the Expression Tree Visualizer

The latest in Charlie's LINQ Farm Seed series is LINQ Farm Seed: Using the Expression Tree Visualizer of February 13, 2008. This post gives detailed instructions on how to download the visalizer and use it to view the expression tree for a simple lambda function.

Added: 2/13/2008

Julie Lerman Records Entity Framework Podcast for .NET Rocks!

Julie's Just recorded DotNetRocks about Entity Framework post of February 12, 2008 describes her one-hour interview with Carl Franklin and Richard Campbell about the Entity Framework.

A major concern voiced in her podcast: DBAs' reluctance to accept dynamic SQL queries from Entity Framework or LINQ to SQL and ignorance of both implementations' capability to use stored procedures for all CRUD operations.

Mark your calendars: The podcast will be aired on February 26, 2008.

Added: 2/13/2008

James Newton-King Publishes LINQ to JSON Beta

According to James' LINQ to JSON beta post of February 11, 2008, LINQ to JSON is an API that's similar to LINQ to XML, not a LINQ provider. It enables a declarative syntax for writing JSON code and a query syntax for reading it. The post provides a simple RSS channel/items structure as an example.

LINQ to JSON is part of James' larger Json.NET project on CodePlex. According to the project's home page, Json.NET offers:

  • Lightning fast JsonReader and JsonWriter
  • The JsonSerializer for quickly converting your .NET objects to JSON and back again
  • Json.NET can optionally produce well formatted, indented JSON for debugging or display
  • Attributes like JsonIgnore and JsonProperty can be added to a class to customize how a class is serialized
  • Ability to convert JSON to and from XML

Json.NET 2.0 Beta 1 released on 2/11/2008 includes the following new features, changes, and bug fixes:

  • New feature - LINQ to JSON!
  • New feature - Ability to specify how JsonSerializer handles null values when serializing/deserializing.
  • New feature - Ability to specify how JsonSerializer handles missing values when deserializing.
  • Change - Improved support for reading and writing JavaScript constructors.
  • Change - A new JsonWriter can now write additional tokens without erroring.
  • Bug fix - JsonSerializer handles deserializing dictionaries where the key isn't a string.
  • Bug fix - JsonReader now correctly parses hex character codes.

Added: 2/13/2008

Bill Horst Struggles to Emulate a FULL OUTER JOIN with LINQ to SQL

Bill's Converting SQL to LINQ, Part 9: Full Outer Join post of February 12, 2008 demonstrates creating the equivalent of a T-SQL FULL OUTER JOIN command with one initial LINQ query and two concatenated subqueries.

Apparently, this is the last of Bill's posts unless users suggest a different topic.

Links to all nine topics are available here.

Added: 2/12/2008

Alex James Reaches Part 4 of His Bulk Updates with Entity Framework Saga

If you want to avoid bringing a large number business objects into memory for bulk updating, Alex has the solution. In his Rolling your own SQL Update on top of the Entity Framework - Part 4 post of February 12, 2008, he shows you how he builds the final UPDATE [Table] SET Column = value, .... WHERE Key IN (...) query from Part 3's GetQueryInfo() method.

What would be interesting is a comparison of the performance of Alex's approach with the out-of-the-box bring-the-objects-into-memory-make-changes-and-invoke-SaveChanges() approach.

Following are links to Alex's three previous parts:

Added: 2/12/2008

Kathleen Dollard Moves from XSLT to LINQ to XML Literals for Code Generation

dnrTV's Kathleen Dollard on Code Generation in .NET video of February 1, 2008 describes Kathleen's recent development efforts in moving from XSLT to LINQ to XML literals. You can see an example of a language specific LINQ to XML literal template starting at 15:30. Language-independent (C# and VB) template generation examples follow.

Despite the fact that Kathleen doesn't like Microsoft's SingleFileGenerator approach, she plans to move to .edmx files for templates because they enable mapping between tables and objects.

Beth's two earlier dnrTV videos about VB 9.0 and C# 3.0 are at:

Thanks to Beth Massi for the heads up.

Added: 2/12/2008

Brian Dawson Continues His Entity Framework Performance Series

In his Exploring the Performance of the ADO.NET Entity Framework – Part 2 post of February 11, 2008, Brian starts by describing caching of Entity SQL queries and its dramatic affect on query execution time: 179 ms. for the first vs. 15 ms. for subsequent executions. The same LINQ to Entities query executes in 202 ms. vs. 18 ms.

Brian then goes on to describe the benefit of compiling LINQ queries, which increases the initial execution time to 305 ms. and reduces subsequent execution times to 15 ms. (Brian says "It’s true that in this case subsequent executions of the compiled LINQ query are slower than those of the non-compiled LINQ query," however, the data indicates otherwise.

Changing the MergeOption to no tracking has a dramatic affect on subsequent executions of compiled queries: 3 ms. vs 15 to 17 ms.

Brian concludes:

  • Initial creation of the ObjectContext includes the cost of loading and validating the metadata.
  • Initial execution of any query includes the costs of building up a query cache to enable faster execution of subsequent queries.
  • Compiled LINQ queries are faster than Non-compiled LINQ queries.
  • Queries executed with a NoTracking merge option work well for streaming large data objects or when changes and relationships do not need to be tracked.

Added: 2/12/2008

Stefan Sedich Discovers Cause of Slow LINQ to SQL Performance with VB

Stefan's Different SQL between VB.NET and C# LINQ to SQL SOLVED!! post of February 12, 2008 describes the cause of VB's added (COALESCE(CASE ... )) clauses: VB treats database fields that allow nulls differently from C#. Note that Northwind tables allow nulls in foreign-key fields that would throw referential integrity exceptions if set to null.

See the "Dramatically Poorer Performance of VB Than C# LINQ to SQL Queries" topic of LINQ and Entity Framework Posts for 2/4/2008+.

Added: 2/12/2008

DinnerNow.NET Updated for VS 2008 RTM

The open-source DinnerNow.NET sample ASP.NET application has been updated to .NET 3.5 and VS 2008 RTM. According to the CodePlex release site, the following new .NET 3.5/VS 2008 features have been added:

  • LINQ to SQL
  • WF/WCF integration
  • IIS7 & Windows Activation Services (WAS) hosting for WCF services
  • WCF 3.5 Syndication Services
  • WCF on the .NET Compact Framework 3.5

The DinnerNow 2.5 release has also been tested for compatib[ility] with Windows Vista, Windows Vista SP1, and Windows Server 2008 RTM.

You can download the bits from DinnerNow.net on CodePlex.

The Visual Studio 2008 Training Kit uses DinnerNow.NET in several of its online Labs.

Thanks to Guy Burstein for the heads-up.

Jim Wooley Upgrades the Personal Web Starter Kit's Data Tier from ADO.NET to LINQ to SQL

The Personal Web Starter Kit (PWSK) is a sample ASP.NET application that lets users organize individual photo files into albums stored in an SQL Server Express database. You can download the original version here.

Jim's Personal Web Starter Kit LINQed up post of February 10, 2008 shows you how he refactored PhotoManager.vb to replace ADO.NET SqlClient stored procedure calls with LINQ to SQL's parameterized dynamic queries. You can read more about Jim's PSWK version and download the upgraded project at his LINQ enabled Personal Web Site Starter kit site in the MSDN Code Gallery.

Jim also provides a set of guidelines for migration from SqlClient to LINQ to SQL in his LINQ Migration hints of the same date.

Thursday, February 14, 2008

Pablo Castro: ADO.NET Data Services and the Atom Publishing Protocol

Pablo explains the Project Astoria team's approach to adopting the Atom Syndication Protocol (Atom, RFC 4287) and Atom Publishing Protocol (APP, RFC 5023) as data-exchange (payload) formats in his AtomPub support in the ADO.NET Data Services Framework post of February 13, 2008.

Pablo says the following under the heading "Why are we looking at AtomPub?:

Astoria data services can work with different payload formats and to some level different user-level details of the protocol on top of HTTP. For example, we support a JSON payload format that should make the life of folks writing AJAX applications a bit easier. While we have a couple of these kind of ad-hoc formats, we wanted to support a pre-established format and protocol as our primary interface.

If you look at the underlying data model for Astoria, it boils down to two constructs: resources (addressable using URLs) and links between those resources. The resources are grouped into containers that are also addressable. The mapping to Atom entries, links and feeds is so straightforward that [it's] hard to ignore. Of course, the devil is in the details and we'll get to that later on.

Here's part of a sample query result against the Northwind.Orders EntityCollection delivered in the December 2007 CTP's Atom payload format:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<feed xml:base="http://localhost:50539/Northwind.svc/" xmlns:ads="http://schemas.microsoft.com/ado/2007/08/dataweb" xmlns:adsm="http://schemas.microsoft.com/ado/2007/08/dataweb/metadata" xmlns="http://www.w3.org/2005/Atom">
  <entry adsm:type="NorthwindModel.Orders">
    <id>http://localhost:50539/Northwind.svc/Orders(11077)</id>
    <updated />
    <title />
    <author>
      <name />
    </author>
    <link rel="edit" href="Orders(11077)" title="Orders" />
    <content type="application/xml">
      <ads:OrderID adsm:type="Int32">11077</ads:OrderID>
      <ads:OrderDate adsm:type="Nullable`1[System.DateTime]">2007-05-06T00:00:00</ads:OrderDate>
      <ads:RequiredDate adsm:type="Nullable`1[System.DateTime]">2007-06-03T00:00:00</ads:RequiredDate>
      <ads:ShippedDate adsm:type="Nullable`1[System.DateTime]" ads:null="true" />
      <ads:Freight adsm:type="Nullable`1[System.Decimal]">8.5300</ads:Freight>
      <ads:ShipName>Rattlesnake Canyon Grocery</ads:ShipName>
      <ads:ShipAddress>2817 Milton Dr.</ads:ShipAddress>
      <ads:ShipCity>Albuquerque</ads:ShipCity>
      <ads:ShipRegion>NM</ads:ShipRegion>
      <ads:ShipPostalCode>87110</ads:ShipPostalCode>
      <ads:ShipCountry>USA</ads:ShipCountry>
    </content>
    <link rel="related" title="Customers" href="Orders(11077)/Customers" type="application/atom+xml;type=entry" />
    <link rel="related" title="Employees" href="Orders(11077)/Employees" type="application/atom+xml;type=entry" />
    <link rel="related" title="Order_Details" href="Orders(11077)/Order_Details" type="application/atom+xml;type=feed" />
    <link rel="related" title="Shippers" href="Orders(11077)/Shippers" type="application/atom+xml;type=entry" />
  </entry>
  <!-- ... -->
</feed>

Using Atom and APP as data exchange formats has a precedent. Google was one of the early proponents of Atom as a replacement for RSS and specified Atom 0.3 as Blogger's sole syndication format. The Google data APIs (GData) are based on either Atom or RSS for responses to HTTP GET requests and APP for updating data with POST (insert), PUT (update), or DELETE requests. GData uses an Atom extension for queries, while Astoria uses an expressive Universal Resource Identifiier (URI) syntax for GET requests. (The Google Base data API has a simple URI-based GET syntax for querying items by attribute values.) A more distinguishing characteristic is ADO.NET Data Services use of LINQ to REST to generate URI-based WebDataQueries.

It appears that the following issues with implementing AtomPub as an Astoria protocol are still open for discussion:

  1. How does a client send a set of PUT/POST/DELETE operations to the server in a single go (request batching)?
  2. How is metadata that describes the structure of a service end points exposed?
  3. How do we deal with aspects that AtomPub does not handle by design or just because it has not been needed so far?
  4. What to do with fields that may not have a backing value in the input source (e.g. [title], updated, author)?
  5. How high-level can we make clients so they can consume AtomPub-based Astoria services but still feel that they are working against regular objects and have general integration with the development environment?

The Astoria Team plans to post its proposed extensions to and application-level features for Atom for discussion on the atom-syntax and atom-protocol mailing lists.

Update 2/16/2008:

In his ADO.NET Data Services (Astoria) Adopts AtomPub post of February 16, 2008, Dare Obasanjo says:

I'm glad to see Microsoft making a huge bet on standards based, RESTful protocols especially given our recent history where we foisted Snakes On APlane on the industry. [Emphasis Dare's.]

However since AtomPub is intended to be an extensible protocol, Astoria has added certain extensions to make the service work for their scenarios while staying within the letter and spirit of the spec.

Dare then goes on to analyze in his Thoughts on Google's Proposal for Granular Updates in AtomPub post of February 16, 2008 Joe Gregorio's How to do RESTful Partial Updates proposal of February 15, 2008 for updating specific properties of an entry without retrieving and returning the entire entry. (Joe Gregorio

See the "Dare Obasanjo Seconds the ADO.NET Entity Data Team's Decision to Adopt AtomPub for Updates" topic of LINQ and Entity Framework Posts for 2/11/2008+ for more details about solving the partial updates problem.

Wednesday, February 13, 2008

Fix for "Known Bug" when Creating ADO.NET Data Services with Visual Basic and a Web Application

My ASP.NET 3.5 Extensions Preview Released post of December 10, 2008 included a "Data Services (Project Astoria)" topic that contains the following details for creating an ADO.NET Data Service:

The Creating ADO.NET Data Services QuickStart instructions for ASP.NET Data Services show you how to get an Entity Framework-based data service up and running.

Note: There is a typo and an error in the QuickStart instructions for the "Create the Data Service" topic.

  • In "When the Add New Item window appears, select SimpleDataService," replace SimpleDataService with ADO.NET Data Service.
  • The following instruction doesn't work for VB services. "In this example, the database was called Northwind so the namespace is NorthwindModel," you must replace NorthwindModel with SimpleDataService.NorthwindModel in the Imports statement to make NorthwindModel visible.
  • The VB version then fails with an "The member with identity 'NorthwindModel.Categories' does not exist in the MetadataCollection" error message written to Default.aspx.

It turns out that the VB version of this ADO.NET Data Services CTP fails due to a known—but apparently unreported—bug when starting with an ASP.NET Web Application, which I reported in the Creating a New ADO.NET DataService with VB Fails, C# is OK post of February 12, 2008 in the ADO.NET Data Services (Pre-release) forum. According to Microsoft's Pratik Patel:

This is a known bug. For the time being, just clear the root namespace in the VB project. [Emphasis added.]

It turns out that you also must clear the namespace prefix, which was NwindDataServiceVB for my project, from the Imports directive (Imports NwindDataServiceVB.NorthwindModel) and the Service attribute in the

<%@ ServiceHost Language="VB" Factory="Microsoft.Data.Web.DataServiceHostFactory, Microsoft.Data.Web, Version=3.5.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089" Service="NwindDataServiceVB.Northwind" %>

directive of the Northwind.svc file. (Open it with the HTML editor for editing).

It behooves the ADO.NET team to be proactive in reporting named bugs in the appropriate blog (Astoria Project Team Blog for this example), rather than mystifying CTP pilgrims.

Tuesday, February 12, 2008

Will Access 14 be a "Tool for the Web" or SharePoint Server Front End?

Julie Lerman turned me on to Dan Fernandez's Channel9 interview of Bill Gates at the Office Developer Conference (ODC) 2008. What caught my eye were Dan's comments about the next version of Microsoft Access gaining Web capabilities:

The other cool part is hearing Bill hint about how Access will now be a tool for the Web. Many a developer has cut his teeth building Access applications and seeing Access go beyond desktop applications is huge. [Italics added for emphasis.]

Microsoft made a previous attempt to make Access "a tool for the Web" with Data Access Pages (DAP). However very few DAP projects offered Internet connectivity because securing them was exceedingly complicated. Microsoft Office Access 2007 (a.k.a. Access 12) doesn't support creating or even editing DAP.

I cut my Windows database development teeth building Access applications, and writing Using Microsoft Access [1.0] for Windows and 10 successive editions got me into the computer book authoring business in earnest. I was surprised to hear that the next version of Access (14.0, there won't be an Office 13) would be Web-enabled because the preceding attempt failed. So I transcribed BillG's comments on Access, which start at about 06:00 into the 11:42 interview:

Access, of course, has been phenomenally successful, but you think of it as just client forms and client data. Now [with Access] 2007, they did a great job where you could replicate from SharePoint down to Access lists back and forth, but that still didn't let you run your logic up on the server. So the next step is to take that base of Access users and literally let them write things that connect directly up to SharePoint and so it's server-based. So it's a logical step for Access. There's a lot of smart people working on that, so in no sense are we leaving the Access people behind. The same way we moved Excel up to the server, now we're moving Access up there as well.

There's not a single mention of the Web in the Access segment. Bill's comments were all about SharePoint. What's more, Access already has a free server; it's called SQL Server 2005 [Express Edition] or its predecessors, MSDE 1.0 and 2.0. If Access has a server, what constitutes "moving Access up there as well?"

Mene Mene Tekel Upharsin: Excel Services Beget Access Services

Access 12 currently enables importing, exporting, and moving Access tables from traditional .mdb and new .accdb files to or from SharePoint lists. The .accdb format was created expressly to support SharePoint's multi-select lists. You can store Access front ends in SharePoint document libraries, but opening them requires a licensed copy of Access 12 on the desktop. SharePoint has an Access DataSheet ActiveX control that's only accessible if the user has a license for a recent Access version.

An abandoned Excel Blog entry describes Excel Services as follows (slightly paraphrased):

Excel Services is brand new server technology that will ship with Office 12. Excel Services supports loading, calculating, and rendering Excel spreadsheets on servers. There are two primary interfaces: a web-based UI that lets you view spreadsheets in the browser, and a web services interface for programmatic access.

So what happens, exactly, to get the spreadsheet in the browser?  Behind the scenes, Excel Services opens the file the sales analyst saved to SharePoint, refreshes any external data in the spreadsheet, calculates any formulas, and renders the results in the browser. Specifically, Excel services sends only DHTML to the browser (no ActiveX), so the sales manager can be using any modern browser. The result is a very high-fidelity version of the analysis that the sales manager can interact with in the browser or, if they have permissions to do so, open up back in Excel. Excel 12 is the authoring tool for spreadsheets that run on Excel Services.

Note: SharePoint Server 2007's Creating Custom Solutions with Excel Services topic offers more up-to-date and detailed information about Excel Services.

Access 14 might become a "tool for the Web" (or at least intranets) by abandoning any pretense of remaining a desktop relational database management system (RDBMS), and becoming a list management query and reporting tool for SharePoint. This implies a set of Access Services running on the next version of Microsoft Office SharePoint Server (MOSS).

The Access Services might parallel Excel Services:

Access Web Services based on the SharePoint Lists Web Service or, less likely, a dedicated SOAP API similar to Excel Web Services.

Access Web Access provided by a simple forms engine, perhaps derived from the existing InfoPath forms engine, which renders customized forms and a DHTML version of the Access DataSheet control in a browser. Excel Services doesn't support VBA so Access Web Access might embargo VBA, too.

Access Query Services based on LINQ to Access (or the like), corresponding to Excel Calculation Services and modeled on Bart De Smet's LINQ to SharePoint project. SharePoint isn't an RDBMS, so support for SQL is superfluous.

Access User-Defined Functions for queries, roughly corresponding to Excel User Defined Functions, provided by the long-awaited Access application add-in for Visual Studio Tools for Office (VSTO).

Update 2/13/2008: InfoWorld's James Niccolai writes in his "'Office 14' to be more Web-friendly, Gates says" article of February 11 from ODC 2008:

Microsoft envisions the next version of Office [will] have partial online functionality similar to how Outlook Web Access works. ... "As we look at all the modules [in Office 14], we have in mind the equivalent of Outlook Web Access," Gates said. ...

It will take another step in that direction with Office 14 by offering Outlook Web Access equivalents of other Office applications such as Excel. "If you look at spreadsheets, maybe you'll not be able to set up all the data models [online], but you'll be able to read documents, change a few assumptions, and try things out," Gates said. ...

Gates did say that SharePoint Server, which is becoming more closely aligned with Office, "will be able to render a greater set of Office documents in an HTML environment."

A Pricey Transition to Access as a "Web Tool"

Web (Internet) access would require a MOSS for Internet Sites license (estimated price $40,943 for MOSS 2007) in addition to a MOSS Server License (est. $4,424 plus $94/user CAL for MOSs 2007) for non-public content.

Bloomberg's "Microsoft's SharePoint Exceeds $800 Million in Sales (Update2)" article reported that SharePoint grew faster than any other piece of software in company history, according to former Business Division President Jeff Raikes at the Microsoft Financial Analysts meeting on July 26, 2007. Microsoft said it has sold more than 85 million licenses to 17,000 customers.

According to eWeek's "Ballmer Details Ways Yahoo Deal Would Challenge Google" article of Feburary 4, 2008 by Clint Boulton:

Ballmer assured analysts that Microsoft is a provider of hosted software plus services for the Internet age.

He said that while people associate online advertising with the cloud computing trend that Google, Salesforce.com and other smaller vendors are leveraging, Microsoft plans to make all of its products available in the cloud as a SAAS (software-as-a-service). That includes the Windows operating system and major money-making products such as Office, in addition to the ad-driven Windows Live suite. [Emphasis added.]

"Each and every one of these businesses on top of a consistent cloud platform transitions to have additional revenue and profit opportunities based on this transformation to the cloud," Ballmer said.

A $40,943 per server price for enabling Access to become a Web tool certainly qualifies as a "revenue and profit opportunit[y]" for Microsoft but doesn't bode well for a "developer [who] has cut his teeth building Access applications" and wants to continue doing the same. This is especially the case when you consider that the runtime version of Access 2007 is a free download.

Update 2/13/2008: Mary Jo Foley's Office 14 to add more online document sharing article of February 13, 2008 sheds additional light on Office 14's "webification."

Update 2/14/2008: CIO Magazine's SharePoint 2007 Demystified: How to Cash in on Collaboration Tools article offers a relatively unbiased review of MOSS for enterprise use. However Sahil Malik contests the article's "Beware Microsoft Baggage" section; see his CIO Magazine - Journalistic ethics? post of this date.

Saturday, February 09, 2008

LINQ and Entity Framework Posts for 2/4/2008+

Note: This post is updated daily or more often, depending on the availability of new articles.

Mike Tulty Defines Update Functions in SSDL, a QueryView in CSDL, and Maps the QueryView and Update Functions in MSL

Once you review the schema additions that Mike makes to define functions to return, insert, update, or delete QueryView entities, the inline functions appear straightforward. His ADO.NET Entity Framework - QueryViews, Inline Functions post of February 8, 2008 shows the modifications to the SSDL, CSDL and MSL files required to create a custom QueryView as the UkCustomers EntitySet and provides InsUkCustomers, DelUkCustomers, and UpdUkCustomers functions to update the EntitySet.

Mike points to Julie Lerman's EDM QueryViews vs Defining Queries (and read-only views)treatise of November 2, 2007 about the difference between QueryViews and DefiningViews.

Added: February 9, 2008  Clarified: February 10, 2008

Eden Ridgway Updates Emile Bosch's LINQ to WMI Provider to .NET 3.5 RTM

In his An Updated LINQ to WMI Implementation post of January 2, 2008 (which I missed in the first linkblog of the new year, Eden describes his update to Emile Bosch's 2005 implementation (Query your WMI with ease using WMILinq! of December 12, 2005) which was based on the May 2005 CTP running in VS 2005.

Added: February 9, 2008

SQL Server Compact Edition Team Posts SSCE 3.5 Build and Availability List

Laxmi Narsimha Rao Oruganti's SQL Server Compact Release Versions post of February 8, 2008 lists the product name, release vehicle, and version numbers for each release of SQL Server 2005 Mobile 3.0 and Compact 3.0 and 3.5 Editions.

SQL Server Compact 3.5 SP1 Beta for ADO.NET Entity Framework Provider Beta (v.3.5.5626) is of particular interest to Entity Framework pilgrims, which Laxmi's post says available from the Download Center. The latest version I can find that's available for download is Microsoft SQL Server Compact 3.5 and Microsoft Synchronization Services for ADO.Net v1.0 for Windows Desktop (v.3.5.5386.0).

A question about availability of v.3.5.5626 is pending as a comment.

Added: February 8, 2008 

Bart De Smet Updates LINQ to SharePoint Alpha for VS 2008 RTM

Bart's LINQ to SharePoint 0.2.4 alpha release for .NET Framework 3.5 and WSS 3.0 CodePlex release offers these features

  • LINQ query provider for runtime translation of LINQ queries to CAML queries
  • Support for both C# 3.0 and VB 9.0
  • Debugger visualizer for LINQ to SharePoint queries
  • Entity creation wizard for Visual Studio 2008
  • SpMetal tool for SharePoint list entity creation
  • MSI setup

updated to Visual Studio 2008 RTM bits.

Important: Read the Release Note for instructions to load the four LINQ to SharePoint libraries into the GAC, which is required for this drop.

Added: February 7, 2008 

Dramatically Poorer Performance of VB Than C# LINQ to SQL Queries

Stefan Sedich's Different SQL between C# and vb.net using LINQ to SQL causes performance issues post of January 7, 2008 demonstrates a major difference between the T-SQL generated by VB & C# versions of the same (somewhat complex) LINQ to SQL query.

Subtree cost for the C# version was 0.1; the VB version's subtree cost was 7.3.

Similarly, the execution plan reports that the C# query's cost was 2% of the batch, which the VB query was 98% of the batch.

Frans Bouma's comment suggests reasons for overall poor query performance as well as the cause of the VB query's problem (the WHERE(COALESCE(CASE ... )) clauses.

Added: February 7, 2008 

Update February 12, 2008: Stefan's Different SQL between VB.NET and C# LINQ to SQL SOLVED!! post of February 12, 2008 describes the cause of VB's added (COALESCE(CASE ... )) clauses: VB treats database fields that allow nulls differently from C#. Note that Northwind tables allow nulls in foreign-key fields that would throw referential integrity exceptions if set to null.

Beth Massi Demonstrates Populating Combo Boxes with LINQ to SQL

Beth's Related Data Binding and ComboBoxes with LINQ to SQL post of February 6, 2008 describes how to substitute a combo box bound to an Object Data Source to serve as the master navigation tool to populate a DataGridView with related (child) records. The example she uses assigns Regions to the combo box and Territories to the DataGridView. Making a selection from the combo box's list is similar to doing the same with a BindingNavigator or a master DataGridView.

Added: February 6, 2008 

Open Source Profiler for SQL Server 2005 Express

Stefan Cruysberghs' .NET - ADO.NET Entity Framework & LINQ to Entities - part 2 post of February 6, 2008 covers the following topics:

  • SQL profiling tools
    • Microsoft SQL Server Profiler
    • AnfiniL's SQL Express Profiler
  • ToTraceString method
    • Entity SQL : EntityCommand.ToTraceString()
    • Entity SQL : ObjectQuery.ToTraceString()
    • LINQ to Entities : (query as ObjectQuery).ToTraceString()
  • eSqlBlast
  • LINQPad

I'd never heard of AnfiniL's SQL Express Profiler so decided to give it a try. It's relatively Spartan, as indicated by the following screen capture of a LINQ to REST query, but it does the trick for LINQ implementations that don't offer LINQ to SQL's Log property or where you want a quick way to display autogenerated T-SQL.

David Hayden Demonstrates LINQ to SQL and Repository Pattern with ASP.NET MVC

David's ASP.NET MVC Framework and AJAX Using jQuery, JavaScriptSerializer and LINQ post of February 6, 2008 is based on a presentation he gave at the South Florida Code Camp.

The presentation included a "sample ASP.NET MVC Application in question helps developers manage and promote their developer events." David says:

As part of the application's functionality, you can view the attendees for a given event by clicking on the Attendees Link in the Event List. As we will discuss, the Attendees Link makes a javascript call to the server using jQuery. The call is picked up by the Attendees Controller and List Action, which uses LINQ To SQL ( wrapped using the Repository Pattern ) and returns JSON back to the web client.

He also includes the code for the AttendeesController and List action.

Added: February 6, 2008

Scott Walker Implements the Specification Pattern for Validation and Object Selection with LINQ

Scott's Specifications, LINQ, and Lambdas post of February 5, 2008, traces his evolution "to a more DDD oriented model than a CSLA oriented one" and "the main challenges [in finding] the right way to do validation." Scott says,

This got me to thinking about the specification pattern as this type of validation is just one of the cases that pattern can be used for. The other great use of this pattern is for selecting a subset of objects from a master list, which ties in very well with the repository pattern for object retrieval. This line of thinking eventually lead me to the idea of somehow creating an implementation of the specification pattern that could be used for both validation and entity selection (using LINQ) types of scenarios. It seemed like a reasonable goal and one that could be very beneficial if done correctly.

Scott has posted the source code here.

Added: February 6, 2008

Rick Strahl Updates His LINQ to SQL Business Object Wrapper

Rick posted an Update to LINQ to SQL Business Object Wrapper Sample for .NET 3.5 RTM on February 5, 2008. The lightweight wrapper and a simple TimerTracker demo progect originally appeared in his A simple Business Object Wrapper for LINQ to SQL post of September 27, 2007. You can download the updated code here.

Added: February 6, 2008

Julie Lerman: Instructions for Precompiling Entity Framework Views for Faster Startup

Brian Dawson (see "Brian Dawson Begins Series to Analyze Entity Framework's Performance Problems" below) and Julie Lerman (see Entity Framework Performance of February 5, 2008) observe that precompiling the view reduces Entity Framework's start up query time. This feature is new in Entity Framework Beta 3.

Jaroslav Kowalski provided the step-by-step instructions and the XML precompile piece of XML in an answer to Julie's EF: Best way to compare apples to apples when comparing perf of materialized objects vs datareader? December 13, 2007 question in the ADO.NET (Prerelease) forum.

Danny Simmons concluded:

[Julie's] timings are pretty reasonable.  The exact numbers and ratios between the various techniques will depend a lot on the complexity of the model and other factors, but view generation is typically very time consuming (which is why we made it possible to do it at compile time) while executing queries through object services has improved dramatically to the point where for some scenarios it is the same speed or better than using the DataReader on EntityClient (as you observed).

Added: February 6, 2008

Jim Wooley's Extension Method Creates LINQ to DataReader

Chalk up another LINQ to Whatever implementation: LINQ to DataReader. Jim provides the details and a sample C# extension method with a VB test harness in his Consuming a DataReader with LINQ post of February 5, 2008.

Neither Jim nor I made comparative timings, but I'd bet the process is several times slower than reading the DataReader directly.

Added: February 6, 2008

Charlie Calvert's LINQ Farm Seed 1 and 2: The Aggregate Operator

Charlie continues his series with LINQ Farm Seed I: Aggregate Operator Part I of February 5, 2008 and LINQ Farm Seed 02: Aggregate Operator Part II of February 6. Source code for both episodes is available from CodePlex.

Added and Updated: February 6, 2008

Rick Strahl Tackles LINQ to SQL DataContext Persistence Issues

Rick's Linq to SQL DataContext Lifetime Management post of February 5, 2008 discusses trade-offs with LINQ to SQL's top-level DataContext object. He analyzes the following options for persisting the DataContext for multiple operations (or not):

  • Create a new Context for each atomic operation (Application level management)
  • Create a global DataContext and handle all operations against this single DataContext object
  • Create a thread specific DataContext
  • Create a per business object DataContext

and provides source code for the last two approaches. Here's Rick's conclusion:

In the end the key for DataContext change management is to ensure that the DataContext is always accessible and doesn't need to be recreated. As long as you can find some way to hang on to a single data context it can be used and be expected to correctly deal with change management.

Unfortunately as I pointed here and here , once you disconnect from the DataContext, the story with LINQ to SQL reconnecting and synching with another DataContext is bleak at best. So the best strategy is to keep hanging on to to the DataContext if you at all can <shrug>. Not always possible when you really need to cross machine boundaries with Web Services, but at least in that scenario you do get a fresh de-serialized instance of an object that can potentially be resynched by using Table.Attach().

For anything else it'll be better to implement something like one of the strategies above to hang on to your DataContext.

Entity Framework's ObjectContext has characteristics that are similar to LINQ to SQL's DataContext object. Therefore, Rick's conclusion probably applies equally to EF implementations.

Any .NET developer who has adopted or is considering implementing either LINQ to SQL or Entity Framework should read this well-researched post (and its comments) carefully.

Brian Dawson Begins Series to Analyze Entity Framework's Performance Problems

The ADO.NET Team's Exploring the Performance of the ADO.NET Entity Framework - Part 1 post of February 4, 2008 by Brian Dawson, an ADO.NET program manager, starts by analyzing what occurs in the 4.241 seconds required to run an initial ObjectQuery. View generation (creating data store views) is the primary performance culprit, consuming 56% of initial query time or 2.375 seconds.

My tests confirm Brian's 4+ second initial query times for simple LINQ to REST queries against basic ADO.NET Data Services with Entity Framework back ends, as well as WCF Web Services that use EntityBag objects. (See my "Notice the 4-second delay between completion of the request and response messages." at the end of my Logging EntityBag's SOAP Request and Response Messages post of January 29, 2008).

Update 2/5/2008: Added WCF Web service quote to the above.

Brian says, "The good news is there is a way of making view generation part of the build process so that this step can be avoided at run time." If the step were eliminated, the query time would be 1.866 seconds.

However, he goes on to say:

When I run the EDM generator (EdmGen.exe) command line tool with the view generation command parameter (/mode:ViewGeneration), the output is a code file (either C# or Visual Basic) that I can include in my project. Having the view pre-generated reduces the startup time down to 2933 milliseconds or about a 28% decrease.

2.933 seconds for an initial query doesn't mesh with the 1.866-second result calculated with the step eliminated. This issue needs further investigation.

The problem is that even 1.866 seconds is too long an initial spin up time for conventional WebForm apps.

Successive queries execute, for the most part, in 13 or 14 ms. However, one query took 24 ms. to execute. I add timers to all my WinForm Entity Framework clients and find the same anomalies in my tests.

Brian promises to "show some of the performance improvements that can be made on the query itself and how Entity SQL and LINQ to Entities perform" in his next post.

Mike Flasko Explains "Why ADO.NET Data Services?"

Mike's Why ADO.NET Data Services Framework (aka Astoria)? is a compilation of his replies to questions in the ADO.NET (Pre-Release) forum about the ADO.NET Team's "motivations for ... building Astoria."

My take is that the Entity Framework folks needed a quick and easy demo of a practical use of the Entity Data Model, just as LINQ to SQL began as a demonstration of the capabilities of LINQ as a whole. The problem, of course, is the four-second delay in delivery of the initial query result that's described in the preceding topic.

Charlie Calvert Starts "LINQ Farm Seeds" Series by Showing How to Enumerate LINQ Standard Query Operators

Charlie's LINQ Farm Seed 0: Operators in LINQ to Objects post of February 4, 2008 demonstrates the use of "LINQ to Reflection" to enumerate LINQ to Objects' Standard Query Operators. Source code is available from CodePlex.

I believe the first use of the term "LINQ to Reflection" to refer to LINQ queries that use reflection occurred in the book Introducing Microsoft® LINQ by Paolo Pialorsi and Marco Russo, which is now available as a free download from Microsoft Press. A "LINQ to Reflection" Google search receives only 6 legitimate hits, so the term isn't in widespread use.

Pablo Castro Delivers Entity Framework and ADO.NET Data Services Podcast

SQL Server MVP Greg Low, who runs the SQL Down Under podcast site, offers Pablo Castro in a 1-hour podcast described as follows:

SQL Server Product Group member Pablo Castro discusses LINQ, Entity Framework and ADO.NET Data Services for the DBA and database developer.

Listen to an MP3 or WMA version.

Off-Topic: Jacob Profitt Contests Conclusions of Canadian Test-Drive Development Paper and Its Analysis by Phil Haack

Test-driven development (TDD) is ingrained in most professional .NET programmers' approach to enterprise development. TDD implies a test-first, code-later approach, which was the subject of a Canadian research report, "On the Effectiveness of Test-first Approach to Programming." Microsoft senior program manager Phil Haack based his Research Supports The Effectiveness of TDD post of January 22, 2008 on the conclusions of the report.

However, developer Jacob Profitt more closely analyzed the study's findings and found that they lacked support for Phil's post. In fact, Jacob concluded that the study actually found:

  • The control group (non-TDD or "Test Last") had higher quality in every dimension—they had higher floor, ceiling, mean, and median quality.
  • The control group produced higher quality with consistently fewer tests.
  • Quality was better correlated to number of tests for the TDD group (an interesting point of differentiation that I'm not sure the authors caught).
  • The control group's productivity was highly predictable as a function of number of tests and had a stronger correlation than the TDD group.

So TDD's relationship to quality is problematic at best. Its relationship to productivity is more interesting. I hope there's a follow-up study because the productivity numbers simply don't add up very well to me. There is an undeniable correlation between productivity and the number of tests, but that correlation is actually stronger in the non-TDD group (which had a single outlier compared to roughly half of the TDD group being outside the 95% band).

Jacob then goes on with a detailed interpretation of the report.

I tend to code first and test later because it allows me to refine the tests to more precisely suit the project's objectives and implementation.