Wednesday, July 16, 2008

“Data Dilemma” Cover Story for Redmond Developer News’ July 15 Issue

My “Data Dilemma” cover story, sub-titled “Mapping a strategy for Microsoft's new data-programming models” for the July 15, 2008 issue of Redmond Developer News covers the ADO.NET data technologies included in Visual Studio 2008’s forthcoming Service Pack 1: ADO.NET Entity Framework, ADO.NET Data Services (formerly codenamed “Astoria”), and ASP.NET Dynamic Data.

The article includes a couple of sidebars:

The issue also has related items:

Mike’s Q & A session outlines plans for the use of Entity Framework as a future data source for data synchronization and reporting services.

For more than the single anonymous reader’s response to the “ADO.NET Entity Framework Vote of No Confidence,” see Current Commentary About the “ADO.NET Entity Framework Vote of No Confidence” Manifesto of June 28, 2008 (updated 7/8/2008.)

Enjoy!

Tuesday, July 15, 2008

LINQ and Entity Framework Posts for 7/14/2008+

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Soumitra Sengupta Explains Issues with Aligning SSDS and ADO.NET Data Services APIs

Soumitra’s Why Astoria alignment is not that trivial post of July 14, 2008 describes the pain points of aligning the SQL Server Data Services (SSDS) and ADO.NET Data Services (Astoria) APIs to simplify the transition from off-premises (cloud) to on-premises (or vice-versa) implementations of SQL Server. Following are my paraphrased versions of his three primary issues:

  1. Single (Astoria databases) versus multiple (SSDS containers) consistency domains raise issues with cross-container queries, inter alia.
  2. Multitenancy (SSDS) versus single tenancy (Astoria) require different security models.
  3. Astoria with the Entity Data Model (EDM) as the data layer requires schemas; SSDS doesn’t (but optional EDM or EDM-like schemas are planned)

Readers of this blog know I’ve been lobbying for more background details about the promised alignment of the SSDS and ADO.NET Data Services (Astoria) clients since Francois Ajenstadt, then director of project management for SQL Server, said in early March 2008: "The goal is to bring the APIs between SSDS and Astoria closer together closer to Release." Soumitra’s post is a step in the right direction.

Note: Gianpaolo Carraro, who wrote the preceding definition of multitenancy, is Director of SaaS Architecture, in the Architecture Strategy team at Microsoft. Gianpaulo’s blog is a very useful source of architectural data about Software as a Service and cloud computing. His Cloudy Future for the Enterprise and most likely for ISVs too post of June 19, 2008 is a good starting point.

Added: 7/15/2008

Marcelo Lopez Ruiz Reviews Support for the $value Keyword by ADO.NET Data Services

Marcelo originally discussed the $value near the end of his Service Operations in ADO.NET Data Services post of January 8, 2008. In his ADO.NET Data Services support for $value post of January 13, 2008, Marcelo digs deeper into the use of $value:

Let's say that customers have a "CompanyName" property. A URL such as service.svc/Customers('ALFKI')/CompanyName would refer to a resource that is an XML document with a single element called CompanyName, enclosing the value. The service will also accept service.svc/Customers('ALFKI')/CompanyName/$value, in which case the representation for the resource is just the value for the company name, with a MIME type of text/plain.

Added: 7/15/2008

Ruby Developer Has Doubts About Google App Engine

The July 14, 2008 Test Drive Google App Engine post by Juixe Software cites issues with GQL, which was “more strict and temperamental” than SQL, case sensitivity of table names, and indexes that you must define and then wait “anywhere from a few minutes to hours” to build. Juixe concludes:

This experience has t[a]ught me that learning a new platform requires learning a whole new set of known issues and limitations, and to discover new ones in the process. Anyone eager to deploy a web application on a new framework is a masochist.

Added: 7/15/2008

Eric White Shows You How to Reduce Bugs in LINQ to XML Code for Modifying XML Trees

His Writing Robust LINQ to XML Code that Performs Well post of July 13, 2008 begins:

I've made a few observations about how to write LINQ to XML code when modifying an XML tree in such a way that it becomes harder to introduce bugs.  In addition, I've also mentioned a couple of other points that you can take into consideration - you can write code that performs better if you know a bit about how LINQ to XML operates under the covers.

And offers links to three examples that use LINQ to XML to modify Open XML documents.

Added: 7/15/2008

Sybase to Release SQL Anywhere 11 in July 2008 with LINQ and Entity Framework Support

From the “Better Late Than Never Department”

A Sybase press release dated June 2, 2008 and issued at Tech*Ed Developers 2008 announced:

SQL Anywhere 11 provides several new key features for .NET developers:

  • Entity Framework and LINQ support
  • ADO.NET 3.5 Provider
  • Visual Studio 2008 Integration
  • .NET CLR stored procedures
  • Integrated full-text search and regular expression search

SQL Anywhere 11 general availability is currently scheduled for July 2008.

Prior to the release date, developers can download the Panorama beta build.

Sybase was mentioned in the “Updated Third-Party EntityClient Data Provider List” topic of my LINQ and Entity Framework Posts for 6/5/2008+ post as being “available within three months of the Entity Framework's RTM date.” However, the iAnywhere folks appear to be releasing a production managed data provider for a technology that’s still in beta.

Added: 7/15/2008

Kris Andersson Raises Issues with LINQ to SQL’s Handling of Grouped Queries with Multiple Aggregates

One of the problems with LINQ to SQL (and Entity Framework) is generating sub-optimal T-SQL syntax for “edge case” queries. (O/RM developers often use the term “edge case” for any query construct that generates poorly performing SQL.) As Kris points out in his Application Architecture - Part 4 - Data Access Layer - LINQ and Linq2SQL Links post of July 14, 2008 that:

[LINQ to SQL has a few] childhood diseases. … One of those is that grouped queries with multiple aggregates are less than optimal; the Linq2SQL provider translates such queries into one subquery for each aggregate resulting in poor I/O statistics.

Microsoft Regional Director Steven Forte, who’s a member of the newlyformed Data Programmability Group’s Advisory Council, has similar concerns. (See the “Stephen Forte Embraces the Impedance Mismatch” topic that follows.

However, Kris points out that Matt Warren’s Part X – GroupBy and Aggregates post of July 8, 2008 has the potential to solve the multiple aggregates problem by eliminating subqueries altogether. However, Kris shares my concern that LINQ to SQL might on life support and heading for the hospice.

Update 7/15/2008: Reworded first paragraph for clarity.

Stephen Forte Embraces the Impedance Mismatch

From the “How Did I Miss This?” Department

Poor-quality autogenerated SQL is just one of the issues that Stephen Forte raised in his Impedance Mismatch post of June 27, 2008 (following the “ADO.NET Framework Vote of No Confidence” ruckus.) Steve isn’t sanguine about the prospects for solving the object-relational “impedance mismatch” with O/RM tools. Steve says:

The ironic thing I’m now seeing is developers who are lazy and don't want to learn SQL using tools that will produce SQL for them. The SQL is bad, and now those same anti-SQL lazy developers are struggling to read pages of generated and near-unreadable SQL trying to solve performance problems. They’re dealing with SQL that’s more verbose and orders of magnitude harder to understand than what was needed in the first place! …

What I am saying (and have been saying for a long time) is that we should accept, no, embrace the impedance mismatch!  While others are saying we should eradicate it, I say embrace it.

ORM tools should evolve to get closer to the database, not further away.

Steve’s dismissal of O/RM tools in general and adoption of a data-first stance in particular makes me wonder why he’s on the Data Programmability Group’s Advisory Council whose other members predominately advocate domain-driven design.

Credit: I found the preceding thanks to a link from Kris Andersson’s post above to Jonas Follesoe’s ORM – We’re still stuck in Vietnam post of July 1, 2008 that contained a link to Steve’s post. Both posts have been added to my Current Commentary About the “ADO.NET Entity Framework Vote of No Confidence” Manifesto list.

Matt Warren Continues His IQuerable Coding Rampage with Part 11

Matt’s LINQ: Building an IQueryable Provider - Part XI post of July 14, 2008 is drawing near the finish line. This episode covers:

  • More Query Operators
  • Distinct
    • Skip, Take,
    • First, FirstOrDefault,
    • Single, SingleOrDefalut
    • Any, All
    • Contains
  • Framework Methods and Properties
  • Parameter[ized Queries]
  • Simpler Queries
  • Compiled Queries
  • Unit Tests[!]

Matt says, there’s “More posts to come.” The question now is: “Who will be first with a third-party, LINQ to SQL-enabled data provider?”

The ADO.NET Team Completes the EDM Tools Series with Part 4 for EdmGen2.exe

The EDM Tools | Options (Part 4 of 4) post of July 14, 2008 says:

Now, we combine the code examples in the preceding parts, along with a few extra functions, to make a command-line tool we’ll call EdmGen2.exe.

EdmGen2.exe is similar in functionality to EdmGen.exe, but understands how to read & write the EDMX file format.

EdmGen2.exe’s source code is available from the MSDN Code Gallery.

Scott Guthrie Finally Surfaces with the “Higher Level Features” Added by ASP.NET MVC Preview 4

Following a one-month+ hiatus, Scott’s ASP.NET MVC Preview 4 Release (Part 1) post of July 14, 2008 begins:

The ASP.NET MVC team is in the final stages of finishing up a new "Preview 4" release that they hope to ship later this week.  The Preview 3 release focused on finishing up a lot of the underlying core APIs and extensibility points in ASP.NET MVC.  Starting with Preview 4 this week you'll start to see more and more higher level features begin to appear that build on top of the core foundation and add nice productivity.

There are a bunch of new features and capabilities in this new build - so much in fact that I decided I needed two posts to cover them all.  This first post will cover the new Caching, Error Handling and Security features in Preview 4, as well as some testing improvements it brings.  My next post will cover the new AJAX features being added with this release as well.

And then goes on for several feet of illustrated demonstrations of Filter Interceptors and the following filters: OutputCache, HandleError, and Authorize. There’s also an AccountController class and help for unit testing TempData.

Sunday, July 13, 2008

LINQ and Entity Framework Posts for 7/7/2008+

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Comparison of Entity Framework and LINQ to SQL Projects Using a 120-Table Database

Kristofer Andersson of Huagati Systems, Bangkok, is in the process of developing an airline passenger reservation system (APRS) example and has posted a database diagram and T-SQL DDL script to create the 120 tables of its data model in his Data Model - Basics - Part 10 - Overview #2 and SQL-DDL script post of June 29, 2008.

I created the database in a local SQL Server 2005 Express instance and wrote simple LINQ to SQL and ADO.NET Entity Framework (EF) projects to compare the characteristics of the two data access layer approaches. Tests showed that it took 24 times longer to initialize the EF than the LINQ to SQL version (9.426 vs. 0.394 seconds).

Precompiling the view with a T4 template from the ADO.NET Team’s How to use a T4 template for View Generation post reduced the initialization time to 2.063 seconds, which is still about five times longer than LINQ to SQL. However, a substantial workaround is required to prevent an exception when running the EF project from its Release (or Debug) *.exe file. See my Comparison of Entity Framework and LINQ to SQL Projects Using a 120-Table Database post of July 12, 2008 for more details.

Added: 7/13/2008

Charlie Calvert Continues LINQ Farm Renaissance with More on Set Operators

Charlie’s LINQ Farm: More on Set Operators post of July 12, 2008 provides “simplified examples of how to use each of the operators:” Union, Intersect, Distinct, and Except. “[A]nd then ends with a more complex example that shows how the operators might be used in a real world setting.”

Added: 7/13/2008

Guy Burstein Delivers EntityDataSource Screencast from Microsoft’s Silverlight Streaming Site

Guy’s ‘EntityDataSource Screencast – Hosted on Silverlight Streaming post of July 12, 2008 demonstrates how to use the ASP.NET EntityDataSource. The associated How To: Host Your Screencasts on Silverlight Streaming post of the same date is an illustrated tutorial for taking advantage of Microsoft’s free Silverlight Streaming service.

Added: 7/13/2008

Mike Flasko Alex James Promotes ADO.NET Data Services’ Uniform Interface with Varying Data Source Schemas

Mike Alex says in Using the Uniform Interface Exposed By Data Services of July 11, 2008:

In some of our past posts, we've discussed one of the benefits of data services is that the interface to such a service is uniform.  This means that how you interact with each service (issue a query, insert a new entity, etc) is the same across all services regardless of the schema being exposed.  This uniform interface leads to one of the value propositions of data services, which is the ability to create reusable code for service consumption.  That is, one can now write reusable client libraries, LINQ providers, app/UI components, widgets, etc that are able to work across any data service.

Hopefully, the preceding will apply to the promised unified client library for SQL Server Data Services.

Added: 7/13/2008 Updated 7/21/2008: Attribution error

Use the ASP.NET Cache for LINQ to SQL Generic Lists

J Sawyer, a Microsoft developer evangelist in Houston demonstrates a Cool way to do ASP.NET Caching with Linq with C# and VB in his detailed post of July 11, 2008. I haven’t tried it, but the technique should apply equally to Lists created from Entity Framework’s ObjectContexts.

Added: 7/13/2008

Pablo Castro Responds with His Software Development Meme

Julie Lerman tagged me and I tagged Pablo. Here’s his Software Development Meme of July 10, 2008.

I notice Pablo included SQL in his list of programming languages he has used. Neither Julie nor I included SQL (which we both use constantly) in our lists. I’ve never thought of SQL as a programming language. I’ve always considered SQL as a set-based data manipulation language, although T-SQL stored procedures probably qualify it as a PL. What’s your take?

Added: 7/13/2008

Maíra Wenzel Starts ASP.NET Dynamic Data - Tips and Tricks Series

Maíra’s first episode, ASP.NET Dynamic Data - Tips and Tricks #1 of July 11, 2008, covers the dreaded The control 'DataSource1' does not have a TableName property and a table name could not be inferred from the URL exception.

Added: 7/13/2008

Samuel Jack Uses a Hydraulic Analogy for Reporting Progress of and Canceling LINQ Queries

His Reporting Progress During LINQ Queries of July 7, 2008 and Canceling Long-Running LINQ Queries of July 10 use “LINQ Pipeline Monitors,” which Sam analogizes as:

One way to think of LINQ queries with sequences is as a pipeline, or an oil refinery. You have the data source (often a collection) acting like the oil well, but producing items. Items flow along the pipeline (think of the "."s in the expression as the pipe). Some items are removed from the pipeline by filters - Where clauses; some items are converted to other products - Select clauses; finally, just as oil is cracked, and graded into gasoline and diesel etc. items in LINQ queries are often grouped and sorted. In oil pipelines there are also flow meters, measuring how much oil is going through the pipes. If we can create the equivalent of a flow-meter for LINQ queries, then we've got our means of reporting progress.

Having been in the petroleum flowmeter business for many years (in an earlier avatar), the analogy appealed to me.

Added: 7/13/2008

Tech*Ed 2008 IT Pro Session DAT251: Introduction to SQL Server Data Services (Video)

From the session’s description:

Cloud computing offers many benefits to business customers looking to increase operational efficiencies, lower operational costs and/or move to a Software as a Service (SaaS) model. This session provides an overview of Microsoft SQL Server Data Services (SSDS) and walks through on-boarding and typical usage scenarios to show how SSDS brings the power of scale-free storage, distributed computing, and reliable data processing to new and existing business solutions. Speaker: Soumitra Sengupta.

For more details and a link to the one-hour video segment: Tech*Ed 2008 IT Pro Session DAT251: Introduction to SQL Server Data Services (Video).

Alex James on Entity Framework v2 Design: Query Caching

Alex’s Transparent Caching Support in the Entity Framework post of July 9, 2008 lays out the design of a proposed query caching implementation. From the introduction:

Entity Framework can be extended to handle data caching in a transparent way, so that any application using it can take advantage of caching with little or no modification. In Entity Framework V1 it is possible to implement transparent caching using a custom provider as demonstrated in EFCachingProvider sample (TBD). We are considering adding caching as a first-class concept in Entity Framework V2 so that it will no longer be necessary to use a wrapper provider approach.

The design caches queries, not EntityObject instances, so the queries can be reused.

Steve Naughton Starts a ASP.NET Dynamic Data Series for Custom Pages

Steve concludes that there are three now four types of ASP.NET Dynamic Data custom pages:

  • Standard Custom Page based on an existing PageTemplate and customised in the DynamicData\CustomPages folder.
  • A completely Custom Page again in the DynamicData\CustomPages folder.
  • Standard ASP.Net Page with Dynamic Data features added to take advantage of the FieldTemplates.
  • Added 7/13/2008: A DetailsView and a GridView using Validation Groups

Following are posts with instructions and sample code for the first two four types:

I’ll update this post when Steve completes Part 3.

Updated: 7/13/2008

David Hayden Demonstrates LLBLGen Pro New LINQ to SQL Templates with an Update Model Feature

Entity Framework has an Update Model from Database feature but LINQ to SQL doesn’t (and probably never will.) Dave’s LLBLGen Pro LINQ To SQL Templates with Model Refresh from Database - O/R Mappers post of July 9, 2008 explains the benefit of the template’s “update model” feature:

The attraction to using something like the LLBLGen Pro Designer instead of the Visual Studio 2008 LINQ To SQL Designer with LINQ To SQL is that the LLBLGen Pro Designer will pick up changes to the database and update the model accordingly. For very few tables this is not a big deal, but as the number of tables grows and you have DBA's making changes to the database schema it is very easy to get your model out-of-sync with the database. With LLBLGen Pro you can just refresh the catalog and it will pick up those changes for you. With the LINQ To SQL Visual Designer in Visual Studio 2008 I don't know of a real easy way to find those changes and update your model.

And demonstrates how to use it.

Jonas Stawski: Use the Code-Generated Initialize() Method to Detach and Initialize Each Entity

Jonas is “working on an article on how to use LINQ to SQL in an N-Tier environment with ASP.NET (without WCF),” so he needs to detach and reinitialize each entity. His LINQ To SQL Generic Detach post of July 9, 2008 explains a simple approach to doing the job.

Jonas (an MVP) promises to post a notice when the article is published.

Sprint 3 for SQL Server Data Services Will Add Several New Features

An e-mail from the SQL Data Services (SSDS) team dated July 8, 2008 announces that Sprint 3 will add support for Blobs, timestamp versioning, ETag, HEAD, If-Match, Accept and a new content type when the latest update goes live.

Get more details in Sprint 3 for SQL Server Data Services Will Add Several New Features of July 9, 2008.

Matt Warren Surfaces with Part 10 of His LINQ: Building an IQueryable Provider Saga

After a six-month hiatus, Matt posted a very lengthy LINQ: Building an IQueryable Provider - Part X chapter on July 8, 2008. Topics covered are:

  • Grappling with GroupBy and Aggregates
  • The GroupBy Operator
  • Aggregate Binding
  • Aggregate Rewriting
  • Taking it for a Spin

As usual, a C# tour de force with downloadable source code.

Elisa Flasko’s “Achieve Flexible Data Modeling With The Entity Framework” Article in MSDN Magazine for July 2008

Elisa’s “Achieve Flexible Data Modeling With The Entity Framework” is an overview of the Entity Framework and Entity Data Model that will be distributed with Visual Studio 2008 SP1.

The blurb says “Here the author answers questions regarding the Entity Framework and provides an understanding of how and why it was developed” and the article covers:

    • The philosophy behind the Entity Framework
    • The Entity Data Model
    • Querying, mapping, and n-tier development

Eric White Posts Updated LINQ to XML Code for the Open XML SDK

Eric describes is Open XML SDK and LINQ to XML post of July 8, 2008 as follows:

In this post, I’m presenting some code that uses the Open XML SDK and LINQ to XML to query an Open XML document. I’ve posted on this before, but this version is cleaner and smaller. It also is an example of code written in the functional style.

The most common need is to retrieve the paragraphs of a word processing document. But I've also had the need to dig into a spreadsheet, and get the data out for a row, a set of rows, or all the cells in a column. You can think of the code presented here as a (very) small library on top of the Open XML SDK.

LINQ for PHP Clone Sighted

Mike Borozdin asks Is PHPLinq As Cool As Real LINQ? in his post of July 4, 2008. Unfortunately, Mike says the answer is “No” because PHPLinq’s query keywords aren’t part of the language. They’re strings.

Alternative Approaches to Cascading Deletions in LINQ to SQL

“Salyse” analyzes various approaches to implementing cascading deletions on the client or server side in his “Cascading Deletes in LINQ to SQL” article of July 7, 2008 for the Code Project.

Phani Raju Analyzes 1:1 Associations in ADO.NET Data Services

Phani’s Working with Associations in ADO.NET Data Services Beta 1 post of July 2, 2008 provides a detailed analysis of managing 1:1 associations in Astoria.

He promises that his next post will cover the more common 1:n associations.

(Phani is a new arrival on the Astoria team; his previous position was a Microsoft product support specialist (PSS). Thanks to the Astoria Team for the heads up on the blog.)

Live Demo of Custom Pager Implementation for LINQ to SQL

Muhammad M. Mosa Soliman’s Building Custom Paging with LINQ, ListView, DataPager and ObjectDataSource, Different Paging Method post of July 8, 2008 combines techniques from Justin Etheridge’s Efficient Paging in SQL Server 2005 and Kevin Hazzard’s Efficient Paging in SQL Server via LINQ post and provides source code and a live demo of his interpretation.

Steve Naughton Continues His Permutations and Combinations of the FileImage_Edit FieldTemplate

Steve’s Dynamic Data: Part 2 - FileImage_Edit FieldTemplate of July 7, 2008 implements these new ideas:

  1. Image Aspect Ratio lock, keeping the proportions of the image the same when resized.
  2. Doing the same as DynamicDataFutures disabling EnablePartialRendering when the FileImage is in an Edit or Insert page.

Microsoft’s Live Expo Classifieds Database Bites the Dust; Is Google Base Next?

Online classified advertising databases are having a tough time competing with ClickZ’s Craigslist Rivals Struggle to Succeed in Web Classified Space post of July 7. The article says Google Base “garnered about 743,000 unique visitors in May compared to Craigslist's 26.7 million, according to Nielsen Online.” I wrote about what became Live Expo and Google Base during their gestation period (see Windows Live "Fremont" vs. Google Base Classifieds) of November 29, 2005.)

Microsoft will terminate Live Expo on July 31, 2008.

Google shut down it’s Froogle shopping service in April 2007. The http://froogle.google.com now displays an anemic Google Product Search page. URL Perhaps Google should call it quits for Google Base, too.

David Robinson Cooks Concurrent Data Access and Concurrency in Batch Updates Best Practices Soup

Dave’s Best Practice - Take Advantage of Concurrency post of July 6, 2008 uses the term concurrency in an uncommon context: Concurrent CRUD operations against the same authority by multiple front ends. His summary:

  • Multithread your SSDS calls when appropriate
    • Use Asynchronous access patterns
    • Helps combat latency
    • Many front-end servers able to handle many requests
  • Things to watch out for
    • SSDS Supports Simple Concurrency Today
    • Multiple entity updates need to be managed by your application
    • No support for batching today, but its coming

Saturday, July 12, 2008

Comparison of Entity Framework and LINQ to SQL Projects Using a 120-Table Database

Kristofer Andersson of Huagati Systems, Bangkok, is in the process of designing and implementing an airline passenger reservation system (APRS) sample project with Visual Studio 2008 SP1, SQL Server 2005/2008, ASP.NET 3.5, AJAX, LINQ and LINQ to SQL.

His latest Tools - Part 1 - Add-ins - Maintaining naming conventions and keeping the Linq2SQL DBML in sync with the database post of July 7, 2008 describes two add-ins for LINQ to SQL:

Add-in 1. Converts table and column names to .NET naming conventions, such as airport to Airport, and airport_code to AirportCode.

Add-in 2. Applies some database schema changes to the DBML file. The current version supports new or removed columns, column nullability and datatype changes, and tables removed. A future version will support added tables and foreign keys.

You can download the add-in and its DLL from a link on the above page.

Updated 8/19/2008: My Entity Framework Instantiation Times with a 120-Table Database post today provides a comparison with Julie Lerman's model-creation and save times for a 400-table database in her Entity Framework Designer and Large Databases post of the same date.

Earlier Posts about the UI, Application Architecture, and the Data Model

His June 2008 blog entries cover the Web UI, application architecture, and data model:

It’s obvious that Kristofer knows whereof he speaks when it comes to APRS; his CV says:

[For Results Reservation Technologies, Ltd., he] converted an old airline reservation system from 16-bit Windows to 32-bit and subsequently to re-developed key parts of the system, extended and improved functionality. The system as ResultS and has been deployed at Spirit Airlines, Nok Air, FlyNordic, AeroCalifornia, Mango, Viva Aerobus, ExpressJet and several other LCCs [Low-Cost Carriers].

His Data Model - Basics - Part 10 - Overview #2 and SQL-DDL script post of June 29, 2008 includes links to a database diagram as a 6964 x 4770 px Overview2.png file and a 5,138-line CreateResDB_sql.txt T-SQL script to create the final data model, which contains 120 tables.

Using the ResDB Data Model to Compare LINQ to SQL with Entity Framework Project Characteristics

Following are comparisons of executable, XML file and class size, as well as initialization times of simple Windows form LINQ to SQL and Entity Framework projects generated from an SQL Server 2005 Express ResDB database with no sample data:

Comparison

LINQ to SQL

Entity Framework

Executable file size, KB

530

1,124

Memory footprint (private working set), KB

3,425

3,568

XML file (DBML/EDMX) size, KB

213

1,090

XML file(DBML/EDMX) lines

2,176

16,297

Class file (C#) size, KB

1,438

2,220

Class file (C#) lines

50,890

44,313

Application initiation time, s.

0.042

0.039

Data/ObjectContext initiation time (first), s.

0.394

9.426

ObjectContext initiation time (precompiled), s.

N/A

2.063

Data/ObjectContext recreation time (next), s.

0.004

0.021

Notes:

  1. Executable file sizes and times are for Release builds.
  2. Application initiation time starts the Stopwatch timer before invoking InitializeComponent() and stops it in the MainForm_Load() event handler.
  3. Data/ObjectContext initiation time (first) times initially creating, adding, and querying an aircraft entity to the aircraft(s) collection within a using block.
  4. Data/ObjectContext recreation time (next) times repeated instances of the above (cached) operation.
  5. Update 7/12/2008: Precompiled views with the CSharp.Views.tt T4 template from the ADO.NET Team’s How to use a T4 template for View Generation post of June 20, 2008.

Conclusion: Entity Framework takes 24 times longer to execute the first update/query combination and 5 times longer for subsequent (compiled and cached) operations with no sample data loaded in the underlying database. Despite the EF project’s 2:1 ratio of executable file sizes and larger XML and class file sizes, the memory footprint of the two projects is almost identical.

Precompiling Views and Fixing the EntityConnectionString

Update 7/12/2008: Precompiling the views reduces the initiation time from 8.426 to 2.063 seconds and the ratio of EF:LINQ to SQL time from 24:1 to 5.25:1 when built and run with F5. Even 2 seconds is a significant startup hiatus.

However, launching from the Debug or Release Huagati.exe file and clicking the Create Object Context button throws the following exception:

The specified named connection is either not found in the configuration, not intended to be used with the EntityClient provider, or not valid.
  at System.Data.EntityClient.EntityConnection.ChangeConnectionString(String newConnectionString)
  at System.Data.EntityClient.EntityConnection..ctor(String connectionString)
  at System.Data.Objects.ObjectContext.CreateEntityConnection(String connectionString)
  at System.Data.Objects.ObjectContext..ctor(String connectionString, String defaultContainerName)
  at HuagatiModel.HuagatiEntities..ctor() in C:\Huagati\HuagatiEntityCS\Huagati.Designer.cs:line 231

Line 231 is the first line of the default constructor:

        public HuagatiEntities() : 
                base("name=HuagatiEntities", "HuagatiEntities")
        {
            this.OnContextCreated();
        }

which doesn’t work from the Huagati.exe file.

You must build the entity connection string with the following code (from Danny Simmons’ update Entity Framework FAQ #8 post) and pass it to the constructor overload to solve Huagati.exe’s problem:

SqlConnectionStringBuilder sqlBuilder = new SqlConnectionStringBuilder();
sqlBuilder.MultipleActiveResultSets = true;
sqlBuilder.DataSource = @".\SQLEXPRESS";
sqlBuilder.InitialCatalog = "Huagati";
sqlBuilder.IntegratedSecurity = true;

EntityConnectionStringBuilder entityBuilder = new EntityConnectionStringBuilder();
entityBuilder.ProviderConnectionString = sqlBuilder.ToString();
entityBuilder.Metadata = "res://*/";
entityBuilder.Provider = "System.Data.SqlClient";

using (HuagatiEntities hgCtx = new HuagatiEntities(entityBuilder.ConnectionString))

Thursday, July 10, 2008

Tech*Ed 2008 IT Pro Session DAT251: Introduction to SQL Server Data Services (Video)

Microsoft released a one-hour video capture of Soumitra Sengupta’s DAT251 session at Tech*Ed 2008 IT Pro, Introduction to SQL Server Data Services on July 9, 2008. Here’s the session description:

Cloud computing offers many benefits to business customers looking to increase operational efficiencies, lower operational costs and/or move to a Software as a Service (SaaS) model. This session provides an overview of Microsoft SQL Server Data Services (SSDS) and walks through on-boarding and typical usage scenarios to show how SSDS brings the power of scale-free storage, distributed computing, and reliable data processing to new and existing business solutions.

I strongly recommend that anyone interested in SSDS who didn’t attend this session watch the entire segment.

Content and Rating

The agenda for Soumitra’s session followed that of Nigel Ellis’s MIX 08 BT05 presentation of the same name, but with greater concentration on data architecture:

  • Why SQL Server Data Services (SSDS)
    • Value Proposition
    • Example Scenarios
  • SQL Server Data Services (SSDS) Data Architecture
    • Data Model
    • Platform
    • Services Ecosystem
  • SSDS Service Delivery
    • Business SLA
    • Roadmap
    • Resources

Soumitra provided a much more detailed description of SSDS data management, including replication techniques, than I’ve seen or heard previously. I’ve been working with SSDS since early in the beta, but I now have a much clearer understanding of the intricacies of clustering, data partitioning, and transaction processing that distinguish SSDS from other purported “cloud databases,” such as Amazon SimpleDB and the Google App Engine.

He also confirmed that SSDS’s forthcoming schema will be based on the Entity Data Model and mentioned that support for business intelligence operations was in the SSDS team’s future plans.

Session Rating: A

(I was surprised by what appeared to be a sparse audience. I expected more interest in SSDS on the part of IT pros, including DBAs, than developers.)

Wish List

Here’s what I’d like to see after watching Soumitra’s presentation for the first time:

  • More information on the SQL Server Integration Services (SSIS) demo for uploading and downloading data.
  • Downloadable SSDS Management Studio and SSDS Command Line applications.
  • More details on batched operations.

The RSS feed for all Tech*Ed 2008 sessions released in streaming video format (80 as of 7/10/2008) is here.

Recent Related Posts

Wednesday, July 09, 2008

Sprint 3 for SQL Server Data Services Will Add Several New Features

An e-mail from the SQL Data Services (SSDS) team dated July 8, 2008 announces that Sprint 3 will add support for Blobs, timestamp versioning, ETag, HEAD, If-Match, Accept and a new content type when the latest update goes live.

Following are the details (verbatim from the e-mail except for my notes) for those who haven’t yet been inducted into SSDS’s limited beta program:

Blob Support via the REST interface:

  • Support for storing unstructured Blob data has been the most requested feature.
  • Blob support via the SOAP interface is coming.
  • In the beta release, Blobs are limited to 100 megabytes in size.

Version metadata property has been changed from an integer to a large integer:

  • Version numbers are now based on a timestamp and will not monotonically increase as before.

ETAG and HEAD support via REST, with similar functionality via SOAP

  • Support for If-Match and If-None -Match via the REST interface. This will allow you to verify that you have a current Entity enabling you to use resources more efficiently.
  • Support for "Accept" header via the REST interface. This will allow you to denote what content you want returned from the service. For instance when retrieving a Blob, the "Accept" header will allow you to choose between the Blob content, or the metadata properties associated with it.
  • Note: This is consistent with ADO.NET Data Service’s concurrency management approach (see the Astoria team’s Optimistic Concurrency & Data Services post of 4/22/2008.)

A new content type - "application/x-ssds+xml" has been added to the service

  • This content type is for storing XML content in the service.
  • Existing "application/xml" content type is being phased out in a future sprint.
  • The "application/x-ssds+xml" content type should be used for all entities except those containing blob content. Blob entities should instead use a content type value which best reflects the type of blob data stored. It's important to note that the value chosen here will be used later when attempting to retrieve the different parts of an entity by using the, "Accept" header.
  • While the "application/xml" content type will still be supported in the short term, we highly recommend that you begin to migrate your applications to the new "application/x-ssds+xml" content type.

Notes: The SSDS team says the “updates will be coming in a few weeks” and all data uploaded to Authorities will be deleted with one week’s notice. You can expect instructions for implementing the client side of new features to be posted in the SSDS blog, SSDS forum and the main SSDS page on MSDN.

I had expected Sprint 3 to implement full-text search on SSDS’s front-end. It was rolled out on the back end in Sprint 2 (see Soumitra Sengupta’s Confusion about Full Text Search in SSDS post of June 22, 2008).

Hopefully, Ryan Dunn will extend his SSDS client class library (described in Working with Objects in SSDS Part 3 and earlier) to accommodate the new features.

Monday, July 07, 2008

Seven Security Risks the SQL Server Data Services Team Must Mitigate for Enterprise Users

Network World’s Gartner: Seven cloud-computing security risks article of July 2, 2008 carries this lead:

Cloud computing is fraught with security risks, according to analyst firm Gartner. Smart customers will ask tough questions, and consider getting a security assessment from a neutral third party before committing to a cloud vendor, Gartner says in a June report titled “Assessing the Security Risks of Cloud Computing.”

Although the article cites Amazon EC2 and the Google App Engine as examples of cloud computing, the following seven security issues are even more important for data-centric services such as SQL Server Data Services (SSDS):

  1. Privileged user access
  2. Regulatory compliance
  3. Data location
  4. Data segregation
  5. Recovery
  6. Investigative support
  7. Long-term viability

The fact that Microsoft is operating the service probably satisfies long-term viability concerns, although Microsoft has announced that its Live Expo online classifieds database will go dark on July 31, 2008. And the SSDS team has stated that an Authority can specify the data center at which its Containers of Entities are stored. However, promotional material mentions “security features” repeatedly but doesn’t describe what these features are. The team needs to provide potential SSDS users with early insight into how SSDS plans to mitigate the remaining five security issues.

The article’s detailed synopsis of the US$195 report is well worth a read.

Related OakLeaf posts:

Other related posts:

Sunday, July 06, 2008

LINQ and Entity Framework Posts for 7/2/2008+

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Updated: July 4, 2008 1633 PDT: Additions; July 5, 2008 Updates, July 6, 2008 Additions

Soumitra Sengupta: SQL Server Data Services is Built on SQL Server 2005 SP2, not 2008

In an update to his Roger Jennings talks about SSDS in a Visual Studio Magazine Article post of July 2, 2008, Soumitra adds the following in response to a question I posed:

I thought Dave Campbell in one of his press interviews had mentioned it but I could not find the reference.  Sorry Roger.  So here is an answer to your question - SSDS is built using SQL Server 2005, SP2 as the starting code base.  That is the starting point and we made changes to it.  Over time, some of these changes will make its way into the SQL Server mainline and SQL Server 2008 will make its way into SSDS.  We wanted to proceed in parallel as fast as we could and this was the best way to do it.

I found a reference to SQL Server 2005’s use with SSDS in SQL Server Data Services: Good news, bad news for partners of March 14, 2008 by Barbara Darrow (at MIX 07):

The services are based on the current SQL Server 2005 release. SQL Server 2008 is due later this year.

Apparently she was quoting Niraj Nagrani, senior marketing manager for SQL Server.

This means that references to SQL Server 2008 and its features in my “Test-Drive SQL Server Data Services” cover article for the July 2008 issue of Visual Studio Magazine are misleading. Sorry about that; mea culpa. The fact that “SQL Server 2008 will make its way into SSDS” ameliorates the error somewhat.

Jamie Thompson Analyzes Query Resultset Paging by SQL Server Data Services

SQL Server Data Services (SSDS) batches query resultsets that return more than 500 entities. Jamie’s SSDS: I'm gonna get 500 results and then I'll get 500 more... post of June 27, 2008 answers several common questions about the paging process.

Note: If you’re wondering why SSDS posts qualify for inclusion in this blog, here’s the answer:

  • SSDS uses a LINQ-like query language (but supports only a few LINQ Standard Query Operators)
  • SSDS schemas will use the Entity Data Model when schemas are enabled later this year. Use of the entity data model will make implementing the promised on-premises SSDS clone practical.

Shawn Wildermuth Continues His IUpdatable Implementation for NHibernate Project

As Shawn observed in the first member of this series, Implementing IUpdatable (Part 1) of July 1, 2008:

[O]nly the Entity Framework currently support[s] the IUpdatable interface (and [it] is in fact implemented inside of ADO.NET Data Services not directly in Entity Framework).  This means that DataSets and LINQ to SQL do not support it either.

IUpdatable is a member of the System.Data.Services namespace, which VS 2008 SP1’s System.Data.Services.dll provides. The Project Astoria Team Blog’s IUpdatable & ADO.NET Data Services Framework post of April 10, 2008 provides more information on the IUpdatable interface.

In his Implementing IUpdatable (Part 2) post of July 4, 2008, Shawn finds NHibernate’s “type information to be more complete and faster” than his cached reflection data, so he refactors his GetResource() and CreateResource() method calls and completes the easy parts with GetProperty() and SetProperty().

He promises to deal with the “hard parts” in his next post.

(Shawn has “been looking at the story around non-Entity Framework models through a Data Service and thought that NHibernate through a Data Service would be a great example” for a forthcoming article about Silverlight 2 and ADO.NET Data Services.)

Added: July 4, 2008 1630 PDT Updated: July 5, 2008

Nigel Ellis: Role-Based Authorization Plans for SQL Server Data Services

SQL Server Data Services (SSDS) relies on Secure Socket Layer (SSL) v3 encryption to secure an authority’s user ID and password for authentication, as well as to protect data transfer from exposure by interception. Delegated authorization is required to restrict additional authenticated users to read-only or read-write access. Several potential SSDS testers (including me) have requested (but not received) details on plans for implementing role-based authorization.

Channel9’s Istvan Cseri and Nigel Ellis: SQL Server Data Services Architecture interview by SSDS evangelist Ryan Dunn posted April 7, 2008 contains quite a bit of useful (and previously unpublished) information about SSDS architecture. My Nigel Ellis Says SQL Server Data Services Will Use the Entity Data Model For Schemas post of July 3, 2008 is a transcript of Nigel Ellis’s comments about SSDS’s future schema model.

My Nigel Ellis Discusses Role-Based Authorization Plans for SQL Server Data Services post of July 4, 2008 contains a transcript I made of Nigel’s remarks about SSDS security. The transcript starts at 00:31:05, immediately following the schema discussion.

Added: July 4, 2008 1111 PDT

Mike Amundsen Posts Sample SQL Server Data Services Provisioning Application

Mike’s New Provisioning Client Example Web App thread of July 4, 2008 in the SQL Server Data Services (SSDS) - Getting Started forum includes a link to the SSDS Provisioning Client Web service that lets you manage authorities, containers, and entities.

My Mike Amundsen Posts Sample SQL Server Data Services Provisioning Application post includes a couple of screen captures for those of you without an SSDS beta account.

Added: July 4, 2008 1105 PDT

Stefan Tilkov’s REST Anti-Patterns Essay

InfoQ’s blurb for this article offers a brief description of Stefan’s REST Anti-Patterns essay of July 2, 2008:

In this article, Stefan Tilkov explains some of the most common anti-patterns found in applications that claim to follow a "RESTful" design and suggests ways to avoid them.

If you’re using REST protocols with SQL Server Data Services or ADO.NET Data Services, this article is a must-read. 

Added: July 4, 2008 1105 PDT

Vote for Mike Taulty as Computer Weekly’s King of Programming and Technical Blogs

Click the screen capture at Mike’s Outrageous Self-Promotion :-) post of July 3, 2008.

Added: July 3, 2008 1700 PDT

Julie Lerman’s Software Development Meme

Julie says about her DevSource Software Development Meme post of July 3, 2008:

There's a new meme going around the blogs and I've been called out by Shawn Wildermuth to participate - it's the "my programming career" questionnaire. My history is not quite as illustrious as Shawn's when it comes to programming, but I do have a few very unique skeletons in my closet!

Added: July 3, 2008 1700 PDT

Frans Bouma Continues LINQ to LLBLGen Pro Feature Highlights Series

In Linq to LLBLGen Pro: feature highlights, part 2 of July 3, 2008 Frans continues from where he left off in Linq to LLBLGen Pro: feature highlights, part 1 with a discussion of:

  • Exclusion / inclusion of entity fields in a query
  • Hierarchical fetching of entity graphs using Prefetch Paths

This post is recommended reading for the ADO.NET team’s EF program managers and developers.

Added: July 3, 2008 1700PDT

David Robinson: SQL Server Data Services Article in MSDN Magazine’s July Issue

Develop Robust and Scalable Apps with SQL Server Data Services” is a lengthy article by SSDS evangelist Dave Robinson for MSDN Magazine’s July 2008 issue that covers these topics:

  • The SSDS data model
  • Managing entities, containers, and authorities
  • Creating a sample Web application
  • Class serialization and deserialization

Coverage is about the same as that of my article.

Added: July 3, 2008 1700 PDT

John Papa: n-Tier Architecture for Entity Framework with WCF and an MVC Site with a WPF Presentation Layer in MSDN July Issue

John’s Using the Entity Framework in a Layered Architecture post of July 3, 2008 points to his “The Entity Framework In Layered Architectures” article for MSDN Magazine’s “DataPoints” July 2008 column. The article includes downloadable code for a Northwind-based demonstration site that uses WPF forms for views of a ASP.NET MVC site.

Updates require updating the table each time an entity is added, updated, or deleted because the ObjectContext can’t be serialized. Inability to serialize the ObjectContext prevents accumulating changes in a disconnected fashion and applying them all at once with a SaveChange() invocation.

Replacing “Layered” with “Tiered” would improve the post and article titles’ accuracy.

Added: July 3, 2008 1300 PDT

Nigel Ellis: Entity Data Model Schemas Coming for SQL Server Data Services

My Nigel Ellis Says SQL Server Data Services Will Use the Entity Data Model For Schemas post of July 3, 2008 includes a partial transcription from Channel9’s Istvan Cseri and Nigel Ellis: SQL Server Data Services Architecture interview by SSDS evangelist Ryan Dunn.

The transcription covers Nigel’s commentary about the schema feature coming in a future version of SSDS.

Added: July 3, 2008 1300 PDT

Steve Naughton Codes ASP.NET Dynamic Data’s Missing FileImage_Edit FieldTemplate

His Dynamic Data: FileImage_Edit FieldTemplate post of July 3, 2008 offers two alternatives for the missing FileImage_Edit FieldTemplate:

  1. Get a list of images from the specified folder [ImageUrl("~/images/{0}.png")] and let the user choose
  2. Let the user upload the file to the [ImageUrl("~/images/{0}.png")] folder and the pass the filename to the DB field.

Scott Hanselman Shows How to Take Advantage of XML Literals with ASP.NET MVC

Scott’s The Weekly Source Code 30 - VB.NET with XML Literals as a View Engine for ASP.NET MVC post of July 2, 2008 uses VB’s LINQ to XML literals to generate XML lists for an MVC editlink class. (Of course, all other .aspx pages used C# in the codebehind.)

You also can watch the XML Literals Panel from TechEd 2008 video on a similar topic.

Added: July 3, 2008 1300 PDT

Marcin Dobosz Posts ASP.NET Dynamic Data July 2 (2008) Update

His Dynamic Data Futures 7/2 update posted item of July 2, 2008 lists the new features added to the Dynamic Data Futures section on CodePlex:

  • Added DynamicHyperLink control. It generates links to Dynamic Data tables for all actions and works both in data-binding and declarative scenarios. 
  • Added Url and EmailAddress field templates. 
  • Modified ForeignKey_Edit field template to display a [Not Set] entry for required columns in insert mode. Added a RequiredFieldValidator to ensure that an entry is selected if the column is required.
  • Modified routes in Global.asax to generate pretty URLs using the new PrettyDynamicDataRoute.
  • Changed FilterAttribute.Order property to match the behavior of ColumnOrderAttribute.Order property. Now each filter has a default Order of 0; negative numbers can be used to push filters to the front, while positive numbers can be used to push filters to the back.

Added: July 3, 2008 1300 PDT

Ryan Dunn Completes a REST Library for CRUD Operations on SQL Server Data Services

The default SQL Server Data Services page on MSDN promises in it’s “Application Agility” section that:

A rich client library in C# or VB providing LINQ query support will be available.

So far, no such “rich client library” has appeared. However, Ryan Dunn is rolling his own client API for REST library in his “Working with Objects in SSDS” series. His Working with Objects in SSDS Part 3 post of July 2, 2008 completes the library by adding querying capability with an expression tree and expression visitor, as well as paging.

Sample code is available from the MSDN Code Gallery.

Added: July 3, 2008 1300 PDT

Shawn Wildermuth’s New Blog Uses Entity Framework as the Data Source and the ASP.NET Routing Framework Instead of .aspx Links

Shawn says in his Links and New Site Technology post of July 2, 2008, the first on his new blog that replaces adoguy.com:

The use of the Entity Framework and LINQ was pretty straightforward. I used LLBLGen Pro in my old site to do the data access and it held up great.  I only switched so I could test out the Entity Framework on a production-ish system, not because of any limitations in the old code. Creating a model with the Entity Framework was a snap. My data is not complicated so I didn't have any complex scenarios.  The only thing I did do was change the collection names from singular to plural. Being able to use LINQ to do my queries, search and paging was just spectacularly useful. I am completely in the LINQ camp now.

ASP.NET Dynamic Data also uses the new routing library.

Added: 7/2/2008 1740 PDT

“Test-Drive SQL Server Data Services” Article Published

My “Test-Drive SQL Server Data Services” cover article for the July 2008 issue of Visual Studio Magazine is available from the above link or by clicking the cover image.

Here’s the deck: Microsoft readies its new "cloud database" by hosting beta versions of REST and SOAP protocols for performing CRUD operations on clustered, customized SQL Server 2008 instances. These instances are organized as massively scalable, super-reliable, highly available Web services.

My “Test-Drive SQL Server Data Services” Article Published post of July 2, 2008 has more background data about the article, sample code, and an interesting question about the architecture of SSDS.

Ian Cooper Requests the Data Programmability Group to Increase Support for LINQ to SQL

Ian is just one of a number of independent .NET developers who recommend LINQ to SQL as the better choice for object-oriented data-intensive .NET applications. He concludes his Showing some support for LINQ to SQL post of July 2, 2008 with:

Today, while I would not recommend using the Entity Framework I would recommend looking at LINQ to SQL. Everything needs evaluation for your own needs, but unlike EF, LINQ to SQL is a better contender for OO approaches today.

Sasha [Goldshtein] points out when looking at how LINQ to SQL is suprising people who had been misinformed as to what if offered and how we should use it, the noise in the blogsphere from Entity Framework supporters seems to have drowned out the value of LINQ to SQL as an ORM. Indeed I believe the Data Team's own pitching of LINQ to SQL as a RAD tool is an underestimates the product.

As I mentioned in my “Sasha Goldshtein Believes LINQ to SQL is Under-appreciated” topic of my LINQ and Entity Framework Posts for 6/30/2008+ post, it’s my opinion that:

  • Microsoft’s SQL Server Data Programmability (DP) group is fully committed to Entity Framework as its data platform going forward
  • The DP group is in the process of deprecating LINQ to SQL by not-so-benign neglect.

Mike Taulty Substitutes Entity Framework for LINQ to Entities in His Concurrency Management Analysis

In On Entity Framework, Concurrency of June 2, 2008, Mike tackles the same issues for EF that he handled for LINQ to SQL in his On LINQ to SQL, Concurrency and Timestamps post of July 1:

[H]ow we can detect concurrency problems with Entity Framework in order to ensure that when we submit changes to the DB we first;

  1. Present the user with a list of all the concurrency errors in one go
  2. Present the user with their changes versus the DB's current data
  3. Ask the user how to go about resolving each issue
  4. Repeat until all the changes get to the DB one way or another

Similar (but not identical) problems occur for transaction rollback with both data providers.

Simon Segal Implements the Specification Pattern for LINQ to Objects and LINQ to SQL with Dependency Injection

Simon’s LINQ to SQL, the Specification Pattern and Dynamic Queries post of July 2, 2008 begins:

I have just finished working on an implementation of the Specification Pattern that will work with LINQ (to objects) using Lambdas to provide the test logic and in the case of LINQ to SQL using the same testing predicates as search predicates when querying the database.

And provides a starting code snippet. He continues:

My next goal is to enable the eager and lazy loading in these scenarios dealt with by using something similar to Fetching Strategies and have the Repository load them using dependency injection which will enable me to plug in the correct fetching strategy at will.

Shortly I will post some code that demonstrates all this with a Repository and not the standard .DBML file, DataContext way of doing things as per above.

The later episodes sound interesting.