Saturday, April 12, 2008

LINQ and Entity Framework Posts for 4/7/2008+

Note: This post is updated daily or more frequently, depending on the availability of new posts.

Comparison of "DataStores in the Cloud" Started

My Comparing Google App Engine, Amazon SimpleDB and Microsoft SQL Server Data Services post of April 9, 2008 is the beginning of a full-length comparison of Amazon SimpleDB, SQL Server Data Services, and Google App Engine beta versions.

As of April 10, 2008, the introduction and Data Model sections are complete.

Update 4/12/2008: Security and authentication topics expanded and moved to the "API" section, "API" section updated with query syntax and API limitations.

Added: 4/10/2008, Updated 4/12/2008

Jeff Currier Explains How to Use the cURL Utility with SQL Server Data Services

According to its author, Daniel Stenberg:

cURL is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS and FILE. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, Kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.

Jeff's cURL'ing up with SQL Server Data Services post of April 12, 2008 shows you how to use cURL at the command prompt to query an SSDS authority, as well as create containers and create, update and delete entities from file-based templates.

Added: 4/13/2008

Rick Strahl Encounters an Anomaly in LINQ to SQL Behavior with Object Tracking Disabled

Object tracking creates a very small performance hit, so it's a common practice to set DataContext.ObjectTrackingEnabled = false if you don't intend to update entities. However, you receive an "Object Reference not set to an instance of an object" exception if the entity has deferred loading associations. Eager loading is required in this case.

Rick's LINQ to SQL ObjectTrackingEnabled and Deferred Loading of Related Data of April 12, 2008 notes:

It turns out that this is a known issue that has been reported on Connect before.

and he goes on to quote Alex Turner's response to the Connect Bug report, which explains the reason for throwing the exception.

Added: 4/13/2008

Sahil Malik Writes a Windows Communication Foundation Tutorial

You need to know how to create WCF services and clients if you intend to try n-tier deployment of LINQ to SQL or the Entity Framework. Sahil's irreverent approach to his tutorials makes them more interesting (and informative) than the ordinary step-by-steps from MSDN and elsewhere.

He starts the series with four (count 'em, four) posts on April 12, 2008:

  • What is WCF? explains the basics of Service Oriented Architecture (SOA) and what makes WCF different from its Web service predecessors.
  • Writing the WCF Hello World App shows you how to create a WCF service class library, add a WCF service, and edit and implement the ServiceContract interface. Sahil also explains the mysteries of the App.config file (for a class library, yet!)
  • Writing your first WCF client demonstrates using dynamic proxy generation with metadata from the mex endpoint or metadata predistributed to the client in a DLL.
  • Host a WCF Service in IIS 7 & Windows 2008 - The right way shows you how to deploy a WCF service to IIS 7.0, a topic that Sahil says appears "nowhere else on the web." The process is by no means straightforward.

It will be interesting to see if more episodes follow.

Added: 4/13/2008

Beth Massi Starts LINQ to SQL N-Tier Smart Client Series

Beth's LINQ to SQL N-Tier Smart Client - Part 1 Building the Middle-Tier post of April 12, 2008 covers the following topics about the middle tier:

  • The Great Debate - DataSets or objects?
  • Going N-Tier with LINQ to SQL
  • Contract First
  • Disconnected Retrieval
  • Disconnected Updates and Deletes

Full VB 2008 code for the middle tier is included. Beth promises that her next post in the series "we'll build our client form and implement a simple technique for change tracking."

Added: April 12, 2008

Guy Burstein: Download the .NET 3.5 Enhancements Training Kit Preview

Guy's .NET 3.5 Enhancements Training Kit is available for Download says:

The Developer and Platform Evangelism Group in Microsoft has created a new training kit for all the enhancements for the .Net Framework 3.5:

  • ASP.NET MVC
  • ASP.NET Dynamic Data
  • ASP.NET AJAX History
  • ASP.NET Silverlight controls
  • ADO.NET Data Services
  • ADO.NET Entity Framework

Download the training kit (34.9 MB)

The April Content Preview contains Developer and Platform Evangelism Group's elementary Hands-on Labs for ADO.NET Entity Framework, ADO.NET Data Services, and ASP.NET Dynamic Data. Content for the Presentations, Demos, Screencasts, and Resources "will be added in a future release of the training kit.

Added: April 12, 2008

Upcoming Microsoft Developer Webcasts about LINQ and ADO.NET Data Services

From Scott Lum's MSDN Online Media Blog: Upcoming Microsoft Developer Webcasts: WCF, WF, Linq, AJAX, IIS 7.0, MS Sync Framework post:

MSDN Webcast: geekSpeak: Inside LINQ to XML with Paul Sheriff (Level 200)
In this episode of geekSpeak, Microsoft Regional Director Paul Sheriff explains how to put LINQ to XML to work in your applications. Guest Presenters: Paul Sheriff, President, PDSA, Inc. He has authored several books, webcasts, videos, and articles on the Microsoft .NET Framework, Microsoft SQL Server, and Windows SharePoint Services.
4/16/2008 12:00 PM - 1:00 PM Pacific Time | Duration: 60 Minutes

MSDN Webcast: LINQ Features in Visual Studio 2008 (Level 200)
Think you have heard all there is to know about Microsoft .NET Language-Integrated Query (LINQ)? Think again! In this webcast, we dazzle and amaze you with our staggering array of wondrous tools to make your LINQ experience a memorable one! Presenter:: Zain Naboulsi, Developer Evangelist, Microsoft CorporationTechnology is useless if it does not serve a clear purpose. That's Zain's philosophy. In fact, showing people how technology can make their lives better delivers his greatest on-the-job thrills
4/16/2008 1:00 PM - 2:30 PM Pacific Time | Duration: 90 Minutes

MSDN Webcast: Offline Data Synchronization with the Microsoft Sync Framework (Level 100)
In this webcast, we give an overview of the Microsoft Sync Framework and describe its key components that enable any data store to exchange information. We examine the built-in synchronization providers, Sync Services for ADO.NET, Sync Services for File Systems, and Sync Services for FeedSync, in addition to explaining how to implement custom synchronization providers for custom data sources. Lastly, we take a look at an example of using the Microsoft Sync Framework along with the new Microsoft SQL Server Data Services to provide synchronization capabilities across multiple applications, like Microsoft Office Access and Outlook. Presenter: Lindsay Rutter, Developer Evangelist.
4/17/2008 10:00 AM - 11:00 AM Pacific Time | Duration: 60 Minutes

Related Topic

MSDN Webcast: Database Features in Visual Studio 2008 (Level 200)
Like working with data? The Microsoft Visual Studio 2008 development system comes with many new and improved data tools. Who could say no to those cute little controls? Join this webcast for the ultimate database tools experience that you can have without a laser light show! Presenter: Zain Naboulsi, Developer Evangelist.
4/17/2008 1:00 PM - 2:30 PM Pacific Time | Duration: 90 Minutes

Added: April 12, 2008

Pratik Patel Describes the IUpdatable Interface for ADO.NET Data Services

The new IUpdatable interface supports create, update, and delete (CUD) operations on ADO.NET Data Services entities. Pratik's IUpdatable & ADO.NET Data Services Framework post of April 10, 2008 to the Astoria Team Blog describes the interface's following 10 methods:

  • CreateResource()
  • GetResource()
  • SetValue()
  • GetValue()
  • SetReference()
  • AddReferenceToCollection()
  • RemoveReferenceFromCollection()
  • DeleteResource()
  • SaveChanges()
  • ResolveResource()

Pratik promises "some examples and the sequence of calls made on the IUpdatable interface for each of them" in a follow-on post, as well as "features like ETag, Update interceptors" in later posts.

Added: April 10, 2008

White Paper: How to Choose a Data Synchronization Technology

Liam Cavanaugh's How to Choose a Data Synchronization Solution for Offline and Collaboration post of April 10, 2008 says the following about this Microsoft whitepaper about choosing between multiple sync technologies;

[W]e have posted a document titled "How to Choose a Data Synchronization Solution" which is an attempt to differentiate the offline and collaborations solutions available from Microsoft.  Hopefully this will help you better understand the various data sync technologies as well as to help answer the question of which technology is best suited for your particular needs.

Added: April 10, 2008

Rob Conery Starts a Series of MVC Storefront Tutorials

Rob describes his series as follows:

I'm creating an ongoing series of webcasts and blog posts, documenting the building of an eCommerce storefront using ASP.NET MVC. It's important to understand that this application has not been built yet - you're gonna help me do it :).

The goal is to build this application as openly, and "currently" as possible using an Agile approach with TDD. My goal is to post to this series at least twice a week as I build this application, taking your feedback and adjusting/discussing issues as they come up.

The project uses LINQ to SQL as its default data source, but Rob plans to plug in NHibernate and Subsonic later in the series.

Here are the episodes as of April 10, 2008:

So far, Rob has received plenty feedback on his "Agile approach with TDD" from TDD purists. His MVC Storefront: Intermission post of April 8, 2008 documents his reaction to the comments to the first two posts.

Added: April 10, 2008

Scott Guthrie: ASP.NET Dynamic Data Preview Available

Scott's ASP.NET Dynamic Data Preview Available post of April 10, 2008 includes the following description:

This new dynamic data preview now works with the standard built-in ASP.NET data controls (GridView, ListView, FormView, DetailsView, etc).  The dynamic data support enables these controls to automatically handle foreign-key relationships.  For example, on a gridview you'll now get automatic friendly name display of foreign key column values and automatic drop-down list selection support of these values when in edit mode.

The preview doesn't yet support Entity Framework. According to David Ebbo's day-earlier post of the same name:

Note that even though the drop has the ability to support Entity Framework (in addition to Linq To Sql), you will not quite be able to use it yet because it targets an Entity Framework build that is newer than the latest public build.  But don't worry, it won't be long before the missing pieces that will allow this are made available.

Added: April 10, 2008

Chris Rock Continues His LINQ to SQL Dynamic Queries Series

His LINQ TO SQL Dynamic Queries #3 (And's & Or's Together) post of April 10, 2008 extends his quest for dynamic Where clauses in LINQ to SQL expression. His previous two posts on this topic are:

LINQ TO SQL Dynamic Queries (And Operators)
LINQ TO SQL Dynamic Queries (Or Operators)

Added: April 10, 2008

Damien Guard Creates a Tiny Class to Enumerate Enums with LINQ

Damien's Using LINQ to foreach over an enum in C# post of April 10, 2008 describes a small Enums class to enable:

foreach(CustomerTypes customerType in Enums.Get<CustomerTypes>())

Cool.

--rj

Calais.NET Abstracts RDF Metadata from HTML, XML and Plain Text Documents with LINQ to XML

Chris Fulstow, an ASP.NET developer in Sydney, has written a API wrapper for the Calais Web service "that abstracts away web service details like complex XML input parameters and verbose RDF output data by processing the data with LINQ to XML." Chris's Calais.NET - generating metadata using the Calais web service post of March 21, 2008 explains his open-source project, which is licensed under the Microsoft Reciprocal License (Ms-RL) and available for download from Codeplex.

You pass the URL for an HTML, XML or plain text document to the Calais.NET API wrapper and receive a IEnumerable<string> of Resource Description Format (Semantic Web) entities that the Calais Web service detects in the document. For the current list of entities that Calais claims to detect, go to Calais Semantic Metadata.

Calais, which is sponsored by Reuters, delivers a free Web service that has this lofty goal and components:

The Calais initiative seeks to help make all the worlds content more accessible, interoperable and valuable via the automated generation of rich semantic metadata, the incorporation of user defined metadata, the transportation of those metadata resources throughout the content ecosystem and the extension of it’s capabilities by user-contributed components.

The Calais initiative has three major components:

  • The Calais Web Service is the core. The Web service provides for the automated generation of rich semantic metadata in RDF format.
  • A series of sample applications to demonstrate how the web service can be utilized and serve as a starting point for other’s development activities.
  • Active support for developers that want to incorporate Calais capabilities in their applications and web sites.

The Calais initiative is sponsored by Reuters and built on ClearForest technology.

Calais was the primary topic of The semantic web: Start making sense story for The Economist's "Technology Monitor" column of April 9, 2008. The column also mentions a number of other startups whose business plans relate to extracting metadata from ordinary text

Added: April 10, 2008

Danny Simmons: Visual Studio 2008/.NET Framework 3.5 SP1 to be Entity Framework's Shipping Vehicle

Danny wouldn't provide a date, but advises in EF will ship with SP1 of VS 2008/.Net Framework 3.5 (April 9, 2008) that EF will ship in VS2008 SP1 "this summer" and that EF beta bits will be in the forthcoming VS 2008 SP1 beta.

Danny also says that he'll be at DevConnections in Orlando in a couple of weeks giving these two EF presentations:

VMD314: Entity Framework in the Real World
Daniel Simmons
Come see the Entity Framework in action! Check out an exciting new open source application built on the Entity Framework. During this session we’ll discuss and demo the use of the Entity Framework to build a LOB application in the healthcare vertical.

VDM215: Entity Framework: Application Patterns
Daniel Simmons
Microsoft is introducing the ADO.NET Entity Framework and the Entity Data Model to help developers code against first-class business objects when creating business applications. Like any new technology, most of the information available on the topic focuses on what the Entity Framework is, what are its constituting components and related aspects. This session takes the audience beyond the “what“ of the Entity Framework, instead delving into various application scenarios and approaches to application architecture to show how one can use the Entity Framework today. We shall discuss the role of Entity Framework in Web, rich-client, and service-oriented applications.

Ryan Dunn Posts First SQL Server Data Services Sample App Available to Download or Test

It might have a funky name but Ryan's PhluffyFotos Sample Available post of April 9, 2008 is the first of what I hope will be several sample applications that use SSDS as their data source. Following are highlights:

    • This sample has a LINQ-to-SSDS provider in it.  You will notice we don't use any strings for queries, but rather lambda expressions. I had a lot of fun writing the first version of this and I would expect that there are a few more revisions here to go. Of course, Matt Warren should get a ton of credit here for providing the base implementation.
    • This sample also uses a very simplistic ASP.NET Role provider for SSDS. Likely updates here will include encryption and hashing support.
    • We have a number of Powershell cmdlets included for managing authorities and containers.

You must have credentials for the SSDS limited beta to run the sample code. If you don't, here's a link to an online version.

Beth Massi Offers Link List for VS 2008 N-tier Application Resources

Beth has accumulated a feed of Resources for Building N-tier Applications (April 9, 2008), which includes N-Tier and Remote Applications with LINQ to SQL.

Liam Cavanagh Demonstrates FeedSync Between Machines with the Sync Framework

Using the Microsoft Sync Framework to upload data to SQL Server Data Services (SSDS) isn't a built-in feature yet, but the following sample project will give you a leg up when Sync becomes the favorite way of keeping offline data sources synchronized.

According to Liam's Synchronization of Browser Favorites using FeedSync and the Microsoft Sync Framework of April 8, 2008:

At first, creating a new provider and figuring out how to make it sync can be a little intimidating. In order to help you through the process, we will walk through the creation of a simple provider that can sync between directories that contain Favorites. We will start first with the metadata store that the provider will use, since considering how metadata will be stored is the first step a provider writer should take when writing a provider. On top of this, we will build the provider, which will provide our actual sync capability. Then, we will use this provider to produce and consume feeds so that we can share our favorites between computers.

Liam presented Sync to the Web - Enable Offline Access to Web Data from any Data Store, which covers the use of Sync Framework with SSDS, at MIX 08.

Pablo Castro wants Input for Batching ADO.NET Data Services Requests

Pablo's Batching Data Service Requests post of April 6, 2008 casts a net for suggestions from developers and data architects for the syntax of batched CRUD operations with Astoria.

Ryan Dunn Interviews SSDS Architects Istvan Cseri and Nigel Ellis for Channel9

The blurb for the Istvan Cseri and Nigel Ellis: SQL Server Data Services Architecture interview posted April 7, 2008 says:

SQL Server Technical Evangelist Ryan Dunn sits down and talks turkey with two of the creators of SQL Server Data Services (SSDS), Architect Istvan Cseri and Development Manager Nigel Ellis. Istvan and Nigel dig into how to design applications for SSDS and cover a number of the interesting aspects of working with SSDS in terms of features, design, and security.

If you're interested in working with SSDS, don't miss this video segment.

Debra Dove: Avoid SQL Server 2008 (Katmai) Connection Problems

If you're using a pre-release version of SQL Server 2008 (Katmai) you've probably performed the required patches. If not, read Debra's Connecting to Pre-Release Versions of SQL Server 2008 – Part Deux of April 7, 2008.

Note that neither LINQ to SQL or Entity Framework will support SQL Server 2008's GEOMETRY or GEOGRAPHY datatypes or SPARSE columns.

Paul Stovell Demonstrates SyncLINQ for Silverlight in a Live Application

Paul's SyncLINQ for Silverlight post of April 7, 2008 has an embedded Silverlight app that demos data binding with his open source SyncLINQ project.

Note that the entries you add don't persist.

Updated Version of "Rumor: Google to Compete with SimpleDB and SSDS with Bigtable as a Web Service"

My post about the Google App Engine, which undoubtedly will compete with SQL Server Data Services, has been extensively updated to cover it's release to limited beta status on Monday night, April 7, 2008.

Tuesday, April 08, 2008

Rumor: Google to Compete with SimpleDB and SSDS with Bigtable as a Web Service

Update 4/8/2008: It's no longer a rumor.

Google announced the new Google App Engine service last night at their second invitation-only, outdoor Campfire One event. The rumor that Bigtable would become accessible to developers as a highly-scalable data store in the cloud was confirmed and a demarcation line established between free and paid services. What surprised most observers was support for custom Python Web applications as the front end to the Bigtable implementation on the Google File System (GFS).

Here's how the first post on the Google App Engine blog describes the development environment's features:

  • Dynamic webserving, with full support of common web technologies
  • Persistent storage (powered by Bigtable and GFS with queries, sorting, and transactions)
  • Automatic scaling and load balancing
  • Google APIs for authenticating users and sending email
  • Fully featured local development environment

By the time I signed up, the initial 10,000 invitations were taken, but the App Engine SDK includes a Cassini-like Web server for local development and lets you define data stores locally. I've downloaded the SDK and have it running the Greetings app (HelloWorld with Python 2.5.2 under Windows Vista Ultimate.

I'm exploring the persistent storage features of the SDK in the docs now and will be comparing them to Amazon SimpleDB and SSDS in a forthcoming post. In the meantime, it's interesting to note that Google's data-access wrapper to Bigtable features transactions for entity updates, a rich collection of data types for properties (with Google Base overtones), an SQL-like query language (GQL), plus emulations of many:one associations with the Reference data type and one:many associations with the List data type.

There's no question that the Google App Engine will compete with SSDS. The question is: Will Google enhance their Datastore API with a Web-facing front-end that doesn't require custom Python code to provision domains and perform CRUD operations?

Mary Jo Foley's Google App Engine: When will Microsoft field a competitor? post of April 8, 2008 discusses the Google App Engine with an SSDS bent.

Later Updates

Update 4/10/2008: App Engine World, which claims to be "Your one-stop resource for Google App Engine has launched with the following announcement: "Coming soon! A complete community for Google App Engine Developers." Until then, the site offers a few links through DZone and a few other A-list sites. Amazingly, the site doesn't have an Atom or RSS feed.

The Official Google Data API Blog's Release the hounds: Support for App Engine and Contacts API post of April 8, 2009 announces:

The 1.0.12 release of the gdata-python-client has a couple of nifty new features which I thought are worth mentioning. First, is a new module which allows the Python library to be used on Google App Engine. Since the App Engine runs a sandboxed Python interpreter, HTTP calls must be made using one of the App Engine's APIs: urlfetch. After some refactoring of the library's code, I was able to write a drop-in replacement for making HTTP requests, so that the gdata.service module can be used on Google App Engine.

It looks to me like this will enable direct HTTP request/response operations against GAE's Datastore API. Confirmation by a GData expert would be appreciated.

Update 4/9/2008: Henry Work posted TechCrunch Labs: Our Experience Building And Launching An App On Google App Engine on 4/8/2008; it describes creating and uploading in four hours a simple Python application that accesses the Data Store API. You can try the app Henry built at appengine.crunchbase.com. The post also has a screen capture of GAE's dashboard.

Redmonk analyst Michael Coté's Your Data in the Cloud - URL-based computing, SimpleDB, Astoria, etc. post of December 23, 2007 echoed my observation in Amazon Announces Beta of SimpleDB Web Services in the Cloud (December 14, 2008) that the hosted version of ADO.NET Data Services (Astoria) represented an interesting model for a future Microsoft service. Michael also notes that:

[T]hese new technologies need to be completely self-service. If a developer has to ever talk with a human from the company or team offering the project, something has gone wrong. You can sling out all sorts of “complex problems need complex solutions” chaff, but the historic fact remains that the new, simple solution tends to win out versus the new, “complete but complex” solution. And of course, there’s some wiggle room with the difference between “complex solution” and “easy to use complex solution.”

The need to write complex Python apps to provision and conduct CRUD operations on "data in the cloud" will be a major issue with GAE.

Henry Blodgett chimes in with Google, Amazon Lead Disruptive Cloud Computing Wave, Microsoft Again Behind Curve of April 9, 2008. He said that the Google App Engine was "not disruptive technology" in his Google Taking On Amazon With App Engine, because:

Google isn't aiming at the heart of of a competitor's business. Office is core to what Microsoft (MSFT) does, but Amazon's services are sidelights designed to take advantage of the company's prodigious resources -- when they work.

Update 4/7/2008: According to TechCrunch's Michael Arrington in his Major Google Announcement Monday Evening: Is It BigTable? post of 4/6/2008:

Google is hosting the second of their developer events, called “Campfire One,” on Monday (April 7) evening. Multiple people have forwarded email invitations to me for the event, where Google promises they’ll be “unveiling another exciting technology” to the developer community. ...

My guess is that Google will be announcing the launch of web services that will compete head on with those offered by Amazon and others. The anchor for these services, we hear, is their internal database system called BigTable. Google has definitely briefed press on the imminent launch of BigTable as a web service, although as we said last week we haven’t been contacted.

Original 4/5/2007 story starts here:

TechCrunch blogger Mark Hendrickson relies on "a source with knowledge of the launch" to report the following rumors in Source: Google To Launch BigTable As Web Service of April 4, 2008:

Google may be releasing BigTable, its internal database system, as a web service to compete with Amazon SimpleDB, according to a source with knowledge of the launch. There are also rumors that press is being pre-briefed on the product, although we haven’t been contacted by Google. ...

The decision to open up BigTable would seem to mark Google’s challenge to Amazon Web Services (AWS) suite, which also includes the Elastic Compute Cloud (EC2) for cloud processing power and Simple Storage Service (S3) for cloud storage.

If Google does indeed announce public access to BigTable next week, expect the company to follow up with cloud storage and processing solutions as well, since there are substantial synergies between the three.

Bigtable: A Distributed Storage System for Structured Data is a paper by nine Google Employees that describes Bigtable in detail. (The paper was one of two Best Papers at the 7th USENIX Symposium on Operating Systems Design and Implementation, OSDI 2006.)

An interesting fact from the paper's Table 2: Google Base runs on Bigtable and, when the paper was prepared in mid-2006, Google Base had 10 terabytes of data in 10 billion cells. Google Base, which is still in beta, became publicly available in November 2005 but is not as widely used as other free Google apps.

My conclusion based on the Bigtable paper and SimpleDB documentation: Bigtable's architecture and implementation have more in common with Amazon's Simple DB than either database has with SSDS:

  • Bigtable and SimpleDB aren't relational database management systems (RDBMSs); both resemble multidimensional indexed maps of attribute/value pairs. SSDS attempts to hide the fact that it's an RDBMS.
  • Bigtable and SimpleDB have latency, which results in eventual consistency, although it appears from Table 2 of the Bigtable paper that no-latency operation is an option; SSDS has no latency by design.
  • Bigtable values are an "uninterpreted array of bytes" and SimpleDB stores only strings; SSDS has string, number, datetime, binary and boolean datatypes.

Update 4/6/2008: This post became a Techmeme discussion item on April 5, 2008 and resulted in the blog's highest Saturday traffic in several months.

Dare Obasanjo's Amazon SimpleDB: The Good, the Bad and the Ugly post of December 21, 2007 provides a more detailed comparison of the structure of SimpleDB and Bigtable.

Dan Farber adds his 2 cents in Is Google bringing Bigtable out of the closet? of April 4/ 2008.

Dave Winer Reads the Tea Leaves

Charging for Web services would be a new business model for Google, but Dave Winer's prescient mock interview with a pig at a Walnut Creek stoplight on March 29, 2008 went (in part) like this:

Pig: You know how Amazon has all those great web services.

Dave: Yes, ... I use them and they're great.

Pig: Well how would you like to get all those services and more, and get to run software in Google's cloud, just like all the people at Google do?

Dave: Yes, I would. ... How much would it cost?" I asked

Pig: That's the best part. ... For a guy like you, a blogger, with modest needs, it would be free.

The obvious implication is that the service wouldn't be free to all users.

Dave continued the next day in Why would Google Web Services cost $0?:

Google Web Services, or GWS, is the hypothetical competitor to Amazon Web Services that I wrote about yesterday.

The first question that comes up is how can they afford to give it away? That came up in yesterday's comments and the answer is important enough to deserve its own blog post. ...

Flipped around, I don't see why Amazon charges me to use AWS. I think I produce as much value for them as I use just by writing about it, but they haven't been willing to bend (not that I've asked them to). If there was no cost to it, I'd use their services for new things that I'm not willing to try as long as I have to pay. I know that because there are projects I've not attempted because the cost was prohibitive. ...

Dave went on to surmise that Google gives away its services to increase the value of acquisitions that already use its technology and concluded:

What's hard to believe is how much of a running start Microsoft, Yahoo, and Google have been willing to let Amazon have.

Google might start with a free service like Google Base, but I'd bet that they would establish a tiered tariff with only heavy hitters paying a SimpleDB-like fee. The approach would be similar to Google Enterprise's Google Apps Premier Edition, which costs $50/year for an account. Google claims to have 100,000 Google Apps customers.

Note: Dave said in a 4/4/2008 comment to the TechCrunch post:

Mark, pretty sure the price will be $0, with some ceiling that keeps other infrastructure vendors from reselling Google’s service.

An exciting development for sure!

Update 4/6/2008: GigaOm's Kevin Kelleher comments on Dave's post in How Google Can Eat Amazon’s Lunch of March 31, 2008. Dave responds in a comment, Matthew Ingram posts Google: Why not make the cloud free?, and links to James Hamilton's Third Party Service Platform From Google? post of March 30, 2008. (James is an architect on the Windows Live Platform team. His Microsoft Research Bio says "Before Windows Live, James was General Manager of the Microsoft Exchange Hosted Services team which was formed as part of the FrontBridge Technologies acquisition.")

Al the Folknologist analyzes Dave's post in Is there a silver lining in Google's Cloud? of April 1, 2008 with the assumption that Google would offer an EC2 clone with a Cloud Virtual Machine, which could "run any suitable Java byte code that has been built with the CVM toolkit." His New value, old money, familiar issue? post of the next day suggests that cloud-based business data storage is analogous to storing money in banks.

On the Other Hand ...

Kevin Burton said on December 14, 2007 (the day that Amazon announced SimpleDB) in Google vs Amazon in Open Infrastructure:

Last time I checked, Amazon’s bandwidth pricing was insane. It would literally cost us 3x more to host Spinn3r. Granted, we process a LOT of data (from 60-160Mbits per month) but when your startup is successful you don’t want to burn it all your AdSense revenues on bandwidth invoices courtesy of Amazon.

It looks to me as if there's something wrong with Kevin's data volume figure (I process more Mbits than that per month) but it's doubtful if Google's terms of service would let folks use BigTable in direct competition. (Spinn3r is a pay-to-play blog search API.)

Friday, April 04, 2008

LINQ and Entity Framework Posts for 4/3/2008+

Note: This post is updated daily or more frequently, depending on the availability of new posts.

John Papa Writes Yet Another Article on LINQ Query Operators

John's Standard Query Operators with LINQ post of April 5, 2008 points to his eponymous article for his "Data Points" column in MSDN Magazine. I thought the market for magazine articles that demo the Standard Query Operators with arrays of numbers and Northwind entities was saturated. Obviously I was wrong.

Added: 4/6/2008

Rumor: Google to Compete Against SimpleDB and SSDS with Bigtable as a Web Service

TechCrunch blogger Mark Hendrickson relies on "a source with knowledge of the launch" to report rumors about Google opening Bigtable to developers in his Source: Google To Launch BigTable As Web Service post of April 4, 2008. Mark says the announcement is scheduled for the week of April 6.

Dave Winer must have had premonitions of the possible event when he wrote a pair of prescient posts that I quote in my Rumor: Google to Compete with SimpleDB and SSDS with Bigtable as a Web Service post of April 5, 2008.

My conclusion: Bigtable's architecture and implementation have more in common with Amazon's Simple DB than either database has with SSDS:

  • Bigtable and SimpleDB aren't relational database management systems (RDBMSs); both resemble multidimensional indexed maps of attribute/value pairs. SSDS attempts to hide the fact that it's an RDBMS.
  • Bigtable and SimpleDB have latency, which results in eventual consistency, although it appears from Table 2 of the Bigtable technical paper that no-latency operation is an option; SSDS has no latency by design.
  • Bigtable values are an "uninterpreted array of bytes" and SimpleDB stores only strings; SSDS has string, number, datetime, binary and boolean datatypes.

Added: 4/5/2008

Updated: 4/6/2008

Randolph Cabral Compares ActiveRecord and Unit of Work Patterns with LINQ to SQL

Randolph's Active Record Vs. Unit of Work post of April 3, 2008 is an essay on the relative merits of and problems with the two design patterns. He cites cardinality issues with updates to associated entities as an Active Record problem. The additional code required to reattach detached objects with LINQ to SQL in Web applications is the major hurdle with the Unit of Work pattern.

Added: 4/5/2008

Julie Lerman Finds Anomaly in the EDM Designer's Update Model from Database Feature

Julie describes in her  EF UpdateModel and naming conflicts post of April 5, 2008 the agonies of creating entities in the EDM Designer, then adding tables for them to a persistence database, and trying to use the Update Model from Database feature to hook the model to the store. In this case, a table and association with identical names blocked the update.

Julie reports that she gave up on the process of updating a simple model from an existing database in her Model first with EDM? post a few hours later.

A message by a member of the ADO.NET team in the ADO.NET Entity Framework and LINQ to Entities (Pre-release) forum, which I no longer can find, mentioned that the team was working on a separate application to create a (presumably SQL Server-only) database from an EDM. This is good news.

However, Julie was attempting to define a CSDL file that can be used with multiple sets of SSDL and MSL files to accommodate different back end databases, so building the database from the model wouldn't appear to solve her problem.

Added: 4/5/2008

Update 4/6/2008: Julie added the following in a comment at the end of the post: "I actually had some success with the model first tests and updated my blog post."

Emad Ibrahim Creates a LINQ to SQL Unit Test with LINQ to Mock (Moq)

Emad's Unit Test Linq to Sql in ASP.Net MVC with Moq post of April 4, 2008 shows how he:

    1. Modified the Moq version of the MvcMockHelpers class from Scott Hanselman and added two more methods to mock an authenticated user
    2. Added a couple of properties to [the] test class (MessageControllerTest) to make it easy ... to access the controller and view engine in all the test methods
    3. Created [a] test method which is going to call a Create method in [the] MessageController and pass it a string.

His post also include links to related resources.

Added: 4/4/2008

Julie Lerman Posts Eight Entity Framework Tutorials on DataDeveloper.net

In addition to writing her book, Julie found time to write eight Entity Framework tutorials. Here's the list with links:

  1. 101 Tutorial:Creating an ADO.NET Entity Framework Entity Data Model
  2. 101 Tutorial: Use an Entity Framework Entity as a WinForms Data Source
  3. 101 Tutorial: ASP.NET DataBinding with the Entity Framework
  4. 101 Tutorials: WPF Databinding with Entity Framework
  5. Many to Many Relationships in the Entity Data Model
  6. Using Stored Procedures for Insert, Update & Delete in an Entity Data Model
  7. Naming conventions in an Entity Data Model: Cleaning up the wizard-generated EDM
  8. Adding items to a LINQ Query of anonymous types - after the fact

Julie says in her Eight Entity Framework Tutorials on DataDeveloper.NET post of April 4, 2008 that the "'101 Tutorials'" are designed to make sure new programmers don't get completely lost."

Added: 4/4/2008

Jesse Liberty: Send LINQ to SQL Data via WCF to a WPF DataGrid in a Silverlight 2.0 Project

The Silverlight.net site's Displaying SQL Database Data in a DataGrid using LINQ and WCF post of April 4, 2008 is a detailed, fully illustrated tutorial for integrating LINQ to SQL, Windows Communication Foundation (WCF) asynchronous service calls, and the new Windows Presentation Foundation (WPF) DataGrid in a Silverlight 2.0 Beta 1 project.

Here are Jesse's links to the source code and a version of the tutorial in PDF.

According to the tutorial's introduction:

The application we’ll build will combine a number of different new skills:

  • Connecting [asynchronously] to a WCF Web Service
  • Using LINQ to query and retrieve data that our Silverlight Application can use
  • Using the DataGrid control to display data

From a strictly Silverlight perspective, building the Web Service and using LINQ are related skills but beyond the scope of these tutorials. That said, we’ll examine how they intersect with Silverlight.

The tutorial is based on material contributed by the Swiss MSDN Team who've produced a 15:00 video segment on it that you can watch at Silverlight 2 Beta1 + WCF + LINQ to SQL = a powerfull combination.

Jesse offers a simpler LINQ to SQL to WPF demo (not Silverlight) in his LINQ To SQL blog post of March 26, 2008.

It should be easy to substitute Entity Framework and LINQ to Entities for LINQ to SQL.

Added: April 4, 2007

Greg Beech Maps LINQ C# Query Expressions into the Underlying LINQ to Objects Method Calls

Greg is a UK developer who's a Development Lead at blinkBox Entertainment currently focusing on the middle tier who's started a "Translating C# 3.0 query syntax for Linq-to-Objects" series. Greg observes:

I haven't found any that go into the gritty details of how the query syntax is translated into the underlying method calls. It's a gap this series aims to fill.

After an explanation of how the compiler handles the translation from query expressions to method calls, he maps the two in a small diagram near the end of each post:

Following are the first three members of the series:

  • Translating C# 3.0 query syntax for Linq-to-Objects, Part 3: Where (4/3/2008)
  • Translating C# 3.0 query syntax for Linq-to-Objects, Part 2: SelectMany (3/29/2008)
  • Translating C# 3.0 query syntax for Linq-to-Objects, Part 1: Select (3/25/2008)

    I'll update this post as Greg posts additional translations.

    Austrailia's Zu Shoe Web Site Uses LINQ to SQL as its Data Access Layer

    Stefan Sedich, an ASP.NET developer for the Vivid Group in Perth, reports in his Our First ASP.NET 3.5 Site Goes Live!! post of April 3, 2008 that Zu Shoe's new site uses the latest technologies:

    • Visual Studio 2008
    • ASP.NET 3.5
    • LINQ to SQL
    • ASP.NET AJAX

    Stefan says:

    All in all this project was great to work on, being able to use .NET 3.5 and Visual Studio 2008 saved us a lot of time, and using LINQ to SQL cut down creating the DAL quite a bit. Of course with all new techs we have had challenges along the way, but still developed it a lot quicker using VS 2008 and .NET 3.5.

    This is the first large commercial site that I've heard uses LINQ to SQL as its data source. Vivid Group has two more ASP.NET 3.5 sites in the works.

    The site has flair; check it out!

    Jeff Currier Demonstrates How to Use SQL Server Data Services' SOAP Client

    Jeff's Interacting with SQL Server Data Services using SOAP post of April 2, 2008 describes SSDS's SOAP client and the generated C# code for the Scope object, which has the following four settings:

    Service Scope - This the scope when a scope object instance is passed to the service without any of the properties set.  This would typically be the case when the intent is to create a new authority object in the service.

    Authority Scope - When just the authority id is specified on the scope object.  This is generally the case when the caller is attempting to create a container, retrieve the set (or a restricted set) of containers from a given authority or finally when attempting to simply retrieve a particular authority instance.

    Container Scope - As you might imagine this scope is established when both the authority and container id properties are set on the scope object.  This scope is typically used when querying within a container for a set of entities (or all entities), creating new entities, or deleting a specific container.

    Entity Scope - Used to narrow the focus of the operation to be performed down to the entity level.  This is typically used when the caller is attempting to update, retrieve or delete a specific entity within the service.

    He then goes on to provide examples of creating a new Authority, Container and Entity, populating the Entity and finally reading its content.

    Following are Jeff's previous posts that cover Java and .NET REST clients:

    SQL Server Data Services Meet Java (Jakarta REST client, 3/19/2008)

    Some SQL Server Data Services Coding Examples (REST client, 3/17/2008)

    The SSDS Team Wants Your Input on SOAP Versus REST Wire Formats

    In the SSDS Team's Jeff has posted some SOAP samples post of April 3, 2008, kellyalt asks:

    We are interested in finding out how many out there will build against REST and how many will build using SOAP.  What are the considerations when you think about using one or the other?  How can we make these interfaces better over time?  These are some of the questions we have in our mind.  Would love to hear from you.

    My reply was 10:1 REST:SOAP because of simplicity and low overhead for public catalog data that doesn't require the WS-* security features afforded by the WSHttpBinding default.

    I'd need to have the REST and SOAP client bits and admission to the beta services to answer the "How can we make these interfaces better over time?" question.

    Overall, I'd say both questions are a bit premature.

    Julie Lerman Explains How to Leverage ObjectQuery Options on a LINQ to Entitities Query

    Julie observes:

    At design time, [a LINQ to Entitities] query is an IQueryable, not an ObjectQuery and doesn't have MergeOptions.

    So you can cast the LINQ to Entities query to an ObjectQuery, set the MergeOption and, as Jeffrey Palermo would say, party on the LINQ to Entities query.

    The whole story is in Julie's ObjectQuery, LINQ to Entities and IQueryable post of April 2, 2008.

    Taking Advantage of Stored Procedures in the Entity Framework

    The Entity Framework (EF) defaults to generating dynamic parameterized Entity SQL (eSQL) statements to populate (hydrate) entities from tables in the underlying relational data store and parameterized data manipulation language (DML) statements to create, update and delete entities and their associated tables.

    Many organizations and individual DBAs prefer or insist on the use of stored procedures rather than dynamic SQL for SELECT, INSERT, UPDATE or DELETE operations. Stored procedures prevent the need to grant users direct access to tables, which could compromise database security. In many cases, the capability to efficiently execute stored procedures instead of dynamic SQL is the principal determining factor in choosing an object/relational mapping (O/RM) tool.

    EF has built-in support for stored procedures but there's no central source of complete, up-to-date documentation and how-to information for using stored procedures with the current Beta 3 version and its Entity Data Model (EDM) Designer CTP 2. The EF Extensions, which add flexibility in the use of custom stored procedures and speed their execution, hasn't been widely used or thoroughly documented. (There were only 181 EF Extension downloads from the MSDN Code Gallery as of 3/31/2008, and Colin Meek only recently published his ADO.Entity Framework: Stored Procedure Customization article on 3/26/2008)

    The following three articles contain detailed instructions with screen captures and code examples for creating entities, changing from dynamic SQL to stored procedures, testing EDMs with stored procedures, as well as minimizing the number of stored procedure calls and improving performance with the EF Extensions:

    Migrating to SQL Server Stored Procedures with the EDM Designer December CTP 2 (March 27, 2008)

    Testing Stored Procedure Replacements for Entity SQL Statements (March 29, 2008)

    Minimize Stored Procedure Calls and Improve Performance with EF Extensions (March 30, 2008)

    Repeated from LINQ and Entity Framework Posts for 3/31/2008+.

  • Wednesday, April 02, 2008

    LINQ and Entity Framework Posts for 3/31/2008+

    Note: This post is updated daily or more frequently, depending on the availability of new posts.

    Microsoft France Academic Interns Create Visual LINQ Query Designer (VLinq)

    Mitsuru Furuta, Microsoft France's manager and technical lead of the VLinq project, recruited two talented students, Simon Ferquel and Johanna Piou, who was a French Imagine Cup participant, to write a Visual Studio designer template that helps LINQ naifs write LINQ queries on a graphical design surface implemented in WPF.

    Mitsu's Visual Linq query builder for Linq to Sql: VLinq post of April 2, 2008, offers a brief history of the VLinq project and a step-by-step demonstration with LINQ to SQL as the data source. He says the project took "almost one year of work and organization." The student internships lasted six months.

    Here's the Visual LINQ Query Designer surface running a simple query:

    Do you see something missing in the generated T-SQL prepared statement?

    You can download the add-in template from MSDN Code Gallery.

    Added: April 2, 2008

    Microsoft Data Centers Expand the Unit of Deployment from Bricks to ISO 668 20-Foot Intermodal Containers

    Mary Jo Foley reports in her Microsoft builds out its first containerized datacenter post of April 2, 2008 that the new Northlake, Ill. data center will use containers holding 1,000 to 2,000 servers as the basic unit of deployment.

    Mike Manos who leads the Microsoft Global Foundations Data Center team made the first public announcement of a containerized production data center at Data Center World. According to Microsoft Data Architect James Hamilton:

    The Microsoft Chicago facility is a two-floor design where the first floor is a containerized design housing 150 to 220 40’ containers each [holding] 1,000 to 2,000 servers. Chicago is a large facility with the low end of the ranges Mike quoted yielding 150k serves and the high end running to 440k servers.

    The Database Bricks/Containers Backstory and Their Relation to SSDS

    I wrote about bricks and containers as units of data-center deployment almost two years ago (to the day) in my Very Large Databases: Bricks, BitVault and BigTable post of April 6, 2006. That post's "Really Big Smart Bricks and Databases" topic starts with:

    The ultimate brick is a Google data center in a shipping container, as postulated by PBS's Robert X. Cringely (Mark Stephens) in his 11/17/2005 column, "Google-Mart: Sam Walton Taught Google More About How to Dominate the Internet Than Microsoft Ever Did."

    I concluded:

    Clearly, large distributed file systems and databases must run on very big smart bricks or hardware of similar scale. Google probably probably runs the world's largest distributed file system.

    It's a reasonably safe bet that future Midwest users of SQL Server Data Services will be accessing data in the container instead of data in the cloud.

    The "VLDB Implementation/Deployment Issues" topic quoted Dare Obasanjo's "Greg Linden on SQL Databases and Internet-Scale Applications" post, which in turn quotes Greg Linden:

    What I want is a robust, high performance virtual relational database that runs transparently over a cluster, nodes dropping in an out of service at will, read-write replication and data migration all done automatically. I want to be able to install a database on a server cloud and use it like it was all running on one machine.

    and goes on to describe Adam Bosworth's database requirements of the time:

    Adam Bosworth, who managed the intial development of Microsoft Access, went on to found CrossGain (together with Tod Neilsen, Access marketing honcho), sold CrossGain to BEA and became BEA's chief architect and senior vice president, and [was then] VP Engineering at Google, lists these three features that database users want but database vendors don't suppply: Dynamic schema, dynamic partitioning of data across large dynamic numbers of machines, and modern [Googlesque] indexing. Adam wants the the Open Source community to "[g]ive us systems that scale linearly, are flexible and dynamically reconfigurable and load balanced and easy to use." Adam does mean give, not sell.

    It seems to me that BitVault's smart bricks with appropriate deployment and management applications would fulfill Greg's and all but Adam's economic desires for reference data, which now constitutes more than half of the data stored by North American firms

    The tragedy is that Jim Gray didn't live to witness the massive scaleup and scaleout of his original CyberBrick concept.

    Added: April 2, 2008

    DevExpress Implements LINQ Data Sources for ASPxGrid View and XtraGrid

    The Server mode using LINQ? Let's wax rhapsodic post of April 1, 2008 by DevExpress CTO Julian M. Buckness describes how DevExpress refactored its ASP.NET and Windows form grid controls from dependence on eXpress Persistent Objects (XPO) to support for any LINQ provider that returns an IQueryable<T> type.

    The DevExpress's 2008 Volume 1 release adds LinqServerModeDataSource for pageable ASPxGridView controls for ASP.NET, which appears to emulate the ASP.NET LinqDataSource. The LinqServerModeSource supports the XtraGrid and adds similar paging capability. You can view a short screen cast that demos LinqServerModeDataSource here.

    The "Developer Express uses LINQ for grid controls" article for SDTimes by David Worthington became one of the magazine's Top Stories by April 2, 2008.

    Added: April 2, 2008

    Eugenio Pace Implements a Mock SSDS for LitwareHR with SQL Server Express

    Eugenio's LitwareHR on SSDS - Part IV - Data access enhancements 2: developing offline post of April 1, 2008 describes how the new SQL Server Data Services (SSDS) version of LitwareHR implements a proxy to let developers switch between SSDS and a local SQL Server Express instance as the data source for the demo project.

    Eugenio's post describes his objective in implementing the proxy:

    Our goal while developing LitwareHR was to actually make the dev team as independent and autonomous as possible. Notice I say the dev team, not the application itself. We were comfortable in taking a dependency with SSDS for runtime, that is when LitwareHR would be deployed in a production data center. But we wanted developers to be able to work even if they are flying on a plane with no connectivity (like I was!). In one sentence we wanted a "mock SSDS".

    With this in mind, we developed a new proxy implementing the same interface the real proxy implements, but against a local SQL Server Express database.

    Following are previous posts in Eugenio's LitwareHR for SSDS series:

    Added: April 2, 2008

    Dave Robinson Writes in 20 Minutes an SSDS Client That Adds and Retrieves Red Bull Recipes to/from SSDS

    Challenged to write a "full-featured" SSDS app in the 20 minutes before his CodeTrip presentation last Friday at the Red Bull headquarters in Santa Monica, Dave came up with the project he describes in his SSDS @ Red Bull post of April 1, 2008. The project uses the Excel Add-in for SSDS that Dave describes in his Using Excel with SSDS post of March 21, 2008.

    [Dave's link to SSDS @ RedBull in his SSDS, The Code Trip & Red Bull post of April 1, 2008 doesn't work for me; it's fixed above.] Dave fixed his link.

    Update April 3, 2008: You can watch a 09:54 TechjZulu video segment that includes an interview with Dave, which begins at 02:40.

    TechZulu claims to be "one of the first weblog that introduces southern California's companies to the technology community."

    Added: April 1, 2008 Updated: April 2, 2008

    Noam Ben-Ami Describes Entity Framework's Future Update Model from Database Feature

    Noam's Update Model From DB post of April 1, 2008 describes the post-CTP2 version of the Entity Data Model (EDM) Designer's Update Model from Database feature that detects changes to the data store's schema and modifies the physical schema (SSDL) and the mapping (MSL) layer as follows:

    1. Adds EntityTypes, Properties and/or Associations for objects that have tables or columns in the database but are missing from the EDM. You can select which tables, columns, or both to add.
    2. Modifies EntityTypes, Properties and/or Associations where database tables or columns have changed from the structure represented by the current SSDL schema.
    3. Removes EntityTypes, Property and/or Associations from the SSDL only whose database tables or columns have been deleted from the database. This process doesn't delete the element from the model (CSDL section).

    The post-CTP2 EDM Designer handles renamed entities or properties, but not renamed tables or columns.

    Noam says that these are the limitations of the forthcoming designer:

    • It does not update the types of properties when the corresponding database columns change.
    • It does not “resurrect” entity types – once you have deleted a type, the only way to get it back is to recreate it manually, or delete the corresponding SSDL by hand in the XML editor.
    • It will also not “resurrect” properties.
    • If you change the keys that define your type, all current associations lose their identity and new associations will be brought in – you will need to delete old associations.
    • It cannot detect database object renaming – renames will show up as the deletion of an old object and the addition of a new one.

    Of all the improvements, my favorite is the ability to handle renamed entities because I work primarily with tables having plural names which I always singularize for entity names.

    Added: April 1, 2008

    Oren Eini: NHibernate 2.0 Alpha 1 Released

    Ayende says in his NHibernate 2.0 Alpha is out! post of March 31, 2008:

    It gives me great pleasure to announce that NHibernate 2.0 Alpha 1 was released last night and can be downloaded from this location.

    We call this alpha, but many of us are using this in production, so we are really certain in its stability. The reason that this is an alpha is that we have made a lot of changes in the last nine months (since the last release), and we want to get more real world experience before we ship this. Recent estimates are of about 100,000 lines of code has changed since the last release.

    NHibernate is the standard of comparison for Windows object/relational mapping (O/RM) tools. The "NHibernate Mafia" (James Kovacs, Scott Bellware, Jeffrey Palermo, and Jean-Paul Boodhoo) were responsible for making the Entity Framework folks finally understand what persistence ignorance is all about.

    Added: April 1, 2008

    SQL Server Data Services Team Seeks Ruby on Rails, PHP and Java Developers for April SDR

    Ryan Dunn's Interested in SQL Server Data Services? post of March 31, 2008 touts early access to SSDS and a seat at an SSDS software design review (SDR) at the Microsoft Silicon Valley Campus on April 24 - 25, 2008 for developers using non-Microsoft technologies, such as Ruby on Rails, PHP and Java.

    PS: Watch for my cover story about SSDS coming in the July 2008 issue of Visual Studio Magazine.

    Query Caching In ADO.NET Data Services Feedback Wanted

    Pablo Castro is seeking feedback from users of ADO.NET Data Services (Astoria) about the tradeoff between the performance benefit of compiling/caching data retrieval queries and the code complexity of defining the queries when taking advantage of Astoria's query interceptor feature. Pablo's Looking for feedback: query caching in data services post of March 31, 2008 dives deep into the caching issues.

    Data retrieval queries involve elaborate, provider-specific composition. For example, Pablo cites this flow for Astoria with Entity Framework as the data provider:

    URL -> Expression Tree -> Canonical Query Tree -> View expansion/Query simplification -> Canonical Query Tree -> SQL -> Rows (DataReader) -> Entities (DataReader) -> Objects -> Serialization (Atom/JSON)

    Thus compiling and caching these queries, especially complex ones that involve deeply nested data structures resulting from use of the $expand keyword, has a very beneficial effect on performance. Undoubtedly the perf improvement will be substantially greater than that reported for LINQ to SQL by Rico Mariani in his famous five-part DLinq (LINQ to SQL) Performance series, as well as his more recent Performance Quiz #13 -- Linq to SQL compiled queries cost of January 11, 2008 and Performance Quiz #13 -- Linq to SQL compiled query cost -- solution of January 14.

    Data interceptors, which enable adding a custom filter predicate to the query, are commonly used to implement data access control but complicate the compiling/caching process. What Pablo is seeking is opinions on the relative importance of query performance versus code complexity for implementing data inceptors. Without knowing how much the perf benefit is affected by the two options he offers at the end of his post's Part 2, it's difficult to make a recommendation. I'm inclined to favor performance because the interceptor code is likely to be suited to cookbook-style documentation.

    The ultimate decision is likely to affect SQL Server Data Services, also.

    PS: Watch for my cover article, "Retrieve and Update Data in the Cloud with Astoria," about ADO.NET Data Services coming in the May 2008 issue of Visual Studio Magazine.

    PPS: My "ASP.NET MVC: Is the new MVC pattern right for you?" TechBrief is in the March 15, 2008 issue of Redmond Developer News. Unlike on p. 27 of the print edition, the electronic version has the right blog address.

    Taking Advantage of Stored Procedures in the Entity Framework

    The Entity Framework (EF) defaults to generating dynamic parameterized Entity SQL (eSQL) statements to populate (hydrate) entities from tables in the underlying relational data store and parameterized data manipulation language (DML) statements to create, update and delete entities and their associated tables.

    Many organizations and individual DBAs prefer or insist on the use of stored procedures rather than dynamic SQL for SELECT, INSERT, UPDATE or DELETE operations. Stored procedures prevent the need to grant users direct access to tables, which could compromise database security. In many cases, the capability to efficiently execute stored procedures instead of dynamic SQL is the principal determining factor in choosing an object/relational mapping (O/RM) tool.

    EF has built-in support for stored procedures but there's no central source of complete, up-to-date documentation and how-to information for using stored procedures with the current Beta 3 version and its Entity Data Model (EDM) Designer CTP 2. The EF Extensions, which add flexibility in the use of custom stored procedures and speed their execution, hasn't been widely used or thoroughly documented. (There were only 181 EF Extension downloads from the MSDN Code Gallery as of 3/31/2008, and Colin Meek only recently published his ADO.Entity Framework: Stored Procedure Customization article on 3/26/2008)

    The following three articles contain detailed instructions with screen captures and code examples for creating entities, changing from dynamic SQL to stored procedures, testing EDMs with stored procedures, as well as minimizing the number of stored procedure calls and improving performance with the EF Extensions:

    Migrating to SQL Server Stored Procedures with the EDM Designer December CTP 2 (March 27, 2008)

    Testing Stored Procedure Replacements for Entity SQL Statements (March 29, 2008)

    Minimize Stored Procedure Calls and Improve Performance with EF Extensions (March 30, 2008)

    Jim Zemlin: "The Linux ecosystem should pay attention to" SQL Server Data Services

    Jim Zemlin is the Executive Director of the Linux Foundation so his advice is well taken, especially by Microsoft. In his Cloud Computing - Did anybody notice Microsoft’s SQL Server Data Services Announcement? post of March 14, 2008, Jim says:

    Linux has an early lead in this area with service offerings like Amazon’s Linux based S3 and IBM’s Blue Cloud which uses Xen and PowerVM virtualized Linux operating-system images.

    Today Web 2.0 start ups are flocking to these services as a way to reduce their cost, have world class infrastructure, and most importantly to be able to scale up and down based on demand. It won’t be long before mainstream enterprises follow this trend. It turns out that IT operations combined with convenient tools is something that will be the core competency of companies like Google, IBM, Amazon, and Microsoft in the future. ...

    A product from Microsoft that doesn’t require their tools or lock you into their platform offered as a service over the web? Time[s] are indeed a changing. The interesting thing about this is that the folks who need to adjust for this change the most are software companies that don’t have broad IT operations infrastructure and management competencies similar to the likes of Google and Amazon. Software companies like Microsoft are waking up to this. Are other going to follow suit? [Emphasis added.]

    Like many others, Jim identifies the wrong services as SSDS competitors. Amazon's S3 stores raw data (blobs) and IBM touts Blue Cloud as providing on demand computing services, similar to Amazon's EC2. SSDS's only direct competitor is Amazon's SimpleDB, as far as I've been able to determine.

    Thanks to the SSDS Team for the heads up.

    Added: April 1, 2008 [Not an April Fool]

    Video: Liam Cavanaugh and Neil Padgett Demonstrate Synchronizing SSDS Data with Access and Outlook

    From the Microsoft Sync Framework blog's Sync to the Web - Enable Offline Access to Web Data from any Data Store post of March 12, 2008:

    At MIX 2008 Liam Cavanagh discusses how to use the Microsoft Sync Framework along with SQL Server Data Services to provide synchronization cabability across multiple applications.

    In the demonstration, he synchronizes contacts between Microsoft Access, Outlook and SQL Server Data Services. However, by building new providers, any application can participate in synchronization.

    Later in the presentation Neil Padgett goes through a detailed internals discussion of the Microsoft Sync Framework metadata and provider models.

    You can view the WMV (high or low bitrate) version of the 41:00 informal presentation. Here's another link to Neil's formal MIX 08 session: Using Microsoft Sync Framework and FeedSync (T32).

    The OakLeaf Systems blog will start carrying more Sync Framework/Services content because of its importance to SSDS and its pending integration with Astoria to create "Astoria Offline."

    Here's a link to an earlier "Introducing Microsoft Sync Framework" presentation by Moe Khosravy, the lead PM for the Sync Framework.

    Added: April 1, 2008