Showing posts with label Entity Data Platform. Show all posts
Showing posts with label Entity Data Platform. Show all posts

Friday, July 30, 2010

Rob Sanders’ Working with Entity Framework v4 and SQL Azure: Parts 1 through 4

image Rob Sanders (@AusRob), who writes the Sanders Technology Blog, wrote a four-part blog series (and an addendum) about using SQL Azure to persist .NET entities with Entity Framework v4 in early 2010, just after SQL Azure was released for general use. He originally posted these articles to his internationalized.spaces.live.com blog from which they’re no longer available. I’m interested in this topic and decided to plug the linkage gap.

Following is a list of the four parts and addendum with links to posts on Rob’s new blog:

Update 8/1/2010: Rob posted the following tweet on 7/31/2010:

@rogerjenn Cheers for the plug :) I'm going to also investigate whether SQLAzure can be used with the new Code First approach (EFv4)

and I added Rob’s recent How To Choose a WCF Binding post and some background posts about use of EF4 with SQL Azure. See “Related Posts” below.

Following are the introductory section(s) of each post for reference:

Working with Entity Framework v4 and SQL Azure

Introduction

Well, it’s been “out there” for a while, and now it is time I gave you a taste of how to combine some of the “Next Generation” data access technologies, today.  SQL Azure is Microsoft’s “Database in the Cloud” and represents a complete makeover (replacement) of the first attempt, SQL Data Services (SDS).  SDS is no longer, SQL Azure is the future.

One of the biggest knocks on SDS was the lack of a relational database.  Everything was done according to vertical data “silos” and cross-silo querying was painful (and almost impossible).  Unfortunately, administration was also tricky (well, worse) and it veered quite severely from the established SQL world which we know and love (or submit to).

Now, I’m going to walk you through the process of establishing a working Entity Framework (v4) model which runs against a SQL Azure database – a database in the “Cloud”.  This is not, by any means, a definitive guide.  In fact, I only just got this working myself this morning.  It relies on a few creative “workarounds”, which I will explain in due course.

This example is only intended as a “Proof of Concept” and has not exhaustively been tested. 
Attempt at your own peril (usual conditions and waivers apply)!

Prerequisites

What you will need prior to following along:

All of the above can be obtained for free, from the links above.  You may have to wait for a SQL Azure invitation or token.  My tokens were obtained a little while ago.

I’ll consider writing a separate article on how to set up SQL Azure, so if you are interested send me an email or leave a comment here. …

Next Steps with SQL Azure

Introduction

So now that we’ve had a look at some of the basics around SQL Azure, Microsoft’s Database in the cloud, let us take a look at something more practical.  For our next example, I’m going to refer to a schema I designed and populated last year – my vinyl record collection.  You can have a look at the details of the schema in a previous entry – Project Sneak Peek.

The project was previously defined in this entry – New Project Announcement: SqlAzure Database and Azure Application (though we may skip the Azure application part, depending on time).

Given the schema is for SQL Server 2008, one of our first tasks will be to script the database and existing data, and then convert it to be conform with SQL Azure’s T-SQL restrictions. …

Putting it all together – SQL Azure

Now we should have a working data model which has been created in your SQL Azure (Cloud) database.  Taking lessons learned from Part 2, we’re going to create an Entity Framework (v4) model of the SQL Azure tables using Visual Studio 2010 Beta 2.

This entry is sort of a rehash of the first and second entries, but the outcome should provide you with a fully functioning Entity Framework (v4) data model, and a Unit Test to prove it works! …

A Dynamic Data Website with SQL Azure and the Entity Framework

Continuing along…

Our next step is to create a Dynamic Data website.  If you haven’t come across this yet, it’s most likely because you haven’t been using Visual Studio 2010 or the .Net Framework 4.0.  Recently introduced and compatible with both LINQ-to-SQL and the Entity Framework, this nice site template makes use of the dynamic nature of both LINQ-to-SQL [.dbml] (SqlMetal) and Entity Framework [.edmx] data models.

Continuing within the solution we created in Part 3, we shall now add a Dynamic Data website to our solution.  Open the solution in Visual Studio 2010 Beta 2 and then Right click the solution, “Add –> New Project..”.

Please note: this portion of the solution will also work in conjunction with any other database provider supported by the Entity Framework, not just SQL Azure.  To use a Dynamic Data site all you need is a LINQ-To-SQL or Entity Framework Data Context!  For details, read on. ...

Getting a SQL Azure Database

Introduction

This article will describe the steps you need to take to obtain a SQL Azure database.  The main website for SQL Azure is http://www.microsoft.com/windowsazure/sqlazure/ and for developers http://www.microsoft.com/windowsazure/developers/sqlazure/

The Microsoft Connect site (https://connect.microsoft.com/SQLAzure ) is now closed, but you may still find useful information here, such as links to the team blog.  There are SQL Azure forums for community discussions located here.

To get started with SQL Azure, you must have a valid Windows Live ID.  You can sign up for a Live ID with a valid email address by visiting https://accountservices.passport.net/reg.srf?sl=1&lc=1033

You’ll need to create an account with Microsoft Online Services at https://mocp.microsoftonline.com/site/default.aspx
Sign in with a valid Windows Live ID, and then you will be prompted for some profile information. …

Parts of the addendum apply only to residents of Australia. According to his new Sanders Technology Blog, Rob now lives in HangZhou, China.

Related posts:

Saturday, November 08, 2008

Professional ADO.NET 3.5 with LINQ and the Entity Framework: Table of Contents

Following is the final table of contents for Professional ADO.NET 3.5 with LINQ and the Entity Framework from Wiley, which will be published in January or February 2009:

  Introduction
Part I   Getting a Grip on ADO.NET 3.5
Chapter 1   Taking a New Approach to Data Access in ADO.NET 3.5
Part II   Introducing Language Integrated Query
    Introduction to Part II
Chapter 2   Understanding LINQ Architecture and Implementation
Chapter 3   Executing LINQ Query Expressions with LINQ to Objects
Chapter 4   Working with Advanced Query Operators and Expressions
Part III   Applying Domain-Specific LINQ Implementations
    Introduction to Part III
Chapter 5   Using LINQ to SQL and the LinqDataSource
Chapter 6   Querying DataTables with LINQ to DataSets
Chapter 7   Manipulating Documents with LINQ to XML
Chapter 8   Exploring Third-Party and Emerging LINQ Implementations
Part IV   Introducing the ADO.NET Entity Framework
    Introduction to Part IV
Chapter   9   Raising the Level of Data Abstraction with the Entity Data Model
Chapter 10   Defining Conceptual, Mapping, and Storage Layers
Chapter 11   Introducing Entity SQL
Part V   Implementing the ADO.NET Entity Framework
    Introduction to Part V
Chapter 12   Taking Advantage of Object Services and LINQ to Entities
Chapter 13   Updating Entities and Complex Types
Chapter 14   Binding Data Controls to the ObjectContext
Chapter 15   Using the Entity Framework As a Data Source

I’ll be reviewing final page layout during the next week or two. I estimate the final page count will be close to 650 (not 550) pages.

Thursday, July 03, 2008

Nigel Ellis Says SQL Server Data Services Will Use the Entity Data Model For Schemas

The SQL Server Data Services (SSDS) team promotes free-form (schemaless) containers and entities as one of SSDS’s primary features.

However, there have been many brief mentions of schemas being included in future SSDS versions (sprints). For example, the SSDS team’s Sprint 2 goes live post of May 26, 2008 includes the following bulletpoint for key features added in Sprint 2:

b. Multiple B-Trees - this will allow us to deploy schemas as we move toward supporting schemas in SSDS

(The SSDS team’s Soumitra Sengupta mentioned in his Philosophy behind the design of SSDS and some personal thoughts post of June 27, 2008 that “Sprint 3 [is] winding down.”)

Channel9’s Istvan Cseri and Nigel Ellis: SQL Server Data Services Architecture interview by SSDS evangelist Ryan Dunn posted April 7, 2008 contains quite a bit of useful (and previously unpublished) information about SSDS architecture.

Following is a transcript I made of Nigel Ellis’s remarks about SSDS’s schemas:

00:29.08: Over the past three or four years, there's been a big investment in the SQL [Server] space in something called the Entity Data Model, where we've got the regular relational model and and we have a high-level model built on top of that. We think of modeling customers, purchase orders, contacts and things like that. There's a whole set of tooling and schema design work that's going on [in] the SQL Server division to make those things approachable and easy to use. ...

For those of you who know the ADO.NET Data Services Framework, that is essentially using the EDM Data Model. You take something that's expressible in EDM terms and make it accessible to open Internet protocols.

00:30:00 So as we evolve our service around the notion of what we call flexible entities, those are the same entities that are in EDM. EDM today doesn't really speak very much to flexibility and open content—Property Bags, if you will—but we’ve been working with the EDM architects as to how that can be extended and accommodated with the same set of with the same set of schema design tools and protocols. And, as I mentioned, the schemas in the SSDS service heads are essentially going to be EDM schemas, EDM types, and EDM relationships. [Emphasis added.]

That statement leads me to believe that SSDS will support complete object graphs in the (hopefully near) future. It also reinforces my prediction that Entity Framework and EDM will dominate Microsoft’s data platform strategy for the next several years.

Related posts:

Friday, March 07, 2008

SQL Server Data Services to Deliver Entities from the Cloud

Complementing its lowering the size barrier to the Office Live Workspace, SharePoint Online and Exchange Online services, Microsoft announced a SQL Server Data Services (SSDS) beta on March 5, 2008. Ray Ozzie mentioned SSDS in passing during his MIX08 keynote.

Updated 3/6/2008 and 3/7/2008

SSDS provides an on-demand, pay-per-use data model, including the following features:

  • Flexible, schemaless entity model, which lets users add new attributes to the entity type when needed; the system indexes them automatically.
  • Simplified data types for properties: string, numeric, datetime, boolean and binary
  • REST-based and LINQ-like queries; SOAP is supported also
  • Plain old XML (POX) is the only wire format currently available; AtomPub is promised
  • No projections; the unit of storage is a complete entity
  • Highly scalable by partitioning of sizes in the 10s of GB
  • Secure data access with SSL, LiveID authentication, and authorization at the account (billing unit), authority (collection of containers), container (set of entity sets), type (entity type) levels.
  • Data synchronization and bulk uploads with the Microsoft Sync Services.
  • On-premises version (presumably ADO.NET Data Services, a.k.a. Astoria)

SSDS is scheduled for 2009H1 deployment with an invitation-only beta open for registration now; an open beta will start in July 2008. During the beta period, the SSDS team intends to update the services every eight weeks with features added or altered in response to user feedback.

The following additional documentation was available today: SQL Server Data Services FAQ and SQL Server Data Services overview. There's no mention of Entity Framework, ADO.NET Data Services or Entity Data Model in the Web-based documents currently available.

The strangely named SSDS blog (The Long Term Store cast) is now live with autobiographical posts by two SSDS architects. The Long Term Store moniker probably derives from Microsoft's belief that archiving will be a major SSDS use case. There's a placeholder for an SSDS blog on MSDN with a single Coming Soon post.

Update 3/7/2008: The following MIX 08 presentation on SSDS is available online:

BT05 - Introducing SQL Server Data Services (WMV)

Thursday, March 6 8:30 AM - 9:45 AM, Delfino 4005
Speaker(s): Nigel Ellis
Audience(s): Business, Technical
Session Type: Breakout

Come learn about our new cloud-based storage service and how it addresses a number of business scenarios. This session introduces the new Microsoft SQL Server Data Services, as well as outlines business models and terms.

The documentation uses http://mydomain.stratus.microsoft.com/ChildrensBooksContainer1 as a sample REST request for an EntitySet, so it's reasonable to assume that the service is named Stratus. (Cirrus was the codename for Microsoft Access and Cumulus was the codename for Windows NT's erstwhile Object File System (OFS)).

Update 3/6/2008: Microsoft's Eugenio Pace says the codename was Sitka (a city on the Alaskan Panhandle) in his SQL Server Data Services - SDSDS - New version of LitwareHR post of March 5, 2008.

Astoria Similarities and Differences

It's obvious that ADO.NET Data Services and SQL Server Data Sources are related but they don't appear to share a common API. The current Astoria CTP supports Atom/AtomPub and JSON but not POX; SSDS only supports POX. SSDS promises AtomPub for the future and the Astoria team promises POX.

Update 3/6/2008: Francois Ajenstadt, director of project management for SQL Server, said the following in reply to my "SSDS's API appears to have diverged from Astoria's, which is surprising" comment to the Data Platform Insider blog's Announcing SQL Server Data Services post:

The goal is to bring the API's between SSDS and Astoria closer together closer to Release. Stay tuned.

There's no indication that SSDS support associations or relationships, which is surprising.

Information Week's J. Nicholas Hoover's Microsoft Extends SQL Server To The Web With Data Services article of March 5, 2008 states on page 2:

Microsoft's wrapping the "entity data model" into SQL Server Data Services, meaning that in addition to the traditional data model, developers will be able to easily create containers for a set of data and define data entities like "Customer" or "Order" that contain all the data types relevant to a customer or an order and can be altered at will, using technology Microsoft developed in a project code-named Astoria.

However, the preceding paragraph has no attribution. As I read the tea leaves, SSDS doesn't support SQL Server's "traditional data model."

Hoover compares ADO.NET data sources with Amazon S3's unstructured storage instead of their SimpleDB. My Amazon Announces Beta of SimpleDB Web Services in the Cloud post of December 14, 2007 and Dare Obasanjo's Amazon SimpleDB: The Good, the Bad and the Ugly post of December 21, 2007 analyze SimpleDB's feature set.

Update 3/7/2008: Microsoft's party line appears to be: "SSDS is not a SimpleDB compete," which was Nigel Ellis's answer to a question after his presentation. I doubt if anyone buys that line. The primary differences at present between the two services are SSDS's use of an industrial-strength database engine instead of a Erlang application, LINQ-like queries, data synchronization, and simple data types: string, numeric, date/time, and binary (base64-encoded).

(Queries use LINQ syntax but appear from Nigel Ellis's demo to be late-bound and thus untyped.)

It's now obvious why the Astoria team didn't rejuvenate the free Live Labs ADO.NET Data Services Online Service demonstration. Its schema wizard made designing entities relatively easy, but SSDS automatically builds its schema under the covers (and keeps it there).

Entity Data Platform Backstory from the Microsoft Business Intelligence Conference

The SQL Server "Katmai" to Deliver Entity Data Platform and Support LINQ post of May 25, 2007 covers references to the Entity Data Platform, which Data Architect Michael Pizzo described as follows:

Microsoft envisions an Entity Data Platform that enables customers to define a common Entity Data Model across data services and applications. The Entity Data Platform is a multi-release vision, with future versions of reporting tools, replication, data definition, security, etc. all being built around a common Entity Data Model. [Emphasis added.]

The Katmai CTP Scheduled for June post of May 23, 2007, includes a link to the video of Ted Kummert's keynote to the Microsoft Business Intelligence Conference. Comments about LINQ and the Entity Data Platform start at 46:00 into the presentation.

Register for the Beta

SSDS received brief coverage in today's Microsoft Unlocks the Power of the Web for Connected Customer Experiences press release.

Register for the SSDS beta at http://www.microsoft.com/sql/dataservices/default.mspx.

You must enter 5000 or more as the number of employees to register. This restriction formerly applied to Microsoft's SharePoint Online and Exchange Online services but was removed on March 2, 2008, according to the Microsoft to Expand Microsoft Online Services for Businesses of All Sizes press release. I was surprised to see the restriction for this beta.

Update 3/5/2008 1630 PST: An anonymous commenter says:

There is no restriction to the size of your company... big or small can sign up.

Please make this correction...

which I've done. I've verified that the number of employees and other company details no longer are marked as required fields.

Update 3/6/2008: The commentator turned out to be the Microsoft person in charge of the signup form who said it was a bug, which they fixed.

Wednesday, March 05, 2008

LINQ and Entity Framework Posts for 3/2/2008+

Note: Items for March 6, 2008 were moved to LINQ and Entity Framework Posts for 3/6/2008+ due to excessive length of this post.

Microsoft Announces SQL Server Data Services at MIX08

Ray Ozzie's brief announcement of SQL Server Data Services (SSDS) at MIX08 might have been the source for Nick Carr's Rumor: Microsoft about to unveil web-apps strategy post of March 1, 2008 and follow-on Rumor: Microsoft set for vast data-center push of the next day. Other than SSDS, I'd have to agree with Joe Wilcox that Ray Ozzie Says Nothing New, Again.

For a synopsis of the scant documentation for and other stories about Microsoft's data-centric complement to Online Exchange and Online SharePoint, see my SQL Server Data Services to Deliver Entities from the Cloud of March 5, 2008. My conclusion is that the SSDS and .NET Data Services/Entity Framework APIs have diverged.

Added: 3/5/2008 Post updated: 3/8/22008

"Indefinite Hiatus" for Dare Obasanjo's Blog

Dare announced that his Indefinite Hiatus post of March 4, 2008 would be the last to his Carnage4Life blog that started as his kuro5hin diary in 1991. I've been an avid reader of Dare's blog since my early work with and articles and book about XML Web services. I'll miss reading his blog every morning.

Dare was the XML Schema program manager on Microsoft's XML team before moving to the Windows Live group. He's now a program manager on the Contacts & Storage team in Windows Live and has been writing primarily about social networking. I've been most interested in Dare's occasional posts about LINQ, RESTful interfaces, Atom, and other topics of interest to potential users of ADO.NET Data Services.

His early (1991) comment about XML while interning at Microsoft is interesting:

I've picked up XSD, XSLT & XPath while I've been here [at Microsoft] and am stunned by how complicated using XML has been made that almost every spec that deals with it is looks like it can only be fully understood by a Ph.D and is a few hundred pages when printed. Sheesh.

Ain't it the truth, especially for WS-*. (Dare says SOAP = Snakes On A Plane).

There was no word about the future of his Windows Live Spaces blog in the swan song, but Dare mentioned in an IM that it was only echoing his Carnage4Life posts, so it will probably go dark too.

Added: 3/5/2007

Bob Beauchemin Discounts Lack of Blind or Bulk Updates in LINQ to SQL and Entity Framework

Bob outlined six potential problems with LINQ to SQL and Entity Framework as "worry points" in his initial MHO: LINQ to SQL and Entity Framework: Panacea or evil incarnate? Part 1 post of February 13, 2008. The LINQ and Entity Framework Posts for 2/25/2008+ post has pointers to analyses of the first two:

1. LINQ to SQL and EF will proliferate dynamic SQL, and will almost surely produce suboptimal dynamic SQL, causing database performance problems and plan cache pollution. And dynamic SQL is evil to start with.

2. LINQ to SQL and EF will encourage "SELECT * FROM..." style coding because you get back a nice, clean object this way instead of the less-useful anonymous type you receive by doing projection. This will also make most/all covering indexes useless.

Bob's MHO: LINQ to SQL and Entity Framework: Panacea or evil incarnate? Part 3 post of March 1, 2008 tackles the third worry point:

3. LINQ to SQL and EF will encourage "SELECT to middle-tier, then update/delete" rather than issuing direct SQL UPDATE/DELETE statements that are set-based.

I, too, was concerned that EF's lack of DML for blind or bulk updates and deletions would impede its adoption. Bob concludes:

With stored procedures as needed and the realization that most apps use "GET then UPDATE" anyway, I think I'll dismiss this worry.

I agree. Blind and bulk updates in production online transaction processing applications are very uncommon. Because of the potential for serious data corruption, bulk updates should be done with stored procedures and preferably be planned and executed by a DBA.

Added: 3/5/2007

Note: Beginning with this post, links to topics in "LINQ and Entity Framework Posts for MM/DD/YYYY+" posts will include links to intra-page HTML anchors for the related topic.

I'll also mark individual topics as WebSlices when IE 8 gains a wider audience. Jon Udell's WebSlices can help popularize feed syndication post of March 5, 2007 has a great demo of the new WebSlice feature.

Sahil Malik Predicts: Entity Framework Will Become the Standard Architecture for Data-Intensive Applications

SharePoint and ADO.NET guru Sahil Malik's I told you so!! post of March 2, 2008 contains an important prediction:

ADO.NET eF - Will receive significant criticism, but will become the de facto way we write applications going forward.

I assume Sahil means "data-intensive .NET applications."

He goes on to explain his doubts about LINQ to SQL:

Microsoft's data access story: About 2 years ago, when I first played with DLINQ (now rechristened to LINQ to SQL), I basically flat out said - This is for RAD development/prototyping - not good enough for production environments. To be very specific, I had said ...

".... where your  logical model could be sufficiently disjoint from the conceptual model. So DLINQ (now LINQ to SQL) was ideal for situations where your logical model was very close to the conceptual model. Of course, you had better chances of finding a hot bikini model who was also a C++ programmer, than to find an application where the logical and conceptual model would be the same....".

I also expressed a more comprehensive opinion not too long ago.

Sahil also predicts the demise of technical books, that SharePoint vNext "will blow your mind," and Silverlight 2 won't succeed because it doesn't support search engine optimization (SEO). His other 10 predictions are far more vague.

Danny Simmons: More on the ObjectContext Lifecycle and Need for a Separate Data Layer

Danny's More about how to fit the ObjectContext into apps post of March 2, 2008 responds to Rick Strahl's suggestion to use one ObjectContext per business object, which raises obvious issues with associated entities. Danny recommends a single ObjectContext per session (or per thread for multithreaded apps).

He also says "no" to the question of "should you explicitly write a layer that hides all data access like we used to do?" The Business Objects layer he describes contains a combination of generated code for the ObjectContext partial classes and custom partial classes to implement business logic.

Danny concludes:

Of course there's nothing that says you can't write your own data layer which encapsulates the EF code that I described as the data layer, but for many applications I tend to think that will make your life harder for relatively little benefit.  One of the main reasons we would write a completely separate data layer in the past is that we wanted to isolate the business logic from changes in database schema and the like.  The EF now does that for you with its mapping layer.

The problem with this architecture is isolating persistence operations from unit tests for Test Driven Development, as Diego Vega discusses below.

Diego Vega on Test Driven Development with the Entity Framework

Diego's Unit Testing Your Entity Framework Domain Classes post of March 2, 2008 provides the following general guidelines for performing unit tests on an Entity Data Model (EDM):

  • Encapsulate domain classes by factoring "all code that deals with IQueryable<T>, ObjectQuery<T>, and IRelatedEnd.Load()" into a "separate DAL component."
  • Expose explicit data retrieval functions instead of ObjectContext<T> or IQueryable<T> objects.
  • Eager-load associated entities for mocking (but lazy loading is OK for production).
  • Mock with a "metadata-only" ObjectContext to gain persistence ignorance.
  • Redirect SaveChanges() invocations to AcceptAllChanges() for tests.
  • Create a new ObjectContext after the preceding action.
  • Don't involve EntityKeys in business logic.

A few of Diego's recommendations appear to me to have internal or external inconsistencies.

Ian Cooper demonstrates the complexity of mocking LINQ to SQL DataContext objects with a repository model in his Architecting LINQ To SQL Applications, part 5 post of February 17, 2008. Similar problems are inherent to mocking the persistence features of an EDM. Ian provides more background on swapping from LINQ to SQL to LINQ to Objects and vice versa in his original Being Ignorant with LINQ to SQL post of June 10, 2007. See also my Ian Cooper Takes on DDD, TDD and PI with LINQ to SQL post of June 11, 2007, and the earlier (and longer) Persistence Ignorance Is Bliss, but Is It Missing from the Entity Framework? post of March 14, 2007.

Diego concludes:

It is actually premature to use the word “conclusion”. Mixing EF and TDD in the same pan is something I am only starting to think about. This is a set of scenarios that I want to see among our priorities for future versions.

In order to come to a real conclusion, I need to at least develop a sample application in which I apply and distill the approaches I am suggesting in this post. I hope I will find the time to do it soon.

I, too, hope he finds the time to do it soon.

New "Model Domain Objects with the Entity Framework" Article in Visual Studio Magazine for March 2008

Visual Studio Magazine's March 2008 cover story is my "Model Domain Objects with the Entity Framework" article. Here's the deck:

Microsoft's ADO.NET Team readies Entity Framework and Tools 1.0 for release as a VS 2008 add-in with enterprise-level features that LINQ to SQL doesn't offer—domain object modeling, flexible inheritance techniques, multiple database vendors, and do-it-yourself n-tier deployment.

The story covers new and modified EF features since my "Objectify Data with ADO.NET vNext" article in the October 2006 issue.

A sidebar describes Danny Simmons' EntityBag (Perseus) project for "do-it-yourself n-tier deployment."

Erratum: Entity SQL does have a JOIN command but it's used to join unrelated entities only. The eSQL NAVIGATE command is used with associations and navigation properties, which represent INNER JOINs at the conceptual level.

Repeated from previous week's post.

Monday, July 09, 2007

Orcas and Entity Framework June 2007 CTPs - New Features

Despite Microsoft's deafening silence (so far) about the availability of the Orcas June 2007 CTP, a few pilgrims are spelunking the VM for new features. Folks are using Danny Simmons' new features list to test-drive the Entity Framework June 2007 CTP under the original Visual Web Developer Express (VWDX) June 2007 CTP as well as the full Orcas June 2007 CTP.

Update 7/10/2007: Here's the apparent explanation for Microsoft's deafening silence. The original Microsoft UK Visual Studio "Orcas" June 2007 CTP post reappeared this morning with the following caveat:

The Product Group have released an newer version of Visual Studio, but they aren't releasing it as a CTP. It is aimed at fixing a couple of problems for some targeted customers. The reality of this isn't tested or up to the quality of a proper CTP. You can still use this download, but it has some problems and it isn't supported for TFS. I would recommend waiting until Beta 2. Sorry for any confusion I have caused. [Spelling corrected.]

If the download page says it's a CTP and the four Readmes say it's a CTP, they released it as a CTP. The post's headline now calls it a "pre-CTP."

Tamir Khason of Microsoft Israel posted Orcas June CTP for limited beta is available on July 4, 2007 with this announcement:

After releasing June CTP of .NET framework 3.5, June CTP bits of Visual Studio 2008 (Team Suite and Visual Web Developer) are also ready for limited beta as Layout or VPC image.

and invited readers to contact him for downloading details.

Visual Basic 9.0 Language Features

VB MVP Bill McCarthy's What's new in June Orcas CTP for VB post lists the following new language features:

  • Lambda functions
  • Nullable types get ? shorthand for Nullable(Of ...) and operator elevation
  • Ternary If() function
  • JoinGroup By and other missing LINQ Standard Query Operators

Bill also mentions differences in C# 3.0 and VB 9.0 behavior in his Having fun with "Group-Ease", Change notification with Orcas data model and Keyed Anonymous types in VB9 posts.

VB 9.0 Lambda Functions and LINQ Method Call Syntax Issues

Online help's "Lambda Expressions [VB]" topic provides the following as a code example in the "Lambda Expressions in Queries" subtopic for method call query syntax:

Dim londonCusts As IQueryable(Of Customer) = _
    db.Customers.Where(Function(c) c.City = "London").Select(c)

If you create a new NorthwindDataContext that contains a Customer object and assign it to db, then add the above code to a Sub or Function, it won't compile. The c in the Select(c) selector isn't visible, which results in a "Name 'c' is not declared" error.

The following version with type inference compiles and returns the expected result to an iterator:

Dim londonCusts = _
    db.Customers.Where(Function(ByVal c) c.City = "London") _
                .Select(Function(ByVal c) c)

Notice that the ByVal directive is missing from the help topic's sample; if you don't type it, the compiler adds ByVal for you.

The preceding code seems absurdly verbose compared to the corresponding C# method call syntax:

var londonCusts = db.Customers.Where(c => c.City == "London")
                              .Select(c => c);

The following C# query chains an OrderBy lambda:

var londonCusts = db.Customers.Where(c => c.City == "London")
                              .OrderBy(c => c.ContactName)
                              .Select(c => c);

Here's the corresponding VB version:

Dim londonCusts = _
    db.Customers.Where(Function(ByVal c) c.City = "London") _
                .OrderBy(Function(ByVal c) c.ContactName)
                .Select(Function(ByVal c) c)

The problem with chaining the additional VB lambda expression is that you get the wrong IntelliSense's List Members collection (Equals, GetHashCode, GetType, ReferenceEquals and ToString instead of the Customer object's properties) for the intermediate OrderBy and other chained functions between Where and Select. C# IntelliSense has no such defect.

Join Operator Works But Union Method Is Still Broken

VB 9.0 finally acquired the Join operator, but the following code still returns the literal 'Customer' for every row, not a mix of 'Customer' and 'Supplier' values. I described this problem in my May 15, 2007 Changes Coming to LINQ for SQL post. Here's the VB code with Joins:

Dim sbQuery As New StringBuilder

Dim Query = ( _
From c In dcNwind.Customers _
Join s In dcNwind.Suppliers On c.City Equals s.City _
Select New With {c.City, c.CompanyName, .Type = "Customer"}) _
.Union( _
From c In dcNwind.Customers _
Join s In dcNwind.Suppliers On c.City Equals s.City _
Select New With {s.City, s.CompanyName, .Type = "Supplier"})

' Append list header
sbQuery.Append("City" + vbTab + "Company Name" + vbTab + "Type" + vbCrLf)
' Append query results
For Each u In Query
    sbQuery.Append(u.City + vbTab + u.CompanyName + vbTab + u.Type + vbCrLf)
Next

Matt Warren said on May 15, 2007 in response to my Strange Behavior of LINQ Union Operator message in the LINQ Project General forum, "[T]here is a bug on how we treat literals in a union.  We are fixing it."

I plan to do more testing of JoinGroup By and other previously missing LINQ Standard Query Operators and will update this post accordingly.

Update 7/11/2007: Extensive tests of multiple Joins with typed and untyped DataSets indicate that it behaves as expected.

Julie Lerman Explores the Entity Framework's New Feature Set

MVP Julie Lerman provides in her illustrated Entity Framework, Some Before & After tidbits post:

  • Illustrated examples of fixes for the missing foreign key values,
  • Views and stored procedures now appear in the Entity Data Model Wizard
  • New support for stored procedures
  • Replacement of default constructors with Public Shared Function CreateObjectName functions.

Breaking Changes in .NET Fx Libraries (Update: 7/11/2007)

LINQ to SQL: INotifyPropertyChanged and Column properties have changed, which generates 100+ errors in small projects. You'll need to recreate your LINQ to SQL classes with the renamed LINQ to SQL Classes template.

LINQ to SQL: Jim Wooley reports in a comment that the System.Data.Linq.AttributeMappingSource and System.Data.Linq.XmlMappingSource classes have moved to System.Data.Linq.Mapping.AttributeMappingSource and System.Data.Linq.Mapping.XmlMappingSource respectively. [Note: MappingSource changed to AttributeMappingSource and XmlMapping changed to XmlMappingSource, and namespace prefixes added.]

LINQ to DataSet: The System.Data.DataTableExtensions namespace has moved from System.Data.Entities.dll to System.Data.DataSetExtensions.dll so you must change references.

LINQ to XML: Fixing "Could not load type 'System.Linq.Func`2' from assembly 'System.Core, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089'." exceptions requires adding a reference to System.Linq.DataSetExtensions.dll. This is probably a temporary expedient for the June 2007 CTP.

Problems with the Orcas June 2007 CTP

Julie reported problems running the Entity Framework June 2007 CTP and I encountered an issue running the extended Northwind database for the C# LINQ to Entity samples under the Orcas June 2007 CTP  in this thread in the ADO.NET Orcas forum.

If you've found and tested new language features (especially that support LINQ), please add a comment with a link to your post.

Sunday, July 08, 2007

Michael Stonebraker on Accessing Data as Entities and Low-Latency Streaming

The May/June 2007 issue of the Association for Computing Machinery's ACM QUEUE magazine includes a wide ranging interview: "A Conversation with Michael Stonebraker and Margo Seltzer: Relating to Databases." Stonebraker was the primary force behind UC Berkeley's development of the Ingres relational database, which provided the foundation for the PostgreSQL open-source database, and is an adjunct professor of computer science at MIT. Margo Seltzer is professor of computer science at Harvard University and one of the founders of Sleepycat Software, publishers of the Berkeley DB embedded database engine. Ingres Corporation is owned mostly by Computer Associates and Oracle acquired Sleepycat Software in 2006.

Stonebraker's major premise is that today's one-size-fits-all relational OLTP and OLAP databases/applications, which he calls legacy code sold by elephants, will be displaced by in important vertical markets by new technology sold by start-ups, which Seltzer calls mice. Along the way, he attributes the lack of object-oriented databases' market success to their failure to gain a foothold in the computer-aided-design (CAD) industry.

Accessing Data as Entities

Stonebraker isn't sanguine about the long-term prospects for SQL, which he characterizes as a language that "started out as a simple standard and [grew] into a huge thing, with layer upon layer of junk."

However, he likes Ruby on Rails' incorporation of data access in the language itself—a la LINQ—and implementation of the entity-relationship model as the ADO.NET Entity Framework does. Stonebraker says:

C++ and C# are really big, hard languages with all kinds of stuff in them. I’m a huge fan of little languages, such as PHP and Python. Look at a language such as Ruby on Rails. It has been extended to have database capabilities built into the language. You don’t make a call out to SQL; you say, “for E in employee do” and language constructs and variables are used for database access. ...

Let’s look at Ruby on Rails again. It does not look like SQL. If you do clean extensions of interesting languages, those aren’t SQL and they look nothing like SQL. So I think SQL could well go away. More generally, Ruby on Rails implements an entity-relationship model and then basically compiles it down into ODBC. It papers over ODBC with a clean entity-relationship language embedding.

I don't agree with Stonebraker's description of Ruby on Rails compiling the E-R model to ODBC, which—along with JDBC—he characterizes as "the worst interfaces on the planet. I mean, they are so ugly, you wouldn’t wish them on your worst enemy." However, I share his disdain for ODBC, which is two generations older than SqlClient.

In reality, Stonebraker is describing the Entity Framework and LINQ to Entities.

Although LINQ has an SQL-like syntax, I certainly prefer it to the following Ruby on Rails examples:

class BlogComment < ActiveRecord::Base
  belongs_to :blog_post
end

and

class BlogPost < ActiveRecord::Base
  has_many :blog_comments, :order => "date"
end

or

class BlogPost < ActiveRecord::Base
  has_many :blog_comments, :order => "date",
  :dependent => true
end

Even LINQ's chained method syntax is easier to read than the preceding snippets from Exploring Ruby on Rails. Another benefit of the EF, as well as LINQ to SQL, is that you aren't locked into the ActiveRecord model.

Low-Latency Streaming

Another issue that Stonebraker raises is the issue of latency when processing streaming data, especially in the financial markets. Financial data feeds "are going through the roof" and "legacy infrastructures weren’t built for sub-millisecond latency, which is what everyone is moving toward." Stonebraker observes:

Until recently, everyone was using composite feeds from companies such as Reuters and Bloomberg. These feeds, however, have latency, measured in hundreds of milliseconds, from when the tick actually happens until you get it from one of the composite-feed vendors.

Direct feeds from the exchanges are much faster. Composite feeds have too much latency for the current requirements of electronic trading, so people are getting rid of them in favor of direct feeds.

They are also starting to collocate computers next to the exchanges, again, just to knock down latency. Anything you can do to reduce latency is viewed as a competitive advantage.

Let’s say you have an architecture where you process the data from the wire and then use your favorite messaging middleware to send it to the next machine, where you clean the data. People just line up software architectures with a bunch of steps, often on separate machines, and often on separate processes. And they just get clobbered by latency. ...

If I want to be able to read and write a data element in less than a millisecond, there is no possible way that I can do that from an application program to any one of the elephant databases, because you have to do a process switch, a message to get into their systems. You’ve got to have an embedded database, or you lose.

In the stream processing market, the only kinds of databases that make any sense are ones that are embedded. With all the other types, the latency is just too high.

Of course the same requirement holds true for processing digital sensor streams. These are the types of applications that might benefit from LINQ's expression tree approach. Oren Novotny's LINQ to Streams (SLinq) and Aaron Erickson's LINQ to Expressions (MetaLinq) are interesting steps in this direction. Stonebraker's comment about the reduced latency of embedded databases adds support for my contention that LINQ to SQL should support SQL Server Compact Edition (SSCE).

Entity Framework and Orcas Team Suite June 2007 CTPs

Microsoft just announced availability of the Microsoft Visual Studio Code Name "Orcas" Team Suite June 2007 Community Technology Preview (CTP) virtual hard disk (VHD) for Virtual PC 2007 and Virtual Server 2005 R2, which I'm currently downloading.

The Visual Studio Code Name "Orcas" June 2007 Community Technology Preview (CTP) Readme has a link to an Express Editions Readme, so it's probably safe to assume that the Visual Web Developer Express June 2007 CTP download for the Entity Framework June 2007 CTP is a member of this update wave. The Readmes provide no information on new features.

There was only one Microsoft blog post about the SDK drop as of Sunday morning. It was a Friday morning post from the Microsoft UK Developer Tools Team, which was subsequently deleted (but it's cached). Reports of the CTP's availability have come from the UK and Europe primarily.

Update 7/10/2007: The Microsoft UK post reappeared this morning with the following caveat:

The Product Group have released an newer version of Visual Studio, but they aren't releasing it as a CTP. It is aimed at fixing a couple of problems for some targeted customers. The reality of this isn't tested or up to the quality of a proper CTP. You can still use this download, but it has some problems and it isn't supported for TFS. I would recommend waiting until Beta 2. Sorry for any confusion I have caused. [Spelling corrected.]

If the download page says it's a CTP and the four Readmes say it's a CTP, they released it as a CTP.

Tamir Khason of Microsoft Israel posted Orcas June CTP for limited beta is available on July 4, 2007 with this announcement:

After releasing June CTP of .NET framework 3.5, June CTP bits of Visual Studio 2008 (Team Suite and Visual Web Developer) are also ready for limited beta as Layout or VPC image.

and invited readers to contact him for downloading details.

I intend to test the EF June 2007 CTP with WinForms as soon as I get the VHD installed. I've also posted a message in the ADO.NET vNext forum concerning compatibility of the EF drop with the Team Suite and VB/C# Express versions. I'll update this post as I learn more about compatibility.

I've also posted requests for lists of new and improved features, bug fixes, and (for Visual Basic) promised features dropped from Orcas in the LINQ Project General and Visual Basic Orcas forums.

There also is a Microsoft Visual Studio Code Name "Orcas" SDK - June 2007 Community Technology Preview download (127.5 MB) available but it's date is 6/1/2007 and appears to be dedicated to Orcas Beta 1.

Saturday, July 07, 2007

Entity Framework June 2007 CTP Visual Basic Samples

As usual, the sample code included with the Entity Framework June 2007 CTP was written in C#. So I ported the basic elements of the C# EFJuneCTPSamples Web Site project to the VB EFJuneCTPwithVB ASP.NET Web Site (file system) project to determine if there were unexpected issues with VB 9.0. Updated 7/6/2007: Added detail text and a GridView page populated by an ObjectQuery from an Entity SQL statement. Note: John Papa's ADO.NET Entity Framework Overview in MSDN Magazine's July 2007 Data Points column provides a summary of EF features of the Orcas Beta 1 version. LINQ to Entities ObjectQueries The LINQGrid.aspx page executes a relatively simple LINQ to Entities ObjectQuery query against the C# sample's default mapping of an extended version of the Northwind Sample database. ObjectQuery objects implement IOrderedQueryable, IQueryable, IEnumerable(Of T), IQueryProvider, IOrderedQueryable, IQueryable, and IEnumerable, so you can bind them to GridView and other databound Web controls:

Click images to display full-size versions

Notice that the GridView maps only String data to columns automatically but the CustomerID foreign-key value is missing. If you want to display foreign key values, you must obtain them from the primary key of the related entities. (There is a request pending to provide read-only access to foreign-key values so associated entities don't need to be loaded to obtain key values.) You can bind GridView columns to OrderDate, RequiredDate, ShippedDate and Freight properties manually. The Country value isn't passed as a parameter to keep the query expression simple.

ObjectQuery also supports the ToBindingList method, which (theoretically) enables two-way (read/write) data binding with DataGridView controls.

The LINQText.aspx page has a more complex query with a specific projection and uses the Take(10) partition operator to display the last 10 orders from Mexico with a VB clone of the ObjectDumper utility.

The preceding page displays the same fields as those of the WinForms test harness for LINQ to SQL performance tests in my earlier posts here and here.

References and Namespaces for LINQ to Entities Queries

The sample Web Site project adds references to System.Data.Entity.dll and System.Data.DataSetExtensions.dll in the \Program Files\Reference Assemblies\Microsoft\Framework\v3.5 folder. System.Data.Entity.dll contains all EF-specific namespaces.

Only the default Imports System.Linq directive from System.Core.dll is needed for ordinary queries.

Using Entity SQL Statements with ObjectQuery Objects

The following page demonstrates the current state of Entity SQL (eSQL) syntax for ObjectQuery. Use of the related Customer entity's Country property as the restriction is contrived because ShipCountry could have provided this information (although it might differ from the Customer.Country value.)

ObjectQuery objects returned by eSQL queries that specify projections of properties with SELECT (instead of SELECT VALUE) return MaterializedDataRecords, not a type that implements IEnumerable. Dealing with DbDataRecord types isn't fun by any stretch of the imagination. Stick with LINQ for Entities queries when you want to return specified property (field) values.

Note: For a short course on Entity SQL, read the ADO.NET Team's EntitySQL post of May 30, 2007 by Mark Shields and Fabio Valbuena. Mike Dodaro's Entity Data Model 101: Part 1 (January 30, 2007) and Entity Data Model 101 - Part 2 (February 12, 2007) explain the EDM.

The following page substitutes a simplified Entity SQL query for the LINQ to Entities expression to verify that the GridView control responds to the ObjectQuery identically.

The GridView is missing the same fields as the first example.

Namespaces for Entity SQL ObjectQuery Operations

In addition to the default Imports System.Linq directive used with LINQ to Entities queries, add Imports System.Data.EntityClient and Imports System.Data.Objects directives.

Monday, July 02, 2007

ADO.NET Entity Framework June 2007 CTP Available for Download

The ADO.NET team missed their June target by only one working day for the first CTP of the ADO.NET Entity Framework after it's "liberation" from Visual Studio 2008. The announcement appeared about an hour ago.

You can download the CTP bits from this link on the ADO.NET Team blog: ADO.NET Entity Framework June 2007 CTP now available.

The CTP works only with Visual Web Developer Codename "Orcas" Express Edition June 2007 CTP. (Sigh!)

Update 2/8/2007: The CTP should work with the recently-released Microsoft Visual Studio Code Name "Orcas" Team Suite June 2007 Community Technology Preview (CTP). See the Entity Framework and Orcas Team Suite June 2007 CTPs post for more details.

The Visual Web Developer Express Edition only section says:

This is a CTP and the only edition of Visual Studio that works for this release is Visual Web Developer Express. This means that you will only be able to author Web sites in Visual Studio. Nothing prevents you from building console applications or WinForms applications outside of Visual Studio and building them with MSBuild and a .NET Framework compiler, but you will not be able to work on those project types inside of Visual Web Developer Express.

Update 7/3/2007: Danny Simmons' "June" CTP of the Entity Framework is finally out item (posted at 3:13 a.m. today)  describes the new features of the June CTP in more detail than the ADO.NET Team's list.

Download the sample code from the CodePlex ADO.NET Samples home page.

Danny Simmons and Tommy Williams explain why the Visual Web Developer June 2007 CTP is required to run ADO.NET EF June 2007 CTP in this ADO.NET Orcas forum post. Tommy says:

We do plan to align our CTPs and Betas as closely as we can to releases of Orcas so you can expect something that lines up with Orcas Beta 2 about the time that it comes out.

Update 7/4/2007: The ADO.NET Team revised the installation instructions to add a step that appears to be required only for Windows Vista. The instructions omit this admonition. which affects all OSes:

You must uninstall Orcas Beta 1 in order to install the June CTP bits.

The above is from Tommy Williams in yesterday's 10:35 PM response to my VWD Express June 2007 CTP Side-by-Side with VS 2008 Beta 1 Crashes Opening Any .aspx File post in the forum. He also notes that:

We are also currently investigating a problem where the Visual Web Developer Express installer (or possibly the .NET Framework v3.5 bootstrapper that it includes) is downloading the Orcas Beta 1 version of .NET Fx 3.5 rather than the June CTP version on Windows Vista. The installation works correctly on Windows XP but we are seeing failures on Windows Vista.

Detailed Steps for Running the Sample Files Under Virtualized Windows Vista

For Windows Vista running under Virtual Server 2005 RS (and presumably Virtual PC 2007), here's the full drill:

  1. Create a new virtual machine from an existing image or from scratch. It's not necessary to enable IIS 7.0.
  2. Run Windows Update. If you create a Vista virtual machine from scratch, there are 19 critical and recommended updates (as of 7/4/2007) of which 18 install. A reboot is required.
  3. Download and install SQL Server 2005 Express Edition (SSX) SP2 and SQL Server Management Studio Express (SSMX). Alternatively, install SSX with Advanced Services SP2.
  4. Optionally, download and install the case-insensitive version of the AdventureWorks sample database (AdventureWorksDBCI.msi) and the latest (May 2007) SQL Server Books Online. (If you download the Northwind sample database, don't attach it. The sample files include NORTHWND_101.MDB and NORTHWND_101_LOG.LDB in the App_Data folder.)
  5. Download and install the Microsoft .NET Framework 3.5 June 2007 Community Technology Preview (CTP). This undoubtedly is the culprit that prevents side-by-side installation of the VWDX CTP.
  6. Download and install VWDX June 2007 CTP (1-GB), which consists of the following four elements:
    • Microsoft Web Designer Tools
    • Microsoft .NET Framework v3.5
    • Windows SDK Tools for Visual Studio Express (x86)
    • Microsoft Visual Web Developer Codename Orcas Express Edition
    and requires a reboot after installation of the Windows SDK Tools for Visual Studio Express (x86). Installing the Express Edition documentation is optional.
  7. Optionally, create, build and run a default C# or VB WebSite project. Doing this will confirm that you have the correct .NETFx v3.5 CTP installed. (If you don't, errors result from assemblies that won't compile.)
  8. Download and install the ADO.NET Entity Framework June 2007 CTP.
  9. Download the EFLinqJuneCTPSamples.zip file to a temporary folder and extract all files to the C:\EFJuneCTPSamples folder. This is the root location that the configuration string in the \App_Code\BasePage.cs folder uses to attach the Northwind sample database files. (The folder name is hard coded.)
  10. Run VWDX as an Administrator, choose Open Web Site, navigate to the C:\EFJuneCTPSamples folder and click open.
  11. When the .NET Framework 2.0 Web Site Found message box opens, make sure the Do the Same for All .NET Framework 2.0 Web Sites in the Solution checkbox is marked and click OK.
  12. Press F5 to build the project and display Default.aspx page with links to 10 categories of LINQ to Entities queries. 
  13. If you incur an "It is an error to use a section registered as
    allowDefinition='MachineToApplication' beyond application level. This
    error can be caused by a virtual directory not being configured as an application in IIS" error, open the Web.config file in the root folder and comment out the <authentication mode="Windows"/> element on line 37.

As far as I can determine, the current LINQ to Entities queries don't demonstrate any new features that Danny mentions. The project's pages and navigation techniques won't win any design awards, either.

Update 7/6/2007: Installing EF adds a Program Files | ADO.NET Entity Framework menu with Class Reference, Help Overview, Readme, Samples and Tools Help choices. The ADO.NET Team's July 5, 2007 Entity Framework June CTP Documentation post describes problems with cross-references between the three help files.

Many of the documentation's EntitySQL query examples won't compile because of unresolved type references to Customer, Product, and the like. An example of a VB Web Site project with an eSQL query that will compile and execute is at Entity Framework June 2007 CTP Visual Basic Samples.

Note: As I mentioned in my June 17, 2007 Entity Framework: Complex Types Redux in Beta 2 post, Danny Simmons said on June 16:

Fortunately, we did decide to implement complex types, and they will appear in an upcoming CTP.

However, they aren't in this CTP. Danny says:

P.S. One other small bit of bad news: Complex Types didn't make it into this CTP.  So if you are holding out for them, unfortunately you'll have to wait for the next round.

The LinqDataSource control is missing from the VWDX CTP's toolbox, so there's still no opportunity to test it.