Sunday, April 26, 2009

Windows Azure and Cloud Computing Posts for 4/20/2009+

Windows Azure, Azure Data Services, SQL Data Services and related cloud computing topics now appear in this weekly series.

• Updated 4/24-26/2009: Additions, especially the RSA Conference 2009 and IBM DB2, Informix and other AMIs for Amazon EC2.

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Azure Blob, Table and Queue Services

Brent Stineman delivers his promised Azure Storage – Hands on with Queues, Part 2 post of 4/26/2009 with the details of how to get a list of queues, put a message into a queue, and read/peek at messages with RESTful HTTP Request and Response messages. (Brent doesn’t use the StorageClient class library.)

Part 1 is at Azure Storage – Hands on with Queues, Part 1.

My Thumbnails.sln Original and Enhanced Versions – Flow Diagrams post of 4/24/2009 provides flow diagrams of the original Thumbnails.sln project from the Windows Azure SDK (March 2009 CTP) with and without fixes for high bandwidth requirements by Steve Marx.

My Scalability and Cost Issues with Windows Azure Web and Worker Role Projects – Live Demo post of 4/20/2009 (updated 4/21/2009) analyzes performance problems and excessive cloud-services costs as the result of excessively heavy-duty AJAX page postbacks in the Thumbnails.sln sample project from the Windows Azure SDK (March 2009 CTP).

Microsoft’s Steve Marx, the author of the Thumbnails sample, added a comment to the post that showed how to fix the problem (albeit by sacrificing immediate visibility of added images for all users).

SQL Data Services (SDS)

No posts of significance as of 4/21/2009 at 1:43 PM.

.NET Services: Access Control, Service Bus and Workflow

• My Creating CardSpace Credentials at Microsoft’s Identity Labs Web Site of 4/25/2009 is an illustrated walk-through for creating CardSpace credentials for the Windows Cardspace “Geneva” contol panel tool that appears after you install the Geneva SDK.

• Ryan Dunn says “Azure Issue Tracker is temp[orarily] broken with March changes to .NET Services ACS API. Will be fixed soon-ish (crosses fingers)...” in a 4/25/2009 tweet. I should have re-read his Windows Azure and Geneva post of 4/2/2009, which says:

I don't like to basically re-blog what others have written, but I will make a minor exception today, as it is important enough to repeat.  My friend and colleague, Vittorio [Bertocci] has explained the current status of the Geneva framework running on Windows Azure today.

The short and sweet is that we know it is not working 100% today and we are working on a solution.  This is actually the main reason that you do not see a Windows Azure version of Azure Issue Tracker today.

• Matias Woloski’s Multi tenant federation with Geneva Framework and Microsoft .NET Services Access Control post of 4/23/2009 provides detailed diagrams and sample code to enable the following requirement:

"I want to enable single sign on and allow enterprises that have their own STS to integrate with us. For companies that don’t have any identity infrastructure in place we want to allow them to login with an ubiqu[it]ous credential like Windows LiveID. How do we do that without spending three months with a security guru?"

Live Windows Azure Apps, Tools and Test Harnesses

• David Pallmann continues his Azure grid-computing series with Grid Computing on the Azure Cloud Computing Platform, Part 3: Running a Grid Application of 4/25/2009, which uses the Azure Grid, the community edition of the Neudesic Grid Computing Framework. The two earlier posts are:

Jim Nakashima’s The Easy Way to Install the Windows Azure Tools and SDK Pre-Requisites shows you how to use the Microsoft Platform installer to set up the prerequisites for Windows Azure SDK CTPs.

Azure Infrastructure

• James Hamilton’s Randy Katz on High Scale Data Centers describes highlights of Randy Katz’s Tech Titans Building Boom, which focuses on data center infrastructure and the Google, Microsoft, Amazon, and Yahoo data center building boom. James notes:

Microsoft Chicago will have 200 containers in its lower floor (it’s a two floor facility) and it’s expected to be well over 45MW and will be 75MW if built out to the full 200 containers planned (First Containerized Data Center Announcement). The Chicago, Dublin, and Des Moines facilities have all been delayed by Microsoft presumably due to economic conditions: Microsoft Delays Chicago, Dublin, and Des Moines Data Centers.

Brandon Watson compares Amazon, Google, Microsoft - Big Three Cloud Providers Examined of 11/13/2008 reproduced in this 4/21/2009 post for Cloud Computing Magazine.

Input Industry Insight’s Evolution of the Cloud: The Future of Cloud Computing in Government research report of 4/21/2009 “projects the federal cloud market to expand to $800 million by 2013 while the state and local market for cloud applications could swell to $635 million by 2013.”

Datacenter Team posts Microsoft’s Top 10 Business Practices for Environmentally Sustainable Data Centers (celebrating Earth Day) on 4/21/2009.

Erik Sherman’s Cloud Computing Facing Stormy Weather post of 4/21/2009 casts a jaundiced eye on the subject:

The level of hype on cloud computing is heavy, though not surprising if you have any memory for industry trends. MRP, the paperless office, TQM, client-server computing, object-oriented programming, supply chain management, ERP, the Internet — all have been technical advances during whose introductions vendors queued to prove themselves on the leading edge. All eventually became important tools for corporate computing, but none offered the polished silver bullet they promised. Right now, cloud computing is thoroughly in the introductory stage.

“The first guy in gets to establish the space,” says Lynda Stadtmueller, a senior research analyst at Stratecast, a division of Frost & Sullivan. “That’s been the assumption.”

and then concludes with this quote:

“[Cloud computing] is part of our language today,” Stadtmueller says. “You’d better stick the word in somewhere or you’re going to be at a competitive disadvantage.”

Nicole Schepker reports in her Azure Cloud OS Journal Launched on Ulitzer post of 4/21/2009 that Brandon Watson, Microsoft’s Director of the Azure Services Platform Ecosystem, will be Editor and a Topic Contributor to the new Azure Cloud OS Journal.

Brandon Watson adds his US$0.02 in his McKinsey, The Cloud, and Fuzzy Calculations post of 4/21/2009 to the Azure Cloud OS Journal. Brandon writes:

They call Windows Azure a cloud example, and not Azure Services Platform. This confusion is consistent with customers and press/blogger sentiment that I am seeing.  Windows Azure is a piece of the overall Microsoft cloud play. It’s an application hosting environment, which serves as the foundation, though not required, layer for other code execution paths in the Azure Services Platform.  One can build applications that live completely on-premises without using Windows Azure, but utilize other pieces of the Azure Services Platform.

Mike Cameron and Rod Fontecilla offer their views on the March 2009 McKinsey & Co. report entitled "Clearing the air on cloud computing" in their Cloud Computing Debate: Booz Allen Hamilton Comments on Recent McKinsey & Co. Report post of 4/21/2009 by Kevin Jackson. Cameron and Fontecilla are Booz Allen Hamilton Principals.

James Hamilton analyzes the McKinsey report in his McKinsey Speculates that Cloud Computing May Be More Expensive than Internal IT post of 4/20/2008. His conclusion:

[A]ny company not fully understanding cloud computing economics and not having cloud computing as a tool to deploy where it makes sense is giving up a very valuable competitive edge. No matter how large the IT group, if I led the team, I would be experimenting with cloud computing and deploying where it make sense. I would want my team to know it well and to be deploying to the cloud when the work done is not differentiated or when the capital was better leveraged elsewhere

IT is complex and a single glib answer is almost always wrong. My recommendation is to start testing and learning about cloud services, to take a closer look at your current IT costs, and to compare the advantages of using a cloud service offering with both internal hosting and mixed hosting models.

Roger Strukhoff says Cloud is Bigger Than the Internet on 4/20/2009 in this part I of a two-part pseudo-analysis. Part II is here, for whatever it’s worth.

Cloud Computing Events

• IEEE Computer Socity announced on 4/25/2009 the IEEE 2009 International Conference on Cloud Computing (CLOUD-II 2009) to be held September 21-25, 2009 in Bangalore, India. The conference’s theme is “Changes we can lead” (whatever that means) and here’s the promo paragraph:

CLOUD 2009 is the identified hot-topic conference by the 2009 World Congress on Services (SERVICES 2009). The two well-established theme conferences identified by SERVICES 2009 are the 2009 IEEE International Conference on Web Services (ICWS 2009) in July 2009 in USA and the 2009 IEEE International Conference on Services Computing (SCC 2009) in September 2009 in India. 

Brian Prince reports 4/24/2009 for eWeek from the RSA Conference: Security Vendors Keep Head in the Cloud at RSA Conference:

Every RSA Conference has a popular buzzword or phrase. This year it was "the cloud."

In one way or another, vendors were pushing their answer to handling security in the cloud. Cisco unveiled a number of tools and services in the cloud April 21, even though a day later Cisco CEO John Chambers described the idea of securing a virtual cloud network as “a security nightmare.”

The Cloud Security Alliance published an 83-page Security Guidance for Critical Areas of Focus in Cloud Computing at the RSA Conference. From the Overview:

Covers key issues and provides advice for both Cloud Computing customers and providers within 15 strategic domains.

More information on the Alliance is here.

Gregory T. Huang’s User Interfaces, Cloud Computing, and Ray Ozzie—A Guide to the Season’s Tech Events post of 4/21/2009 reports:

On April 30, Microsoft is hosting an event organized by the WTIA where it will answer those questions, and more. Doug Hauger, Microsoft’s general manager of cloud infrastructure services, will give an overview of Azure and Microsoft’s view of cloud computing software and services.

A bonus speaker will be Ian Knox, director of product management at Skytap, a Seattle-based cloud computing and virtual lab startup. Knox will talk about using cloud computing for Windows applications and lowering IT lab costs, among other things. All in all, it’s essential stuff for anyone interested in shaping the future of the cloud.

The Workshop on Software Engineering Challenges of Cloud Computing @ ICSE 2009 issued on 4/21/2009 a “Call for participation: Workshop on Software Engineering Challenges of Cloud Computing” in conjunction with the 31st International Conference on Software Engineering (ICSE) to be held May 23rd, 2009 in Vancouver, Canada.

Graeme Thickins reports on the Minnesota High-Tech Association's annual spring conference and CloudCampMSP in his It Was Cloud Week in Minneapolis, and All the Cool Kids (and Old Guys!) Were There post of 4/21/2009.

António Costa quotes David Carrera, director of the Cloud Computing (CC) research team at Spain’s Universitat Politecnica de Catalunya (UPC) in his Cloud computing: changing the way we work post of 4/21/2009.

Nicole Schepker’s Burton Group Announces In-Depth Cloud Coverage post of 4/20/2009 announces a “new research coverage area focused on understanding, defining, integration, and the optimization of cloud computing.” The group says:

Burton Group provides practical advice for the underlying architectures, technologies, and platforms necessary, including:

  • Building cloud-friendly applications
  • Public, private, and internal clouds
  • Server virtualization, migration, and fail-over strategies
  • Segmenting data to leverage outsourced storage
  • Understanding software-as-a-service (SaaS) , platform-as-a-service (PaaS), and infrastructure as a service (IaaS).
  • Improving business continuity by exploiting cloud-based architectures

and will offer a Cloud Computing track at its forthcoming Catalyst conference 7/27 – 7/31/2009 in San Diego, CA.

Other Cloud Computing Platforms and Services

ABS*CBN News (Manila) reports IBM plans cloud computing services for 2009 from Reuters on 4/26/2009:

IBM got its feet wet in the field last year when it launched an Internet-based data backup and recovery service.

In addition to the new service for developers, IBM also plans to introduce clouds that let businesses run applications and to virtualize personal computers, Sims said.

Like IBM, Microsoft is viewed as having been slow to enter the field. It plans to launch its cloud services platform, dubbed Azure, late this year or early in 2010

Amazon Web Services announces in Amazon Elastic Compute Cloud (EC2) running IBM of 4/24/2009, a set of IBM services similar to that for Windows Server 2003 and SQL Server [Express]. The following IBM AMIs 

“are ready to run in Amazon EC2 with Novell SuSE Linux and the associated IBM products” at rates ranging from US$0.38/hour for a Standard Small (default) instance type for DB2 Express edition to US$24.35/hour for High CPU Extra Large instances Running IBM WebSphere Portal Server and IBM Web Content Management Server Standard Edition.

Paul Miller says he was Talking to Simon Wardley about Ubuntu and the Cloud on 4/21/2009. Paul writes:

Earlier today I spoke with Simon Wardley of Canonical (the commercial organisation that sells support and consultancy for Ubuntu) to hear a little more about what those downloading Ubuntu will get… and what it might mean for the rapidly shifting Cloud landscape.

Maureen O’Gara’s VMware’s Newfangled Cloud OS Positioned for Takeoff post of 4/21/2009 provides a brief synopsis of VMware’s vSphere 4 offering.

Alin Irimie analyzes VMware’s press release in his VMware vSphere 4 - First Operating System For Building The Internal Cloud post of 4/21/2009.

Kris Tuttle’s Winners and Losers from Oracle / Sun post of 4/21/2009 to the Seeking Alpha blog analyzes the forthcoming Sun Microsystems purchase by Oracle (instead of IBM). Kris lede’s off:

Primarily this is a defensive (meaning not very creative) move by Oracle (ORCL). IBM would have been a measurably stronger competitor for Oracle with Java and MySQL added to their formidable software stack. So the real value for Oracle is more in IBM not having Sun rather than direct benefits for Oracle. All their statements to the contrary are only corporate posturing. That fact is not going to be good news for Sun employees, who can expect Oracle to make the Sun acquisition highly profitable by dramatically eliminating costs, shutting down initiatives and exiting businesses.

and concludes that Oracle / Sun is better for Microsoft than IBM / Sun.

Darryl Plummer asks Night of the Living Dead: Will Oracle buying Sun Raise the Specter of “Network Computing” Again? and on 4/20/2009. He writes:

Today, Oracle announced plans to acquire Sun Microsystems. Gartner is in the process of generating an official position on what this means. You’re going to want to see what we have to say since there will be multiple pieces dissecting this acquisition announcement.

Roger Strukhoff writes Oracle-Sun Nice Cloud Computing Fit, But Maybe Not on 4/20/2009.

Paulo Calçada analyzes from the cloud-computing standpoint Oracle’s pending purchase of Sun Microssystems in his ORACLE + SUN: The Cloud is growing, or losing one important player? of 4/20/2009.

Creating CardSpace Credentials at Microsoft’s Identity Labs Web Site

Microsoft’s Identity Lab (Identity Protocols Security Token Service, ipsts) is a set of hosted security token services to support testing of Identity Protocols. The goal of the lab is to provide a set of custom test endpoints to evaluate the interoperability of Identity Protocols amongst multiple partners and vendors. Microsoft promotes its CardSpace credentials as an industry-standard, SAML v1.1-compliant source of identity information.

Note: For an overview of Windows “Geneva” and CardSpace, read David Chappell & Associates’ “Introducing ‘Geneva’” white paper and Matias Woloski’s Multi tenant federation with Geneva Framework and Microsoft .NET Services Access Control post of 4/23/2009, from which the following illustration was taken:

There are no help buttons on the pages for creating a CardSpace credential, so I captured the following page views to help readers of Chapter 9, “Authenticating Users with .NET Access Control Services” add a CardSpace credential to their computer.

Browse to the Microsoft Identity Lab’s Microsoft Identity Interop Sts Logon page and click the Sign Up button to open the Registration page. Type a fictitious name in the UserName text box, a password in the Password and Confirm Password text boxes, and mark the Accept Terms of Use check box:

Click Submit to open the Claims Configuration page. Accept the default (marked) setting for the By Default, Release the Following Claims to Any Relying Party check box. These are the minimum claims required by most relying parties. Type fictitious names in the First Name and LastName text boxes, accept the referring party’s Email Address (UserName@ipsts.federatedidentity.net).

Click Continue to open the Edit Profile Information/Manage Relying Party Policies page (see Figure 9-09.) The Edit Profile Information link opens a page that lets you add to and edit the information you entered previously; the Manage Relying Party Policies page enables selecting the profile information you release to relying parties:

Click the Edit Profile Information Link to open an expanded version of the Claims Configuration page. Complete the entries with fictitious information and enable their selection for your profile by marking the associated check box:

Click Submit to return to the Edit Profile Information/Manage Relying Party Policies page, click Save to return to the Manage Relying Party Policies and click the Manage Relying Party Policies link to open the Relying Parties page which contains Edit/View buttons for HTTPS and HTTP policies.

Click the EditView button for the https://relyingparty.federatedidentity.net party to open the Edit a Policy page. Mark the check boxes for the profile items you want to release and select the public key file (CertName.cer) for your localhost or other certificate you created for authenticating users:

Click Save to save your changes and return to the Edit Profile Information/Manage Relying Party Policies page.

Click the Download Your Username/Password card button to open the File Download dialog for the InformationCard.crd file, and click Yes when asked if you want to save the card with Windows CardSpace “Geneva” on your local computer. This will add the CardSpace credential to the Windows CardSpace “Geneva” Beta Control Panel applet (Control Panel –> Windows Cardspace “Geneva” in Vista):

For more information on the CardSpace Control Panel applet, read Oren Melzer’s The CardSpace “Geneva” Selection Experience post of 11/18/2008 to the somnolent “Geneva” Team Blog.

The OfficeLive team announced on 8/27/2007 that Windows Live ID adds Beta support for Information Cards with Windows CardSpace! However, I haven’t been able to get the Windows Live ID Information Card management page to work, regardless of whether I use saved username/password credentials or type them in manually. After almost two years, you would think their process would be out of beta and work reliably.

Mary Branscombe's When will Windows Live stop treating CardSpace as the unwanted stepchild? post of 10/29/2008 laments that “Windows Live [ID is] ignoring CardSpace.”

Friday, April 24, 2009

Thumbnails.sln Original and Enhanced Versions – Flow Diagrams

Additional diagrams of Steve Marx’s fix for extreme polling bandwidth requirements (Thumbnails2.sln) and the final version with the GridView and thumbnail deletion features (PhotoGallery.sln) will be added as I draw them.

The diagrams are from Chapter 8, “Messaging with Azure Queues,” of my forthcoming Cloud Computing with the Microsoft Azure Services Platform book for WROX.

Click the images to download from Windows Live SkyDrive and display a full-size PDF version.

Original Thumbnails.sln from the Windows Azure SDK (March 2009 CTP)

Thumbnails2 with JavaScript Modifications by Steve Marx to Reduce Traffic Shaded

Steve described his modifications in a extensive comment to my Scalability and Cost Issues with Windows Azure Web and Worker Role Projects – Live Demo post of 4/22/2009. You can run a live demo of this version at http://oakleaf6.cloudapp.net/Default.aspx. Launch Fiddler2 to prove there’s no traffic when the client is quiescent (just connected to the Azure service.)

Wednesday, April 22, 2009

Scalability and Cost Issues with Windows Azure Web and Worker Role Projects – Live Demo

The Thumbnails sample project from the Windows Azure SDK (March 2009 CTP)’s Samples.zip file illustrates combining Web and Worker Roles to upload graphics files from the user’s local file system into Azure Blobs and display thumbnail images from these in a bound ListView control. An AJAX UpdatePanel contains the ListView to minimize UI flashing on postbacks. A Worker Role handles the thumbnail generation process asynchronously by polling the ‘thumbnailmaker’ queue at one-second intervals.

Update 4/21/2009: Comment added by Microsoft’s Steve Marx, who wrote the original Thumbnails.sln sample project; details on the method causing the problem; and hourly network usage data from Windows Azure Metrics. See end of this post.

Update 4/22/2009: Simple modification to eliminate JavaScript exception in Steve Marx’s fix.

Update 4/24/2009: Visit My Thumbnails.sln Original and Enhanced Versions – Flow Diagrams post for flow diagrams of the original and fixed versions of the Thumbnails projects.


Here’s a screen capture of the original Thumbnails.sln’s Web Role with 15 *.jpg thumbnails from *.png images used in previous OakLeaf posts. (Click images to display full-size 1,024 x 768 captures.)

You can run the original Thumbnails.sln project at http://oakleaf6.cloudapp.net/Default.aspx.

Following is the WorkerRole’s HTTP ASYNC Request message with 3,110 bytes of ViewState:

POST /Default.aspx HTTP/1.1
Accept: */*
Accept-Language: en-us
Referer: http://1f33a544-f6d1-431f-bf6d-42fe54904995.cloudapp.net/Default.aspx
x-microsoftajax: Delta=true
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Cache-Control: no-cache
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30428; .NET CLR 3.5.30729; .NET CLR 3.0.30618; MS-RTC LM 8; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.1.3)
Host: 1f33a544-f6d1-431f-bf6d-42fe54904995.cloudapp.net
Content-Length: 3110
Connection: Keep-Alive
Pragma: no-cache

sm1=up1%7Ctimer1&__EVENTTARGET=timer1&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwUJOTA5OTI0ODEyD2QWAgIDDxYCHgdlbmN0eXBlBRNtdWx0aXBhcnQvZm9ybS1kYXRhFgICBw9kFgJmD2QWA
...
yM2RkGAEFCnRodW1ibmFpbHMPPCsACgIHPCsADwAIAg9kum72snd50MHlpjvMEsnb17lu6dc%3D&__EVENTVALIDATION=%2FwEWAgL094j7BwKSuuDUC8HaRKdWg3LeOCFwZBwLVxzVxzF2&__ASYNCPOST=true&

and here’s the HTTP Response message with a list of blobs having a ‘thumbnails/’ prefix and ViewState, which total 7,217 bytes:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/plain; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Mon, 20 Apr 2009 19:39:51 GMT
Content-Length: 7217

3488|updatePanel|up1|
        
                
                <img id="thumbnails_ctrl0_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633755652469388030_95a72280-fd3e-438a-9624-73b7cfc5e109" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl1_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633756498271797216_1f06998c-d6f9-423c-b8ba-66277e7aebc6" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl2_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757268772274027_d895bb79-8a41-44c4-9514-d391faff878d" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl3_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757440636206298_114f3d2f-4fbc-4d83-9831-954ae0693734" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl4_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758473471680000_549f111f-f1e4-4652-b886-d1b703740797" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl5_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758476929180000_5cc14527-64be-4e19-ba2a-5b19fd1f51dd" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl6_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758477105033750_e25614be-d2a6-46d9-87f3-a0eb63a7b392" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl7_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478020658750_d56f76ad-9a26-46f2-ae0c-c7ce30d228ef" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl8_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478137773750_923ab9f2-3607-4665-8f59-45df1fcc3a77" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl9_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479545430000_74ff03eb-878b-490e-808a-0f6792f726f7" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl10_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479706752500_9a5a7c6f-1dd9-48bc-82b5-aaaa2c431b78" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl11_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758480860273750_e54eee86-8ddb-44cb-82b8-badbaa1b2a8d" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl12_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482608086250_bfbd7e5a-026c-45ce-88f7-b6eb563af89b" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl13_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482981283750_d41a21a7-def6-4688-ae29-74109b6050a7" style="border-width:0px;" />
            
                <img id="thumbnails_ctrl14_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758484541283750_6d03a107-dfc7-4551-8201-8052ddb9ec23" style="border-width:0px;" />
            
            
                   <span id="timer1" style="visibility:hidden;display:none;"></span>
        |0|hiddenField|__EVENTTARGET||0|hiddenField|__EVENTARGUMENT||2952|hiddenField|__VIEWSTATE|/wEPDwUJOTA5OTI0ODEyD2QWAgIDDxYCHgdlbmN0eXBlBRNtdWx0aXBhcnQvZm9ybS1kYXRhFgICBw9kFgJmD2QWAg
...
M2RkGAEFCnRodW1ibmFpbHMPPCsACgIHPCsADwAIAg9kum72snd50MHlpjvMEsnb17lu6dc=|48|hiddenField|__EVENTVALIDATION|/wEWAgL094j7BwKSuuDUC8HaRKdWg3LeOCFwZBwLVxzVxzF2|0|asyncPostBackControlIDs|||0|postBackControlIDs|||4|updatePanelIDs||tup1|0|childUpdatePanelIDs|||3|panelsToRefreshIDs||up1|2|asyncPostBackTimeout||90|12|formAction||Default.aspx|13|pageTitle||Photo Gallery|149|scriptBlock|ScriptPath|/ScriptResource.axd?d=9YtLxwU-nmRnm7oJ9nOfEaztduuiriMqe964NLCEARKknzUa7EJGulUSq-QKJyN3_XQG98ij__ElfezvDJkF-OgjZtff28LO1U1_MHbdocg1&t=ffffffff9a77c993|155|scriptStartupBlock|ScriptContentNoTags|Sys.Application.add_init(function() {
    $create(Sys.UI._Timer, {"enabled":true,"interval":1000,"uniqueID":"timer1"}, null, null, $get("timer1"));
});
|

Note: Run your own tests with the live demo and Fiddler2 if you’d like to see the unexpurgated viewstate.


Following is a screen capture of an upgraded version of the Thumbnails.sln project from Chapter 8, “Messaging with Azure Queues,” of my forthcoming Cloud Computing with the Microsoft Azure Services Platform book for WROX:

Modifications include the addition of a GridView control to display five attributes of the PhotoGallery blobs having a ‘thumbnails/’ prefix. (The ContentEncoding and ContentLanguage attributes aren’t populated) Another important feature is the ability to delete all but three thumbnails and their source blobs.

You can run the upgraded Thumbnails.sln project at http://oakleaf5.cloudapp.net/Default.aspx.

Here’s the HTTP Request message with 9,179 payload bytes (The GridView increases the ViewState payload):

POST /Default.aspx HTTP/1.1
Accept: */*
Accept-Language: en-us
Referer: http://oakleaf5.cloudapp.net/Default.aspx
x-microsoftajax: Delta=true
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Cache-Control: no-cache
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30428; .NET CLR 3.5.30729; .NET CLR 3.0.30618; MS-RTC LM 8; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.1.3)
Host: oakleaf5.cloudapp.net
Content-Length: 9179
Connection: Keep-Alive
Pragma: no-cache

sm1=up1%7Ctimer1&txtTime=&__EVENTTARGET=timer1&__EVENTARGUMENT=&__VIEWSTATE=WR%2BUOP22CwPyX73JtlDmUmmPrRsHh%2BiBwiRDWCm%2BViU00I%2B4hxdW5gvjpq9nCGyVwHj1CdJ8bwxR3oBOYshQUQi2TSlwFRCsOrbowEk4cyE7Ykxo3FcFv9q0k7COA092NHbEls1
...
2FxpttnPCr4zm6hlQbOPlbRXR0yso6CJk9acjrOxqRWOw5l15ypXkVJ0jXv7svxpJ34Ob%2BUGRboSIp6t8xMB7n5mB7cnUqBIsqW7bFnn01Z5gvCxUajLJFx1mUs7XkUr4Yfsbj9w0xTzcL7IMJenC%2BriHNm7GSCJnONw%3D&__VIEWSTATEENCRYPTED=&__ASYNCPOST=true&

and the HTTP Response with payload bytes:

HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/plain; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Mon, 20 Apr 2009 20:51:21 GMT
Content-Length: 18849

9586|updatePanel|up1|
                    
                            
                            <img id="thumbnails_ctrl0_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633755652469388030_95a72280-fd3e-438a-9624-73b7cfc5e109" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl1_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633756498271797216_1f06998c-d6f9-423c-b8ba-66277e7aebc6" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl2_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757268772274027_d895bb79-8a41-44c4-9514-d391faff878d" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl3_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757440636206298_114f3d2f-4fbc-4d83-9831-954ae0693734" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl4_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758473471680000_549f111f-f1e4-4652-b886-d1b703740797" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl5_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758476929180000_5cc14527-64be-4e19-ba2a-5b19fd1f51dd" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl6_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758477105033750_e25614be-d2a6-46d9-87f3-a0eb63a7b392" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl7_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478020658750_d56f76ad-9a26-46f2-ae0c-c7ce30d228ef" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl8_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478137773750_923ab9f2-3607-4665-8f59-45df1fcc3a77" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl9_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479545430000_74ff03eb-878b-490e-808a-0f6792f726f7" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl10_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479706752500_9a5a7c6f-1dd9-48bc-82b5-aaaa2c431b78" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl11_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758480860273750_e54eee86-8ddb-44cb-82b8-badbaa1b2a8d" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl12_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482608086250_bfbd7e5a-026c-45ce-88f7-b6eb563af89b" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl13_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482981283750_d41a21a7-def6-4688-ae29-74109b6050a7" style="border-width:0px;" />
                        
                            <img id="thumbnails_ctrl14_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758484541283750_6d03a107-dfc7-4551-8201-8052ddb9ec23" style="border-width:0px;" />
                        
                        
                    <div id="timer" style="height: 37px">
                        Create Thumbnail Time, seconds:&nbsp;
                        <input name="txtTime" type="text" id="txtTime" style="width:67px;" />
                    </div>
                    <div id="divGridView"">
<div>
<table title="Displays the current thumbnail blobs." cellspacing="0" cellpadding="4" rules="cols" border="1" id="gvBlobs" style="color:#6B696B;background-color:White;border-color:#DEDFDE;border-width:1px;border-style:None;border-collapse:collapse;">
        <tr style="color:White;background-color:#6B696B;font-weight:bold;">
            <th scope="col">Blobs</th><th scope="col">Name</th><th scope="col">ContentType</th><th scope="col">ContentLength</th><th scope="col">LastModifiedTime</th><th scope="col">ETag</th>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$0')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633755652469388030_95a72280-fd3e-438a-9624-73b7cfc5e109</td><td>image/jpeg</td><td>3993</td><td>4/17/2009 6:39:39 PM</td><td>0x8CB8DA2FFD51660</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$1')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633756498271797216_1f06998c-d6f9-423c-b8ba-66277e7aebc6</td><td>image/jpeg</td><td>4269</td><td>4/18/2009 6:10:01 PM</td><td>0x8CB8E6806E01B60</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$2')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633757268772274027_d895bb79-8a41-44c4-9514-d391faff878d</td><td>image/jpeg</td><td>4803</td><td>4/19/2009 3:33:40 PM</td><td>0x8CB8F1B598712C0</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$3')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633757440636206298_114f3d2f-4fbc-4d83-9831-954ae0693734</td><td>image/jpeg</td><td>3761</td><td>4/19/2009 8:20:12 PM</td><td>0x8CB8F4360FB7640</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$4')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758473471680000_549f111f-f1e4-4652-b886-d1b703740797</td><td>image/jpeg</td><td>3433</td><td>4/20/2009 6:01:35 PM</td><td>0x8CB8FF92DB10050</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$5')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758476929180000_5cc14527-64be-4e19-ba2a-5b19fd1f51dd</td><td>image/jpeg</td><td>3584</td><td>4/20/2009 6:07:20 PM</td><td>0x8CB8FF9FBC05040</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$6')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758477105033750_e25614be-d2a6-46d9-87f3-a0eb63a7b392</td><td>image/jpeg</td><td>4523</td><td>4/20/2009 6:08:42 PM</td><td>0x8CB8FFA2C29CEA0</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$7')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758478020658750_d56f76ad-9a26-46f2-ae0c-c7ce30d228ef</td><td>image/jpeg</td><td>5346</td><td>4/20/2009 6:10:13 PM</td><td>0x8CB8FFA62786520</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$8')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758478137773750_923ab9f2-3607-4665-8f59-45df1fcc3a77</td><td>image/jpeg</td><td>4205</td><td>4/20/2009 6:09:22 PM</td><td>0x8CB8FFA440486F0</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$9')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758479545430000_74ff03eb-878b-490e-808a-0f6792f726f7</td><td>image/jpeg</td><td>5058</td><td>4/20/2009 6:11:43 PM</td><td>0x8CB8FFA986DB810</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$10')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758479706752500_9a5a7c6f-1dd9-48bc-82b5-aaaa2c431b78</td><td>image/jpeg</td><td>3675</td><td>4/20/2009 6:13:01 PM</td><td>0x8CB8FFAC69E8480</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$11')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758480860273750_e54eee86-8ddb-44cb-82b8-badbaa1b2a8d</td><td>image/jpeg</td><td>3218</td><td>4/20/2009 6:13:54 PM</td><td>0x8CB8FFAE65235B0</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$12')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758482608086250_bfbd7e5a-026c-45ce-88f7-b6eb563af89b</td><td>image/jpeg</td><td>4567</td><td>4/20/2009 6:16:49 PM</td><td>0x8CB8FFB4E5FC870</td>
        </tr><tr style="background-color:White;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$13')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758482981283750_d41a21a7-def6-4688-ae29-74109b6050a7</td><td>image/jpeg</td><td>2909</td><td>4/20/2009 6:18:28 PM</td><td>0x8CB8FFB89EB7740</td>
        </tr><tr style="background-color:#F7F7DE;">
            <td><a href="javascript:__doPostBack('gvBlobs','Delete$14')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758484541283750_6d03a107-dfc7-4551-8201-8052ddb9ec23</td><td>image/jpeg</td><td>4005</td><td>4/20/2009 6:21:04 PM</td><td>0x8CB8FFBE6BEBDB0</td>
        </tr>
    </table>
</div><span id="statusMessage" style="color:Red;"></span>
</div><span id="timer1" style="visibility:hidden;display:none;"></span>                    
                |0|hiddenField|__EVENTTARGET||0|hiddenField|__EVENTARGUMENT||8300|hiddenField|__VIEWSTATE|WR+UOP22CwPyX73JtlDmUmmPrRsHh+iBwiRDWCm+ViU00I+
...
+W3PbzKDhmGYLycE6Aw4kYh4TL842jUgYDkeXmfSwpIUW621XJxMRbpw6x4fJPslo2aJZPiGi6OuPuXXTGS0ozejw6pMwaDuWvI9EPxEnMQ+gojkKoAusxU6q+lA0benFiF9OlukXDn68uEK+oso0hoMV8wfDVBDWqG6bJi2kIUNPCiaNAzVRW1rxLJETlwtbl6d6LA=|0|hiddenField|__VIEWSTATEENCRYPTED||172|hiddenField|__EVENTVALIDATION|Hi0WJQfF/xpttnPCr4zm6hlQbOPlbRXR0yso6CJk9acjrOxqRWOw5l15ypXkVJ0jXv7svxpJ34Ob+UGRboSIp6t8xMB7n5mB7cnUqBIsqW7bFnn01Z5gvCxUajLJFx1mUs7XkUr4Yfsbj9w0xTzcL7IMJenC+riHNm7GSCJnONw=|0|asyncPostBackControlIDs|||0|postBackControlIDs|||4|updatePanelIDs||tup1|0|childUpdatePanelIDs|||3|panelsToRefreshIDs||up1|2|asyncPostBackTimeout||90|12|formAction||Default.aspx|38|pageTitle||Photo Gallery Azure Queue Test Harness|149|scriptBlock|ScriptPath|/ScriptResource.axd?d=9YtLxwU-nmRnm7oJ9nOfEaztduuiriMqe964NLCEARKknzUa7EJGulUSq-QKJyN3_XQG98ij__ElfezvDJkF-OgjZtff28LO1U1_MHbdocg1&t=ffffffff9a77c993|155|scriptStartupBlock|ScriptContentNoTags|Sys.Application.add_init(function() {
    $create(Sys.UI._Timer, {"enabled":true,"interval":1000,"uniqueID":"timer1"}, null, null, $get("timer1"));
});
|

The following table compares the Request, Response and total message sizes in bytes of the two Photo Gallery versions:

Project Version Request Header Request Payload Response Header Response Payload Total Message
Original

702

3,110

221

7,217

11,250

Enhanced

646

9,179

224

18,849

28,898

Estimated Data Egress and Ingress Charges Based on Amazon Web Services EC2

Microsoft hasn’t announced its pricing but says Azure Services will be “competitive.” Therefore, AWS posted prices are used to estimate data transfer charges.

1,000 simultaneous users of the enhanced version with an average of 15 saved thumbnails would generate (646 + 9,179) = 9.825 MB/s of data egress and (224 + 18,849) = 19.073 MB/s of data ingress from polling postbacks alone. This corresponds to (9.825 * 60 * 60 * 24)/1000 = 848.880 GB/day egress and (19.073 * 60 * 60 * 24)/1000 = 1647.9 GB/day ingress.

The AWS price for EC2 or S3 data ingress is US$ 0.10 per GB = $164.79/day.

Pricing for EC2 or S3 data egress is based on a monthly volume of (848.9 + 1647.9) GB * 30 = 74.9 GB/month = < 1 TB/month. The first 10 TB/month is US$ 0.17/GB for 848.9 GB/day = US$ 144.31/day.

Daily data transfer costs are US$ 164.79 + $ 144.31 = US$ 309.10/day = US$ 9273.09/month per 1,000 average simultaneous users.

Increasing the polling interval from 1 to 10 seconds would reduce the cost to US$ 927.31 /month but might cause unacceptable latency.

Estimated Data Storage Charges Based on Amazon Web Services S3

The average size of the *.png images is ~400 KB and of the thumbnails ~4 KB for a total of 15 * 404 KB = 6.060 MB/user * 1,000 users = 6.06 GB average. AWS charges $0.15/GB*month for S3 storage, so the monthly storage cost would be $0.91/month, which is not significant with 1,000 users.

I’ll run the enhanced version for 24 hours or so and report in an update the stats from the Azure Analytics data for Hosted Services and Storage Accounts.

I’d be very grateful if someone would check my calculations.


Updates 4/21/2009

Microsoft developer-evangelist Steve Marx added a comment to this post and a reply to my Scalability and Cost Issues with Windows Azure Web and Worker Role Projects – Live Demo thread in the Windows Azure forum. Steve’s point is:

[T]his has nothing to do with Windows Azure; it's just a question of how best to use AJAX to make a responsive UI.  The same optimizations would apply if you're hosting somewhere other than Windows Azure.

My point is that most Web developers don’t write polling apps very often and that Azure Queues with poorly designed code can lead to serious performance and cost problems. Microsoft has been taking a lot of heat lately for sample ASP.NET code and architectural recommendations that don’t meet the .NET Developer community’s definitions of best practices. This is another example.

Here’s the culprit method in the WebRole’s original _Default class:

protected void Page_PreRender(object sender, EventArgs e)
{    
    BlobContainer testBlob = GetPhotoGalleryContainer();
    thumbnails.DataSource = from o in GetPhotoGalleryContainer().ListBlobs("thumbnails/", false) select new { Url = ((BlobProperties)o).Uri };
    thumbnails.DataBind();
}

I tried wrapping the operative statements in an

if (GlobalClass.refreshData || !Page.IsPostBack)

block where GlobalClass.refreshData boolean variable is set true by the gvBlobs.RowDeleting() event handler and the WorkerRole’s GetMessage() loop and is set false after invoking the DataBind() method.

This updates the page immediately after deleting a thumbnail but, due to timing problems, doesn’t solve the problem with displaying additions immediately. In fact, added thumbnails never appear unless you close and reopen the service connection.

Following are the Hourly Network Usage metrics for 18 hours of continuous operation with (presumably) a single user connected to the oakleaf5 host:

The average for AJAX polling postbacks for the 18 hours is about 30 MB/hour * 24 = 720 MB per day, which is off by a factor of about 2.4 from the previously calculated 848.880 GB/day egress and (19.073 * 60 * 60 * 24)/1000 = 1647.9 GB/day ingress for 1,000 users = 2496.8 MB/day for a single user. Thus, I’m taking a second look at my calculations. Watch for an update later today.


Updates 4/22-24/2009

To make Steve Marx’s alternate polling approach work without throwing a JavaScript exception when adding a thumbnail, add EnablePageMethods=”true” to the ScriptManager declaration:

<asp:ScriptManager ID="sm1" runat="server" EnablePageMethods="true" />

To see your deletions occur immediately, you’ll need to set refreshData = true in your deletion method code.

Thanks to Steve Marx for the heads up on the missing EnablePageMethods attribute.

You can run a live demo of this version at http://oakleaf6.cloudapp.net/Default.aspx. Launch Fiddler2 to prove there’s no traffic when the client is quiescent (just connected to the Azure service.)

The 2.4x overcalculation of data ingress and egress load appears to be due to the actual average polling rate being 2.4 seconds, not the expected 1.0 seconds. More investigation as to the reason follows, but it’s probably due to the slow upstream speed of my AT&T DSL connection to the data center.

Tuesday, April 21, 2009

Windows Azure and Cloud Computing Posts for 4/13/2009+

Windows Azure, Azure Data Services, SQL Data Services and related cloud computing topics now appear in this weekly series.

• Updated 4/21/2009 for posts through 4/19/2009.
Updated 4/15/2009 3:40 PM PDT for 33-page McKinsey & Co. Clearing the Air on Cloud Computing report and Nicholas Carr’s The big company and the cloud response (see the “Azure Infrastructure” section).

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Azure Blob, Table and Queue Services

Joe Gregorio, the primary contributor to the AtomPub protocol, concludes on 4/18/2009 that The Atom Publishing Protocol is a failure in the light of “browsers are much more powerful, Javascript compatibility is increasing among them, there are more libraries to smooth over the differences, and connectivity is on the rise.” He also attributes AtomPub’s failure to JavaScript Object Notation:

The idea was that with a common format you could build up libraries and make it easy to move information around. The 'problem' in this case is that a better format came along in the interim: JSON. JSON, born of Javascript, born of the browser, is the perfect 'data' interchange format, and here I am distinguishing between 'data' interchange and 'document' interchange. If all you want to do is get data from point A to B then JSON is a much easier format to generate and consume as it maps directly into data structures, as opposed to a document oriented format like Atom, which has to be mapped manually into data structures and that mapping will be different from library to library.

Dare Obasanjo adds his commentary on Joe’s post in Joe Gregorio on why the Atom Publishing Protocol (AtomPub) is a failure of 4/18/2009. Dare’s conclusion:

In my opinion, the growth in popularity of object-centric JSON over document-centric XML as the way to expose APIs on the Web has been the real stake in the heart for the Atom Publishing Protocol.

Rick Strahl discusses browser support for JSON in his Native JSON Parsing: What does it mean? post of 4/19/2009.

Brent Stineman’s detailed Azure Storage – Hands on with Queues, Part 1 of 4/16/2009 is the first member of a series on writing HTTPWebRequest calls to Azure Queues without using the sample ServiceClient library.

Rob Bagby proceeds with his detailed, lavishly illustrated Azure Web Services project with Azure Application Part 3: Expose (REST) Web Service And Consume in Silverlight of 4/17/2009. The screencast is here.

Rob Bagby continues his Azure shopping cart project with a 10-foot-long Azure Application Part 2: Access Azure Table Storage post of 4/14/2009:

This is part 2 in this series where I am building an Azure shopping cart application from the ground up.  In this post, I will create a simplified ASP.NET version of the wine catalog.  We will create a table in developer storage (the local version of Azure Storage) to store our wines and write 2 web pages: 1 to view all wines and another to add a wine.  We will then access the same table in the cloud in Azure Table Storage.

Here’s a link to the screencast: deCast - Building an Azure App Part II: Azure Table Storage

SQL Data Services (SDS)

Ryan Dunn asks Why does Windows Azure use a cscfg file? and provides the answer in this post of 4/16/2009.

Microsoft reminds us that Microsoft SQL Server Data Mining Services for the cloud are available for SDS. There’s more information at the home page: http://www.sqlserverdatamining.com/ssdm/.

.NET Services: Access Control, Service Bus and Workflow

Don’t forget the New Azure Training Kit [is] Available with

    • 11 hands-on labs – including new hands-on labs for PHP and Native Code on Windows Azure.
    • 18 demo scripts – These demo scripts are designed to provide detailed walkthroughs of key features so that someone can easily give a demo of a service
    • 9 presentations – the presentations used for our 3 day training workshops including speaker notes

You can download it from http://go.microsoft.com/fwlink/?LinkID=130354

Live Windows Azure Apps, Tools and Test Harnesses

Aleksy Savateyev describes his forthcoming Azurelight project that combines a Silverlight front end with an Azure Services Platform back end in his Announcing Azurelight post of 4/10/2009:

Microsoft codename "Azurelight" is an easy-to-use application for providing basic product support in the cloud, collecting feedback about products and exchanging opinions with other users. It's also intended to be used by developers as a reference application utilizing both Windows Azure and Silverlight for rich yet scalable and highly available business solutions.

Thanks to Mary Jo Foley for the heads-up in her Aleksey Savateyev's Blog- Web, S+S, etc. - Announcing Azurelight post of 4/13/2009.

Note: As of 4/15/2009, 12:00 noon Aleksy’s blog has reappeared after being 404 all morning.

Azure Infrastructure

Mike Amundsen announced in RESTful Web Services Cookbook of 4/19/2009 that he and Subbu Allamaraju are writing a book titled RESTful Web Services Cookbook, which will be published by O’Reilly by the end of 2009. Mike describes the book:

Each recipe in this book will tackle one or more related design problems, and then discuss solutions and trade-offs. We think that learning the trade-offs is sometimes more important than just learning the right way, because an important part of software development is about making judicious trade-offs. In this book, we will therefore try to emphasize pragmatism over principles.

Amy Wohl chimes in with her McKinsey Got It Wrong: Cloud Computing is for Enterprises summary post of 4/18/2009. Amy’s full post is here.

James Hamilton’s SSD versus Enterprise SATA and SAS disks post of 4/18/2009 provides a detailed analysis of the economics of solid-state disks (SSDs) vs. conventional mechanical hard disk drives (HDDs). James concludes:

If you are storing data where you need 1.109 MB/s of 8k I/Os per GB of capacity or better, then the Intel device will be I/O bound and you won’t be able to use all the capacity. If the workload requires less than this number, then it is capacity bound and you won’t be able to use all the IOPS on the device. For very low access rate data, HDDs are a win. For very high access rate data, SSDs will be a better price performer.

Mike Walker reports that a “Mapping Applications to the Cloud” article will appear in Architecture Journal 19: TechEd 2009 Special Edition, which you can download here.

• John Foley says Cloud Computing Gets A Much-Needed Reality Check by McKinsey & Co. in this 4/17/2009 InformationWeek post. Michael Hickens does the same in his Anti-Cloud Hype Is Hype post of 4/16/2009.

• Jim Nakashima details the contents of *.cspkg files in his Digging in to the Windows Azure Service Package post of 4/16/2009.

• Leena Rao concludes that McKinsey’s Cloud Computing Report Is Partly Cloudy on 4/16/2009 in this TechCrunch post.

Steve Lohr chimes in with another McKinsey review in the NY TimesWhen Cloud Computing Doesn’t Make Sense article of 4/15/2009.

Instead of chasing cloudy visions, McKinsey suggests, corporate technology managers should focus mainly on adopting one building-block technology of the cloud model, virtualization. Such virtualization allows server computers to juggle more software tasks, and thus increase utilization, reducing capital and energy costs.

Steve Nagy “dive[s] a little deeper into the lifetime of a server in the Azure Fabric and the process of deploying resources on demand” in his Azure Server Virtualization Provisioning and Multitenancy post of 4/18/2009.

Steve Nagy’s Multitenancy And The Cloud post of 4/17/2009 describes why the definition of the term “multitenancy” has changed over time.

Nicholas Carr critiques the McKinsey & Co. report (see below) in his The big company and the cloud post of 4/15/2009:

[I]t has to be said, the numbers McKinsey presents seem a bit skewed, probably understating some of the savings or other benefits of moving to a cloud. For instance, again drawing on a "disguised client example," McKinsey suggests that replacing an in-house data center with cloud services would reduce IT labor costs by only about 10 to 15 percent.

I look forward to further analyses of these numbers. I'd be particularly interested in hearing Amazon's perspective on McKinsey's comparisons.

Will Forrest defines cloud computing as “hardware-based services
offering compute, network and storage capacity where:

    1. Hardware management is highly abstracted from the
      buyer
    2. Buyers incur infrastructure costs as variable OPEX
    3. Infrastructure capacity is highly elastic (up or down)”

in a detailed, 33-page McKinsey & Co. final report, Clearing the Air on Cloud Computing, dated 4/14/2009. Don’t miss it!

Steve Marx’s Does Windows Azure Support Java? post of 4/15/2009 answers the question with:

Windows Azure doesn’t support Java today. … But, you can run whatever you want.

Sun Microsystems reports about Cloud Security Presentation at the 10th Annual CERIAS Symposium on 4/15/2009:

A panel of four experts addressed the issue of “Security in the Cloud” at the recent symposium hosted by The Center for Education and Research in Information Assurance and Security (CERIAS). The speakers included Lorenzo D. Martino, College of Technology, Purdue University; Keith Watson, CERIAS, Purdue University; Dennis R. Moreau, Configuresoft Inc.; and Christoph Schuba, Sun Microsystems.

Cost Savings Will Drive Users to the Cloud; Providers Will Take Care of Security

Bill McColl’s What's Really Industry-Changing About Cloud Computing? post of 4/15/2009 describes “Four exciting new directions in massively parallel cloud computing:”

    • Cloudbursting.
    • Libraries and App Stores.
    • Live Data.
    • Domain Specific Development Tools.

Kym Jones discusses Legal Issues Surrounding Cloud Computing in this 4/15/2009 Cloud Computing Journal post. Jones writes:

Deploying cloud resources requires a different legal analysis than using, or selling, traditional Internet services. This session [at Cloud Computing Expo Europe] will focus on the interconnected nature, but nationless state, of cloud computing. Delegates will come away with: Five legal theories that will minimize risk regardless of the nation in which you live; a legal toolkit to address thorny contract issues in the United States and European Union; and a comparison of the risks of doing business with several major grid providers.

Dave Linthicum “talks about the use of architectural approaches for cloud computing, and issues with moving towards the clouds” in this Moving Towards Cloud Computing podcast of 4/15/2009.

Cath Jennings’ (no relation) Cloud computing: The answer to supply chain woes post of 4/15/2009 claims:

The adoption of cloud computing services in a supply chain context will mirror the former uptake patterns of on-premise enterprise resource planning (ERP) software. This means that such offerings are unlikely to be used to support non-commodity, core business processes for at least another three years.

Gordon Haff’s Thoughts on the cloud from Razorfish CNet News post of 4/15/2009 describes Microsoft’s Razorfish organization’s:

[W]ork with Rackspace's cloud hosting division, Mosso, to "build consumer-facing websites and web applications that can handle large traffic spikes during promotions and product launches of brands." However, we mostly discussed cloud computing more broadly. Here were some of the takeaways of which that I took particular note.

Razorfish is operating its internal systems more and more in the style of public cloud providers. Almost nothing runs on a dedicated physical server. They're also consolidating down from 25 datacenters to three. Klauder told me that big enablers here were cheap bandwidth and wide area network (WAN) acceleration--for which Razorfish uses Riverbed products. Razorfish also uses co-location facilities to optimize bandwidth use.

I wonder what the Azure team has to say about this turn of events.

Andrew Nusca asks Is adopting the cloud a money-losing mistake? in this 4/15/2009 post to ZDNet’s Between the Lines blog and reports:

But new research from McKinsey & Co. says that trying to adopt the cloud model would be a money-losing mistake for most large corporations. The research is being presented at a symposium this afternoon sponsored by the Uptime Institute, an organization that focuses on improving the efficiency of data centers.

The McKinsey study, “Clearing the Air on Cloud Computing,” concludes that outsourcing a typical corporate data center to the cloud would more than double the cost. The study uses Amazon’s well-known Web Services as the model for the price of outsourced cloud computing. According to McKinsey, the total cost of the data center functions would be $366 a month per unit of computing output, compared with $150 a month for the conventional data center. …

On the other hand, the cloud can be beneficial for small and medium-sized companies, typically with revenues of $500 million or less.

Dustin Amrhein’s Enhancing Development and Test with Cloud Computing post of 4/15/2009 describes “Five ways cloud computing strengthens IT development and test efforts.”

Maureen O’Gara reports EMC Creates Whopping Big Cloud Storage on 4/14/2009: EMC also has the first product derived from this new architecture, the world's biggest high-end storage array ever:

Dubbed the Virtual Matrix Architecture, it promises storage that scale to hundreds of thousand of terabytes and tens of million of IOPS supporting hundreds of thousands of virtual machines in a single federated storage infrastructure.

Dmitry Sotnikov’s Will Cloud make SOA mainstream? post of 4/14/2009 comments on Gartner’s report on SOA and Cloud Computing: “Cloud Computing Will Cement the Mainstream Role of SOA”.

In my opinion, on the one hand this is all common sense, and web interfaces are the obvious API approach for cloud/SaaS applications, and thus indeed in that way make SOA finally become widely spread. …

Overall, this is a very short report (less than one page of actual text) with a $195 price tag, so you might want to spend the money elsewhere. However, obviously do buy it if you need an analyst-approved document to prove to your boss that SOA is important.

Kyle Gabhart asserts Owning Hardware is Soooooo 2008 in this 4/14/2009 review of Fortune Magazine’s recent Tech Daily post: "Goodbye hardware. Hello, services". Kyle quotes Fortune:

"As Salesforce CEO Marc Benioff said back in November: In this economy, ‘customers are not going to bring out their checkbooks for the cost and risk and complexity of big database purchases, or application server purchases, or data center purchases.' They're buying more services, and fewer servers."

Krishnan Subramanian starts a new series with Scientists And Cloud Computing – Part 1 of 4/14/2009:

Again and again in this space, I have pushed the idea of scientific community tapping into Cloud Computing for their academic research. I have argued that it will save tons of money and time for the scientists. Of late, we are hearing many success stories about scientists and other academics tapping into the Clouds for their research. In this two part series, I want to highlight how scientists are taking advantage of the Clouds. In this first article, I want to, once again, emphasize the importance of Cloud Computing for scientists and in the second part, I will list out some of the successful use cases.

Reuven Cohen makes The Case for a Cloud Computing Trade Association in this lengthy 4/13/2009 post:

I believe the association should focus on the commonalities we share -- accelerating the adoption of cloud computing through a consensus view of the general opportunities cloud based technology brings to customers. I'm not speaking about defining what cloud computing is so much as defining the problems it solves and opportunities it enables. The things we can actually agree on.

Ruv also reports that as of 4/15/2009 his Open Cloud Manifesto Reaches 150 Supporting Companies.

Mary Jo Foley asks What is Microsoft doing to add Java support to Azure? in this 4/13/2009 post as the result of “Google’s recent announcement that is allowing developers writing for the Google App Engine cloud platform to develop in Java:

The answer? Not much that company officials haven’t said before. 

Cloud Computing Events

Markus Klems reports on 4/20/2009 from CloudSlam about Francis Carden’s OpenSpan - Workflow automization on the presentation layer presentation.

Mike Ormond’s UK Azure NET Usergroup post of 4/17/2009 is a brief postmortem of the first meeting of a new Windows Azure community in the UK.

Rob Bagby announced on 4/14/2009 MSDN Events Presents: The Best Of MIX, which will deliver “3 great sessions to you, including ‘What’s New in Silverlight 3?’, ‘Building Web Applications with Windows Azure’ and ‘MVC 1.0 vs ASP.Net Webforms’” (emphasis added). Where and When (registration links):

Joseph F. Fovar reports VMware Outlines Cloud Vision, Previews vSphere, At Partner Conference on 4/14/2009:

VMware opened its annual partner summit Tuesday by introducing how partners can take advantage of new VMware technology to help start moving customers to the compute cloud. [Link added.]

The company also unveiled a new partner initiative aimed at making it easier for solution providers to work with its technology.

David Linthicum chimes in a bit late with his 'Open Cloud Manifesto?' Just Stop! post of 4/13/2009: Dave writes (in part):

Manifestos are nothing new; I've been dealing with them since I started in IT. The core notion is that my ideas count and yours don't, and what I say is the way it should be. At least, that's the impression I get. I recall manifestos issued around relational database technology in the late '80s and around other topical IT trends. The trouble is, manifestos have the opposite of the desired effect, serving to polarize rather than bring together. This manifesto was no different.

Aaron Skonnard’s Speaking on Windows Azure at VSLive! Las Vegas in June post of 4/13/2009 announces that he’ll be presenting:

  • Windows Azure: A New Era of Cloud Computing, 8:30 a.m.–9:45 a.m., Wednesday, June 10
  • Codename “Dublin”: Windows Application Server, 10:00 a.m.–11:15 a.m., Wednesday, June 10
  • Workshop: A Day of Windows Azure, 9:00 a.m.–6:00 p.m.,Thursday, June 11

at VSLive! Las Vegas in the Venetian.

SDForum describes its Shaping the New Age of Application Development Developers Conference as follows:

It’s no longer about a sweet web interface.  Instead it’s about applications with collaboration and analytics at the core and built from the ground up to easily accommodate elastic demand, community driven functionality and pay-per-use models. Along with the technology, profitable business models and the funding environment has also changed. 

The rules have changed. It’s a new age. SDForum’s “Shaping the New Age of Application Development” 2-day conference for developers can help you make sense of what’s changing, plot a winning course and keep you on the cutting edge. 

When: 8:30AM April 17 - 3:00PM April 18, 2009

Where: The Tech Mart, 5201 Great America Parkway, Santa Clara, CA  95054

Other Cloud Computing Platforms and Services

Dennis Howlett’s RightNow's DoD play: a good antidote to the cloud security naysayers post of 4/15/2009 details:

RightNow’s new hosting capabilities use DITSCAP/DIACAP to ensure compliance with DoD Instruction 8500.2, meet US Federal security standard FISMA (NIST 800-53) and include a 24×7 dedicated security and information assurance team. For civilian agencies with requirements similar to the DoD, RightNow now offers a second, highly secure hosting environment to meet their needs.

Alin Arimie reports on “automated, high-definition video encoding in the cloud for large-scale media companies” by HDCloud and Amazon EC2 in his HD Cloud - The FedEx Of Web Video. How Do They Do It? post of 4/15/2009:

This is another success story in the “cloud”. By leveraging public and proprietary cloud technologies, HDCloud is able to scale horizontally without limits. Remember, video processing requires lots of processing power and results must be delivered in timely manner.

More information is available from the HD Cloud Launches Next-Generation Video Transcoding Service post of 4/15/2009.

Reuven Cohen reposted on 4/15/2009 his The Unified Data Center: Unified Computing Perspectives article for the Cisco Data Center Networks Blog of 3/19/2009. Ruv claims:

For some the biggest buzz word so far in 2009 is “cloud”, for Cisco it’s “unified”. Today Cisco announced a new server centric strategy which is underpinned by the use of a “unified computing” methodology. This new unified approach to computing represents a radical shift in how we as an industry both visualize and manage a modern virtualized data center.

For Cisco Unified Computing seems to be an overarching mantra being applied to the broader management of data center resources (compute, storage, and network elements) through a singular virtualized point of interaction. In a sense they are attempting the unification of the the entire infrastructure stack in what some are calling a unified infrastructure fabric. …

What’s been exciting for us at Enomaly is envisioning the potential for a singular infrastructure abstraction that can encompass the entire infrastructure stack as well as emerging cloud centric technologies through a unified application interface (API). At Enomaly we have been saying this for awhile and fully believe that this model represents the future of computing.

James Hamilton goes Under the Covers of Google App Engine Datastore in this 4/14/2009 post that includes his “notes from an older talk done by Ryan Barrett on the Google App Engine Data store at Google IO last year (5/28/2008). Ryan is a co-founder of the App Engine team.”

John Foley reports in his Microsoft To Amazon: We'll Fix Windows Licensing post of 4/13/2009:

In a surprising admission, Microsoft president Bob Muglia says Microsoft's licensing arrangements with Amazon Web Services and other cloud service providers are both too complicated and too expensive. "We'll fix that," Muglia promises.

The subject came up in a recent 90-minute interview in InformationWeek's New York offices. I asked Muglia, president of Microsoft's Server and Tools division, about what I see as a growing rivalry between Microsoft and Amazon in the cloud computing market. "There are people at Microsoft who think of Amazon as a competitor," he admitted. "I don't. I think of them as a customer."

James Urquhart’s Maybe "cloud-computing" hasn't lost its VC luster post of 4/13/2009 for CNet News reports:

In reviewing the companies I will be judging [for Under the Radar] this year, it feels like the term "cloud" covers way too much ground to be useful in a venture pitch. In fact, a few weeks back I wrote a post that built on a conversation I had with venture capitalist Lars Leckie of Hummer-Winblad Venture Partners, in which I asked the question, "Has 'cloud computing' lost its VC luster?".

It's possible I'm being a little too harsh on the term, however. Soon after writing that post, I exchanged emails with good friend and sometimes mentor, Gamiel Gran, Vice President of Business Development at Sierra Ventures. I asked Gamiel what he thought of the term "cloud computing" as it is applied to start-up pitches.

His response frankly surprised me. Far from being a confirmation that the use of term has gotten out of control, Gamiel embraces "cloud" for all its worth (and its worth multiple trillions of dollars in Sierra's estimation). He is excited about the opportunities that cloud computing presents for new businesses, and wants to see more of it--lots more of it.

James goes on to quote the entire text of Gamiel’s response.

Joe Weinman claims in his 6 Half-Truths About the Cloud guest post of 4/11/2009 on GigaOM:

The following “six commonly held views that, while not wrong, are just not entirely accurate:”

  1. Economies of scale are the key to cloud benefits
  2. All IT will move to the cloud
  3. Clouds generate value by replacing capital expenditures with operating expenditures
  4. Private clouds are as effective as public clouds
  5. Cloud = virtualization
  6. Clouds are greener

Joe Weinman is Strategy and Business Development V-P for AT&T Business Solutions.

Peter Kim’s AT&T has Cloud Computing Advantages: What They Are is the Issue post of 4/13/2009 takes issue with Joe Weinman’s article for GigaOM. Kim wrote:

Weinman suggests that telcos will have an edge in "statistics of scale," essentially the ability to smooth out demand peaks created by customers in different verticals and time zones. …

Actually, a telco's potential advantages in cloud computing are interesting in an economist's sense, as the smoothing of demand over time and between verticals might be considered derivative from "scale" or a derivative from "scope economics.”

Wednesday, April 15, 2009

LINQ and Entity Framework Posts for 4/13/2009+

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Entity Framework and Entity Data Model (EF/EDM)

Özgür Aytekin’s Practical Entity Framework for C#: Compiled Queries in Entity Framework of 4/14/2009 includes this suggestion:

With any new technology, performance implications are important to consider. If you want to get the best performance from Entity Framework queries, you should use the Compile function of the CompiledQuery class. In this video, Todd Miranda demonstrates how to use compiled queries in the Entity Framework: http://msdn.microsoft.com/en-us/vcsharp/dd565858.aspx

Alex JamesTip 12 - Choosing an Inheritance Strategy post of 4/14/2009 describes EF’s support for Table Per Hierarchy (TPH), Table Per Type (TPT), and Table Per Concrete Class (TPC) inheritance models. Alex notes that the EDM Designer doesn’t support TPC, but he supplies suggestions for choosing between TPH and TPT.

Mariuz reports in his Entity Framework and Firebird Embedded post of 4/13/2009 that Firebird Embedded works with Entity framework.

Sankarsan’s four-part A Layered ASP.NET MVC Application uses EF as the data access layer:

  1. A Layered ASP.NET MVC Application - Part I
  2. A Layered ASP.NET MVC Application - Part II
  3. A Layered ASP.NET MVC Application - Part III
  4. A Layered ASP.NET MVC Application - Part IV

LINQ to SQL

Damien Guard’s LINQ to SQL tips and tricks #2 of 4/13/2009 provides a “few more useful and lesser-known techniques for using LINQ to SQL:”

    • Take full control of  the TSQL
    • Complex stored procedures
    • Cloning an entity

Denny Tuppeny rehashes the Eager-Fetching of Relationships with LINQ to SQL topic in this 4/13/2009 post.

LINQ to Objects, LINQ to XML, et al.

Stephan Cruysberghs writes in his New connection providers in LINQPad post of 4/14/2009:

A few weeks ago I started testing the beta version of LINQPad 1.35. Some days ago it has been officially released. This great tool from Joe Albahari was introduced in 2007 and nowadays it has become a very mature tool which should be in the toolbox of every .NET developer.

Version 1.35 offers new connection providers and it has native support for LINQ to SQL and the Entity Framework models. It also provides new query types like SQL and Entity-SQL.

I am a fan of this tool and I already demonstrated its features in many of my articles. I was probably one of the first developers who used LINQPad to execute LINQ to Entities queries. In the past you needed to add some references and create the ObjectContext in each script. It was a bit of work but it functioned fine. Now you can do the same thing with the new connection providers but they also offer many advantages. In a small article which can found on my website I will highlight some of the new features.

ADO.NET Data Services (Astoria)

No significant new posts as of 4/15/2009 12:00 noon

ASP.NET Dynamic Data (DD)

No significant new posts as of 4/15/2009 12:00 noon 

SQL Data Services (SDS) and Cloud Computing

This topic moved on 1/3/2009 to Windows Azure and Cloud Computing Posts for 1/5/2009+.

Miscellaneous (WPF, WCF, MVC, Silverlight, etc.)

Guy Burstein continues his series about .NET RIA Services with:

LINQ and Entity Framework Posts for 4/6/2009+

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Entity Framework and Entity Data Model (EF/EDM)

Matthieu Mezil reports multiple EDM Designer: one more new feature[s]:

  • On 4/12/2009: [W]hen an entity type isn't mapped or is only partially mapped (so we aren't able to use our edmx), I reduce its opacity.
  • On 4/11/2009: [T]he ability to see with context menu in which views the selected type (entity type or complex type) is present and the ability to navigate to this view.
  • On 4/7/2009: Published EDM Designer: new version

Gil Fink reminds EF developers on 4/10/2009 that the ADO.NET Entity Framework Extensions Library includes:

[U]tilities that make querying stored procedures,
creating typed results from DB data readers and state tracking external data much easier in the Entity Framework. A sample application demonstrates several patterns using these
utilities, including stored procedures with multiple result sets, materialization of CLR types, and registering entities in the Entity Framework state manager” (the description is taken from the library’s page). The following utilities are included in the current release of the
ADO.NET Entity Framework Extensions library:

  • Execution of store commands via the ObjectContext.
  • Connection lifetime management.
  • State management of entities from external sources.
  • Materialization of arbitrary CLR types given a data reader or DB command.
  • Stored procedure mapping:
    • Multiple result sets.
    • Column renames, polymorphic results and nested structures via the
      materialization service.
  • Getting and setting key values for entity references.
  • Rewrite InvocationExpressions in LINQ queries and expressions.

You can download the library from here.

Danny Simmons attacks Building N-Tier applications with the EF – The Basics on 4/8/2009 with more about new features in EF v4 as well as suggestions for n-tier apps in EF v3.5.

Danny SimmonsD3: Using T4 to Generate Entity and Context Classes post of 4/8/2009 explains how to use T4 templates “to create the DPMud model using the EF Designer and then generate both the classes and the database from the model (database generation has been discussed previously on the EF Design blog).”

Goodbye Linq to SQL POCO, Hello nHibernate

Simon Segal reports on 4/7/2009 that his Entity Framework Profiler hosts IronRuby and IronPython.

LINQ to SQL

Jason Young’s Unit Testing a LINQ to SQL or EF Query post of 4/9/2009 begins:

I was writing a slightly non-trivial method to query a database to find a record matching a certain time range. It quickly became clear that it would be nice to write some automated unit tests against it. Integration tests would be less than ideal because of the execution time and complexity. I ended up with a way to test the code without jumping through too many hoops.

Steve from TeamDataLogic’s Goodbye Linq to SQL POCO, Hello nHibernate post of 4/8/2009 announces:

After a long time trying to get model first poco working with Linq to SQL (using xml mapping files) we have finally given up and have moved over to nHibernate.

Matt Warren surfaces on 4/8/2009 with Building a LINQ IQueryable provider - Part XIV (IQToolkit v3), “the fourteenth in a series of posts on how to build a LINQ IQueryable provider.” Matt suggests:

If you have not read the previous posts you might request a week’s vacation, sit back, relax with a mochacino in one hand a netbook in the other, or if you've got better things to do with your time print them all out and stuff them under your pillow.

.NET Junkie’s Integrating Enterprise Library Validation Application Block With LINQ to SQL and Entity Framework Part 1: Basic Integration post of 4/7/2009 “describes how to integrate the Enterprise Library Validation Application Block in conjunction with an O/RM technology such as LINQ to SQL and Entity Framework.”

David DeWinter’s LINQ to SQL: Updating Entities post of 4/7/2009 begins:

Updating entities using any object-relational mapper can be difficult for at least two reasons: (1) the number of update options that these ORM frameworks provide and (2) the complexity of requirements in today’s business scenarios. With this post I want to discuss a few of the ways that you can update entities in LINQ to SQL to fit your particular scenario.

LINQ to Objects, LINQ to XML, et al.

Suprotim Agarwal explains how to Return the First Element of a Sequence in LINQ in this brief 4/10/2009 article.

LinqMaster shows you How to Use LINQ GroupBy in this 4/8/2009 post.

ADO.NET Data Services (Astoria)

My Uploading Entities for Storage in Azure Tables of 4/9/2009 discusses “the ease, speed, accuracy and reliability of uploading typical entities to persistent storage, as well as the cost of data ingress/egress and long-term storage.”

ASP.NET Dynamic Data (DD)

Steve Naughton’s Hiding Foreign Key column Globally in Dynamic Data post of 4/11/2009 “is based on a question in the Dynamic Data forum Hide Foreign Key Column:”

And so I thought I’d document what I did for posterity or at least so I can find it again if ever the question arises again. So I decided  here is what I would need:

  1. An Attribute to mark FK relation ships as hidden.
  2. Some Extension methods to extract and test the attribute
  3. An IAutoFieldGenerator to filter the Columns on a page

SQL Data Services (SDS) and Cloud Computing

This topic moved on 1/3/2009 to Windows Azure and Cloud Computing Posts for 1/5/2009+.

SQL Server Compact (SSCE) 3.5 and Sync Services

Shawn Kelley announced on 4/7/2009 a Custom Conflict Resolution Survey for the SyncFramework and requests answers to three questions as comments to the post.

Miscellaneous (WPF, WCF, MVC, Silverlight, etc.)

Guy Burstein brings developers up to date on .NET RIA Services with:

Charlie Calvert’s Creating a New Silverlight Project in Visual Studio 2008 post shows you how to “start a new Silverlight project or how to add a new or existing Silverlight project to an existing ASP.NET Web Application.”

Sunday, April 12, 2009

Windows Azure and Cloud Computing Posts for 4/6/2009+

Windows Azure, Azure Data Services, SQL Data Services and related cloud computing topics now appear in this weekly series.

Note: This post is updated daily or more frequently, depending on the availability of new articles.

  Update 4/10 to 4/12/2009: Rob Bagby starts an Azure shopping cart, Pat Helland’s new cloud presentation, other additions
• Update 4/8 and 4/9/2009: Big news from Google for GAE, other additions
§ indicates item repeated from Windows Azure and Cloud Computing Posts for 3/30/2009+

Azure Blob, Table and Queue Services

I asked Will Azure Attempt to Store Multiple Tables with the Same PartitionKey Value on the Same Node? in the Windows Azure forum on Easter Sunday afternoon. Specificially, I asked:

My question is will Azure attempt to store all entities with the same PartitionKey value in multiple tables on the same node?

Microsoft’s Brad Calder replied about 4-1/2 hours later with a detailed answer about controlling locality, which I recommend to anyone wondering about similar topics. The answer, by the way, was no. The entities must have the same PartitionKey and be in the same table to be localized on a single node.

Werner VogelsGood Advice on Keeping Your Database Simple and Fast post of 3/25/2009 posits:

The AWS services Amazon S3 and Amazon SimpleDB were designed to handle the dominant storage usage patterns within Amazon and they greatly reduced our need to rely on relational storage for scaling our systems. But it is almost never the case that a single storage technique is used in applications and services that need to operate at enterprise scale. For example it is a common pattern that objects stored in S3 using a primary key, have a collection of secondary keys (e.g. metadata) stored in SimpleDB. SimpleDB provides very fast indexing for querying of the metadata that will return primary keys of objects located in S3.

Azure and SimpleDB tables are quite similar and Azure Blob Services resemble S3. Amazon recently upgraded Simple Queue Services (SQS) to match Azure Queue Services features.

Erik Carlin, Rackspace Cloud’s Chief Architect explains Cloud Servers and EC2: Why Persistence Matters in this 4/7/2009 post. Erik contrasts ephemeral (volatile) EC2 instance memory with (non-volatile) Elastic Block Store (EBS) and S3 memory. The analogy also applies to ephemeral Azure instances and memory and non-volatile Azure Data Services: tables, blobs, and queues.

Rob Bagby presents Screencast Published - Understanding The Azure Table Storage API, a 00:26:14 Channel9 screencast to accompany his earlier Azure Table Storage, the REST and ADO.NET Data Services Story post of 4/3/2009:

In this screencast, I illustrate consuming Azure Table Storage from a variety of means.  I show you the easy way, using the StorageClient sample application along with the ADO.NET Data Services Client Libraries, as well as making calls directly to the REST API using Fiddler as the client.

Said Syed’s Future of Block Storage in the Cloud post of 4/7/2009 asks “Are Storage Clouds or Compute Cloud for everyone?” Said’s answer:

Probably not. For transactional type databases or I/O, cloud is a very bad idea. For those who want to run 7/24/265 operations, it could actually be more expensive to run the operation on the cloud but that is relative as some offer as low as 10 cents per hour (or was it a month???) for compute with a nice size disk drive (not cloud storage mind you). Again, still not high performance, high capacity, high I/O capable, especially over the cloud (S3 that is).

SQL Data Services (SDS)

The SQL Server Team announced SQL Server 2008 Enterprise Edition Receives Common Criteria Certification on 4/10/2009. SDS v2 will be based on SQL Server 2008 EE, so I wonder if the the certification will apply to SDS. (I’ve asked.)

Emil Sayegh’s Today from Mosso: Red Hat Enterprise Available on Cloud Servers and MS SQL 2008 on Cloud Sites post of 4/7/2009 announces that SQL Server 2008 is now available on Rackspace Cloud Sites (but not Cloud Servers). Emil claims the benefits are:

• Lower storage costs through SQL Server 2008’s data compression features
• Increased security and data protection features through encryption
• Improved scalability and performance through resource governing

Emil promises:

Following closely behind are APIs from Cloud Servers. Watch for several partners who will soon actively support The Rackspace Cloud including Aptana, Encoding, rPath, RightScale, SOASTA, and Sonian. RightScale announced it again yesterday at Cloud Computing Expo.

.NET Services: Access Control, Service Bus and Workflow

Bruno Tekaly starts a series about .NET Services with his Azure - Microsoft .NET Services- Step 01 - The Service Bus - Blog Post about Setup post of 4/10/2009. Bruno says:

This blog post is all about getting ready to learn about .NET Services, and the Service Bus specifically. This blog is based on the Azure Services Toolit and adds additional background information.

Bruno’s a Microsoft developer-evangelist.

Clemens Vasters’ .NET Services March 2009 CTP - Service Bus Routers and Queues - Part 5: The Queue API for the rest of us and .NET Services March 2009 CTP - Service Bus Routers and Queues - Part 4: The REST Queue Protocol in Code Snippets posts of 4/6/2009 adds two new chapters to his .NET Service bus saga.

§ Clemens Vasters continues on his .NET Service Bus roll with his .NET Services March 2009 CTP: Host a Public Website At The Kitchen Table or from a Coffee Shop! No Kidding post of 4/5/2009:

Using the application/service built from the sample linked at the top of this post you can host a publicly discoverable and accessible website or Web service from your Windows notebook or desktop machine from within most network environments without having to open up a port on the firewall, mapping a port on your NAT, or using some type of dynamic DNS service to make the site discoverable. All those essential connectivity features are provided by the .NET Service Bus and with the help of the included sample code. [Emphasis Clemens’.]

Live Windows Azure Apps, Tools and Test Harnesses

•• Gaurav Mantri announces Cerebrata Software’s Cloud Storage Studio for Azure Storage - A browser based client for Azure Storage in this thread added to the Windows Azure forum on 4/9/2009.

•• Rob Bagby starts “building an Azure shopping cart application from the ground up” with his lengthy and fully illustrated Azure Application Part 1: Setup and running “Hello World” post of 4/9/2009. His Screencast Published - Building an Azure App Part I: Setup and Hello World post of 4/10/2009 complements the earlier step-by-step tutorial.

If you new to Azure, this is the place to start.

•• The Azure Team published the following WCF Azure Samples to the MSDN Code Gallery on 3/13/2009:

    1. Service using BasicHttpBinding and .Net client
    2. Service using binary HTTP binding and Silverlight client
    3. Service using WebHttpBinding binding and ASP.NET AJAX client
    4. Service using binary HTTP binding with transport security and message credentials and Silverlight client
    5. Chat service which can push data to Silverlight clients
    6. Service using the AtomPub protocol and a .Net client

and subsequently uploaded live demos of some of the samples to http://wcfazure.cloudapp.net/.

My Uploading Entities for Storage in Azure Tables of 4/9/2009 describes a Windows client test harness for uploading entities from Northwind Orders and Order Details tables, describes the HTTP POST and DELETE request and response messages, and provides some basic timing information on these operations. Additional information on the status of cross-table transactions for Azure tables is included.

Ryan Dunn announces availability of the latest Azure Training Kit and Tools Update in his 4/9/2009 post. The kit includes:

    • 11 hands-on labs - including new hands-on labs for PHP and Native Code on Windows Azure.
    • 18 demo scripts - These demo scripts are designed to provide detailed walkthroughs of key features so that someone can easily give a demo of a service
    • 9 presentations - the presentations used for our 3 day training workshops including speaker notes.

and can be downloaded from here. According to Ryan:

The Azure Services Management Tools include an MMC SnapIn and Windows PowerShell cmdlets that enable a user to configure and manage several Azure Services including .NET Access Control Services, and the .NET Workflow Service. These tools can be helpful when developing and testing applications that use Azure Services. For instance, using these tools you can view and change .NET Access Control Rules, and deploy and view workflows.

You can download the latest management tools from http://code.msdn.microsoft.com/AzureManagementTools.

Mike Ormond’s Windows Azure – A Couple of Learnings of 4/9/2009 describes problems running Azure’s Development Environment on Windows 7. 

§ My Azure Table Test Harness with HTTPS, Encryption, and Membership Services for Authentication/Authorization post of 4/6/2009 shows you how to test drive my live AzureTableTestHarnessSSL.sln project that takes advantage of ASP.NET Membership Services samples from the Windows Azure SDK March 2009 CTP.

§ Clemens Vasters continues on his .NET Service Bus roll with the .NET Services March 2009 CTP: Host a Public Website At The Kitchen Table or from a Coffee Shop! No Kidding post of 4/5/2009 (repeated from the “.NET Services: Access Control, Service Bus and Workflow” section. 

Azure Infrastructure

•• Joe Weinman’s 6 Half-Truths About the Cloud of 4/11/2009 is a guest-post to the GigaOM blog. Here are Joe’s points without the detail in his post:

  1. Economies of scale are the key to cloud benefits
  2. All IT will move to the cloud
  3. Clouds generate value by replacing capital expenditures with operating expenditures
  4. Private clouds are as effective as public clouds
  5. Cloud = virtualization
  6. Clouds are greener

It’s good to see a Strategy and Business Development V-P for AT&T Business Solutions debunk some cloud myths.

•• Pat Helland calls his new Above the Clouds: a Berkeley View of Cloud Computing PowerPoint presentation “kinda’ like a book report” on the UC Berkeley RAD Labs’s Cloud Computing white paper. The 53 slides are a superb graphical rendition of the paper’s primary points from the standpoint of a champion of distributed computing who’s “been there and done that” at Tandem Computers, HaL Computers, Microsoft, and Amazon Web Services. Don’t miss it!

Dare Obasanjo discusses Todd Hoff’s Are Cloud Based Memory Architectures the Next Big Thing? post in a Some thoughts on memory based architectures (aka why memcached isn't good enough) post of 4/11/2009. Dare writes:

The LinkedIn architecture is a great example of this trend. They have servers which they call The Cloud whose job is to cache the site's entire social graph in memory and then have created multiple instances of this cached social graph. Going to disk to satisfy social graph related queries which can require touching data for hundreds to thousands of users is simply never an option. This is different from how you would traditionally treat a caching layer such as ASP.NET caching or typical usage of memcached.

I’m still waiting for the “scalable database” post, Dare.

Art Whittman’s Practical Analysis: Are We Sure This Isn't Clouded Judgment? InformationWeek article of 4/11/2009 carries this deck: “As enticing as cloud computing is, it doesn't change the rules of the game, which call for careful and thoughtful data management.” This week’s cover story is “Private Clouds.”

Dave Malcolm Surgient’s The five defining characteristics of cloud computing article of 4/9/2009 for ZD Net begins: “Interest in cloud computing is rampant across the entire IT industry and everyone has a different perspective and understanding of the technology” and continues with:

    • Characteristic 1: Dynamic computing infrastructure
    • Characteristic 2: IT service-centric approach
    • Characteristic 3: Self-service based usage model
    • Characteristic 4: Minimally or self-managed platform
    • Characteristic 5: Consumption-based billing

Surgient provides a detailed description of each characteristic. If you’re looking for a starter piece for a management or developer presentation, try this.

Eric Knoor and Galen Gruman analyze What cloud computing really means in a 4/7/2009 article for InfoWorld. The deck: “The next big trend sounds nebulous, but it's not so fuzzy when you view the value proposition from the perspective of IT professionals” and the lede:

InfoWorld talked to dozens of vendors, analysts, and IT customers to tease out the various components of cloud computing. Based on those discussions, here's a rough breakdown of what cloud computing is all about:

  1. SaaS
  2. Utility computing
  3. Web services in the cloud
  4. Platform as a service
  5. MSP (managed service providers)
  6. Service commerce platforms

• Tom Bittman, a Gartner Analyst, discusses The Spectrum of Private to Public Cloud Services in this 4/8/2009 post. Bittman writes:

Is it a private cloud service, or a public cloud service? It’s not quite so binary. I first explored this in my post Virtual Cloud Privacy is Gray a few months ago. There are two relative dimensions that determine how “private” or how “public” a cloud service really is:

Service Control/Ownership: There are two ends of a spectrum here – complete implementation ownership, and complete lack of ownership and control of implementation. But there will be many examples in between of partial control, shared ownership, etc.

Service Access: Also two ends to this spectrum – at one end, usage is extremely exclusive, while at the other end, anyone who chooses can access the service. Again, there will be many examples in between of limited access, industry-only access, controlled partner access, etc.

These two dimensions are coupled at the extremes, but there are many variations in between. Each has different security/privacy, cost, customization and elasticity attributes. …

• Microsoft UK’s Arc Magazine issues 1 and 2 provide “insight into the Microsoft 'Software + Services' strategy from an architectural perspective. The 4 issues explore the business case, the implications, why identity matters, Software + Services and the Cloud, supported by case studies,” according to this recent Architecture post.

Nolan M. Goldberg’s and Sharada Devarasetty’s The Cloud: Tomorrow’s EDD Challenge post of 4/9/2009 discuss the legal aspects of electronic data discovery (EDD) and electronically stored information (ESI) in the cloud. (Goldberg is a senior associate in the patent group of New York-based Proskauer Rose LLP and a member of the Litigation Department's e-Discovery Task Force. Devarasetty is an associate in the firm’s patent group.)

• James Hamilton follows up on his posts from the recent Google DataCenter Efficient Summit with Data Center Efficiency Summit Videos Posted of 4/9/2009:

• Kevin Jackson analyzes the Network Centric Operations Industry Consortium (NCOIC)’s Cloud Computing Working Group and its focus on establishing a roadmap for providing value to the industry in his NCOIC and Cloud Computing: An Update post of 4/8/2009.

James Urquhart asks The new cloud infrastructure: Do you care? on 4/8/2009 but then says “At the very least, the future of hardware ought to touch the inner geek in all of us” and talks about Cisco’s Unified Computing Systems, Rackable Systems’ CloudRack C2, and Google servers’ self-contained backup battery.

Mark Wilson attended the inaugural meeting of UK Azure User Group at Microsoft’s London offices, which included a presentation from Microsoft’s James Conard about what Windows Azure really is. Mark’s So, what exactly is Windows Azure? post of 4/7/2009 is a detailed summary of Conard’s introductory session.

Colin McNamara’s Cisco’s Cloud Computing Offering post of 4/7/2009 concludes:

Cisco has to go to market with a Cloud offering to maintain long term viability as a company. When they do they will have the benefit of lower cost of building and operating the grids that their cloud offering will run on. They will be able to leverage millions of Cisco network devices in their current install base as well as provide application centric security integrated with these same devices. And most importantly they will be able to use the lessons learned from running WebEx to ensure flawless delivery of an upcoming cloud computing offering.

James Hamilton suggests using ambient air for datacenter cooling in his 32C (90F) in the Data Center post of 4/7/2009. James concludes:

I recently came across a wonderful study done by the Intel IT department (thanks to Data Center Knowledge): reducing data center cost with an Air Economizer.

In this study Don Atwood and John Miner of Intel IT take the a datacenter module and divide it up into two rooms of 8 racks each. One room is run as a control with re-circulated air the their standard temperatures. The other room is run on pure outside air with the temperature allowed to range between 65F and 90F. If the outside temp falls below 65, server heat is re-circulated to maintain 65F. If over 90F, then the air conditioning system is used to reduced to 90F. The servers ran silicone design simulations at an average utilization rate of 90% for 10 months.

I’m still not sure what’s a “silicone [sic?] design simulation.”

Robert L. Miller’s SaaS integration: Tricky, but manageable ComputerWorld article includes three interesting integration-related sidebars:

    • Tips for successful integration projects
    • Three ways integrations can get tangled up
    • Whose Burden?
    • Who's in Charge?

Miller concludes:

About 80% of integrations use basic technologies such as file transfers, and projects with SaaS applications tend to roll out faster than the 12-to-18-month window that's typical for traditional on-premises applications, says Annrai O'Toole, vice president of integration at Workday Inc., a provider of hosted applications in Pleasanton, Calif. Nonetheless, a typical integration project involving Workday systems, including the migration and cleaning of data, specification of business processes, and systems configuration, still takes around 70 days.

Thanks to David Linthicum for the heads-up on this article.

Tom Lounibos decrees Delivering Reliable Web Services Requires Web Scale Testing in this 4/7/2009 post. Of course, that statement isn’t surprising when you consider that Tom is President and CEO of SOASTA, which is the publisher of CloudTest.

Sam Ruby’s Open Cloud Principles post of 4/6/2009 analyzes Sam Johnston’s Open Cloud Principles (OCP) and the associated Open Cloud Initiative (OCI):

It is getting to be a crowded place in the clouds, what with the Open Cloud Manifesto, the Cloud Bill of Rights, and the Cloud Computing Interoperability Forum.  I’m sure I’ve missed countless others.

I like the OCP’s focus on open APIs, open formats, and open data.  Speaking as an open source person, I think the bullet on Open Source (optional) should simply be dropped.  Always bet on protocols and formats, as long as the focus is on ensuring that the the entire stack can be swapped out or ground up rewritten as the service owner sees fit, the rest will work out.

Sam (Ruby) then goes on to analyze the “concept of Interoperability” and finds it “significantly in need of expansion.”

I agree.

Mark Everett Hall posits Windows Azure: Microsoft Banks on Programmer Loyalty in this 4/7/2009 article for PC World magazine. Mark writes:

Still in beta, Azure features both proprietary tools that Windows developers will recognize and standard technologies that could appeal to programmers outside of Microsoft's orbit. Whether that strategy will work remains to be seen, since Microsoft trails Salesforce.com, Amazon.com, Google and others in entering the cloud, which Merrill-Lynch & Co. has estimated will be a $95 billion market by 2011.

Javier Soltero discusses the developer’s role in his The Cloud Dilemma for Developers post of 2/3/2009:

Developers like the cloud since it lets them bypass operations (i.e., the control agents) and serve their environment needs quickly. In the cloud, developers can build an application and launch it themselves, on their own, without waiting for hardware to be purchased, racked & stacked.  Without waiting for hardware or virtualization capacity planning, electricity consumption plans, cooling or green discussions to finish. If you’re a developer, the cloud offers speed! The cloud is way cool. You’re the man. But developers too often forget to ask themselves what happens afterwards?  Who will make sure the app is always available?  Apps in the cloud will still have performance issues and will still break.

Javier continues with a discussion of two dilemmas for cloud-computing developers.

§ Bernard Golden’s Power: One Cloud Cost Advantage That May Be Irresistible article of 4/6/2009 for CIO’s Virtualization and Cloud Observer blog analyzes Microsoft’s The Cost of a Cloud: Research Problems in Data Center Networks whitepaper and Steve Denegri’s Cloud Computing Dominance Through Renewable Energy paper:

From [Denegri’s] research, he concludes that Microsoft obtains a 22% discount on its property taxes (this is based on a discount extended to companies that make very large real property investments). With regard to power, Microsoft obtains a 45% discount, based on using enough electricity to fall into a "Super Large Power" user category. This advantage may be enhanced in the future, Denegri states, because of potential taxes to be applied to carbon-based power. Microsoft may be able to strike an arrangement to have its data center powered by clean energy sources (e.g., wind or solar), thereby avoiding these additional taxes.

§ Microsoft attempts to explain its ubiquitous Software as a Services strategy with an inspirational[?] intro video, a Ray Ozzie introduction, and pages for potential Xbox, Azure, Windows Live and Mobile, and Office Online services. (This site might have been around for a while; I just found it.)

§ Mary Jo Foley reports Microsoft links HealthVault service with Amalga software on 4/6/2009:

HealthVault is Microsoft’s consumer-focused health-records-management Software+Service platform, which the company unveiled officially in 2007. (The service component of HealthVault is one of a handful of Microsoft services that already is hosted on top of Azure.) Amalga UIS, (one of the products formerly under the Azyxxi brand), is one of the main elements of Microsoft’s enterprise health-information-system platform. [Emphasis and links added.]

§ Dimitry Sotknikov’s Can cloud make you MORE compliant? post of 4/6/2009 reviews a report by Scott Crawford from Enterprise Management Associates - “The Security Paradox of Cloud: Five Questions for Cloud Providers” and adds his comments on the paper.

§ David Pallman begins a new Azure series with Grid Computing on the Azure Cloud Computing Platform, Part 1 of 4/5/2009:

In this series of articles we're going to look at grid computing using the Azure cloud computing platform. In Part 1, we'll look at this from a design pattern and benefits perspective.

Cloud Computing Events

Guy Bunker will present The Darker Sides Of Cloud Computing: Security and Availability at the upcoming Cloud Computing Expo Europe (www.cloudexpo-europe.com) May 18-19, 2009, in Prague, Czech Republic. Here’s the abstract:

Cloud computing offers a fantastic opportunity to businesses of all sizes. However, there are pitfalls that no-one wants to talk about. This session will talk to some of the darker sides to cloud computing - those around security and availability. Understanding where the issues lie will help service providers to create better services and enable customers to ask the right questions of their providers.

• Mike Walker’s Microsoft Architect Insight Conference 2009 post of 4/9/2009 announces that the UK’s fourth annual Microsoft Architect Insight Conference conference will be held 5/8/2009 at Microsoft London.

• Reuven Cohen announces CloudCamp Austin, April 25th at Austin City Limits, 2504 Whitis Ave., Austin, TX 78705 in this 4/9/2009 post. The tentative schedule is Saturday, April 25th (10am - 4pm) to be followed by a Happy Hour at Little Woodrows.

SYS-CON Announces Government IT Conference & Expo, which is being held October 5-6, 2009 in Washington, DC, in this 4/9/2009 post. According to SYS-CON:

Cloud Computing will be one of the main tracks. According to the Washington Post this week, the U.S. Census Bureau is using Salesforce's cloud to manage the activities of about 100,000 partner organizations across the country. And the Defense Department's technology arm has already set up a cloud to let the military rent storage space or use remote software programs. Companies like online application provider NetSuite have shifted their focus to federal sales, on the basis that what works for Enterprise IT can also work for Government.

David Pallman’s Upcoming Orange County Azure User Group April Meeting: "What's New in Azure" post of 4/8/2009 announces that the next meeting of the Orange Couny Azure User Group is Thursday, April 23rd at QuickStart Intelligence.

The Economist memorializes the Open Cloud Manifesto in its Clash of the clouds article of 4/2/2009:

Deck: A familiar-sounding industry spat breaks out over standards.

Just as predictably, the leaders in cloud computing are absent from the list of supporters: Amazon, an online retailer that has successfully branched out into computing services; Google, which is not only a huge cloud unto itself but has built a cloud-computing platform for use by others; Salesforce.com, the biggest provider of software-as-a-service; and Microsoft. Indeed, it was an executive at the world’s biggest software firm, Steven Martin, who first leaked the manifesto, complaining that it had been drawn up in secret. “It appears to us that one company, or just a few companies, would prefer to control the evolution of cloud computing,” he wrote in a blog.

This insignificant “apple pie and motherhood” manifesto provided grist for more that its share of blog mills and even makes The Economist and Financial Times. Seems to me to indicate slow news days.

Other Cloud Computing Platforms and Services

•• Bill McNee’s Saugatuck Trip Report: On the Road in Early 2009 (PDF, requires site registration) reports “What’s Happening”:

Three key recurring themes were evident during the past two months as Saugatuck took to the road at a number of customer and industry events in the US and Western Europe. First and foremost is that interest in On Demand services remains strong despite the tough economic climate that we are in. Second, industry messaging around Cloud Computing is clearly taking center stage over SaaS, which is a good thing in Saugatuck’s opinion. Third, SaaS and Cloud providers are clearly battening down the hatches and slowing key investments that will result in slightly lower growth than they would have otherwise achieved. [Emphasis added.]

•• Jorge Escobar starts the process of Building A Social Application on the Cloud on 4/10/2009 with a description of his objectives and a roadmap to the five steps in his project:

This post will be divided in five parts. As I complete them, they will be hyperlinked below:

  1. Why build it on the cloud?
  2. Amazon EC2
  3. Amazon SimpleDB
  4. Facebook Connect
  5. Next steps

•• William Vambenepe debunks Dave Worthington’s Cloud providers vow interoperability SDTimes article (below) in his Reality check on Cloud portability post of 4/10/2009. Bill begins his post with:

SD Times recently published an interesting article about “cloud interoperability”. It has some well-informed opinions. But, like all Cloud-related discussions, it also suffers from mixing a bunch of things. The word “interoperability” is alternatively applied to the Cloud infrastructure services (in which case this “interoperability” is a way to provide application “portability”) and to the Cloud-hosted applications themselves.

Application-level interoperability (”look, my GAE-hosted app successfully sent an HTTP request to an Azure-hosted app, open the champagne”) is not very new or exciting anymore and is often used as an interoperability smokescreen (hello Salesforce.com). Many of these interop concerns are long solved and the others (like authentication and data migration) need to be solved in ways that don’t care whether the application is hosted in your Silicon Valley garage or near the Columbia river.

•• David Worthington analyzes the outcome of the Open Cloud Manifesto flap in his Cloud providers vow interoperability article of 4/10/2009 for SDTimes. If you missed the blow-by-blow action during the Cloud Computing Expo, you might find this article to be a useful summary:

In a series of interviews, Amazon, Hewlett-Packard, IBM, Microsoft and Salesforce.com detailed how developers can integrate their services, while acknowledging the challenges posed by constructing composite cloud applications.

•• Amazon Web Services introduced the Amazon Elastic WebReduce beta in early April. Amazon’s Hadoop implementation promises:

Using Amazon Elastic MapReduce, you can instantly provision as much or as little capacity as you like to perform data-intensive tasks for applications such as web indexing, data mining, log file analysis, machine learning, financial analysis, scientific simulation, and bioinformatics research. Amazon Elastic MapReduce lets you focus on crunching or analyzing your data without having to worry about time-consuming set-up, management or tuning of Hadoop clusters or the compute capacity upon which they sit.

•• David Dewitt’s MapReduce: A major step backwards post of 1/17/2009 to The Database Column argues that MapReduce is:

  1. A giant step backward in the programming paradigm for large-scale data intensive applications
  2. A sub-optimal implementation, in that it uses brute force instead of indexing
  3. Not novel at all -- it represents a specific implementation of well known techniques developed nearly 25 years ago
  4. Missing most of the features that are routinely included in current DBMS
  5. Incompatible with all of the tools DBMS users have come to depend on

and then goes on to describe MapReduce, as well as to add detail to the preceding arguments.

•• Craig Balding’s enStratus: Confidence in the Cloud (Plus: $100 off Under The Radar VIP Tickets) post of 4/10/2009 is a:

[S]ummary of a very interesting call I had with George Reese, CTO of enStratus and author of the forthcoming “Cloud Application Architectures” book.  Please note: this isn’t a comprehensive review of the full [enStratus] service, rather it reflects the pieces that we delved into based on some of the common concerns we have around Cloud Security (to give you some idea, we spoke for over 90 minutes…).

Filesystem encryption for Amazon Web Services Elastic Block Store (EBS) service is the most interesting aspect of Craig’s summary (at least to me.)

•• J. Nicholas Hoover reports that GE Puts 'Private' Cloud Computing To The Test on 4/11/2009 for Information Week. The deck reads: “It's starting a three-year effort aimed at better efficiency and flexibility.” Hoover writes:

GE is in the early stages of a three-year project to implement technologies that give it the flexibility, automation, and manageability [GE CTO] Simpson seeks. It's evaluating whether to have one expansive internal cloud or multiple discrete clouds dedicated to, say, Web serving or financial systems. Simpson's leaning toward the latter scenario, but either way, he wants to offer IT resources on demand and to charge business units for what they consume.

•• Kevin Jackson’s Cisco's Cloud Computing Strategy post of 4/10/2009 reviews Krishna Sankar’s A Hitchhiker's Guide to the Inter-Cloud presentation which lists the following as Cisco’s cloud computing strategy:

    • Build Right Products (Unified Fabric, Unified Compute, Virtualization Aware)
    • Technology (Enhanced IP core with tight coupling to software)
    • Multiphased (Standalone Clouds to Enterprise-Class to Inter-Cloud)
    • Open Standards (Accelerate Cloud deployment and federation through Cloud standards)
    • Services/References SW (Services-led Cloud blueprints with reference software stacks)

Jackson concludes:

The presentation also describes and end goal as a time "when enterprise applications can seamlessly move between their internal and external clouds leveraging the elasticity and multi-tenancy that a cloud infrastructure offers". An "Inter-cloud" standards and protocol roadmap is also offered.

Craig Balding analyzes Amazon’s new Access Control Policy for Simple Queue Services (SQS) in his Amazon AWS Introduces New Access Policy Language (SQS Today…) post of 4/9/2009.

Ben Kepes takes HP to task for equating the Application Service Provider (ASP) model with SaaS in his SaaS ≠ ASP2.0. A Treatise on FUD post of 4/9/2009.

• Jeff Barr’s Powerful New Amazon SQS Features post of 4/9/2009 discusses SQS’s new features:

  • New AddPermission and RemovePermission functions
  • At the lower level you can use the new Access Policy Language.
  • The new ChangeMessageVisibility function gives you the power to change the read timeout for an individual SQS message.

Craig Balding warns Missile, Chemical and Biological Weapons Developers: Google App Engine Is Not For You in this 4/8/2009 post. Craig writes:

Clause 2.2 just had some text added to [GAE’s] terms of service:

“You agree not to use the Service in the design, development, production, or use of missiles or the design, development, production, stockpiling, or use of chemical or biological weapons.”

I’m glad they cleared that up - now all the bad guys know to use Amazon AWS or Microsoft Azure.

However, there’s no restriction against it’s use for nuclear weapons.

Craig Balding asks Is Amazon AWS Really HIPAA Compliant Today? and then describes these security gaps:

    • There is no customer accessible AWS API call audit log
    • There is no way to restrict the source IP address from which the sacred AWS API key can be used from
    • Each AWS account is limited to a single key - exposure results in total security failure

Hopefully we all know by now that “compliance” does not equal “security”, but HIPAA compliance not withstanding, would you really want your medical data in a Cloud without some or all of these fundamental control gaps resolved or mitigated?  I can’t find anything in this new whitepaper or the old AWS Security Whitepaper (‘Creating HIPAA-compliant Medical Data Applications with AWS’) that speaks directly to these issues. [Whitepaper link added.]

Google App Engine Team announces the Google Secure Data Connector on 4/8/2007. Features include:

  • Access your corporate data in the browser. SDC lets you access your data from within Google Gadgets, Google App Engine, and Google Spreadsheets. SDC provides an agent to connect your Google Apps domain to your behind-the-firewall data sources.
  • Control the use of your data. SDC lets you restrict which users and applications can make requests to your internal services. You can use our partners or your own internal authentication systems to validate and authorize those requests.
  • Build custom apps for your business. SDC lets you extend your enterprise systems into Google Apps. You can easily build gadgets or
    Google App Engine applications that make use of both private and public data.

and CSV bulk data import in the Uploading Data post of the same date. The Scheduled Tasks With Cron for Python post reports that GAE now supports scheduled tasks. Finally, Seriously this time, the new language on App Engine: Java™ of 4/7/2009 squelches the April 1 “announcement” of Fortran for GAE.

Reuven Cohen summarizes the new GAE features in his Google's Cloud Now Bridges To Your Data Center thread of 4/8/2009 in the Cloud Computing Interoperability Forum (CCIF) group and this later, illustrated Google's Cloud Bridges your Data Center post. The App Engine Team’s Google App Engine’s Next Iteration post of 4/8/2009 is the official summary.

Cast Iron Systems announces their support for SDC in this Google Apps charges ahead with improved data security and long-awaited Java support post of 4/8/2009.

§ Larry Dignan analyzes the ramifications of Sun following the IBM deal collapse: Customer confusion en route for ZDNet on 4/6/2009:

[A]ny customer buying from Sun will have to consider the ramifications of a purchase. For instance, if you’re about to buy Sun hardware and HP is in the running why wouldn’t you opt for the latter just to eliminate some uncertainty. After all, HP could buy Sun. What about cloud services? Will you trust your cloud to a company that has a tug-of-war underway over a buyout?

§ Miko Matsumura posits Sun IBM Collapse Heralds the Return of McNealy. Jonathan Schwartz is Toast in this 4/5/2009 post, which includes a link to Steve Gillmor’s Open Source Ponytail Video. Miko concludes:

Mark my words, Schwartz is toast, IBM deal or no deal. If the IBM deal fails completely (most likely outcome), look for Scott McNealy to pull a Michael Dell (or a Jerry Yang, depending on how you look at it) and to appoint himself CEO again. The board of Sun wouldn’t allow such a thing if there were even one viable suitor left. But there isn’t.

§ James Hamilton’s Data Center Efficiency Summit (Posting #4) continues his efficiency analysis series with this 4/5/2009 post. Preceding posts from Google’s Data Center Efficiency Summit are:

  1. Data Center Efficiency Summit
  2. Rough Notes: Data Center Efficiency Summit
  3. Rough Notes: Data Center Efficiency Summit (posting #3)

§ James Urquhart analyzes Internal cloud's big test: Amazon vs. Cloudera on 4/4/2009:

The announcement on Thursday of Amazon's new Hadoop-based Elastic MapReduce service, combined with the introduction of a commercial Hadoop distribution from start-up Cloudera, means that we finally have a reasonable means of watching which directions enterprise IT prefers.

§ Chris Fleck’s Amazon EC2 Reserved Pricing Changes the Equation post of 4/5/2009, subtitled “Cloud Computing Economics Part Three,” analyzes on-premise or colocation versus Amazon EC2 conventional or reserved instance pricing for five quad-core servers.

Thursday, April 09, 2009

Uploading Entities for Storage in Azure Tables

One of the primary considerations when selecting a Platform as a Service (PaaS) cloud provider is the ease, speed, accuracy and reliability of uploading typical entities to persistent storage, as well as the cost of data ingress/egress and long-term storage.

Updated 4/8/2009 3:50 PM PDT: Performance and transaction topics.

Windows Azure, Amazon Web Service’s SimpleDB, and the Google App Engine provide low-cost, highly scalable storage in Entity-Attribute-Value (EAV) tables. To accommodate the widest range of platforms and programming languages for creating, querying and retrieving data from cloud storage, most cloud storage application programming interfaces (APIs) use Representational State Transfer (REST) methods with HTTP as the data transfer protocol and Atom Syndication Format (Atom), Atom Publication Protocol (APP, AtomPub) or a variation of these two Internet standards as the data format on the wire.

The following analysis is adapted from Chapter 7, “Optimizing the Scalability and Performance of Azure Tables,” of my forthcoming Cloud Computing with the Microsoft Azure Services Platform book for WROX.

ADO.NET Data Services HTTP Request and Response Messages in AtomPub Format

Azure Data Services use the HTTP POST method to insert entities in tables. Following is an example of the AtomPub POST Request message to add an entity from the Northwind sample database’s Orders table to an Azure OrderTable:

POST /OrderTable HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Wed, 08 Apr 2009 22:49:49 GMT
Authorization: SharedKeyLite oakleaf3:f3o42-----RedactedKey-----6qT6r9AiYnkRS80=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
Host: oakleaf3.table.core.windows.net
Content-Length: 1383
Expect: 100-continue

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
  <title />
  <updated>2009-04-08T22:49:49.8086948Z</updated>
  <author>
    <name />
  </author>
  <id />
  <content type="application/xml">
    <m:properties>
      <d:CustomerID>BSBEV</d:CustomerID>
      <d:EmployeeID m:type="Edm.Int32">4</d:EmployeeID>
      <d:Freight m:type="Edm.Double">2.17</d:Freight>
      <d:OrderDate m:type="Edm.DateTime">1998-03-11T00:00:00</d:OrderDate>
      <d:OrderID m:type="Edm.Int32">10943</d:OrderID>
      <d:PartitionKey>2147472704</d:PartitionKey>
      <d:RequiredDate m:type="Edm.DateTime">1998-04-08T00:00:00</d:RequiredDate>
      <d:RowKey m:null="false" />
      <d:ShipAddress>Fauntleroy Circus</d:ShipAddress>
      <d:ShipCity>London</d:ShipCity>
      <d:ShipCountry>UK</d:ShipCountry>
      <d:ShipName>B's Beverages</d:ShipName>
      <d:ShipPostalCode>EC2 5NT</d:ShipPostalCode>
      <d:ShipRegion m:null="true" />
      <d:ShipVia m:type="Edm.Int32">2</d:ShipVia>
      <d:ShippedDate m:type="Edm.DateTime">1998-03-19T00:00:00</d:ShippedDate>
      <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>
    </m:properties>
  </content>
</entry>

Azure Tables support a simplified version of the ADO.NET Data Services (formerly the “Astoria Project”) API for create, retrieve, update, and delete (CRUD) operations by means of the Windows Azure SDK’s sample ClientServices class library. The create operation with the POST method returns a confirmation of the data inserted, such as that shown below:

HTTP/1.1 201 Created
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
ETag: W/"datetime'2009-04-08T22%3A48%3A37.9913381Z'"
Location: http://oakleaf3.table.core.windows.net/OrderTable(PartitionKey='2147472704',RowKey='')
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 05b91086-1747-42d6-98ed-1ad6c85537bd
Date: Wed, 08 Apr 2009 22:48:37 GMT
Content-Length: 1774

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="http://oakleaf3.table.core.windows.net/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:etag="W/&quot;datetime'2009-04-08T22%3A48%3A37.9913381Z'&quot;" xmlns="http://www.w3.org/2005/Atom">
  <id>http://oakleaf3.table.core.windows.net/OrderTable(PartitionKey='2147472704',RowKey='')</id>
  <title type="text"></title>
  <updated>2009-04-08T22:48:38Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="OrderTable" href="OrderTable(PartitionKey='2147472704',RowKey='')" />
  <category term="oakleaf3.OrderTable" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:PartitionKey>2147472704</d:PartitionKey>
      <d:RowKey></d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">2009-04-08T22:48:37.9913381Z</d:Timestamp>
      <d:CustomerID>BSBEV</d:CustomerID>
      <d:EmployeeID m:type="Edm.Int32">4</d:EmployeeID>
      <d:Freight m:type="Edm.Double">2.17</d:Freight>
      <d:OrderDate m:type="Edm.DateTime">1998-03-11T00:00:00</d:OrderDate>
      <d:OrderID m:type="Edm.Int32">10943</d:OrderID>
      <d:RequiredDate m:type="Edm.DateTime">1998-04-08T00:00:00</d:RequiredDate>
      <d:ShipAddress>Fauntleroy Circus</d:ShipAddress>
      <d:ShipCity>London</d:ShipCity>
      <d:ShipCountry>UK</d:ShipCountry>
      <d:ShipName>B's Beverages</d:ShipName>
      <d:ShipPostalCode>EC2 5NT</d:ShipPostalCode>
      <d:ShipVia m:type="Edm.Int32">2</d:ShipVia>
      <d:ShippedDate m:type="Edm.DateTime">1998-03-19T00:00:00</d:ShippedDate>
    </m:properties>
  </content>
</entry>

Message Bloat for Order Entities with the AtomPub Wire Format

Notice that the payload for the HTTP 201 Response message is 1.3 times the size of the Request message’s payload. The combined size of the two payloads is 3,157 plus 973 header bytes for a relational table record with a UTF-8 size of about 159 bytes. The update messages are (4130 – 159) * 100/4130 = 96.2% overhead, assuming that the return payload is discarded. In other words, the Request and Response messages are about 26 times the size of uncompressed relational data. The added overhead contributes significantly to increased data ingress charges.

Message Bloat for OrderDetail Entities with the AtomPub Wire Format

Following is an HTTP Request message for a typical OrderDetail entity:

POST /OrderDetailTable HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Thu, 09 Apr 2009 21:02:25 GMT
Authorization: SharedKeyLite oakleaf3:uG/e6awcKFfuy-----RedactedKey-----pp4k+j0msRX0=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
Host: oakleaf3.table.core.windows.net
Content-Length: 886
Expect: 100-continue

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
  <title />
  <updated>2009-04-09T21:02:25.7957487Z</updated>
  <author>
    <name />
  </author>
  <id />
  <content type="application/xml">
    <m:properties>
      <d:Discount m:type="Edm.Double">0.2</d:Discount>
      <d:OrderID m:type="Edm.Int32">11077</d:OrderID>
      <d:PartitionKey>2147472570</d:PartitionKey>
      <d:ProductID m:type="Edm.Int32">2</d:ProductID>
      <d:Quantity m:type="Edm.Int32">24</d:Quantity>
      <d:RowKey>00</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>
      <d:UnitPrice m:type="Edm.Double">19</d:UnitPrice>
    </m:properties>
  </content>
</entry>

and the corresponding 201 Response message:

HTTP/1.1 201 Created
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
ETag: W/"datetime'2009-04-09T21%3A01%3A16.700813Z'"
Location: http://oakleaf3.table.core.windows.net/OrderDetailTable(PartitionKey='2147472570',RowKey='00')
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: f2debcd6-1614-4380-8972-b76f54a73cd1
Date: Thu, 09 Apr 2009 21:01:16 GMT
Content-Length: 1347

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="http://oakleaf3.table.core.windows.net/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:etag="W/&quot;datetime'2009-04-09T21%3A01%3A16.700813Z'&quot;" xmlns="http://www.w3.org/2005/Atom">
  <id>http://oakleaf3.table.core.windows.net/OrderDetailTable(PartitionKey='2147472570',RowKey='00')</id>
  <title type="text"></title>
  <updated>2009-04-09T21:01:16Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="OrderDetailTable" href="OrderDetailTable(PartitionKey='2147472570',RowKey='00')" />
  <category term="oakleaf3.OrderDetailTable" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:PartitionKey>2147472570</d:PartitionKey>
      <d:RowKey>00</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">2009-04-09T21:01:16.700813Z</d:Timestamp>
      <d:Discount m:type="Edm.Double">0.2</d:Discount>
      <d:OrderID m:type="Edm.Int32">11077</d:OrderID>
      <d:ProductID m:type="Edm.Int32">2</d:ProductID>
      <d:Quantity m:type="Edm.Int32">24</d:Quantity>
      <d:UnitPrice m:type="Edm.Double">19</d:UnitPrice>
    </m:properties>
  </content>
</entry>

The two payloads total 2,233 bytes and headers total 886 bytes. The numeric values occupy 47 bytes, which leads to an overhead of (3119 –47)*100/3119 = 98.5% or a message to payload ratio of 66:1. Providing an industry-standard API results in substantially increased data ingress charges.

The Affect of PartitionKey and RowKey Values on Scalability

For this example, each OrderTable entity has a unique PartitionKey value and RowKey zero-length string value to achieve maximum scalability by allowing distribution of entities over a potentially large number of Azure nodes. Default PartitionKey values for this example are 10-digit padded string representations of the maximum int value less the OrderID value. This combination produces the equivalent of a descending index on the primary key, which enables use of the LINQ Take(n) function to return the most recent n orders.

Associated OrderDetailsTable entities carry the parent order’s PartitionKey key value plus a zero-based line item number as the RowKey value to produce a unique object identifier (the equivalent of a relational table’s primary key.) Using the same PartitionKey value for Order and OrderDetail entities assures that a parent and its child entities are stored on the same nodes. (Currently, the original data and two replicas would be stored on three nodes in different failure domains.)

After Microsoft releases geo-location features for Azure, you’ll be able to specify the data center in which to store entities that are coded geographically, such as by state, province or region. The first center to support Azure is NorthwestUS (Quincy, WA). SouthwestUS (San Antonio (Westover Hills), TX) is expected online with Azure in 2009; it opened in September 2008.

Measuring Data Upload Performance with ADO.NET Data Services and AtomPub

Following is a screen capture of the form for conducting timing tests on create operations for OrderTable entities, with and without corresponding OrderDetailTable entities (click the image below for a full-size capture):

Loading the most recent 100 Orders with their 232 Order Details entities requires about 67.75 seconds. The time shown in the preceding screen capture is that for retrieving a single order with four line items. The following table shows the number of seconds to upload or delete 100 Orders, without and with line items (Order Details)

  Upload (Alone) Delete(Alone) Upload (with Details) Delete (with Details)
HTTP (clear) 18.86 13.55 67.75 70.92
HTTPS (TLS) 20.80 13.57 69.18 74.40

What’s interesting about the above data is the longer time to delete than create Order entities with their OrderDetail entities. Client deletions execute a GET request for the last Order entity, which includes PrimaryKey and OrderID values. The client then issues a GET /OrderDetailTable()?$filter=PartitionKey%20eq%20'2147472570' HTTP/1.1 request to obtain associated OrderDetail entities, and follows with DELETE /OrderDetailTable(PartitionKey='2147472570',RowKey='0n') instructions for n = 0 to n OrderDetail entities. Even with the extra GET method calls, the total amount of data on the wire is considerably less for DELETE than POST operations. This implies that the data creation times are more dependent on Azure Table operations in the cloud than message transit times. 

Cross-Table Transactions for Azure Tables

The latest word on transactions for Azure Tables is from Microsoft’s Alex Gershaft in 3/24/2009 answer to Azure table storage and state management questions in the Windows Azure forum:

There is no current support for transactions, but it is something that is on our future feature list.

The question, of course, is “How far in the future?” I’m guessing quite a while because MSFT wants to maximize revenue from SDS surcharges for persistent storage with relational features.

There was discussion of using Azure Queues for Cross-Table Transactions in the early days of Azure tables (see Brad Calder’s answer of 10/29/2008 in the Transactions and Locking? Window Azure forum thread.) A couple of days later, Calder replied to the Design pattern for data structures thread:

Great point.   Yes, you can create a property called “Kind”/“Type” (or whatever you want) for all of the entities stored in your Test Table,  and then treat the entities within a partition as entities belonging to N different types/classes.  This can allow you to model within a partition basically several “mini-tables”, where you can think of each Kind/Type being a mini-table within the partition.    Then once we have Entity Group transactions exposed that will allow you to perform atomic transactions across all of the entities within the partition, which means you can perform atomic transactions across all of these Kind/Type entities (mini-tables) in that partition.

Not much has been heard about queue-based or Kind/Type-assisted transactions recently.

This post will be updated as additional tests under varying conditions are conducted.

Monday, April 06, 2009

Azure Table Test Harness with HTTPS, Encryption, and Membership Services for Authentication/Authorization

This variation on the original AzureTableTestHarnessSSL.sln WebRole project adds customized ASP.NET Membership Services for user/password authentication and role-based authorization is live on Azure. To test-drive the project, go to http://oakleaf4.cloudapp.net/Login.aspx.

Chapter 6, “Authenticating and Authorizing Service Users” of my forthcoming Cloud Computing with the Microsoft Azure Services Platform book for WROX provides full details for quickly integrating ASP.NET Membership Services with Web applications deployed to Windows Azure. Full sample code will be downloadable when WROX publishes the book later this year.

This screen capture shows the first page of 12 Customer entities (click image to display a full-size capture). Personally identifiable information, such as CompanyName, ContactName, and ContactTitle, is stored with 128-bit Advanced Encryption Services (AES) encryption and decrypted for paged display in the DataGrid control. Decryption is the reason for the 2.46-second time to display the first page of Customer entities.

The project includes the Windows Azure SDK March 2009 CTP’s AspProviders class library and customized versions of Login.aspx, ChangePassword.aspx, CreateNewWizard.aspx, ManageRoles.aspx, MyProfile.aspx and MySession.aspx pages, as well as the Web.config file from the AspProviderDemos.sln sample projects.


To test-drive the project, go to http://oakleaf4.cloudapp.net/Login.aspx.

If you haven’t established an account, click the Create a New User Account link to open the CreateNewWizard.aspx page:

Click Create User to open the User roles page and select both DbReader and DbWriter if you want to be able to view and add Customer entities:

Click Finish to open the Default.aspx page (first screen capture above.) If you select only the DbReader role, the FormView for adding new customers, as well as the Delete all Customers and Create Customers buttons are disabled.

If you don’t login and change the initial URL to http://oakleaf4.cloudapp.net/Default.aspx in an attempt to bypass the login or new account process, no information is displayed:

In this case, clicking the Login or one of the three enabled Manage Account links opens the Login.aspx page.

The seven-part tutorial project for the unencrypted version starts here.

Windows Azure and Cloud Computing Posts for 3/30/2009+

Windows Azure, Azure Data Services, SQL Data Services and related cloud computing topics now appear in this weekly series.

Posts about Open Cloud Computing ManifestoGate are in the Cloud Computing Events section temporarily, because all the action is at the Cloud Computing Expo being held in New York City this week (3/30/2009 – 4/1/2009).

Updated 4/6/2009: Additions for 4/5/2009 and some 4/6/2009 (repeated in next edition)
Updated 4/3/2009 – 4/5/2009: Additions
• Updated 4/1/2009 and 4/2/2009: Added .NET Services March CTP info, no tokens for .NET and SQL Services provisioning and the Google Data Center Efficiency Summit. (No 4/1 jokes included intentionally.)

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Azure Blob, Table and Queue Services

Rob Bagby explores Azure Table Storage, the REST and ADO.NET Data Services Story in this lengthy post of 4/3/2009:

Azure Table Storage is exposed via ADO.NET Data Services.  What the heck does that mean?  In this post, I will do my best to answer that question.  I will start with a brief overview of REST, the architectural style underlying ADO.NET Data Services.  As part of that topic, I will illustrate all we are doing when accessing Table Storage is issuing REST calls.  I will then discuss the role ADO.NET Data Services plays, as well as the role of the StorageClient sample application.

SQL Data Services (SDS)

Simon Munro argues that SQL Data Services Does Not Scale in this 3/31/2009 essay based on his “Comparing Azure Storage and SQL Data Services” presentation to the SQLBits conference in Manchester, UK. Simon writes:

The only (and suggested) way of getting scalability out of SDS is to scale out using partitions. Generally this is a really difficult thing for the relational model and SQL to do as a big part of the SQL model is the ability to have consistent data. Brewers CAP conjecture implies that SQL, by being Consistent and Available has to forgo Partition Tolerance – and I believe that that is the case with SDS.

Anyone who has tried to build a solution based on partitioned databases will know that the architecture in the above case study is no simple achievement and takes a lot of work whereby an entire layer needs to exist to allow the system to first identify the correct partition for the data. In the same MIX09 presentation Nigel Ellis hints at efforts that will be made in terms of adding some partition support to SDS, which may, through the use of configuration, provide for better querying across partitions (although ACID may be more difficult).

It’s to be noted that the SDS team now refers to “departmental” sized databases. As an example, the First round of Questions and Answers post of 3/12/2009 to the SQL Data Services Team Blog says:

The database size will be capped. We are still evaluating what the cap will be, but the plan is to ensure that the allowed database size supports most, if not all, departmental and web application workloads. [Emphasis added.]

A video segment of the session might be available shortly here.

I’ve updated my Examples of Scaling Relational Databases Up and Out post of 3/30/2009 with the preceding content.

• Pat Helland, SQL Server architect extraordinaire, discusses cloud computing from the standpoint of a ultra-large database (ULDB) guru who’s an autonomous computing enthusiast. in this ASP.NET Webcast of 4/2/2009 (Remember fiefdoms and emissaries?) Wally McClure’s ASP.NET Podcast Show #139 - David Penton and Pat Helland on Cloud Computing – audio has show notes.

Andrew Brust’s What's Old Is New Again column for Visual Studio Magazine’s April 2009 issue recounts the history of Visual Basic 3.0’s adoption of the Jet (Access) 1.1 relational database and the parallel with the SDS team’s course reversal from the Entity-Attribute-Value data model to a full-featured relational database. Here’s the deck:

Aligning SQL in the cloud to SQL on the ground is about more than just common sense. It's about getting things done.

Andrew concludes:

So Redmond listened to its customers, and the bizarre obsession with copying Amazon's SimpleDB Web service is over. Microsoft has given us a truly simple offering: the SQL Server technology that most Microsoft developers have been using for a decade and some have been using since even before my first column was published.

I couldn’t agree more.

P.S. Andrew and I started writing for VSM’s predecessor about 15 years ago when it had just been renamed from BASIC Pro to Visual Basic Programmers Journal.

• Alin Irimie reports No More Invitation Codes Needed For .NET Services And SLQ Services provisioning as of 4/1/2009. Alin says:

Just go to http://portal.ex.azure.microsoft.com/ and press the Signup button. The following services will be provisioned:

  • Microsoft® SQL Data Service
  • Microsoft® .NET Service Bus
  • Microsoft® .NET Workflow Service
  • Microsoft® .NET Access Control Service

I haven’t tested the new SDS instance yet, but attempting to create a second instance with the same Solution Name fails with a “Solution name has already been used” error message.

My Examples of Scaling Relational Databases Up and Out post of 3/30/2009 draws on presentations by Amazon Web Service architect/VP James Hamilton and Salesforce.com CTO Craig Weissman to demonstrate that relational databases can scale to cloud dimensions.

Mike Flasko’s ADO.NET Data Services and the "new" SQL Data Services post of 3/29/2009 reminds prospective SDS users of the following recent posts about how SDS will support full relational capabilities with ADO.NET:

.NET Services: Access Control, Service Bus and Workflow

Clemens Vasters continues on his .NET Service Bus roll with his .NET Services March 2009 CTP: Host a Public Website At The Kitchen Table or from a Coffee Shop! No Kidding post of 4/5/2009:

Using the application/service built from the sample linked at the top of this post you can host a publicly discoverable and accessible website or Web service from your Windows notebook or desktop machine from within most network environments without having to open up a port on the firewall, mapping a port on your NAT, or using some type of dynamic DNS service to make the site discoverable. All those essential connectivity features are provided by the .NET Service Bus and with the help of the included sample code. [Emphasis Clemens’.]

Clemens VastersXML-RPC with WCF (Updated) post of 4/3/2009 announces the availability of his “new XML-RPC implementation is no longer a binding with a special encoder, but is implemented entirely as a set of behaviors and extensions for the WCF Service Model”:

The resulting Service Model programming experience is completely "normal". That means you can also expose the XML-RPC contracts as SOAP endpoints with all the advanced WCF bindings and features if you like. The behaviors support client and service side. I stripped the config support from this version – I’ll add that back once I get around to it.

•• Ryan Dunn explains why there’s no Windows Azure version of Azure Issue Tracker today [that hosts Geneva] in his Windows Azure and Geneva post of 4/2/2009 by a link to Vittorio Bertocci’s Claims and Cloud: Pardon our Dust post of 4/1/2009. Vittorio writes:

Developing secure applications and providing a hosting service often provide seemingly contrasting requirements. In order to secure communications you take advantage of protected resources, such as private keys; but when you offer a hosting service for multiple tenants, you abstract away the access to those very resources. The net result is that if you combine early versions of products and offerings that evolved under those different requirements, they may not just work with one another out of the box.

This is currently the case with Geneva Framework and Windows Azure: for a variety of reasons, an application that takes advantage of the Geneva Framework will not work “as is” when hosted in Windows Azure, including Microsoft products that were written to use the Geneva Framework. You may have heard that the new full trust settings we announced for Windows Azure at MIX would make the above scenario work, however that’s not the case: there is more than full trust [required] for enabling the complete range of possibilities offered by claims based access. [Emphasis added.] …

•• Clemens Vasters added .NET Services March 2009 CTP - Service Bus Routers And Queues - Part 2: Queue Policies and .NET Services March 2009 CTP - Service Bus Routers And Queues - Part 3: The REST Queue Protocol to his .NET Services series on 4/2/2009

• Matias Woloski has “been working lately with Windows Azure and Geneva in different projects” and posted Windows Azure working with Geneva Framework on 4/2/2009 (although the article is dated 4/5/2009.) Matias writes:

After some days testing different workarounds (and being patient with Azure deploy) I was able to configure Windows Azure web role as a relying party trusting on LiveID through ACS. The biggest challenge to overcome was how to use X509Certificates in the absence of the usual certificate stores.

• Clemens Vasters continues his posts on the new .NET Services features with .NET Services March 2009 CTP - Service Bus Routers and Queues - Part 1: Fundamentals of 4/1/2009.

• Matias Woloski’s Creating Security Token Services with Microsoft Identity Framework (Zermatt) post of 4/1/2009 contains this link:

Sebastian [Iacomuzzi] who has been working with Zermatt for a couple of weeks already, is posting a useful “straight to the point” how to implement active and passive STS’s using Zermatt. The abstractions in Zermatt are making this a joy. I like the separation of the STS from the underlying host (i.e. ASP.NET, WCF, “put-the-name-of-the-next-foundation”) because allows you to reuse the same STS for both the service layer and the presentation layer and have a consistent access control mechanism on both layers using claims.

• Clemens Vasters uses a German children’s song to complain about pseudo-RESTful APIs that aren’t “picking the appropriate predefined HTTP method” in his [Intermission] Auf der Mauer, auf der Lauer sitzt 'ne kleine Wa! - or: When REST isn't REST - or: Why and How I Care About Standards-Compliance post of 4/1/2009.

• Bruno Terkaly’s Azure – Microsoft .NET Services – The Service Bus – Blog Post about Setup post of 4/1/2009 “is all about getting ready to learn about .NET Services, and the Service Bus specifically. This blog is based on the Azure Services Toolit SDK and Training Kit and adds additional background information.”

The second chapter is Azure – Microsoft .NET Services- Step 02 – The Service Bus – Blog Post about Creating a Service Bus Solution of 4/2/2009.

• Alin Irimie reports No More Invitation Codes Needed For .NET Services And SLQ Services provisioning as of 4/1/2009. See the “SQL Data Services (SDS)” section for details.

• Clemens Vasters.NET Services March 2009 CTP - The Service Bus Namespace post of 3/31/2009 explains how the Service Bus’s:

[N]amespace root for each project is taking on a new form that we had already pre-announced into the PDC’08 documentation and that I’ve talked about at PDC and other occasions. Any project’s root URI is now, depending on the chosen transport option, of the form

sb://project-name.servicebus.windows.net/ or
http://project-name.servicebus.windows.net/ or
https://project-name.servicebus.windows.net/

The previous root URI for a project was sb:|http:|https://servicebus.windows.net/services/project-name/, which was clearly sub-optimal for a number of reasons.

And adds details about the technical and philosophical reasons for the change.

His .NET Services March 2009 CTP - "Milestone 5" post of the same date discusses additional March 2009 CTP changes to the .NET Service Bus.

The .NET Services Team’s .NET Services – Announcements post of 4/1/2009 provides more detailed information on the March 2009 .NET Services CTP.

Steven Martin announces in his .NET Services - Cloud Interoperability post of 3/30/2009 that the .NET Services team will release the “M5” (Milestone 5) CTP of .NET Services on 3/31/2009. According to Steven:

This milestone contains enhancements to all of the services including expanded support for standards like REST, ATOM, SOAP and HTTP. As I mentioned previously, we demonstrated at MIX cloud to cloud interop in action. Specifically, we showed how the Access Control Service and Service Bus could be integrated with a Python Application deployed into Google App Engine using just two lines of code. As always, feedback from developers is critical to us. So, please take time to sign up for the CTP, and tell us what you think. We’re on our way to commercial availability later this year and we need your help to get there. [Emphasis Steven’s.]

Update 3/31/2009: You can download the Microsoft .NET Services SDK (March 2009 CTP) from here.

Live Windows Azure Apps, Tools and Test Harnesses

My Azure Table Test Harness with HTTPS, Encryption, and Membership Services for Authentication/Authorization post of 4/6/2009 shows you how to test drive my live AzureTableTestHarnessSSL.sln project that takes advantage of ASP.NET Membership Services samples from the Windows Azure SDK March 2009 CTP.

Clemens Vasters continues on his .NET Service Bus roll with his .NET Services March 2009 CTP: Host a Public Website At The Kitchen Table or from a Coffee Shop! No Kidding post of 4/5/2009 (repeated from the “.NET Services: Access Control, Service Bus and Workflow” section. 

Azure Infrastructure

••• Bernard Golden’s Power: One Cloud Cost Advantage That May Be Irresistible article of 4/6/2009 for CIO’s Virtualization and Cloud Observer blog analyzes Microsoft’s The Cost of a Cloud: Research Problems in Data Center Networks whitepaper and Steve Denegri’s Cloud Computing Dominance Through Renewable Energy paper:

From [Denegri’s] research, he concludes that Microsoft obtains a 22% discount on its property taxes (this is based on a discount extended to companies that make very large real property investments). With regard to power, Microsoft obtains a 45% discount, based on using enough electricity to fall into a "Super Large Power" user category. This advantage may be enhanced in the future, Denegri states, because of potential taxes to be applied to carbon-based power. Microsoft may be able to strike an arrangement to have its data center powered by clean energy sources (e.g., wind or solar), thereby avoiding these additional taxes.

••• Microsoft attempts to explain its ubiquitous Software as a Services strategy with an inspirational[?] intro video, a Ray Ozzie introduction, and pages for potential Xbox, Azure, Windows Live and Mobile, and Office Online services. (This site might have been around for a while; I just found it.)

••• Mary Jo Foley reports Microsoft links HealthVault service with Amalga software on 4/6/2009:

HealthVault is Microsoft’s consumer-focused health-records-management Software+Service platform, which the company unveiled officially in 2007. (The service component of HealthVault is one of a handful of Microsoft services that already is hosted on top of Azure.) Amalga UIS, (one of the products formerly under the Azyxxi brand), is one of the main elements of Microsoft’s enterprise health-information-system platform. [Emphasis and links added.]

Dimitry Sotknikov’s Can cloud make you MORE compliant? post of 4/6/2009 reviews a report by Scott Crawford from Enterprise Management Associates - “The Security Paradox of Cloud: Five Questions for Cloud Providers” and adds his comments on the paper.

••• David Pallman begins a new Azure series with Grid Computing on the Azure Cloud Computing Platform, Part 1 of 4/5/2009:

In this series of articles we're going to look at grid computing using the Azure cloud computing platform. In Part 1, we'll look at this from a design pattern and benefits perspective.

The Windows Azure Team reports on a problem that caused “a number of applications running in Windows Azure [to be] unreachable for a period of time” (~8 hours from 1:15 PM PDT to 9:00 PM PDT) on 4/2/2009 in its Application Accessibility Restored post of the same date. The team writes:

For those who are wondering, though the underlying cause of the March 13th malfunction was very different, the same improvements we're making in detection and response in the Fabric Controller apply.

Remember to change the number of instances to count=”2” in the ServiceConfiguration.cscfg file to minimize the effect of such outages on your services. The Azure team says that the extra instance won’t count against your quotas.

Kevin Jackson’s Former DoT CIO on Cloud Computing post of 4/3/2009 reports that former Transportation Department CIO Dan Mintz offered his views on cloud computing to Eric Chabrow. Kevin adds NSC counter terrorism director Paul Kurtz’s views on the issue:

[D]etermining which information is safe to be accessed over the Internet shouldn't be a problem. He believes the savings to taxpayers to be significant by using software as a service applications and storing non-sensitive data in the clouds. His only concern is what he calls a "disruption in the sky", an Internet outage that would make access to information problematic. Still, he says, it worth doing and calls for lawmakers to address cloud computing in the regulatory reform legislation.

• Bob Muglia says, “Further out, Microsoft will release a version of its System Center management software designed specifically with private clouds in mind,” according to J. Nicholas Hoover’s Microsoft To Enable Private Clouds With Windows Server, System Center article for InformationWeek of 4/2/2009:

Microsoft (NSDQ: MSFT) plans to build features into future versions of Windows Server, as well as a new System Center product, that will help companies deploy and manage private clouds, Microsoft server and tools division president Bob Muglia said in an interview Wednesday.

Muglia’s statements from this interview appear to contradict, at least in part, the protestations of many Microsoft execs that a downsized Azure version for private clouds won’t find life in a SKU because the data center hardware is non-standard.

• Krishnan Subramanian’s Cloud Security Alliance – A Solid First Step post of 4/2/2009 analyzes the tasks awaiting this recently founded industry group.

• Dave Malcom describes The Five Pillars of Cloud Computing in this better-than-average description of cloud computing (4/2/2009).

• Pat Helland, SQL Server architect extraordinaire, discusses cloud computing from the standpoint of a ultra-large database (ULDB) guru who’s an autonomous computing enthusiast. in this ASP.NET Webcast of 4/2/2009 (Remember fiefdoms and emissaries?) Wally McClure’s ASP.NET Podcast Show #139 - David Penton and Pat Helland on Cloud Computing – audio has show notes. (Copied from the “SQL Data Services (SDS)” section.)

• David Linthicum analyzes the downside of service coupling in his Understanding Coupling for the Clouds essay of 4/2/2009 for Cloud Computing Journal.

David Bressler’s The Impact of Cloud Computing on Enterprise Architecture post of 3/31/2009 for Cloud Computing carries this deck:

Cloud computing is set to revolutionize the way enterprises consume IT. In this article, David Bressler of Progress Software investigates the implications of cloud computing on application delivery within the enterprise and outlines best practices for a successful foundation to cloud computing.

Kim Hart reports on current and pending federal adoption of public clouds in the Washington Post’s Tech Firms Seek to Get Agencies on Board With Cloud Computing article of 3/31/2009.

Jon Torresdal’s Article: Microsoft Azure - A new era for software companies? of 4/30/2009 carries the following deck:

This article is a translation of a Norwegian article I wrote earlier that got published on IT Pro a few days ago. It’s targeted towards IT decision makers and leaders, does not contain my usual technical focus, but should still be a valuable read for developers not familiar with Azure or want to know more about cloud computing.

Cloud Computing Events

Cloudforce London will hear from Marc Benioff on 4/7/2009, according to this 4/3/2009 pitch from Salesforce.com:

Salesforce.com the enterprise cloud computing company, today announced that Marc Benioff, chairman and CEO of salesforce.com, will speak at Cloudforce London, UK on Tuesday, April 7th, 2008 at 5:00 am EDT / 10:00 am BST. Mr. Benioff will also speak at a lunch for press and analysts at approximately 12:00 pm BST. An audio webcast of Mr. Benioff's presentations will be available on salesforce.com's website at www.salesforce.com/investor.

Cloud Expo Europe offers free pre-registration for conference sessions with two business and one technical tracks.

Where: Barbican Centre, London, UK When: 20-21 May 2009

TechHermit gives props to Google and wonders “where is Microsoft?” in his Google Data Center Event Impresses and Raises Stakes post of 4/4/2009. He also includes a link to videos of Google’s recent Data Center Efficiency Summit.

Julia Lim’s FOSE 2009 Survey - Part II - Can’t See the Clouds post of 4/2/2009 analyzes her second survey of federal government IT at FOSE (the Federal Office Systems Exposition) and finds cloud computing at the bottom of the important technologies and spending lists:

with only 42% marking it as important or very important to their operations and only 11% already using it (and 8% planning projects for this year). I have to say that I was a bit surprised to see even ITIL/CMDB (at 50% important/very important) beat Cloud Computing out on this list …

Jeffrey Schwartz’s IBM's Cloud CTO On Microsoft Azure: How Open Is It Going To Be? 4/1/2009 post to Redmond Developer News’ Data Driver blog quotes Yousef Khalidi, a distinguished engineer for Microsoft's cloud infrastructure services, who gave a presentation at Cloud Computing Expo this week:

Khalidi emphasized that its forthcoming Azure cloud platform will support both native and managed code, and not just .NET including Java, Ruby and PHP.

"I'm sitting here telling you, you can run anything you want on this platform," Khalidi insisted during a session Monday in response to a question by an attendee about Microsoft's support for different languages on its Azure Services Platform. The attendee seemed taken aback by that.

But Kristof Kloeckner, CTO of enterprise initiatives and VP of cloud computing at IBM wasn’t sanguine about the openness of the Azure Services Platform, according to Schwartz:

"I think the real question is going to be around how open is it going to be? What role are ecosystem partners going to play in Microsoft's environment? Certainly what I heard in the presentation that they want to support multiple languages makes sense. Whether that means a greater openness, I don't know. Let's just wait and see what they come up with."

Kevin Jackson reviews his own Cloud Computing Expo session in his Second Government Cloud Computing Survey post of 4/1/2009:

My presentation, The View from Government Cloud Computing Customers, reviewed findings from the second Government Cloud Computing Survey. Sponsored by Booz Allen Hamilton, 1105 Media and Dataline, LLC, the purpose of this survey was to validate finding of the October 2008 survey and to see if the significant trends noted then had changed.

Kevin’s post includes tables that compare the relatively popularity of concerns between the first and second surveys.

Kevin Jackson reports on the Navy Next Generation Enterprise Network (NGEN) Industry Day festivities in Washington, DC on 3/31/2009:

In case you're not familiar with NGEN, this project will be the follow-on to the Navy Marine Corps Intranet (NMCI) and represents the Navy's first step towards achieving a future vision of a fully integrated Naval Networking Environment (NNE). NMCI itself is the world's largest intranet 700,000 accounts.

Among other things, not the least of which was a year-long program delay, RADM Bill Goodwin, NGEN Program manager, made it a point to highlight an interest in cloud computing.

Sam Johnston’s On the CCIF unkeynote and open cloud community post of 4/1/2009 takes on Reuven Cohen, the CCIF, Manifesto, and the Cloud Computing Alliance. Sam writes:

My point is not to criticise them for the sake of it, rather to encourage fellow community members to critically assess what is forced down their throats proposed for creating a truly open cloud computing community. This is something I've been working towards all along (ironically, against the very people who now propose it) and something I for one have a lot of experience with. While I'm glad Reuven's finally on board with the "open cloud community" idea (a "melting pot" where both users and vendors come together to share ideas") I'll be disappointed if he claims it as his own (like the manifesto which did more damage than good.

David Linthicum discusses the recent issues around the Open Cloud Manifesto and what you need to consider as a cloud computing practitioner in his Cloud Computing Manifestos Manifesto podcast of 4/1/2009, available directly here.

Jon Bodkin’s Cloud computing vendors converge on standard definition, goals NetworkWorld post of 3/31/2009 reviews Werner Vogels’ and Kristof Kloeckner’s Cloud Computing Explo keynotes, notes the formation of the Cloud Security Alliance and the Open Cloud Manifesto controversy.

Cristofer Hoff (@Beaker) announces the formation of another cloud-computing splinter group in his Introducing the Cloud Security Alliance post of 3/31/2009, which he describes as:

[A] member-driven forum for both industry, providers and “consumers” of Cloud Computing services to discuss issues and opportunities for security in this emerging space and help craft awareness, guidance and best practices for secure Cloud adoption.  It’s not a standards body. It’s not a secret cabal of industry-only players shuffling for position.

The website offers a more formal definition.

John Foley calls the Open Cloud Manifesto “anticlimactic” and “embarrassing” in his Cloud Manifesto Lacks Thunder post of 3/30/2009. John concludes:

Where do we go from here? I'm not convinced the computer industry needs a cloud interoperability Manifesto. Everyone already agrees that interoperability is desirable, indeed a necessity; the question is how we get from here to there.

Cloud interoperability will result from the same Darwinian process that drives interoperability in other parts of the computer industry--popular technologies and APIs rise to the top, users seek ways to get platforms and services to work together, vendors (sometimes with arms twisted) work toward making that happen, and standards groups usher the process along where and when needed.

Krishnan Subramanian reports the withdrawal of the Cloud Computing Interoperability Forum (CCIF) and Google from the Open Cloud Manifesto’s signators in his Open Cloud Manifesto - The Fight Takes A Surprise Turn post of 3/30/2009.

Dennis Howlett reports he was surprised when he saw that "SAP had signed up for the Open Cloud Manifesto in his SAP signs up for Open Cloud Manifesto: does anyone care? post of 3/30/2009. Dennis continues:

Despite SAP’s best endeavors, thousands of customers and code jocks, ABAP is not a standard in the same way that Java has become. If you know ABAP the only place you can take it is to another SAP shop. If you know Java - well - you can pretty much go anywhere. So couching the manifesto in terms of ‘open’ seems a little disingenuous.

Brenda Michelson’s Liveblogging: Cloud Computing Expo #1, Werner Vogels on Amazon & Infrastructure Services post of 4/30/2009 indicates that Vogel’s The Power of Infrastructure as a Service session is old had -- a rehash of similar presentations he’s given to all the other cloud-computing conferences this year and last. If you need an example, see The Principles of Distributed Computing and the Birth of AWS for Web 2.0 Expo on 4/2/2009.

Paul Greenberg reports for ZDNet on the formation of the Open Cloud Manifesto (OCM) site, analyzes the OCM’s content, discusses the signators and objectors, and concludes:

I’m not as ready to write the whole thing off that quickly as some others are. I like the idea of some of the tech heavy hitters starting the conversation as they notably put it - even if that conversation is a bit self-serving.  But, the discussion has to start somewhere and this may be where it aggregates - if they can increase their credibility and get at least a few of the biggest cloud players involved.  If they manage that, then, as the old Ella Fitzgerald warbled song goes,  this may be the start of somethin’ big.  I hope so, but its got a long way to go.

(from his The Open Cloud Manifesto: Whats Up With That? post of 3/30/2009).

Cristofer Hoff (@Beaker) contends that Azure can’t be an open and interoperable cloud platform in his Incomplete Thought: Looking At An “Open & Interoperable Cloud” Through Azure-Colored Glasses post of 3/30/2009.

Obviously, definitions of “open” and “interoperable” vary widely, depending on whose ox you’re goring today. Azure Table Services and SDS with RESTful interfaces are certainly open to clients of all platform persuasions.

Steven Martin writes Moving beyond the “Manifesto” in his post of 3/29/2009:

As I indicated on Wednesday night, Microsoft welcomes the opportunity for open dialogue on cloud standards. To that end, we have accepted an invitation to meet on Monday at 4pm in New York at the Cloud Computing Expo with other vendors and members of standards bodies. From our perspective, this represents a fresh start on the conversation – a collaborative “do-over” if you will.

Update 3/31/2009: James Urquhart reports on the meeting in his IBM, Microsoft, others align on open clouds post of 3/31/2009:

IBM, Microsoft, Cisco, Intel, the IEEE/ISTO and key members of the CCIF met to address how they could work with the community to drive cloud computing markets and technologies forward. Jesse Silver spoke to me after the meeting, and Reuven Cohen released a single paragraph of minutes on his blog Tuesday morning.

Paul Miller’s Talking with Reuven Cohen about the Open Cloud Manifesto podcast contains an interview he conducted with @ruv on 3/26/2009.

Daryl Plummer’s The Open Cloud Manifesto: This has all happened before… post of 3/29/2009 compares the OCM slugfest with the WS-* standards battles:

This has all happened before: Remember WS-I? When web Services started, they were supposed to be simple ways to link different systems together using a small set of web compatible technologies. But, the vendors and enterprise IT got involved and decided it had to have standards and along came WS-I. It was supposed to make web Services Interoperable. How has that worked out for most of you? WS-* was born, and now we live in a world of such SOAP based Web Services complexity that the concept is suffering and being assaulted by more simple RESTful approaches. WS-I did not prevent the WS-* glut of standards - mostly aimed at making Web Services “Enterprise Ready”. And lack of Web Services interoperability is often cited as one of the worst problems with the concept. Let’s hope the same thing will not happen to cloud computing.

I said the same thing last week in my “Secret” Cloud Computing Manifesto Dustup post of 3/28/2009.

James Urquhart’s CCIF pulls out of the Open Cloud Manifesto post of 3/29/2009 reports:

In a post to the Cloud Computing Interoperability Forum (copied in full [in James’s Post]), the original organizers of that group--Reuven Cohen, Sam Charrington, Jesse Silver, and David Nielsen--have announced that the CCIF will no longer be a signatory of the controversial Open Cloud Manifesto to be presented Monday:

Other Cloud Computing Platforms and Services

•• Larry Dignan analyzes the ramifications of Sun following the IBM deal collapse: Customer confusion en route for ZDNet on 4/6/2009:

[A]ny customer buying from Sun will have to consider the ramifications of a purchase. For instance, if you’re about to buy Sun hardware and HP is in the running why wouldn’t you opt for the latter just to eliminate some uncertainty. After all, HP could buy Sun. What about cloud services? Will you trust your cloud to a company that has a tug-of-war underway over a buyout?

•• Miko Matsumura posits Sun IBM Collapse Heralds the Return of McNealy. Jonathan Schwartz is Toast in this 4/5/2009 post, which includes a link to Steve Gillmor’s Open Source Ponytail Video. Miko concludes:

Mark my words, Schwartz is toast, IBM deal or no deal. If the IBM deal fails completely (most likely outcome), look for Scott McNealy to pull a Michael Dell (or a Jerry Yang, depending on how you look at it) and to appoint himself CEO again. The board of Sun wouldn’t allow such a thing if there were even one viable suitor left. But there isn’t.

•• James Hamilton’s Data Center Efficiency Summit (Posting #4) continues his efficiency analysis series with this 4/5/2009 post. Preceding posts from Google’s Data Center Efficiency Summit are:

  1. Data Center Efficiency Summit
  2. Rough Notes: Data Center Efficiency Summit
  3. Rough Notes: Data Center Efficiency Summit (posting #3)

••• James Urquhart analyzes Internal cloud's big test: Amazon vs. Cloudera on 4/4/2009:

The announcement on Thursday of Amazon's new Hadoop-based Elastic MapReduce service, combined with the introduction of a commercial Hadoop distribution from start-up Cloudera, means that we finally have a reasonable means of watching which directions enterprise IT prefers.

Chris Fleck’s Amazon EC2 Reserved Pricing Changes the Equation post of 4/5/2009, subtitled “Cloud Computing Economics Part Three,” analyzes on-premise or colocation versus Amazon EC2 conventional or reserved instance pricing for five quad-core servers.

Hamid Motahari-Nezhad, Bryan Stephenson and Sharad Singhal have written Outsourcing Business to Cloud Computing Services: Opportunities and Challenges, a 16-page Hewlett Packard Technical Report dated early February 2009. Here’s the abstract:

Advances in service oriented architecture (SOA) have brought us close to the once imaginary vision of establishing and running a virtual business, a business in which most or all of its business functions are outsourced to online services. Cloud computing offers a realization of SOA in which IT resources are offered as services that are more affordable, flexible and attractive to businesses. In this paper, we briefly study advances in cloud computing, and discuss the benefits of using cloud services for businesses and trade-offs that they have to consider. We then present 1) a layered architecture for the virtual business, and 2) a conceptual architecture for a virtual business operating environment. We discuss the opportunities and research challenges that are ahead of us in realizing the technical components of this conceptual architecture. We conclude by giving the outlook and impact of cloud services on both large and small businesses.

Charles Cooper’s Telcos said testing plan to offer PCs to businesses post for CNet of 4/3/2009 claims:

Telecommunications providers on four continents are testing a plan to provide so-called virtual desktop computing to their business customers.

People familiar with the outlines of the pilot program say the idea is to offer Internet access to companies via dumb terminals connected through the so-called cloud. The tests are said to involve companies in the United States, Europe, Australia, and China.

Anders Bylund claims Amazon Rides Google's Coattails in this Motley Fool post of 4/3/2009:

Amazon.com (Nasdaq: AMZN) thinks that the MapReduce technology that was invented at Google (Nasdaq: GOOG) is good enough to build a moneymaking service on. How come Google didn't think of that first? …

The online retailer is rapidly becoming the go-to name in serious cloud computing, and this service is yet another bright feather in its cap. [Emphasis Bylund’s.]

Maureen O’Gara’s Cloud Computing Expo: Sony Trademarks the Term ‘PS Cloud’ post of 4/3/2009 contains details from Sony’s Trademark application and likens the company’s intention to provide cloud-based gaming to Steve Perlman’s OnLive cloud-gaming startup. O’Gara notes that Perlman “sold his WebTV Networks creation to Microsoft for $425 million back in 1997.”

Amazon Web Services announced a public beta of Amazon Electric Map Reduce on 4/2/2009. Here’s the start of Amazon’s brief description:

Amazon Elastic MapReduce is a web service that enables businesses, researchers, data analysts, and developers to easily and cost-effectively process vast amounts of data. It utilizes a hosted Hadoop framework running on the web-scale infrastructure of Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3).

Jeff Barr adds “how-to” details of Elastic MapReduce in his Announcing Amazon Elastic MapReduce post of 4/2/2009.

James Hamilton offers rough notes from the first two sessions at the Data Center Efficiency Summit at Google Mountain view in his Rough Notes: Data Center Efficiency Summit post of 4/1/2009. He’s presenting the Best Practices session at 1:00 PM, according to his earlier Efficient Data Center Summit post. Rough Notes: Data Center Efficiency Summit (posting #3) contains rough notes about a 10MW Google data center in a container.

Steven Shankland reports Google uncloaks once-secret server “with further details about Google's data center efficiency and shipping containers modules” on 4/1/2009 from the same Google Summit. Shankland writes:

Google's big surprise: each server has its own 12-volt battery to supply power if there's a problem with the main source of electricity. The company also revealed for the first time that since 2005, its data centers have been composed of standard shipping containers--each with 1,160 servers and a power consumption that can reach 250 kilowatts.

David Douglas’s Dave Douglas Sets out Sun's Vision of a Multi-Cloud World keynote at Cloud Computing Expo receives a brief review. Here’s the deck:

In his keynote session at Cloud Computing Expo in New York City, Dave Douglas, Chief Sustainability Officer at Sun Microsystems, investigated how enterprise IT operations can take advantage of this emerging world of many clouds to achieve the cost and flexibility advantages that cloud computing allows while maintaining control of their IT infrastructure.

Jeremy Geelan conducts a Q&A session with Dave Douglas in his A World of Many Clouds at Cloud Computing Expo in New York post of 4/1/2009/

Alin Irimie’s Hewlett-Packard Jumps In The Cloud With HP Cloud Assure describes this new cloud governance offering by HP:

HP Cloud Assure is made up of a suite of the company’s software, including HP Application Security Center, HP Performance Center and HP Business Availability Center. HP will also provide enterprises with a team of engineers to perform security scans, test performance, and monitor availability.

Jeremy Geelan interview of IBM’s Dr. Kristof Kloeckner is captured in this transcript by Susan Wechtler’s  Kristof Kloeckner Delivers Cloud Computing Expo Keynote: Open Cloud Manifesto provides the first draft of really straightforward principles post of 3/31/2009 from the Cloud Computing Expo.

Jem Alexander’s Sony trademark hints at cloud computing post of 3/30/2009 reports:

Sony has trademarked something called "PS Cloud." The trademark was registered "for use with a cloud computing data center management software, communications software, broadcasting services, and a long list of other terms,"

Gregg Ness explores the implications of recent alliances between cloud-computing vendors in his Preparing for the Club Cloud Shift in IT: The IT as a Cruise Ship Model- Will it Float? post of 3/29/2009.

Sunday, April 05, 2009

LINQ and Entity Framework Posts for 3/30/2009+

Note: This post is updated daily or more frequently, depending on the availability of new articles.

Entity Framework and Entity Data Model (EF/EDM)

James Terwilliger, a PostDoc researcher at Microsoft Research, delivers EDMGen2 – Now with Reverse Engineering options, an updated version of EDMGen, on 4/4/2009:

The new version of EdmGen2, recently released on Code Gallery, includes an option “RetrofitModel”. This extension is the result of collaboration with the database team at Microsoft Research, notably summer intern Ankit Malpani. When using this option, EdmGen2 uses data mining techniques to determine if there are TPH, TPT, or vertical partitioning relationships that are inherent in the instance. It then generates a suitable model and mapping in the form of csdl, msl, ssdl, and edmx files.

Mohammed Meligy’s Typed Eager Loading Using Entity Framework (& What is Eager Loading vs Deferred Loading) - Guru Stop post of 4/2/2009 ploughs this fertile ground for those uninitiated in EF’s loading of associated objects.

Gil Fink tackles Supporting Stored Procedures that Return Primitive Types in Entity Framework and Entity Framework Mapping Helper on 4/3/2009 and Creating Complex Types in Entity Framework on 4/2/2009

Alex JamesTip 10 - Understanding Entity Framework jargon of 4/2/2009 translates nine common EF-related abbreviations and acronyms.

Julie Lerman delivers An FAQ about EF/Astoria/LINQ to SQL: Is the data filtered on the server or on the client? on 4/1/2009 observes:

There are people who are concerned that the filters don't get applied until after the data has been returned from the server.

I happen to be one of those people.

Julie Lerman reminds folks in her Using Pre-Compiled LINQ to Entities Queries in Web Apps and Services post of 3/31/2009 about Brian Dawson’s earlier posts about query performance:

  1. Exploring the Performance of the ADO.NET Entity Framework - Part 1
  2. Exploring the Performance of the ADO.NET Entity Framework – Part 2
  3. ADO.NET Entity Framework Performance Comparison

and then adds a few of tricks from other performance types on the ADO.NET team.

Simon Segal’s Entity Framework Profiler – When .ToTraceString() just isn’t enough! post of 3/30/2009 describes how he “set out to build an Entity Framework Profiler to try to address the dislocated nature of having to switch between viewing output from .ToTraceString(), SQL Profiler and LINQPAD.”

Binary Bob compares Julie Lerman’s and John Papa’s approaches to data updates with Entity Framework in Silverlight apps in his 3/25/2009 Disconnected Clients, Changed data and Entity Framework post:

    • Julie Lerman (Programming Entity Framework) goes so far as to create DTOs for each Entity which are passed back and forth over the wire (DTOs are smaller than Entities and can be used by a wide range of clients who do not need to know anything about Entity framework). There is considerable work required in this solution though.
    • John Papa (Data-Driven Services with Silverlight 2) does indeed pass the serialized entities over the wire and then “reattaches” them to the data context and manually sets all the modified properties on the Entity in the update service’s code. The the SaveChanges() method is called to save the updates.

and goes on to suggest:

If you are willing to use Restful techniques as implemented in ADO.Net Data Services, you need not worry about handling changes in server code as ADO.Net Data Services can handle the changes as the changes are in the HTTP request (using the PUT verb), not in serialized Entities. If you are worried about data being sent in clear text (which you should be) then use SSL.

LINQ to SQL

Gunnar Peipman solves Problem when testing LINQ To SQL Classes objects on 3/30/2009 that arose as the result of caching problems:

It turns out that for testing we should also dispose DataContext after each operation so we don’t face caching problems when we want to see if database operations work as expected.

LINQ to Objects, LINQ to XML, et al.

Simon Segal’s LINQ To XML and the Specification Pattern - [In a static world] post of 4/4/2009 continues his infatuation with the Specification pattern. Simon promises:

I am going to have a go shortly at putting together some variations of the Specification<T> class to allow for a more complete use with LINQ in General, remembering that the initial requirement was based around testing objects for equality and then creating a more manageable and malleable system of creating dynamic queries in LINQ To SQL and the Entity Framework. The “new” Specification class will need to support more than the just Func<T, bool> and should prove to be in interesting exercise.

Zeeshan Hirani offers seven “Tips for Delving in to LINQ” in his LINQ Deep Dive: Part I post for ASP.NET Pro Magazine’s April issue:

    1. Apply filters before a join to improve query performance
    2. Filter LINQ queries using object reference comparison
    3. Apply aggregates to empty collections in LINQ to SQL queries
    4. Delay loading a property in LINQ to SQL
    5. Use table-valued functions with eager loading turned on
    6. Put joins in the correct order in a LINQ to Objects query
    7. Compose a LINQ query inside a loop

Matthieu Mezil’s How to use LINQ extension methods on non generic IEnumerable? post of 4/1/2009 contends: 

LINQ extension methods are great but... it's not so cool when we use non generic collections as it is often the case with Sharepoint or TFS APIs.

and then goes on to “redefine LINQ extension methods for non generic IEnumerable.”

ADO.NET Data Services (Astoria)

Mike Flasko’s More on Web Friendly Feeds in ADO.NET Data Services v1.5 CTP1 post of 3/29/2009 offers pointers to Phani Raju’s posts about new Web-Friendly Feeds:

Mike Flasko offers guidance in Using ADO.NET Data Services v1.5 CTP1 with Silverlight 2 and Silverlight 3 in this 3/29/2009 post.

ASP.NET Dynamic Data (DD)

Steve Naughton’s Cascading Filters and Fields – Dynamic Data Entity Framework Version of 4/4/2009 begins:

I wanted to do this in EF and Preview 3 at the same time but I having an issue with that so I’m combining both the Cascading Fields and Filters together and when the bugs are ironed out of the Preview I’ll do it there also.

Miscellaneous (WPF, WCF, MVC, Silverlight, etc.)

Charlie Calvert explains why he’s interested in Silverlight in his Silverlight Revisited post of 4/2/2009:

  1. It allows me to create a rich interface with sophisticated visual techniques including animation.
  2. It provides a thoroughly modern programming model, and a set of rich API’s for performing common programming tasks.

Mike Flasko offers guidance in Using ADO.NET Data Services v1.5 CTP1 with Silverlight 2 and Silverlight 3 in this 3/29/2009 post. (Copied from the “ADO.NET Data Services (Astoria)” section.)