Windows Azure and Cloud Computing Posts for 5/10/2010+

Windows Azure, SQL Azure Database and related cloud computing topics now appear in this weekly series.

 
Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:

• Azure Blob, Table and Queue Services
• SQL Azure Database, Codename “Dallas” and OData
• AppFabric: Access Control and Service Bus
• Live Windows Azure Apps, APIs, Tools and Test Harnesses
• Windows Azure Infrastructure
• Cloud Security and Governance
• Cloud Computing Events
• Other Cloud Computing Platforms and Services

To use the above links, first click the post’s title to display the single article you want to navigate.

Cloud Computing with the Windows Azure Platform published 9/21/2009. Order today from Amazon or Barnes & Noble (in stock.)

Read the detailed TOC here (PDF) and download the sample code here.

Discuss the book on its WROX P2P Forum.

See a short-form TOC, get links to live Azure sample projects, and read a detailed TOC of electronic-only chapters 12 and 13 here.

Wrox’s Web site manager posted on 9/29/2009 a lengthy excerpt from Chapter 4, “Scaling Azure Table and Blob Storage” here.

You can now download and save the following two online-only chapters in Microsoft Office Word 2003 *.doc format by FTP:

• Chapter 12: “Managing SQL Azure Accounts and Databases”
• Chapter 13: “Exploiting SQL Azure Database's Relational Features”

HTTP downloads of the two chapters are available from the book's Code Download page; these chapters will be updated in May 2010 for the January 4, 2010 commercial release. 

Azure Blob, Table and Queue Services

No significant articles today.

<Return to section navigation list> 

SQL Azure Database, Codename “Dallas” and OData

Bill Zack announced the availability of an On-Line Clinic: Introduction to Microsoft SQL Azure on 5/10/2010:

This two-hour on-line SQL Azure clinic explores the capabilities of Microsoft SQL Azure. It covers:

• Understanding the SQL Azure Platform
• Designing Applications for SQL Azure
• Migrating Applications to SQL Azure
• Achieving Scale with SQL Azure

Users completing this collection should have a basic understanding of SQL Server and relational database systems.

At the end of the course, students will be able to:
Understand SQL Server Development
Understand relational database concepts

Find out more and how to take the course here.

Alex Williams wrote How Darwin's Finches and API's Are Connected in the Post 20th Century Economy, which applies to OData.org’s OData Protocol, on 5/10/2010:

A lot get said about the economic value of API's but few have illustrated it as well as Sam Ramji did last week at Web 2.0 in San Francisco.

Ramji, vice president of strategy at Sonoa Systems, used his presentation at the Web 2.0 conference to compare Darwin's theories about finches to the 20th century economy and the rise of API's.

Darwin's Finches, 20th Century Business, and APIs

Darwin made the connection between adaption and evolution by studying the finches on the Galápagos Islands. The Galapagos Islands are a remote place. As Ramji points out, the finches that survived were the ones who adapted best to the severe environmental pressures of the islands hey inhabited.

Much is true for the evolution of 20th century business. In the early 20th century, the economy operated through a direct model. With the evolution of the business economy came a network effect that lead to new channels for physical goods. This shift in the economy meant the emergence of supply chains and ecosystems that ran on infrastructures made of roads, railways and flight paths.

Then the Internet emerged. The Web arrived about 15 years ago. People and the businesses where they worked went online with their own Web sites.

Today, the infrastructure is far deeper with API's connecting loosely joined networks. These API's represent a new indirect economy where business is conducted through interconnected services.

Ramji: I believe that we're going through such a surge right now as the early versions of the web - designed for people using browsers - gives way to the next version: using APIs to design the web for people using applications that communicate on their behalf in complex ways to the services that make up the world's businesses."

Ramji is planning to do a voice over for his slides. But I think you will get the picture once you go through his deck.

We are just at the beginning of this wave. Ramji puts the issues in perspective to form a context for the importance of API's and why they so well represent cloud computing and its importance in the shaping of a new economy.

MSDN’s Online Help for WCF Data Services recently added an Accessing Data Service Resources (WCF Data Services) topic, which begins:

WCF Data Services supports the Open Data Protocol (OData) to expose your data as a feed with resources that are addressable by URIs. These resources are represented according to the entity-relationship conventions of the Entity Data Model. In this model, entities represent operational units of data that are data types in an application domain, such as customers, orders, items, and products. Entity data is accessed and changed by using the semantics of representational state transfer (REST), specifically the standard HTTP verbs of GET, PUT, POST, and DELETE.

and continues with OData syntax examples for:

• Addressing Resources
• System Query Options
• Addressing Relationships
• Consuming the Returned Feed

<Return to section navigation list> 

AppFabric: Access Control and Service Bus

Dave Kearns claims “If project pans out, it could be the greatest thing to happen to directory services since LDAP” as a preface to his Federated identity project on the horizon Security Identity Management Alert of 5/10/2010 for NetworkWorld:

From time to time in this newsletter we take a look back 10 years to see what we were talking about "back in the day." The newsletter was called "Focus on Directory Services" back then. But going back even further, before there were Network World newsletters, directory services often featured prominently in my Wired Windows column. One idea that was near and dear to my heart was the concept of "federated directories."

This was long before the Liberty Alliance (whose 10th anniversary is still over a year away) changed the meaning of federation, for good or ill.

What we were talking about in the mid 90s was a system whereby individual corporate directories -- for the visionaries, even personal directories -- could somehow "plug in" to a world-wide system and have it all look like one big directory. The benefits of having all of that data in a ubiquitous (always available) and pervasive (available anywhere) system seemed enormous. They still do. But the technology wasn't there -- nor, really, was the platform. So over time, it seemed that those of us who had longed for the federated directory model set our sights much lower. One of those who was talking about that federated model back in the mid 90s was Microsoft's Kim Cameron. And Cameron never lost sight of that vision. He recently recognized that the time might be right to start to develop the system that many of us had envisioned oh so many years ago.

Cameron and I were both at the European Identity Conference last week, where he delivered a keynote address on the federated directory project. He later elaborated on the project in what was billed as "A conversation with Kim Cameron" that I conducted as a program session and elaborated even more in a private chat we had.

This is big stuff.

This is the greatest thing to happen to directory services since LDAP. Maybe even bigger.

But, as Cameron readily admits, a few things needed to happen first. Among them are wide acceptance of Active Directory (and the whole LDAP/x.500 method), claims-based (what I call attribute-based) identification and the ubiquitous cloud platform. Without those things, the federated directory project couldn't happen. With them, it's a possibility -- but not a certainty.

So what's it all about? We'll get into that in the next issue.

Clemens Vasters points out in his Hosting Service Bus endpoints in IIS and Server AppFabric (Dublin) post of 5/9/2010 Vishal Chowdhary’s IIS hosting of Wcf Services with Servicebus Endpoints whitepaper in the MSDN Code Gallery:

This whitepaper provides details on how to host WCF services with Windows Azure Platform AppFabric Service Bus endpoints in IIS 7.5 which is available on the Windows Server 2008 R2 and Windows 7 platforms. Windows® Azure™ platform AppFabric provides common building blocks required by .NET applications when extending their functionality to the cloud, and specifically, the Windows Azure platform. We introduce the technical challenges while hosting WCF services on IIS that have a Service Bus endpoint. We then explain how by leveraging IIS 7.5 features we can successfully host WCF services in IIS & the caveats. The remainder of the paper is then divided in the following sections:

1. Hosting WCF Services with Service Bus bindings in IIS 7.5: It details what steps are needed to host a WCF service with service bus endpoints.
2. Service Bus endpoint settings in 4.0: This section describes changes needed for Dublin integration. It’s also a pre-requisite if you want to use Option 2: ASP.NET 4.0 Auto-start.
3. Windows Server AppFabric ‘Dublin’ integration: This section lists additional steps that need to be taken to take benefit of Dublin IIS features for WCF service with service bus endpoints.

Users will be able to leverage Windows® Server AppFabric ‘Dublin’ functionality to configure & monitor method invocations for the Service Bus endpoints.

<Return to section navigation list>

Live Windows Azure Apps, APIs, Tools and Test Harnesses

Bruce Kyle reports QuarkPromote Lights Up Windows 7 with RIA Web Application on Windows Azure in this brief 00:04:54 Channel9 Web interview of 5/10/2010:

QuarkPromote incorporates Windows 7 light up features for its Rich Internet Application (RIA). Users customize promotional material, business cards, and fliers using the QuarkPromote Windows Presentation Foundation (WPF) client that you get from the website. The thick clients allows users to view and customize templates and then print the materials with printer quality at nearby print shops.

John Heal, Sr. Software Developer at Quark, talks with Bruce Kyle, ISV Architect Evangelist, about how he and his team added jump lists, task bar, and progress bar to the application. He explains that it took a half day effort for his team.

And with the new versions of the code pack, you no longer provide code that figures out if it should work in a previous operating system. So without code changes, the applicatoin runs on XP, Vista, and Windows 7.

• One member on his team told me it was literally five lines of code to add support for the progress bar.
• Jump list items that John shows directs the user to various features on the website. items that John shows directs the user to various features on the website.
• Thumbnails show the content user is editing rather than the entire screen, which is the default thumbnail in Windows 7 that shows the entire application window.

John shows the features in action in the latest version of QuarkPromote.

QuarkPromote on Windows Azure

[The] QuarkPromote application is hosted in Windows Azure. For an overview of the project, see the Microsoft Showcase: Quark/Sir Speedy Azure Case Study.

 

Return to section navigation list> 

Windows Azure Infrastructure

Guy Kawasaki lists 8 Tips to Getting Started in Cloud Computing in this 5/10/2010 post to the Open Forum:

In the old days, infrastructure meant buying $5,000 servers, putting them into high-security, cooled rooms, and hiring IT folks to make them all work. And then doing this all over again so that you’d have redundancy. Those days are gone—in fact, if you tell investors that you need money for IT infrastructure, they’re going to question your intelligence.

Cloud computing is what has changed the game. It provides small businesses with the ability to deploy websites and applications quickly, to pay only for what you use, and leave all the management issues to someone else. It makes for a leaner business that can react faster to challenges and opportunities.

But cloud computing also requires understanding a whole new technology and computing philosophy. I was lucky when we put my company’s website, Alltop.com, “into the cloud” because our service provider, Rackspace, hosts hundreds of companies and our developers at Electric Pulp have done this many times. The gang at Rackspace and I have come up with a list of ten tips to help you get started in cloud computing:

1. Know the different options available to you. “Cloud computing” simply means that you pay only for what you use—like electricity, but it can be found many forms. A platform-as-a-service (PaaS for short) works well for front-end applications and websites. It takes care of a lot of the infrastructure you need to get started. An infrastructure as a service (IaaS for short) gives you access to a command line and allows you the flexibility to customize your load balancing, autoscaling, security and failover settings, but requires advanced technical knowledge in system administration.

If you don’t have someone comfortable programming in a terminal, an IaaS is not for you. Infrastructures-as-a-service require some system administration experience. If you don’t have someone with this type of expertise, either find a competent systems administrator or consider a platform-as-a-service instead. A PaaS provider should be able to handle basic but necessary tasks such as load balancing and security.

2. Understand that scaling is a skill, not a default. Cloud computing gives you a lot of flexibility to scale at a lower cost. However, no cloud provider offers “infinite scalability” out of the box though. The world’s most popular websites and applications have professionals working full-time to ensure uptime and speed when they are needed most. Be sure to factor this into your long-term IT costs.

3. Implement a disaster plan. The cloud is not fool proof, but there are ways to protect yourself should it go down. A multi-tenant cloud will go down on occasion. If uptime is crucial to your business, be sure you have a disaster recovery plan, geographic failover, and redundancy set in place.

4. Don’t be naïve. Cloud computing will not make up for a poorly written application or database structure. A hosting provider gets the blame for a lot of performance issues. If a database is not set up properly, or code is not optimized, there is nothing a hosting provider can do to make up for this. When it comes to developers, remember that you often get what you pay for. Be sure to check their resumes and portfolio for other work.

5. Budget for your specific use-case. Calculating your budget is not as simple as reading a provider’s website. Cloud computing treats hosting as a utility. Like other utilities such as electricity, your bill will vary each month with usage. If you see a surge in traffic or users, use more space, or process more information, expect to pay more at the end of the month.

6. Choose a cloud provider on your needs, not its popularity. Do you need something that is highly elastic in a short period of time? Are you going to need support and additional services? High availability? Integration with third party software? Different cloud providers excel at different things. Consider your individual needs, do your homework, and ask cloud providers questions about their availability, speed, security, and integration before you sign up.

7. Remember: some applications are not good fits for cloud. Cloud computing is great for anything you’d need to deploy quickly and at a low cost. However, just like multi-tenant buildings are not good for every business, a multi-tenant cloud is not good for every application. If you have high security or bandwidth needs, you will need to pursue dedicated gear.

For security reasons, any application that requires PCI or HIPPA compliance is not a good fit for cloud computing. A multi-tenant cloud may also not be able to handle extreme performance loads often seen by more resource intensive applications. Evaluate your specific needs and don’t rule out dedicated or hybrid hosting (a combination of cloud and dedicated hosting) if it looks like the right fit.

8. Think outside of the box. When hosting becomes a commodity, it opens your business up to new and exciting things. You can deploy applications or sites on the fly. Consider media rich or real-time elements to your application or website. Set up a server just to comb through customer information or other data your company collects. These possibilities were not as accessible to the masses before cloud computing, so don’t be afraid to try new things and expand how your business operates.

Here is more information about cloud computing from Rackspace.

Disclosure: Rackspace provides hosting for my company’s website, Alltop.

Lori MacVittie warns Don’t Throw the Baby out with the Bath Water “Or in modern technical terms, don’t throw the software out with the hardware” in this 5/10/2010 essay:

Geva Perry recently questioned one of Gartner’s core predictions for 2010, namely that “By 2012, 20 percent of businesses will own no IT assets.” Geva asks a few (very pertinent) questions regarding this prediction that got me re-reading the prediction. Let’s all look at it one more time, shall we?

“By 2012, 20 percent of businesses will own no IT assets. Several interrelated trends are driving the movement toward decreased IT hardware assets, such as virtualization, cloud-enabled services, and employees running personal desktops and notebook systems on corporate networks.

The need for computing hardware, either in a data center or on an employee's desk, will not go away. However, if the ownership of hardware shifts to third parties, then there will be major shifts throughout every facet of the IT hardware industry. For example, enterprise IT budgets will either be shrunk or reallocated to more-strategic projects; enterprise IT staff will either be reduced or reskilled to meet new requirements, and/or hardware distribution will have to change radically to meet the requirements of the new IT hardware buying points. [emphasis added]”

Geva asks: “’IT assets’ - They probably mean IT assets in the data center because aren't personal desktops and notebooks also IT assets?” That would have been my answer at first as well, but the explanation clearly states that “the need for computing hardware either in a data center or on an employee’s desk will not go away.” Is Gartner saying then that “computing hardware” is not an IT asset? If the need for it – in the data center and on the employee’s desk – will not go away, as it asserts, then how can this prediction be accurate? Even if every commoditized business function is enabled via SaaS and any custom solutions are deployed and managed via IaaS or PaaS solutions, employees still need a way to access them, and certainly they’ve got telecommunications equipment of some kind – Blackberries and iPhones abound – and those are, if distributed by the organization, considered IT assets and must be managed accordingly.

As Geva subtly points out, even if an organization moves to a BYOH (bring your own hardware) approach to the problem of client access to remotely (cloud) hosted applications, they still must be – or should be – managed. Without proper management of network access the risk of spreading viruses and other malware to every other employee is much higher. Without proper understanding of what and how the organizational data is being accessed and where it’s being stored, the business is at risk. Without proper controls on employee’s “personal” hardware it is difficult to audit security policies that govern who can and cannot access that laptop and subsequently who might have access to credentials that would allow access to sensitive corporate information.

What Gartner seems to be saying with this prediction is not that hardware will go away, but that the ownership of the hardware will go away. Organizations will still need the hardware – both on the desktop and in the data center – but that they will not need to “own” the IT hardware assets. Notice that it says “will own no IT assets” not “need no IT assets.”  That said, the prediction is still not accurate as it completely ignores that there are more “IT assets” than just hardware. …

Lori concludes with a “WHAT is an IT ASSET ANYWAY?” topic.

James Hamilton’s detailed Inter-Datacenter Replication & Geo-Redundancy essay of 5/10/2010 begins:

Wide area network costs and bandwidth shortage are the single most common reason why many enterprise applications run in a single data center. Single data center failure modes are common. There are many external threats to single data center deployments including utility power loss, tornado strikes, facility fire, network connectivity loss, earthquake, break in, and many others I’ve not yet been “lucky” enough to have seen. And, inside a single facility, there are simply too many ways to shoot one’s own foot. All it takes is one well intentioned networking engineer to black hole the entire facilities networking traffic. Even very high quality power distribution systems can have redundant paths taken out by fires in central switch gear or cascading failure modes. And, even with very highly redundant systems, if the redundant paths aren’t tested often, they won’t work. Even with incredibly redundancy, just having the redundant components in the same room, means that a catastrophic failure of one system, could possibly eliminate the second. It’s very hard to engineer redundancy with high independence and physical separate of all components in a single datacenter.

With incredible redundancy, comes incredible cost. Even with incredible costs, failure modes remain that can eliminate the facility entirely. The only cost effective solution is to run redundantly across multiple data centers. Redundancy without physical separation is not sufficient and making a single facility bullet proof has expenses asymptotically heading towards infinity with only tiny increases in availability as the expense goes up. The only way to get the next nine is have redundancy between two data centers. This approach is both more available and considerably more cost effective.

Given that cross-datacenter redundancy is the only effective way to achieve cost-effective availability, why don’t all workloads run in this mode? There are two main blocker for the customer I’ve spoken with: 1) scale, 2) latency, and 3) WAN bandwidth and costs.

James continues with a description of Infineta System’s Infineta Velocity engine and concludes:

The final [Infineta] technique wasn’t described in enough detail for me to fully understand it. What Infineta is doing is avoiding the cost of fully hashing each packet but taking an approximate signature of carefully chosen packet offsets. Clearly you can take a fast signature on less than the full packet and this signature can be used to know that the packet is not in the index on the other side. But, if the fast hash is present, it doesn’t prove the packet has already been sent. Two different packets can have the same fast hash. Infineta were a bit cagey on this point but what they might be doing is using the very fast approx has to find those that have not yet been sent unambiguously. Using this technique, a fast hash can be used to find those packets that absolutely need to be sent so we can start to compress and send those and waste no more resources on hashing. For those that may not need to be sent, take a full signature and check to see if it is on the remote site. If my guess is correct, the fast hash is being used to avoid spending resources quickly on packets that are not in the index on the other side.

• Infineta looks like an interesting solution. More data on them at:
• Press release: http://www.infineta.com/news/news_releases/press_release:5585,15851,445
• Web site: http://www.infineta.com
• Announcing $15m Series A funding: Infineta Comes Out of Stealth and Closes $15 Million Round of Funding http://tinyurl.com/2g4zhbc

Bernard Gordon points out a recent IDC report by asking Cloud Computing: How big is big data? and delivering IDC's answer to CIO.com.au on 5/8/2010:

I came across a link to a new report from IDC called the "2010 Digital Universe Study". The report echoes what we've been telling our clients for the past year: the projections of the past few years about the growth of data significantly underestimate how much data is going to be created.

Some highlights of the report:

• In 2010, the Digital Universe (a fancy term for all the data created by consumers and businesses on earth, including video, audio, documents, etc.) will grow by 1.2 zettabytes, or 1.2 million petabytes.
• By 2020, the Digital Universe will be 44 times as large as it was in 2009.
• Surprisingly, the number of objects (i.e., files that contain digital data) will increase faster than the total amount of data, due to smaller file sizes - even though lots of large video and audio files are being created, so are massive amounts of small files created by devices, sensors, etc.

The report goes on to highlight some of the biggest issues the future torrent of data will pose:

• Searching: How to find a digital needle in a gigantic data haystack? Most of the data will be unstructured, implying new kinds of searching mechanisms are required.
• Data Tiers: If you thought Hierarchical Storage Management was important before, imagine how necessary it will be in the face of zettabytes of data. A strategy to define a layered approach to storage, based on historical use, immediacy of need, and cost of storage will be necessary.
• Privacy and Compliance: How can the increasing requirements of privacy and compliance be controlled with so much data under management?
• Headcount Mismatch: While the amount of data will increase 44 times, and the number of files will increase 67 times, the number of employees will increase by only 1.4 times.

The report notes that by 2020, much of this data will be held in cloud environments or will be "touched by cloud," which means data that transits through a cloud service or is temporarily held in a cloud application. The report estimates that perhaps 15% of all data will be held in the cloud, and that around one-third will live in or pass through the cloud. Frankly, I think that underestimates what's going to be in the cloud, for this reason:

It's clear that the growth of data is accelerating, which is to say that much of it will be created later in the 2010 - 2020 decade. This means that the average corporation is going to experience an increasing deluge of data - in other words, no matter what level of investment they've already got in storage, it will be accelerating as the decade goes on. This will require ever-increasing amounts of storage and an ever-increasing capital budget for storage devices - not to mention more headcount. There's a truism in economics that something that can't go on, won't go on. I just don't see most companies funding an ever-increasing number of storage devices and employees to manage them, i.e., most companies can't afford the projected growth of storage, so they won't go down the road of on-site storage. Long before they get to the logical conclusion of how much investment, capital, and headcount is required to manage the increased storage, they'll turn to specialized providers who have figured out how to manage enormous amounts of storage more cost-effectively.

Another reason the report underestimates how much data will be in the cloud is that much of the data will, increasingly, originate in the cloud, because of the use of SaaS applications and the hosting of custom applications in IaaS clouds. Just as the rate of change in storage amounts will increase through the decade, so too will the number of cloud applications - which means the data associated with those apps will be created in the cloud to begin with. Another way to look at this is: what proportion of applications do you think will reside in external cloud environments by 2020? I'm betting it's significantly more than 15% of all apps.

The report then turns to privacy and compliance issues and concludes that, despite the best efforts of IT groups, the proportion of data left inadequately protected will increase throughout the next decade, due to the lack of investment made available by the business units that fund central IT expenditure. Unless driven by specific legal requirements (e.g., SarbOx) or actual data breaches, data privacy and compliance runs a poor second (and a long way back) to the functionality requirements of business units. And this is not to mention the use of external cloud providers by business units, which makes eliding IT data security requirements even easier.

The report concludes with some predictions:

• The increased complexity of managing digital information will be an incentive to move to cloud services.
• Within data centers, expect continued pressure for data center automation, consolidation, and virtualization.
• Expect more end-user self-service.
• Expect bottlenecks in key specialties such as security, information management, advanced content management, and real-time processing.

If you work in IT, you owe it to yourself to read this report and consider its implications. I may sound like a broken record, but the future of IT is going to look a lot different than the past -- even the recent past. This report offers information to guide your strategy. We've put together some specific advice about steps you should be taking now to prepare for the very different future heading toward us. If you're interested in getting a jump start on "The Digital Universe," take a look here.

<Return to section navigation list> 

Cloud Security and Governance

Marcia Kaufman asks Are you bypassing CIO policies to access cloud services? in this 5/10/2010 post to Judith Hurwitz's Cloud-Centric Weblog:

I recently spoke with a CIO of a large and highly regulated organization about his company’s experiences with cloud computing. Security and compliance issues are top priorities for this CIO causing the company’s leadership to move with caution into the cloud. He expects that all cloud implementations throughout the enterprise – from Software as a Service (SaaS) to Infrastructure as a Service  (IaaS) and Platform as a Service (PaaS) will receive prior approval from his office. This CIO is implementing the same approach to security and compliance that he has taken with every project undertaken within the company. In other words, security must be implemented following a centralized approach in order to ensure that information governance policies are upheld.   The company’s cloud experiences so far have included the on-demand purchase of extra compute power and storage for development and test on two small projects as well as use of Salesforce.com in several business unit sales teams. Overall, he feels confident about the level of control he has when it comes to managing cloud security issues, and understanding the potential impact of the evolving cost and economic models of cloud computing.

More

Marcia Kaufman, COO and Partner at Hurwitz & Associates has joined my blog as a collaborator. Marcia has great insights into compliance, governance, and security in the cloud.

Reuven Cohen takes on Amazon Web Service’s 5/8/2010 outage in his Failure as a Service post of 5/10/2010:

A recent seven hour outage at Amazon Web Services on Saturday has renewed the discussion about cloud failures and whether the customer or the provider of the services should be held responsible. The conversation stems from two power outages on May 4 and an extended power loss early on Saturday, May 8. Saturday’s outage began at about 12:20 a.m. and lasted until 7:20 a.m., and affected a “set of racks,” according to Amazon, which said the bulk of customers in its U.S. East availability zone remained unaffected.

In one of the most direct posts, Amazon EBS sucks I just lost all my data, Dave Dopson said "they [AWS] promise redundancy, it is BS." Going on to point to AWS's statement " EBS volumes are designed to be highly available and reliable. Amazon EBS volume data is replicated across multiple servers in an Availability Zone to prevent the loss of data from the failure of any single component. The durability of your volume depends both on the size of your volume and the percentage of the data that has changed since your last snapshot. As an example, volumes that operate with 20 GB or lessof modified data since their most recent Amazon EBS snapshot can expect an annual failure rate (AFR) of between 0.1% –0.5%, where failure refers to a complete loss of the volume. This compares with commodity hard disks that will typically fail with an AFR of around 4%, making EBS volumes 10 times more reliable than typical commodity disk drives."

Like many new users to cloud computing, he assumed that he could just use the service and upon failure AWS's redundancy would automatically fix any problems, because they do (sort of) say that they prevent data loss. What Amazon actually states is a little different in that "the durability of your volume depends both on the size of your volume and the percentage of the data that has changed since your last snapshot" placing the responsibility on the customer. On one hand they say that they prevent data loss, but only if you use the AWS cloud correctly, otherwise you're SOL. The reality is that AWS for most users requires significant failure planning -- in this case the use of EBS's snap shot capability. The problem is that most [new] users have a hard time learning the rules of the road. A quick search for AWS failure planning on the AWS forums resulted in little additional insights and really appears to mostly about trial and error.

In the case of hardware failures Amazon expects you to design your architecture correctly for these kinds of events by use of redundancy, for example use mutliple VM's etc. They expect a certain level of knowledge of both system administration as well as how AWS itself has been designed to be used. Newbies need not apply or should use at you're own risk. Which isn't all that clear to a new user, who hears that cloud computing is safe and the answer to all your problems. Which I admit should be a red flag in itself. The problem is two fold, an over hyped technology and unclear failure models which combine to create a perfect storm. You need the late adopters for the real revenue opportunities, but these same late adopters require a different more gentle kind of cloud service, probably one a little more platform than infrastructure focused. As IaaS matures it is becoming obvious that the "Über Geek" developers who first adopted the service is not where the long tail revenue opportunities are. To make IaaS viable to a broader market, AWS and other IaaS vendors need to mature their platforms for a lesser type of user. (A lower or least common denominator) One who is smart enough to be dangerous, otherwise they're doomed to be limited to the only for experts only segment.

The bigger question is should a cloud user have to worry about hardware failures or should these types of failures be the sole responsiblity of the service provider? My opinion is deploying to the cloud should reduce complexity, not increase it. The user should be responsible for what they have access to, so in the case of AWS, they should be responsible for failures that are brought about by the applications and related components they build and deploy, not by the hardware. If hardware fails (which it will) this should be the responsibility of those who manage and provide it. Making things worst is promising to be highly available, reliable and redundant, but with the fine print of "if you are smart enough to use all our services in the proper way" which isn't fair. If EBS is automatically replicated why did Dave lose all his data?

In a optimal cloud environment any single server failures shouldn't matter. But it appears at AWS it does.

Most Platform as a Service (PaaS) vendors, including Microsoft, claim to handle these issues for you by data redundancy. See James Hamilton’s detailed Inter-Datacenter Replication & Geo-Redundancy essay of 5/10/2010 in the Cloud Security and Governance section for another angle on data loss during IaaS (or PaaS) outages.

Eric Engleman reports Amazon.com responds to recent cloud computing disruptions in this 5/10/2010 article for the Puget Sound Business Journal:

Amazon.com is promising unspecified changes to its data center power architecture to reduce the number of disruptions for its cloud computing customers. In a lengthy explanation of a Saturday power outage that caused problems for some users of its EC2 cloud service, Amazon on its web services dashboard writes:

"We realize that unavailability of instances can impact customers' applications, and we design our datacenters to provide highly reliable power. However, as a result of the recent power events, we are working on non-intrusive changes to our power distribution architecture to significantly reduce the number of instances that can be affected by failures like we have seen in the last week. This work is under way and will be carefully performed over the coming months throughout our datacenters."

Amazon did not specify what the "non-intrusive changes" will entail.

According to updates on Amazon's dashboard, a power outage to a "set of racks" in the U.S. East coast region caused disruptions for some seven hours Saturday. There was also a separate and unrelated disruption for "a very small number" of Amazon EBS (Elastic Block Storage) users. The company also reported "power related issues" for some EC2 users on May 4.

Data Center Knowledge has a detailed breakdown of the recent incidents. Amazon reminds customers that they can spread applications across multiple Amazon Web Services "availability zones" and use Amazon CloudWatch and AutoScaling to recover from "instance failures."

Amazon's work on data center power systems comes as it tries to interest more large companies and government agencies in its cloud services. Such institutions often have stringent requirements for security and reliability of data.

<Return to section navigation list> 

Cloud Computing Events

Greg Duncan suggest watching Cloud Ready” on demand - Windows Azure FireStarter event recordings now available (We’re talking 8 hours of event video…) in this 5/10/2010 post:

ISV Developer Community - Windows Azure FireStarter Videos Up on Channel 9

“The videos from the Windows Azure FireStarter event in Redmond are now available for on Channel 9. You can watch or click to see the videos: …”

Windows Azure FireStarter

“… Maybe you've already watched a webcast, attended a recent MSDN Event on the topic, or done your own digging on Azure. Well, here's your chance to go even deeper. This one-of-a-kind event will focus on helping developers get ‘cloud ready’ with concrete details and hands-on tactics. We’ll start by revealing Microsoft’s strategic vision for the cloud, and then offer an end-to-end view of the Windows Azure platform from a developer’s perspective. We’ll also talk about migrating your data and existing applications (regardless of platform) onto the cloud. We’ll finish up with an open panel and lots of time to ask questions. …

• 8:30 am - 9:00 am Event Kickoff View Now >>
• 9:00 am - 10:30 am Platform Overview for Developers View Now >>
• 10:45 am - 11:45 am Windows Azure Storage View Now >>
• 11:45 am - 12:30 pm SQL Azure View Now >>
• 1:00 pm - 2:30 pm Building Applications for the Cloud View Now >>
• 2:45 pm - 4:15 pm Migrating Applications to the Cloud View Now >>
• 4:15 pm - 5:00 pm Panel and Close View Now >>

That’s a good bit of Azure goodness…

Angelbeat announced on 5/10/2010 that its 10th annual tour will hit San Francisco (actually Burlingame) on 6/18/2010. Following is the schedule for cloud-related sessions:

7:30 - 8:30: Breakfast, Registration, Product Demonstration

…

9:15 - 9:40 Cloud Computing, Storage/Server/Data Center Virtualization, Storage/Backup/Disaster Recovery, Data Deduplication

• Virtualization capabilities for storage, servers and overall data centers, to save energy/money and run multiple applications and operating systems (Windows, Linux, Unix) independent of hardware platform
• Server consolidation, reduced infrastructure/support expenses and Green IT
• Cloud Computing architecture, including private/in-house vs public/outsourced clouds
• Options for Cloud Deployment/What to OutSource: All Applications or On-Demand Resources to Handle Peak Loads? All Storage/Data Center/NOC or Just Backup/Archive?
• Strategies for securing, patching and updating virtual and cloud environments
• Integration of storage, server, network and virtualization platforms/architecture
• Disaster recovery, archiving & business continuity; distinction between backing up & actually recovering data
• Data de-duplication deployment on primary and secondary storage, elimination of tape, and corresponding benefits
• Email archiving/storage solutions for compliance with HIPAA, Federal Rules of Civil Procedure, and other regulatory standards

…

10:15 - 10:30: 802.11n High-Speed Wireless, Managed from the Cloud

• 802.11n high-speed wireless
• Transceiver architecture for cost-effective and wide-scale deployment
• Cloud-based wireless management console
• Integrated wired/wireless security10:30 - 11:00: Break, Product Demonstrations

…

11:00 - 11:30: Amazon Web Services
Since early 2006, Amazon Web Services (AWS) has provided companies of all sizes with an infrastructure web services platform in the cloud. With AWS you can requisition compute power, storage, and other services–gaining access to a suite of elastic IT infrastructure services as your business demands them. With AWS you have the flexibility to choose whichever development platform or programming model makes the most sense for the problems you’re trying to solve. You pay only for what you use, with no up-front expenses or long-term commitments, making AWS the most cost-effective way to deliver your application to your customers and clients. And, with AWS, you can take advantage of Amazon.com’s global computing infrastructure, that is the backbone of Amazon.com’s multi-billion retail business and transactional enterprise whose scalable, reliable, and secure distributed computing infrastructure has been honed for over a decade.

Using Amazon Web Services, an e-commerce web site can weather unforeseen demand with ease; a pharmaceutical company can “rent” computing power to execute large-scale simulations; a media company can serve unlimited videos, music, and more; and an enterprise can deploy bandwidth-consuming services and training to its mobile workforce.

11:30 - 12:00: Microsoft Desktop: Windows 7 plus Office 2010 Preview, Integrated Offline, Online, Cloud Functionality
In designing Windows 7, the engineering team had a clear focus on what we call ‘the fundamentals’: performance, application compatibility, device compatibility, reliability, security and battery life. Early reviews of Windows 7 seem to indicate that the choice to focus on the fundamentals is resonating well with many users and professionals. And, IT professionals will further benefit from the enhancements to manageability and security. You’ll also learn how your investments in testing and evaluating Windows Vista will pay off in the transition to Windows 7. Come see firsthand what all the buzz is about in this demo-intensive session where we explore the UI improvements, performance gains, and manageability enhancements in the next client operating system from Microsoft.

Office 2010 includes significant upgrades and enhancements to its main components - Word, Excel, Outlook, Powerpoint - and now provides a consistent experience across the PC, phone and browser. This integrated and seamless strategy allows you to use same application offline or online, individually or in collaboration with others, on the cloud or at your desktop.

12:00 - 12:30: Windows Azure – Microsoft’s Cloud Operating System
The Windows Azure platform offers a flexible, familiar environment for developers to create cloud applications and services. With Windows Azure, you can shorten your time to market and adapt as demand for your service grows. Windows Azure offers a platform that is easily implemented alongside your current environment.

• Windows Azure: operating system as an online service
• Microsoft SQL Azure: fully relational cloud database solution
• Windows Azure platform AppFabric: connects cloud services and on-premises applications

Organizations are already using Windows Azure to:

• Run enterprise workloads in the cloud
• Build, modify, and distribute scalable applications with minimal on-premises resources
• Perform large-volume storage, batch processing, intense or large-volume computations
• Create, test, debug, and distribute Web services quickly and inexpensively

Register:
CLICK HERE TO REGISTER or call our offices at 516-277-2057.
FREE to IT professionals in networking, security, storage, architecture, telecommunications, etc. Click here for more details.

Date: 06/18/2010 from 7:30 AM to 1:00 PM

Location: Hyatt Regency San Francisco Airport (Free Parking & WiFi) 1333 Bayshore Hwy., Burlingame, CA 94010

Bill Zack recommends SQL Azure: Frequently Asked Questions in this 5/10/2010 post to the InnovationShowcase blog:

This paper provides an architectural overview of the SQL Azure Database, and describes how you can use SQL Azure to augment your existing on-premises data infrastructure or as your complete database solution.

SQL Azure Database is a cloud based relational database service from Microsoft. SQL Azure provides relational database functionality as a utility service. Cloud-based database solutions such as SQL Azure can provide many benefits, including rapid provisioning, cost-effective scalability, high availability, and reduced management overhead. This paper provides an architectural overview of SQL Azure Database, and describes how you can use SQL Azure to augment your existing on-premises data infrastructure or as your complete database solution.

You can download the white paper here

CloudCamp announced in a 5/10/2010 CloudCamp Organizer to Hold Agile Operations and Cloud Computing Event in San Francisco press release:

What: OpsCamp San Francisco When: Saturday, May 15, 2010 from 9:00 am to 5:30 pm Where: 139 Townsend St., 3rd, Floor, San Francisco, CA 94107 CloudCamp, an organizer of local events where early adapters of Cloud Computing technologies exchange ideas, will be conducting a cloud operations and IT unconference, OpsCamp San Francisco, on May 15th in San Francisco. The event is free to attend, but all participants must register early.

The OpsCamp event series brings together IT professionals who are interested in the evolution of systems management and application deployment, including emerging Agile software development principles and IT operations. The event will be a participant-driven unconference format, made popular by events like BarCamp, Bloggercon and Mashup Camp. The program commences with a short panel discussion on the current state of IT operations followed by user-led discussions related to deploying applications and managing infrastructure in today's evolving data centers. …

Event Details The event kicks off with a discussion about cloud computing, followed by a self-organizing conference format where topics are proposed and then voted on by the attendees.
Breakfast, lunch, snacks and beverages will be provided. While attendance is free, RSVP is required: http://www.opscamp.org/sf Supporting Quotes from the Sponsors The following organizations have committed their time and resources to help host the event and to facilitate the conversation around Cloud Computing and IT operations management. …

Power and Cooling Capacity Management for Data Centers Learn more, download free white paper.

Bill Zack announced the availability of an On-Line Clinic: Introduction to Microsoft SQL Azure on 5/10/2010:

This two-hour on-line SQL Azure clinic explores the capabilities of Microsoft SQL Azure. It covers:

• Understanding the SQL Azure Platform
• Designing Applications for SQL Azure
• Migrating Applications to SQL Azure
• Achieving Scale with SQL Azure

Users completing this collection should have a basic understanding of SQL Server and relational database systems.

At the end of the course, students will be able to:
Understand SQL Server Development
Understand relational database concepts

Find out more and how to take the course here.

Repeated from the SQL Azure Database, Codename “Dallas” and OData section.

<Return to section navigation list> 

Other Cloud Computing Platforms and Services

Prasanth Rai’s Rackspace Cloud Focused Q1 – 2010 post of 10/1/2010 analyzes Rackspace’s latest quarterly report:

was just looking through the Quarterly results of Rackspace, some interesting excerpts

hosting and cloud computing represent a massive virtually untapped market opportunity for Rackspace. We made good progress in the quarter toward capturing a larger share of its market.

There is a tectonic shift underway that is fundamentally altering the way the world consumes computing resources. We expect the magnitude of this change to eclipse that of past technology era, such as mainframe computers and PCs, simply because there is especially more computing power in the world today.

We are the leader in the hosting and cloud computing industry, and our mission is to aggressively pursue this emerging multi-year,

multi-billion dollar market opportunity.

- Lanham Napier – Rackspace Hosting Inc. – CEO and President

Revenue & Customers

Cloud revenue for the first quarter was $19 million, accelerating to 13% sequential growth from the fourth quarter and 77% year-over-year growth from the first quarter of 2009. Managed hosting revenue increased to $160 million, accelerating to 5% sequential growth from the fourth quarter and a 19% year-over-year from the first quarter of 2009.

We were pleased to see managed hosting customer additions turn positive for the first time in three quarters, even though we actively migrated 135 single server customers to our cloud offering in the quarter.

…

It seems clear from the graphics that the revenue per customer for managed hosting is considerably larger than that for that for cloud customers. It appears to me that cloud customers are simply testing the water and aren’t running highly trafficked apps in Rackspace’s cloud.

Eric Knorrr claims “IBM's cloud computing road map is coming together, with a new Raleigh, N.C., cloud facility set to go live this summer” in his Get ready for the IBM Cloud article for the 5/10/2010 issue of InfoWorld’s CloudComputing blog:

I see IBM as the sleeping giant of cloud service providers. Large enterprises don't take Amazon seriously, and they'll put up with Salesforce if they have to, but IBM is a different story. Anything you pay that much for has got to be beyond reproach.

Last week at the IBM Impact show in Las Vegas, I spoke with Steve Mills, IBM's head of software, and Walter Falk, IBM business development executive for cloud computing, about IBM's plans for cloud services. The news of the week was that the company had acquired Cast Iron Systems, makers of a cloud integration appliance primarily designed to help customers integrate SaaS apps with their local enterprise applications. IBM also opened up a big cloud research facility in Singapore and unveiled a new cloud certification program. …

But for me the big story is that launch time is approaching for IBM Smart Business Development and Test on the IBM Cloud, as it is awkwardly named. As Walter Falk told me, "It's all being built out of the Raleigh data center for the North American market. We are very close to going live. We are right now in beta and we have more than a 100 customers and partners testing it out. It's relatively quiet, I would say, because we really want to build this rock solid."

"Quiet" is the operative word to describe IBM's approach to cloud services. Most people don't know, for example, that IBM offers hosted desktop virtualization in the form of Smart Business Desktop on the IBM Cloud. Why the low-key approach? Well, for one thing, IBM has provided the infrastructure for many SaaS players over the years and probably does not want to be seen as competing with customers. For another, I think IBM wants to be very, very certain its cloud services are seen as enterprise-class.

When I asked Steve Mills about that, he was emphatic: "Yes, we have to provide guarantees. We have to do that. That's what we're expected to do." He is equally insistent that IBM believes in the hybrid cloud model, where customers' private cloud services integrate with the public cloud. No company of any size is going to deep-six the data center and shove everything on the public cloud. …

Eric continues with his paean to Big Blue’s cloud efforts, and concludes:

"This summer you will see the general availability of the IBM Cloud," says Falk, but he's not willing to provide specifics beyond the initial dev and test offering. The most I could get out of him was that the IBM Cloud will be "focusing on industry-oriented services and capabilities." Makes sense, given that over the past few years the company has pushed ever deeper into frameworks and business processes for vertical industries.

So get ready for the IBM Cloud. And it's coming with a big fat blue guarantee.

Guarantee of what? That IBM’s cloud is big and fat?

<Return to section navigation list> 

Technorati Tags: Windows Azure, Windows Azure Platform, Azure Services Platform, Azure Storage Services, Azure Table Services, Azure Blob Services, Azure Queue Services, SQL Azure Database, SADB, OData, Open Data Protocol, Windows Azure AppFabric, Cloud Computing, Amazon Web Services, Rackspace, IBM