OakLeaf Systems: Windows Azure and Cloud Computing Posts for 4/3/2010+

Sunday, April 04, 2010

Windows Azure and Cloud Computing Posts for 4/3/2010+

Windows Azure, SQL Azure Database and related cloud computing topics now appear in this weekly series.

Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:

Azure Blob, Table and Queue Services
SQL Azure Database, Codename “Dallas” and OData
AppFabric: Access Control and Service Bus
Live Windows Azure Apps, APIs, Tools and Test Harnesses
Windows Azure Infrastructure
Cloud Security and Governance
Cloud Computing Events
Other Cloud Computing Platforms and Services

To use the above links, first click the post’s title to display the single article you want to navigate.

Cloud Computing with the Windows Azure Platform published 9/21/2009. Order today from Amazon or Barnes & Noble (in stock.)

Read the detailed TOC here (PDF) and download the sample code here.

Discuss the book on its WROX P2P Forum.

See a short-form TOC, get links to live Azure sample projects, and read a detailed TOC of electronic-only chapters 12 and 13 here.

Wrox’s Web site manager posted on 9/29/2009 a lengthy excerpt from Chapter 4, “Scaling Azure Table and Blob Storage” here.

You can now download and save the following two online-only chapters in Microsoft Office Word 2003 *.doc format by FTP:

Chapter 12: “Managing SQL Azure Accounts and Databases”
Chapter 13: “Exploiting SQL Azure Database's Relational Features”

HTTP downloads of the two chapters are available from the book's Code Download page; these chapters will be updated for the January 4, 2010 commercial release in April 2010.

Azure Blob, Table and Queue Services

No significant articles today.

SQL Azure Database, Codename “Dallas” and OData

Doug Purdy (@douglasp) asks for input on the issue of Should We Open Source OData Server? on 4/3/2010:

Miguel [de Icaza (@migueldeicaza)] has called for us to open source our .NET OData Server implementation.

It is something we have talked about doing in the run up to MIX and something we continue to discuss.

Broadly, I agree with much of Miguel’s argument. Expect to hear more on this topic from us.

As we explore open sourcing our .NET OData server code, the question that I am pondering is if that would be enough to kick-start an ecosystem of OData services on Unix.

Do we need to have an OData implementation for PHP &| Python as well?

What if we both open sourced our .NET implementation and also spooled up a PHP &| Python project?

If we did decide to do something like this and we could only pick one other language/runtime beyond .NET, which one should it be?

Interested in any and all thoughts on this topic.

Here are salient excerpts from Miguel’s post of 3/23/2010:

Microsoft is trying to succeed where web services failed before with their new Open Data Protocol (oData).

A few years ago, I had an epiphany when listening to Don Box talk about data services and web services. Web services and web sites are merely public facades to your internal databases. Web sites expose your database rendered as HTML while web services expose them in some sort of data format, defined on a case-by-case basis.

With Web Services, developers need to define what bits of information should be exposed and how it is exposed (this applies to both SOAP and REST-based services). The problem is that developers might not have the time to spend on exposing every possible bit of interesting information that you maintain in your databases. Most importantly, there are very few web services that provide server-side queries making the information harder to mine.

oData is a little bit different, for a given oData endpoint (like http://odata.netflix.com/Catalog/CatalogTitles) you can request that the server do some work for you, for example, you can pass a query that will limit the results to only those that you are interested in, and you can also specify the information that you are interested in. …

Producing oData: the need to open source System.Data.Services

This is trickier, and I believe that if Microsoft wants to bootstrap the oData universe they need to seed this space not only with some existing services but also with an open source OData server implementation for Unix. [Emphasis added.]

I should further add that they should not wait a single minute and open their server-side .NET implementation if they want to accelerate the oData adoption. …

Miguel is VP Developer Platform at Novell and the leader of the Mono Project, which has “the goal to implement Microsoft's new .NET development platform on Linux and Unix-like platforms” [from Wikipedia.]

I voted for *nix and Java.

Manish Kungwani posted OData: Writing an OData API using Entity Framework & WCF Data Services on 4/4/2010:

In the previous post I talked about OData, what it is, and what it can do …

Now, to use the OData protocol to share data, we need to create an OData service which can make data available to consumers. In this post I will show you how to do this using Entity Framework and WCF Data Services.

Step 1: Create a Database with the schema of the data you want to expose to consumers …

Step 2: Create a Web project and add an Entity Framework Data Model to the project for the database created earlier.

Step 3: Make sure that all the relations and associations from the database are mapped into the data model so that it is ready to be made available to the public.

Step 4: Add a new WCF Data Service as:

Add | New Item | Web | WCF Data Service

Give it the name MyService.svc.

After this, all you need to do to have a full, working OData service is add a class in between the angle brackets (DataService<YourTypeHere>) and include one line for config.EntitySetAccessRule. …

Manish continues with the code for a simple minimal class. His previous post is OData: Oh its Data….. of the same date.

Rob Sanders’ Reviewing SQL Server Modelling CTP (Nov 09) post of 4/4/2010 provides and overview of the “Oslo” suite and the “m” modeling language:

You may have heard of the upcoming SQL Server Modelling suite formerly known as ‘Oslo’. The suite, know known as the SQL Server Modelling CTP, features the ‘m’ modelling language and a graphical workbench known as (or codenamed) ‘Quadrant’. I’ve been meaning to look into codename ‘Oslo’ for a while now, and I’ve finally had a chance to review how it works this weekend.

Designing for SQL Server has always been a fairly subjective art form. Microsoft has not exactly been kind to data architects, continually changing its position on the how and why of data modelling tools. Once upon a time, a database architect could rely on tools like Visio (for Enterprise Architects) for forward and reverse engineering database schema.

Microsoft is endeavouring to build a more comprehensive set of tools for what we term “DSL” languages or domain specific languages. This essentially allows data architects to focus on business specific (or domain specific) architecture and to abstract away from the physical implementation of persisted storage in large enterprise designs. …

Rob continues with the details of his Oslo test drive.

Cihan Biyikoglu describes Handling Error 40552 IN SQL Azure in the 4/3/2010 post:

[I]n the last few customer conversations, error 40552 came up. Hopefully the following tip can help you avoid getting this error.

Why Do I get error 40552?

SQL Azure has mechanism in place to prevent monopolizing of various system resources. One of the safeguards in place watches the active portion of the log space utilized by a transaction. Applications running transactions that use large amount of log space may receive the following error message;

Msg 40552, Level 20, State 1, Line 1
The session has been terminated because of excessive transaction log space usage. Try modifying fewer rows in a single transaction.

Avoiding 40522 when creating, rebuilding and dropping indexes:

Creating, rebuilding and dropping indexes could generate a lot of transaction log records and may hit this error msg on larger tables. You may be able to work around the issue by creating a new table with the desired index layout, and then move the data in smaller chunks over to the new table. However in most cases, you can minimize transaction log space usage with index operations by using the ONLINE option with your statement. Specifying the ONLINE=ON with CREATE, ALTER and DROP INDEX operations change the characteristics of the execution. Instead of a single large transaction, the operation is performed in multiple shorter transactions in the background without holding exclusive locks for extended periods of time on the rows. This both improves concurrency during the execution of the operation, and it eases the transaction log space requirement for the operation. This can help you avoid the 40552 exception.

AppFabric: Access Control and Service Bus

Eugenio Pace’s WIF and MVC – How it works post of 4/3/2010 begins:

I got a few questions from people on how the “Federation with Multiple Partners” sample of the guide works, so I figured I would just write it down in one place for eternity.

The guide explains all sequence in quite some detail (see pages 88 to 97), but sometimes a diagram is more helpful, so here’s a sequence diagram that describes all interactions:

An un-authenticated user browses a protected resource, say the “Shipping” page (which translates into a method in a Controller).
The Controller is decorated with the AuthenticateAndAuthorizeAttribute which implements MVC’s IAuthorizationFilter interface. Because the user is not authenticated yet, it will issue a SignInRequest to the configured issuer (this results in a redirection to the issuer). Among other tings it passes the user original URL in the context (wctx) of the request (in this example is the “Shipping” page).
The Issuer authenticates the user with whatever means and if successful, it issues a token for the user for that application.
The Token and the the context information is passed back to the app to a specific destination (indicated in the realm). In the MVC application, this is just another controller (“Home” in our sample, method “FederationResult”). This controller is not decorated by the AuthenticateAndAuthorizeAttribute.
The request however, does go through the WIF Module (the “FAM” in the diagram above). WIF will validate the token and create a ClaimsPrincipal that is eventually passed on to the controller.
The Home Controller inspects the context parameter, extracts the original URL (remember the context is preserved throughout all hops) and then redirects the user there.
The redirect will go again through the filter, but this time the user is authenticated.
Any authorization rules are performed (in our example, we checked for specific Roles) and if all checks pass…
The specific controller is finally called (e.g. Shipping).

Eugenio continues with detailed notes about his diagram.

The TechNotes blog posted excerpts from Understanding Windows Workflow Foundation by Robert Eisenberg on 4/3/2010:

Excerpted from the book http://www.informit.com/articles/article.aspx?p=1316798

Some of the benefits gained by using most workflow systems to automate the process:

Design-time transparency—It is clear what the expense report process does just by looking at it.

State management—The workflow system manages keeping track of the current step, removing the process from memory when waiting for manager approval, and going back to the approval step when required. A workflow system provides these state management features for you. In contrast, a process automated via a traditional computer language generally requires the developer to create tables and fields to manage the state themselves.

Runtime transparency—The workflow system can also visually show the current step and the execution path of the process. This feature is generally referred to as tracking (you will learn more about it in the “Tracking” section of this hour).

Many think that there is no way to graphically model application logic because any complex process will have so many control flow statements the diagram will be unreadable, eliminating the promised transparency. A process automated with a graphical workflow application should be self-describing. It should serve the same benefit that a flowchart currently supplies (or at least most of it). Just as a flowchart does not include every single control flow statement, neither should a graphical workflow.

Workflow Segmentations

Workflow is also frequently segmented along

in-application

human

integration lines

Traditional languages, like C#, are generally associated with in-application workflow. Human workflow systems arose from the desire to better support human-centric scenarios. Integration systems or integration-centric workflow systems arose from the need to better automate integration scenarios using dedicated integration systems.

Live Windows Azure Apps, APIs, Tools and Test Harnesses

Manashi Narumoto’s Trick in Windows Azure deployment of 4/3/2010 explains:

If you see web roles or worker roles are going back and forth between initializing-busy-stopping, the first thing you should check is the copy local attribute of the assemblies.

All assemblies that aren’t part of .NET framework have to be set as Copy Local=True so that they would be included in your deployment package. Otherwise they aren’t deployed to Azure ending up the problem above. I’m making this mistake just about every day :-)

His earlier Don’t rush! Worker roles will be there for you post of the same date recommends:

In dev fabric, hit F5, wait for a while then start debugging. No wonder if your worker roles don’t respond in the first place. Sometimes it takes over 30 seconds for them to start off. You should better check the dev fabric UI. When it looks like this, it’s still in the middle of its startup.

David Worthington (@dcworthington) quotes Joannes Vormorel (@vermorel) and Me (@rogerjenn) in his Coming to Azure out of the blue article of 4/1/2010 for @SDTimes:

Microsoft's Windows Azure development platform isn't exactly new (it was in beta for almost two years), but many developers are still unsure about how to get a pilot started.

Developers should think hard about what they want to achieve in the cloud and whether they want to start from scratch or migrate applications, said Joannes Vermorel, founder of Lokad, a maker of sales forecasting software. Vermorel is a popular Azure blogger and developer.

The cloud is ideal for Web applications that require a high degree of uptime, Vermorel said. " 'Always on' is nearly impossible in the classic client/server setup." It is also good for organizations that require extra capacity to handle peak loads, he said.

"We [Lokad] rented grids before Azure. We rented half a dozen powerful Web servers from month to month, never had enough processing power at peak [demand], and most of the time processing power was unused," Vermorel said. …

The average .NET developer will have no problem "picking up" basic Azure programming within a day, said Roger Jennings, OakLeaf Systems developer and author of over 30 books on Windows development topics. OakLeaf Systems is a consultancy that specializes in the Azure platform.

Some of its more sophisticated functions, such as the enterprise service bus in AppFabric, will be sticking points that will take "a bit of getting used to," he added. "Differentiating between Web roles and worker roles will take conceptual exercise on the part of the developer."

Jennings compared the difference in programming conventions between the cloud and on-premise servers to an ASP.NET developer having to learn the ASP.NET Model-View-Controller framework. …

Return to section navigation list>

Windows Azure Infrastructure

Ernest De Leon asserts Greenpeace Suffers Major Credibility [Gap] in this 4/3/2010 post:

Let me start this article off by saying that I am all for the ‘green’ movement. If we as a society can reduce what we consume, reuse as much as possible and recycle rather than throw away, I believe we will be much better off than the alternative. If we can reduce global emissions across the board and lean more heavily on renewable energy, all the better. What I don’t agree with, however, is attacking certain industries with only half of the story. It’s one thing if it was due to ignorance and a complete different monster if it was done intentionally. I’m not going to cast judgment as to which one it is, but without a doubt it is one of the two and I’ll leave it to you to decide for yourselves. By the way, I am referring to the Greenpeace report cited at the end of the article. Please look there if you want more information. …

The article to which Ernest refers is here.

Ina Fried’s Microsoft's latest small-business plan article of 3/31/2010 for CNet News’ Beyond Binary column begins:

It happens every couple of years. Microsoft's newly installed head of small-business efforts goes on the road to talk about how the company sees vast potential in the huge numbers of underserved firms that all want the capabilities of big business software without the cost or complexity. The new executive assures me that Microsoft gets it and promises Redmond is rededicating itself to the market.

This time around, the executive was Birger Steen, a Norwegian oil trader who ran Microsoft's Russian subsidiary before moving to Redmond last year to take over the small and midsize business sales effort. In addition to his unusual background, Steen also came in with a different pitch. Refreshingly, Steen said he didn't really think that small businesses are all that poorly treated.

"They are underserved but they are also well served by some people who are not us," Steen said, noting that in every market there tends to be a company like Intuit in North America that does a really good job of crafting accounting and other software products for smaller firms.

What Microsoft has the opportunity to do, he said, is to provide the operating system and Office (and related products like SharePoint and Exchange) and then partner with the Intuits of the world. In addition, he said, Microsoft can help spur the development of the next generation of extremely custom software that a dentist or car repair shop uses to run their core business.

Although Microsoft won't make that last piece of software, Steen said, it could be based on Windows Azure, Microsoft's cloud-based operating system. The cloud, Steen notes, is well suited to smaller businesses because it allows them access to the latest technology from a state-of-the-art data center that they don't have to build. …

Tim O’Reilly’s The State of the Internet Operating System essay of 3/29/2010 is a must-read for anyone interested in cloud computing. Tim asserts the following in his “The Internet Operating System is an Information Operating System” section:

Among many other functions, a traditional operating system coordinates access by applications to the underlying resources of the machine - things like the CPU, memory, disk storage, keyboard and screen. The operating system kernel schedules processes, allocates memory, manages interrupts from devices, handles exceptions, and generally makes it possible for multiple applications to share the same hardware.

As a result, it's easy to jump to the conclusion that "cloud computing" platforms like Amazon Web Services, Google App Engine, or Microsoft Azure, which provide developers with access to storage and computation, are the heart of the emerging Internet Operating System.

Cloud infrastructure services are indeed important, but to focus on them is to make the same mistake as Lotus did when it bet on DOS remaining the operating system standard rather than the new GUI-based interfaces. After all, Graphical User Interfaces weren't part of the "real" operating system, but just another application-level construct. But even though for years, Windows was just a thin shell over DOS, Microsoft understood that moving developers to higher levels of abstraction was the key to making applications easier to use.

But what are these higher levels of abstraction? Are they just features that hide the details of virtual machines in the cloud, insulating the developer from managing scaling or hiding details of 1990s-era operating system instances in cloud virtual machines?

The underlying services accessed by applications today are not just device components and operating system features, but data subsystems: locations, social networks, indexes of web sites, speech recognition, image recognition, automated translation. It's easy to think that it's the sensors in your device - the touch screen, the microphone, the GPS, the magnetometer, the accelerometer - that are enabling their cool new functionality. But really, these sensors are just inputs to massive data subsystems living in the cloud.

When, for example, as an iPhone developer, you use the iPhone's Core Location Framework to establish the phone's location, you aren't just querying the sensor, you're doing a cloud data lookup against the results, transforming GPS coordinates into street addresses, or perhaps transforming WiFi signal strength into GPS coordinates, and then into street addresses. When the Amazon app or Google Goggles scans a barcode, or the cover of a book, it isn't just using the camera with onboard image processing, it's passing the image to much more powerful image processing in the cloud, and then doing a database lookup on the results.

Increasingly, application developers don't do low-level image recognition, speech recognition, location lookup, social network management and friend connect. They place high level function calls to data-rich platforms that provide these services.

With that in mind, let's consider what new subsystems a "modern" Internet Operating System might contain [abridged]:

Search …

Media Access, including Access Control, Caching, and Instrumentation and analytics …

Communications …

Identity and the Social Graph …

Payment …

Advertising …

Location …

Activity Streams …

Time …

Image and Speech Recognition …

Government Data …

Tim continues with in-depth analyses:

What About the Browser? …
What's Still Missing …

and promises:

Next week, I'll handicap the leading players and tell you what I think of their respective strategies. [Emphasis Tim’s.]

Charles Babcock authored Cloud Connect: Grappling with Economics downloadable brief InfomationWeek Analytics:

Although the cloud has come to be viewed as part of IT's future, cloud users and vendors are still debating several key questions. If a company has virtualized its data center, is that "a private cloud?"

The answer that came back over two days of the Cloud Connect show in Santa Clara, Calif., was a definite "no," virtualization by itself is not enough.

"Virtualization is a foundation, but it doesn’t equal cloud," said Valerie Knafo, senior manager of Data Center Solutions at Dell, member of a panel on the pros and cons of adopting cloud computing.

Charles Babcock is an editor-at-large for InformationWeek, having joined the publication in 2003. He is the former editor-in-chief of Digital News, former software editor of Computerworld and former technology editor of Interactive Week.

Cloud Security and Governance

Tanya Forsheit posted Privacy, Privilege, and the Cloud, Oh My: Taking LovingCare to Heart to the InformationLawGroup (@InfoLawGroup) blog on 4/3/2010:

What does workplace privacy have to do with the cloud? Everything. On Tuesday, the New Jersey Supreme Court issued its opinion in Stengart v. LovingCare Agency, Inc., --- A.2d ----, 2010 WL 1189458 (N.J. March 30, 2010), and came out on the side of protecting employee privacy and the attorney-client privilege in personal Yahoo! webmail (a cloud service) even though the employee used a company computer. While everyone has been busy writing about the implications of LovingCare for company policies governing employee expectations of privacy (and for good reason), few have stopped to note that LovingCare is a cloud case. LovingCare is one of only a few published opinions addressing the difficult issues surrounding employee use of webmail and other cloud services on company computers where the attorney-client privilege is at stake, and the impact of the LovingCare decision will undoubtedly be felt for years to come by nearly every employer across the country, both in crafting policies for employee use of company computer systems and in conducting discovery in nearly every employment-related litigation. …

Cloud Computing Events

UBM TechEvents, Cloud Connect and InformationWeek announce The Anatomy of the Cloud: A Look Ahead, a virtual event to be held on 4/20/2010 from 7:00 AM to 3:15 PM PDT. According to Steve Kovsky, Contributing Editor & Virtual Conference Chair:

For many IT veterans, the rapid adoption of cloud computing induces an eerie feeling of déjà vu. Questions quickly arise regarding exactly how today’s cloud computing solutions differ from centralized infrastructure strategies of the past -- technologies such as mainframes, client/server, grids, ASPs and SaaS. Is there more at work here than typical industry hyperbole, and if so, what specific technologies and capabilities does computing in the cloud offer that are truly novel?

In this groundbreaking day-long virtual event, InformationWeek editors and Cloud Connect conference leaders join with internationally recognized experts and top IT practitioners from the public and private sector to dial back the hype surrounding cloud computing and demystify it once and for all. They’ll dig deep to explain how cloud computing is surpassing its architectural predecessors in the IT field, and discuss the specific business advantages that have emerged from these dynamically scalable, highly virtualized environments. …

Registered.

Other Cloud Computing Platforms and Services

Jackson Harper published the source code for his MCloud .NET client library for managing cloud providers (based on opencloud) to github on 4/4/2010. Jackson describes his library:

MCloud is a managed unified interface to various cloud providers. It is heavily based on libcloud (http://www.libcloud.net) with only small changes to make the API more .net friendly. This project is obviously not very far along but checkout test.cs to see what you can do. To run this test you need to have a linode API key. Also you should note that running this test will create/delete a new Linode on your account and that is going to cost you a days worth of Linode service (Aprox $1).

Julie Bort claims “Here's an upside of the newfound chummy relationship between Microsoft and Amazon” as a preface to delivering all the details in her Microsoft, Amazon share Windows Server licensing for the cloud post of 3/25/2010 to NetworkWorld’s Microsoft Subnet blog:

Amazon has worked out a way to run Windows in in its EC2 cloud at roughly the same price as running Linux in said cloud. If you have enterprise license agreements for on-premises Windows Server (and Software Assurance), you can use those licenses with Amazon's EC2 cloud, at least for a trial period. Microsoft announced the pilot program on Thursday.

Updated 0326: It was a rather surprising announcement. Amazon has the most popular cloud and Microsoft would like to change that and bring you to its own cloud, Azure. Although, as a reader below points out, these are apples and oranges, with EC2 being an example of infrastructure as a service (IaaS), and Azure and Google Apps Engine being examples of platform as a service. Looks like Redmond would rather have you try out a competitor the cloud and continue using Windows Server than to have you go wandering off to Linux or the likes of Google, which many Microsoft users say they are thinking of doing. (Google's Google App Engine, is free up to a certain point of usage, and doesn't support Windows or .Net.)

Amazon is also a reseller to Microsoft, and I don't just mean selling Windows 7 and Office to consumers. Amazon already offers Windows Server services on EC2. So today's deal is about allowing you to save money. If you don't have enterprise licenses with software assurance, you can still rent a Windows infrastructure from Amazon but you have to pay a set-up fee roughly equivalent to the license fees for on-prem servers. …

Julie continues with the gory licensing details.

M. Tim Jones’ Anatomy of an open source cloud of 3/9/2010 whitepaper for IBM’s developerWorks carries this summary:

Cloud computing is no longer a technology on the cusp of breaking out but a valuable and important technology that is fundamentally changing the way we use and develop applications. As you would expect, Linux® and open source provide the foundation for the cloud (for public and private infrastructures). Explore the anatomy of the cloud, its architecture, and the open source technologies used to build these dynamic and scalable computing and storage platforms.

He concludes:

Volumes could be written about the leadership role that open source is playing in the cloud and virtualization domain, but this article provided a short introduction to some of the popular and visible solutions available today. Whether you're looking to build a cloud based on your own requirements from individual pieces or simply want a cohesive solution that works out of the box, open source has you covered. See the Resources section for more details on the solutions discussed here and others.

Tim is an embedded firmware architect and the author of Artificial Intelligence: A Systems Approach, GNU/Linux Application Programming, AI Application Programming, and BSD Sockets Programming from a Multi-language Perspective.

Return to section navigation list>

Windows Azure, SQL Azure Database and related cloud computing topics now appear in this weekly series.

Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:

Azure Blob, Table and Queue Services
SQL Azure Database, Codename “Dallas” and OData
AppFabric: Access Control and Service Bus
Live Windows Azure Apps, APIs, Tools and Test Harnesses
Windows Azure Infrastructure
Cloud Security and Governance
Cloud Computing Events
Other Cloud Computing Platforms and Services

To use the above links, first click the post’s title to display the single article you want to navigate.

Cloud Computing with the Windows Azure Platform published 9/21/2009. Order today from Amazon or Barnes & Noble (in stock.)

Read the detailed TOC here (PDF) and download the sample code here.

Discuss the book on its WROX P2P Forum.

See a short-form TOC, get links to live Azure sample projects, and read a detailed TOC of electronic-only chapters 12 and 13 here.

Wrox’s Web site manager posted on 9/29/2009 a lengthy excerpt from Chapter 4, “Scaling Azure Table and Blob Storage” here.

You can now download and save the following two online-only chapters in Microsoft Office Word 2003 *.doc format by FTP:

Chapter 12: “Managing SQL Azure Accounts and Databases”
Chapter 13: “Exploiting SQL Azure Database's Relational Features”

HTTP downloads of the two chapters are available from the book's Code Download page; these chapters will be updated for the January 4, 2010 commercial release in April 2010.

Azure Blob, Table and Queue Services

No significant articles today.

SQL Azure Database, Codename “Dallas” and OData

Doug Purdy (@douglasp) asks for input on the issue of Should We Open Source OData Server? on 4/3/2010:

Miguel [de Icaza (@migueldeicaza)] has called for us to open source our .NET OData Server implementation.

It is something we have talked about doing in the run up to MIX and something we continue to discuss.

Broadly, I agree with much of Miguel’s argument. Expect to hear more on this topic from us.

As we explore open sourcing our .NET OData server code, the question that I am pondering is if that would be enough to kick-start an ecosystem of OData services on Unix.

Do we need to have an OData implementation for PHP &| Python as well?

What if we both open sourced our .NET implementation and also spooled up a PHP &| Python project?

If we did decide to do something like this and we could only pick one other language/runtime beyond .NET, which one should it be?

Interested in any and all thoughts on this topic.

Here are salient excerpts from Miguel’s post of 3/23/2010:

Microsoft is trying to succeed where web services failed before with their new Open Data Protocol (oData).

A few years ago, I had an epiphany when listening to Don Box talk about data services and web services. Web services and web sites are merely public facades to your internal databases. Web sites expose your database rendered as HTML while web services expose them in some sort of data format, defined on a case-by-case basis.

With Web Services, developers need to define what bits of information should be exposed and how it is exposed (this applies to both SOAP and REST-based services). The problem is that developers might not have the time to spend on exposing every possible bit of interesting information that you maintain in your databases. Most importantly, there are very few web services that provide server-side queries making the information harder to mine.

oData is a little bit different, for a given oData endpoint (like http://odata.netflix.com/Catalog/CatalogTitles) you can request that the server do some work for you, for example, you can pass a query that will limit the results to only those that you are interested in, and you can also specify the information that you are interested in. …

Producing oData: the need to open source System.Data.Services

This is trickier, and I believe that if Microsoft wants to bootstrap the oData universe they need to seed this space not only with some existing services but also with an open source OData server implementation for Unix. [Emphasis added.]

I should further add that they should not wait a single minute and open their server-side .NET implementation if they want to accelerate the oData adoption. …

I voted for *nix and Java.

Manish Kungwani posted OData: Writing an OData API using Entity Framework & WCF Data Services on 4/4/2010:

In the previous post I talked about OData, what it is, and what it can do …

Now, to use the OData protocol to share data, we need to create an OData service which can make data available to consumers. In this post I will show you how to do this using Entity Framework and WCF Data Services.

Step 1: Create a Database with the schema of the data you want to expose to consumers …

Step 2: Create a Web project and add an Entity Framework Data Model to the project for the database created earlier.

Step 3: Make sure that all the relations and associations from the database are mapped into the data model so that it is ready to be made available to the public.

Step 4: Add a new WCF Data Service as:

Add | New Item | Web | WCF Data Service

Give it the name MyService.svc.

After this, all you need to do to have a full, working OData service is add a class in between the angle brackets (DataService<YourTypeHere>) and include one line for config.EntitySetAccessRule. …

Manish continues with the code for a simple minimal class. His previous post is OData: Oh its Data….. of the same date.

Rob Sanders’ Reviewing SQL Server Modelling CTP (Nov 09) post of 4/4/2010 provides and overview of the “Oslo” suite and the “m” modeling language:

You may have heard of the upcoming SQL Server Modelling suite formerly known as ‘Oslo’. The suite, know known as the SQL Server Modelling CTP, features the ‘m’ modelling language and a graphical workbench known as (or codenamed) ‘Quadrant’. I’ve been meaning to look into codename ‘Oslo’ for a while now, and I’ve finally had a chance to review how it works this weekend.

Designing for SQL Server has always been a fairly subjective art form. Microsoft has not exactly been kind to data architects, continually changing its position on the how and why of data modelling tools. Once upon a time, a database architect could rely on tools like Visio (for Enterprise Architects) for forward and reverse engineering database schema.

Microsoft is endeavouring to build a more comprehensive set of tools for what we term “DSL” languages or domain specific languages. This essentially allows data architects to focus on business specific (or domain specific) architecture and to abstract away from the physical implementation of persisted storage in large enterprise designs. …

Rob continues with the details of his Oslo test drive.

Cihan Biyikoglu describes Handling Error 40552 IN SQL Azure in the 4/3/2010 post:

[I]n the last few customer conversations, error 40552 came up. Hopefully the following tip can help you avoid getting this error.

Why Do I get error 40552?

SQL Azure has mechanism in place to prevent monopolizing of various system resources. One of the safeguards in place watches the active portion of the log space utilized by a transaction. Applications running transactions that use large amount of log space may receive the following error message;

Msg 40552, Level 20, State 1, Line 1
The session has been terminated because of excessive transaction log space usage. Try modifying fewer rows in a single transaction.

Avoiding 40522 when creating, rebuilding and dropping indexes:

Creating, rebuilding and dropping indexes could generate a lot of transaction log records and may hit this error msg on larger tables. You may be able to work around the issue by creating a new table with the desired index layout, and then move the data in smaller chunks over to the new table. However in most cases, you can minimize transaction log space usage with index operations by using the ONLINE option with your statement. Specifying the ONLINE=ON with CREATE, ALTER and DROP INDEX operations change the characteristics of the execution. Instead of a single large transaction, the operation is performed in multiple shorter transactions in the background without holding exclusive locks for extended periods of time on the rows. This both improves concurrency during the execution of the operation, and it eases the transaction log space requirement for the operation. This can help you avoid the 40552 exception.

AppFabric: Access Control and Service Bus

Eugenio Pace’s WIF and MVC – How it works post of 4/3/2010 begins:

I got a few questions from people on how the “Federation with Multiple Partners” sample of the guide works, so I figured I would just write it down in one place for eternity.

The guide explains all sequence in quite some detail (see pages 88 to 97), but sometimes a diagram is more helpful, so here’s a sequence diagram that describes all interactions:

An un-authenticated user browses a protected resource, say the “Shipping” page (which translates into a method in a Controller).
The Controller is decorated with the AuthenticateAndAuthorizeAttribute which implements MVC’s IAuthorizationFilter interface. Because the user is not authenticated yet, it will issue a SignInRequest to the configured issuer (this results in a redirection to the issuer). Among other tings it passes the user original URL in the context (wctx) of the request (in this example is the “Shipping” page).
The Issuer authenticates the user with whatever means and if successful, it issues a token for the user for that application.
The Token and the the context information is passed back to the app to a specific destination (indicated in the realm). In the MVC application, this is just another controller (“Home” in our sample, method “FederationResult”). This controller is not decorated by the AuthenticateAndAuthorizeAttribute.
The request however, does go through the WIF Module (the “FAM” in the diagram above). WIF will validate the token and create a ClaimsPrincipal that is eventually passed on to the controller.
The Home Controller inspects the context parameter, extracts the original URL (remember the context is preserved throughout all hops) and then redirects the user there.
The redirect will go again through the filter, but this time the user is authenticated.
Any authorization rules are performed (in our example, we checked for specific Roles) and if all checks pass…
The specific controller is finally called (e.g. Shipping).

Eugenio continues with detailed notes about his diagram.

The TechNotes blog posted excerpts from Understanding Windows Workflow Foundation by Robert Eisenberg on 4/3/2010:

Excerpted from the book http://www.informit.com/articles/article.aspx?p=1316798

Some of the benefits gained by using most workflow systems to automate the process:

Design-time transparency—It is clear what the expense report process does just by looking at it.

State management—The workflow system manages keeping track of the current step, removing the process from memory when waiting for manager approval, and going back to the approval step when required. A workflow system provides these state management features for you. In contrast, a process automated via a traditional computer language generally requires the developer to create tables and fields to manage the state themselves.

Runtime transparency—The workflow system can also visually show the current step and the execution path of the process. This feature is generally referred to as tracking (you will learn more about it in the “Tracking” section of this hour).

Many think that there is no way to graphically model application logic because any complex process will have so many control flow statements the diagram will be unreadable, eliminating the promised transparency. A process automated with a graphical workflow application should be self-describing. It should serve the same benefit that a flowchart currently supplies (or at least most of it). Just as a flowchart does not include every single control flow statement, neither should a graphical workflow.

Workflow Segmentations

Workflow is also frequently segmented along

in-application

human

integration lines

Traditional languages, like C#, are generally associated with in-application workflow. Human workflow systems arose from the desire to better support human-centric scenarios. Integration systems or integration-centric workflow systems arose from the need to better automate integration scenarios using dedicated integration systems.

Live Windows Azure Apps, APIs, Tools and Test Harnesses

Manashi Narumoto’s Trick in Windows Azure deployment of 4/3/2010 explains:

If you see web roles or worker roles are going back and forth between initializing-busy-stopping, the first thing you should check is the copy local attribute of the assemblies.

All assemblies that aren’t part of .NET framework have to be set as Copy Local=True so that they would be included in your deployment package. Otherwise they aren’t deployed to Azure ending up the problem above. I’m making this mistake just about every day :-)

His earlier Don’t rush! Worker roles will be there for you post of the same date recommends:

In dev fabric, hit F5, wait for a while then start debugging. No wonder if your worker roles don’t respond in the first place. Sometimes it takes over 30 seconds for them to start off. You should better check the dev fabric UI. When it looks like this, it’s still in the middle of its startup.

David Worthington (@dcworthington) quotes Joannes Vormorel (@vermorel) and Me (@rogerjenn) in his Coming to Azure out of the blue article of 4/1/2010 for @SDTimes:

Microsoft's Windows Azure development platform isn't exactly new (it was in beta for almost two years), but many developers are still unsure about how to get a pilot started.

Developers should think hard about what they want to achieve in the cloud and whether they want to start from scratch or migrate applications, said Joannes Vermorel, founder of Lokad, a maker of sales forecasting software. Vermorel is a popular Azure blogger and developer.

The cloud is ideal for Web applications that require a high degree of uptime, Vermorel said. " 'Always on' is nearly impossible in the classic client/server setup." It is also good for organizations that require extra capacity to handle peak loads, he said.

"We [Lokad] rented grids before Azure. We rented half a dozen powerful Web servers from month to month, never had enough processing power at peak [demand], and most of the time processing power was unused," Vermorel said. …

The average .NET developer will have no problem "picking up" basic Azure programming within a day, said Roger Jennings, OakLeaf Systems developer and author of over 30 books on Windows development topics. OakLeaf Systems is a consultancy that specializes in the Azure platform.

Some of its more sophisticated functions, such as the enterprise service bus in AppFabric, will be sticking points that will take "a bit of getting used to," he added. "Differentiating between Web roles and worker roles will take conceptual exercise on the part of the developer."

Jennings compared the difference in programming conventions between the cloud and on-premise servers to an ASP.NET developer having to learn the ASP.NET Model-View-Controller framework. …

Return to section navigation list>

Windows Azure Infrastructure

Ernest De Leon asserts Greenpeace Suffers Major Credibility [Gap] in this 4/3/2010 post:

Let me start this article off by saying that I am all for the ‘green’ movement. If we as a society can reduce what we consume, reuse as much as possible and recycle rather than throw away, I believe we will be much better off than the alternative. If we can reduce global emissions across the board and lean more heavily on renewable energy, all the better. What I don’t agree with, however, is attacking certain industries with only half of the story. It’s one thing if it was due to ignorance and a complete different monster if it was done intentionally. I’m not going to cast judgment as to which one it is, but without a doubt it is one of the two and I’ll leave it to you to decide for yourselves. By the way, I am referring to the Greenpeace report cited at the end of the article. Please look there if you want more information. …

The article to which Ernest refers is here.

Ina Fried’s Microsoft's latest small-business plan article of 3/31/2010 for CNet News’ Beyond Binary column begins:

It happens every couple of years. Microsoft's newly installed head of small-business efforts goes on the road to talk about how the company sees vast potential in the huge numbers of underserved firms that all want the capabilities of big business software without the cost or complexity. The new executive assures me that Microsoft gets it and promises Redmond is rededicating itself to the market.

This time around, the executive was Birger Steen, a Norwegian oil trader who ran Microsoft's Russian subsidiary before moving to Redmond last year to take over the small and midsize business sales effort. In addition to his unusual background, Steen also came in with a different pitch. Refreshingly, Steen said he didn't really think that small businesses are all that poorly treated.

"They are underserved but they are also well served by some people who are not us," Steen said, noting that in every market there tends to be a company like Intuit in North America that does a really good job of crafting accounting and other software products for smaller firms.

What Microsoft has the opportunity to do, he said, is to provide the operating system and Office (and related products like SharePoint and Exchange) and then partner with the Intuits of the world. In addition, he said, Microsoft can help spur the development of the next generation of extremely custom software that a dentist or car repair shop uses to run their core business.

Although Microsoft won't make that last piece of software, Steen said, it could be based on Windows Azure, Microsoft's cloud-based operating system. The cloud, Steen notes, is well suited to smaller businesses because it allows them access to the latest technology from a state-of-the-art data center that they don't have to build. …

Among many other functions, a traditional operating system coordinates access by applications to the underlying resources of the machine - things like the CPU, memory, disk storage, keyboard and screen. The operating system kernel schedules processes, allocates memory, manages interrupts from devices, handles exceptions, and generally makes it possible for multiple applications to share the same hardware.

As a result, it's easy to jump to the conclusion that "cloud computing" platforms like Amazon Web Services, Google App Engine, or Microsoft Azure, which provide developers with access to storage and computation, are the heart of the emerging Internet Operating System.

Cloud infrastructure services are indeed important, but to focus on them is to make the same mistake as Lotus did when it bet on DOS remaining the operating system standard rather than the new GUI-based interfaces. After all, Graphical User Interfaces weren't part of the "real" operating system, but just another application-level construct. But even though for years, Windows was just a thin shell over DOS, Microsoft understood that moving developers to higher levels of abstraction was the key to making applications easier to use.

But what are these higher levels of abstraction? Are they just features that hide the details of virtual machines in the cloud, insulating the developer from managing scaling or hiding details of 1990s-era operating system instances in cloud virtual machines?

The underlying services accessed by applications today are not just device components and operating system features, but data subsystems: locations, social networks, indexes of web sites, speech recognition, image recognition, automated translation. It's easy to think that it's the sensors in your device - the touch screen, the microphone, the GPS, the magnetometer, the accelerometer - that are enabling their cool new functionality. But really, these sensors are just inputs to massive data subsystems living in the cloud.

When, for example, as an iPhone developer, you use the iPhone's Core Location Framework to establish the phone's location, you aren't just querying the sensor, you're doing a cloud data lookup against the results, transforming GPS coordinates into street addresses, or perhaps transforming WiFi signal strength into GPS coordinates, and then into street addresses. When the Amazon app or Google Goggles scans a barcode, or the cover of a book, it isn't just using the camera with onboard image processing, it's passing the image to much more powerful image processing in the cloud, and then doing a database lookup on the results.

Increasingly, application developers don't do low-level image recognition, speech recognition, location lookup, social network management and friend connect. They place high level function calls to data-rich platforms that provide these services.

With that in mind, let's consider what new subsystems a "modern" Internet Operating System might contain [abridged]:

Search …

Media Access, including Access Control, Caching, and Instrumentation and analytics …

Communications …

Identity and the Social Graph …

Payment …

Advertising …

Location …

Activity Streams …

Time …

Image and Speech Recognition …

Government Data …

Tim continues with in-depth analyses:

What About the Browser? …
What's Still Missing …

and promises:

Next week, I'll handicap the leading players and tell you what I think of their respective strategies. [Emphasis Tim’s.]

Charles Babcock authored Cloud Connect: Grappling with Economics downloadable brief InfomationWeek Analytics:

Although the cloud has come to be viewed as part of IT's future, cloud users and vendors are still debating several key questions. If a company has virtualized its data center, is that "a private cloud?"

The answer that came back over two days of the Cloud Connect show in Santa Clara, Calif., was a definite "no," virtualization by itself is not enough.

"Virtualization is a foundation, but it doesn’t equal cloud," said Valerie Knafo, senior manager of Data Center Solutions at Dell, member of a panel on the pros and cons of adopting cloud computing.

Cloud Security and Governance

Tanya Forsheit posted Privacy, Privilege, and the Cloud, Oh My: Taking LovingCare to Heart to the InformationLawGroup (@InfoLawGroup) blog on 4/3/2010:

What does workplace privacy have to do with the cloud? Everything. On Tuesday, the New Jersey Supreme Court issued its opinion in Stengart v. LovingCare Agency, Inc., --- A.2d ----, 2010 WL 1189458 (N.J. March 30, 2010), and came out on the side of protecting employee privacy and the attorney-client privilege in personal Yahoo! webmail (a cloud service) even though the employee used a company computer. While everyone has been busy writing about the implications of LovingCare for company policies governing employee expectations of privacy (and for good reason), few have stopped to note that LovingCare is a cloud case. LovingCare is one of only a few published opinions addressing the difficult issues surrounding employee use of webmail and other cloud services on company computers where the attorney-client privilege is at stake, and the impact of the LovingCare decision will undoubtedly be felt for years to come by nearly every employer across the country, both in crafting policies for employee use of company computer systems and in conducting discovery in nearly every employment-related litigation. …

Cloud Computing Events

For many IT veterans, the rapid adoption of cloud computing induces an eerie feeling of déjà vu. Questions quickly arise regarding exactly how today’s cloud computing solutions differ from centralized infrastructure strategies of the past -- technologies such as mainframes, client/server, grids, ASPs and SaaS. Is there more at work here than typical industry hyperbole, and if so, what specific technologies and capabilities does computing in the cloud offer that are truly novel?

In this groundbreaking day-long virtual event, InformationWeek editors and Cloud Connect conference leaders join with internationally recognized experts and top IT practitioners from the public and private sector to dial back the hype surrounding cloud computing and demystify it once and for all. They’ll dig deep to explain how cloud computing is surpassing its architectural predecessors in the IT field, and discuss the specific business advantages that have emerged from these dynamically scalable, highly virtualized environments. …

Registered.

Other Cloud Computing Platforms and Services

Jackson Harper published the source code for his MCloud .NET client library for managing cloud providers (based on opencloud) to github on 4/4/2010. Jackson describes his library:

MCloud is a managed unified interface to various cloud providers. It is heavily based on libcloud (http://www.libcloud.net) with only small changes to make the API more .net friendly. This project is obviously not very far along but checkout test.cs to see what you can do. To run this test you need to have a linode API key. Also you should note that running this test will create/delete a new Linode on your account and that is going to cost you a days worth of Linode service (Aprox $1).

Amazon has worked out a way to run Windows in in its EC2 cloud at roughly the same price as running Linux in said cloud. If you have enterprise license agreements for on-premises Windows Server (and Software Assurance), you can use those licenses with Amazon's EC2 cloud, at least for a trial period. Microsoft announced the pilot program on Thursday.

Updated 0326: It was a rather surprising announcement. Amazon has the most popular cloud and Microsoft would like to change that and bring you to its own cloud, Azure. Although, as a reader below points out, these are apples and oranges, with EC2 being an example of infrastructure as a service (IaaS), and Azure and Google Apps Engine being examples of platform as a service. Looks like Redmond would rather have you try out a competitor the cloud and continue using Windows Server than to have you go wandering off to Linux or the likes of Google, which many Microsoft users say they are thinking of doing. (Google's Google App Engine, is free up to a certain point of usage, and doesn't support Windows or .Net.)

Amazon is also a reseller to Microsoft, and I don't just mean selling Windows 7 and Office to consumers. Amazon already offers Windows Server services on EC2. So today's deal is about allowing you to save money. If you don't have enterprise licenses with software assurance, you can still rent a Windows infrastructure from Amazon but you have to pay a set-up fee roughly equivalent to the license fees for on-prem servers. …

Julie continues with the gory licensing details.

M. Tim Jones’ Anatomy of an open source cloud of 3/9/2010 whitepaper for IBM’s developerWorks carries this summary:

Cloud computing is no longer a technology on the cusp of breaking out but a valuable and important technology that is fundamentally changing the way we use and develop applications. As you would expect, Linux® and open source provide the foundation for the cloud (for public and private infrastructures). Explore the anatomy of the cloud, its architecture, and the open source technologies used to build these dynamic and scalable computing and storage platforms.

He concludes:

Volumes could be written about the leadership role that open source is playing in the cloud and virtualization domain, but this article provided a short introduction to some of the popular and visible solutions available today. Whether you're looking to build a cloud based on your own requirements from individual pieces or simply want a cohesive solution that works out of the box, open source has you covered. See the Resources section for more details on the solutions discussed here and others.

Return to section navigation list>