Windows Azure and Cloud Computing Posts for 8/17/2011+
| A compendium of Windows Azure, SQL Azure Database, AppFabric, Windows Azure Platform Appliance and other cloud-computing articles. |  |
Note: This post is updated daily or more frequently, depending on the availability of new articles in the following sections:
- Azure Blob, Drive, Table and Queue Services
- SQL Azure Database and Reporting
- Marketplace DataMarket and OData
- Windows Azure AppFabric: Apps, Access Control, WIF and Service Bus
- Windows Azure VM Role, Virtual Network, Connect, RDP and CDN
- Live Windows Azure Apps, APIs, Tools and Test Harnesses
- Visual Studio LightSwitch and Entity Framework v4+
- Windows Azure Infrastructure and DevOps
- Windows Azure Platform Appliance (WAPA), Hyper-V and Private/Hybrid Clouds
- Cloud Security and Governance
- Cloud Computing Events
- Other Cloud Computing Platforms and Services
Azure Blob, Drive, Table and Queue Services
Steve Marx (@smarx) described Lightweight Tracing to Windows Azure Tables in an 8/17/2011 post:
I wrote a blog post last year called “Printf(‘HERE’) in the Cloud,” and I find it’s one of the posts I reference most frequently when helping people to debug their code in Windows Azure. The technique described in that post is useful because it’s just about the simplest form of logging you can implement. It’s easy to drop the code into a project and to immediately see the output. However, it has the drawback of making it hard to view all the logs together. (Each message logged goes into its own blob.)
To address that, I’ve implemented a custom
TraceListenercalledsmarx.TableTraceListenerthat logs messages immediately to table storage using a simple schema. This is meant to be used for lightweight tracing during the development phase of your project. For serious production logging, you’ll want to invest the time to understand and implement Windows Azure Diagnostics, which is more robust and cost-effective (fewer storage transactions due to batching) but is also harder to configure and introduces a delay between when you log something and when you can see it.Usage
To use the
TraceListenerin your project, just download and reference smarx.TableTraceListener.dll. (Yes, I should make it into a NuGet package. I simply haven’t learned how to do that yet.) Then add the trace listener in yourweb.configorapp.config:<system.diagnostics> <trace> <listeners> <add name="TableTraceListener" type="smarx.TableTraceListener.TableTraceListener, smarx.TableTraceListener" connectionStringName="LogConnectionString" /> </listeners> </trace> </system.diagnostics>Note the
connectionStringNameattribute. That specifies which connection string to use for logging. It defaults to “DataConnectionString,” which is probably what you’re using in your application already, but if you want to log to a different storage account, you can control that.Once you’ve configured the
TraceListener, you can just use normal tracing methods likeTrace.WriteLine(…)in your code, and everything will be logged to a table called “log” in your storage account. To view the logs, I usually just fire up ClumsyLeaf TableXplorer. (Remember, this is supposed to be lightweight!) If you prefer, you can write your own code to query the logs. Here’s an ASP.NET MVC 3 controller action that grabs the top twenty log messages (for the current deployment ID) and sends them to a view:public ActionResult Index() { return View(CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("LogConnectionString")) .CreateCloudTableClient().GetDataServiceContext().CreateQuery<LogMessage>("log") .Where(l => l.PartitionKey == RoleEnvironment.DeploymentId).Take(20)); }Here’s the corresponding view, which just puts the messages into a list:
@using smarx.TableTraceListener; @model IEnumerable<LogMessage> <ul> @foreach (var message in Model) { <li>@message.Time: [@message.InstanceId] (@(message.Category ?? "none")) @message.Message</li> } </ul>Source
The code for the
TraceListeneris short enough that I can share it all here. I first created a class to represent log messages:public class LogMessage : TableServiceEntity { public DateTime Time { get; set; } public string Message { get; set; } public string InstanceId { get; set; } public string Category { get; set; } public LogMessage() { } public LogMessage(string message, string category) { Message = message; Category = category; Time = DateTime.UtcNow; InstanceId = RoleEnvironment.CurrentRoleInstance.Id; PartitionKey = RoleEnvironment.DeploymentId; RowKey = (DateTime.MaxValue.Ticks - Time.Ticks).ToString("d19"); } }Each message includes a timestamp, the ID of the instance that generated the message, and the message itself (with an optional category). The partition key is the app’s deployment ID so it’s easy to find the messages from a specific deployment. (Otherwise messages from a staging deployment and a production deployment get intermixed, possibly even with messages from other applications sharing the same storage account.) The row key is the typical reverse timestamp to make sure new log messages appear on top.
The
TraceListeneritself is as follows:public class TableTraceListener : TraceListener { private TableServiceContext _context = null; private TableServiceContext context { get { if (_context == null) { var tables = CloudStorageAccount .Parse(RoleEnvironment.GetConfigurationSettingValue( Attributes["connectionStringName"] ?? "DataConnectionString")) .CreateCloudTableClient(); tables.CreateTableIfNotExist("log"); _context = tables.GetDataServiceContext(); _context.MergeOption = MergeOption.NoTracking; } return _context; } } protected override string[] GetSupportedAttributes() { return new[] { "connectionStringName" }; } public override void Write(string message, string category) { context.AddObject("log", new LogMessage(message, category)); context.SaveChangesWithRetries(); } public override void WriteLine(string message, string category) { Write(message + "\n", category); } public override void Write(string message) { Write(message, null); } public override void WriteLine(string message) { Write(message + "\n"); } }The actual work is done in the surprisingly short
Writemethod, which creates a newLogMessageobject and adds it to the table. The rest is setup (getting the connection string and ensuring the “log” table exists) and various overloaded methods. That’s it!Downloads
- smarx.TableTraceListener.dll – a .NET 4 assembly you can reference directly
- TableTraceListener_source.zip – the full source code for the
TraceListenerI hope you find this useful, and be sure to check out this Friday’s episode of Cloud Cover, where we’ll talk about this and other logging techniques.
Steve Marx (@smarx) reported the availability of Windows Azure Storage Libraries in Many Languages in an 8/16/2011 post:
A recent question on the Windows Azure forums reminded me of just how many languages have libraries for accessing Windows Azure storage. Here are a few that I know about (some of which I’ve even contributed to):
- C#/.NET: Microsoft.WindowsAzure.dll, azurestoragesamples – The former ships in the Windows Azure SDK and is by far the most popular library to access storage. The latter is written by David Pallmann, one of our Windows Azure MVPs, and it’s a side-by-side comparison of using the client library from the SDK versus building your own HTTP requests.
- Python: winazurestorage.py – Sriram Krishnan (former Windows Azure team member) wrote the first version of this back in 2008, and since then I’ve been committing to it. It supports basic blob upload and download as well as table queries and queue creation/deletion. At this point, it needs something of an overhaul to make the classes more organized and consistent.
- Ruby: waz-storage – This is a well-designed library that’s fairly complete. It supports all the blob operations (including block upload, which I’m proud to say I committed), all table operations (with continuation tokens), and the full queue API as well. I’m not aware of much of anything missing from this library. It’s what I use to power this blog and waz-cmd, my Ruby gem to interact with the Windows Azure Service Management API.
- Perl: waz-storage-perl – This is a very minimal library written by yours truly. It only supports blob upload and download, but as with most of these libraries, the tricky part is constructing a correct authorization header, and I believe that work is done correctly. It should be easy to extend this to support operations like container creation/deletion and queue operations. (Table storage would require a bit more work.)
- JavaScript (Node): waz-storage-js, wastorage.coffee – I don’t have much experience with waz-storage-js, which is a brand new library that supports a minimal set of blob operations. Wastorage.coffee is part of my smarxchat app, which supports table insertion and query and is tuned for speed. (I even made my own HTTP connection pooling library to go with it!) Neither of these is by any means “done,” but they both include some of the tricky parts of getting a library built: authorization and table entity serialization/deserialization.
- Java: WindowsAzure4J – This is the result of a collaboration between Microsoft and Soyatec. It’s a fairly complete library.
- PHP: phpazure – This is another collaboration, this time between Microsoft and RealDolmen. This library is also fairly complete and is based on the work of one of our Windows Azure MVPs, Maarten Balliauw.
- Erlang: winazure.erl – This is a fairly minimal (just blob upload and download) library written in Erlang by Sriram Krishnan (the original author of the Python library).
- Common LISP: cl-azure – Yeah, you read that correctly. Common LISP. I haven’t used this library at all, but I like that it exists.
Do you know of other libraries not on my list? I’ll try to maintain this list as things get added and changed, so send me an email or tweet to me if you have a suggestion. (My contact info is on the right side of this blog.)
Avkash Chauhan posted A Command Line tool for downloading blob from Azure Storage and logging download completed back to Azure Storage on 8/16/2011:
From time to time, I had to download blob from Windows Azure Storage for my Windows Azure Projects but so many time[s], when I was no in Azure VM, there was no way to be sure that my blob was downloaded successfully. To solve this problem I decided to simply write a command line tool which can download any file from Windows Azure storage and after download is completed,
I also wanted togets confirmation that download is completed in a log file at the Azure Storage.
To make it very simple solution, I just created a command line application which use Windows Azure Storage Credentials to download blob from Azure Storage and after download is completed, the same application upload a log file to Azure blob storage to confirm that download is completed.
To use this tool you can create a batch file as below:
@REM Setting up Azure Stroage Credentials
set azurestoragename=happybuddha
set azurestoragekey=*******************************************************==
set storagecontainername=xyz
@REM Download blob name
set filename=apache-tomcat-7.0.19-windows-x64.zip
azurepackagedownloader.exe "%azurestoragename%" "%azurestoragekey%" "%storagecontainername%" "%filename%"
Once you will launch above batch file, the above apache-tomcat-7.0.19-windows-x64.zip will be download from the Azure Storage name "happybuddha" from container xyz. After the download completed, a log file will be uploaded to same "xyz" container in Azure Storage "happybuddha" as below:
If you look at the log file it look like as below:
8/9/2011 5:53:21 AM: Application Started......
8/9/2011 5:53:21 AM: Connection to Azure Storage : SUCCCESS.....
File Download Started....8/9/2011 5:53:38 AM: Transfer progress percentage = 1 - 0.00KB/s
8/9/2011 5:53:38 AM: Transfer progress percentage = 2 - 0.00KB/s
8/9/2011 5:53:38 AM: Transfer progress percentage = 3 - 914.36KB/s
8/9/2011 5:53:38 AM: Transfer progress percentage = 4 - 960.08KB/s
8/9/2011 5:53:38 AM: Transfer progress percentage = 5 - 948.23KB/s
8/9/2011 5:53:39 AM: Transfer progress percentage = 7 - 1,031.11KB/s
8/9/2011 5:53:39 AM: Transfer progress percentage = 9 - 1,174.62KB/s
8/9/2011 5:53:39 AM: Transfer progress percentage = 9 - 1,232.12KB/s
...8/9/2011 5:53:43 AM: Transfer progress percentage = 100 - 1,812.30KB/s
8/9/2011 5:53:43 AM: Transfer completed. Press any key to continue.
8/9/2011 5:53:43 AM: File apache-tomcat-7.0.19-windows-x64.zip Download Completely!!!I got some kick start for this tool from Kevin[‘s] blogs as below:
Using above blog I took the base BlobTransfer class written by Kevin and just added some code to create a commandline application which can satisfy my needs. You can download from the same tool from CodePlex as below:
<Return to section navigation list>
SQL Azure Database and Reporting
Mark Kromer (@mssqldude) described SQL Azure: Scale-Out and Big Data in an 8/17/2011 post:
Perhaps the NoSQL / Big Data trend in high-performance computing, made popular by Hadoop and MapReduce will end up being the “killer app” or at least “killer capability” for cloud databases?
I find it be an interesting thought because the currently available Microsoft cloud databsae, SQL Azure, is a complete SQL Server-based transactional database complete with support for SQL Server tools, T-SQL, ODBC, referential integrity, etc. The current maximum single stand-alone database size is 50 GB.
But Microsoft has recently shown a lot of interest in providing support for scaled-out large database workloads, first with SQL Server Parallel Data Warehouse (PDW) and then the recent announcement of PDW support for Hadoop. Scale-out applications built on traditional SQL Server have been around for some time. The typical mechanisms used to do this are based on partitioning or “sharding” the data to fan-out the data and queries across SQL Servers using MS-DTC, replication or Service Broker.
SQL Azure is coming out with a built-in capability to enable this style of “sharded” partitioning called Database Federations. This is a link to a terrific write-up of using these concepts in a Big-Data application, written by Roger Jennings for Visual Studio Magazine. [Emphasis added.]
Note that this capability is not yet available even in CTP (beta) for SQL Azure yet at the time that I am writing this blog post. I like the fact that these capabilities are being surfaced as a built-in T-SQL command. There will be accompanying ADO.NET library changes with APIs and methods to manipulate the distributed data and to query it appropriately as well.
Very interesting, exciting ways that SQL Azure can be used. Once I get access to the CTPs, I’ll start building out distributed apps using that capability and blog my results here for you. In the meantime, that article link above gives you some code samples to start thinking about your Big Data architectures.
Thanks, Mark, for the kind words about my article.
The SQL Azure Reporting Team sent the following What is New in the Next SQL Azure Reporting CTP? message from SQL Azure Communications (sacomm@microsoft.com) to current Limited CTP members on 8/17/2011:
What is New in the Next SQL Azure Reporting CTP?
Microsoft has extended its best-in-class reporting offering to the cloud with SQL Azure Reporting, enabling you to deliver rich insights to even more users without the need to deploy and maintain reporting infrastructure.
As a selected SQL Azure Reporting CTP customer, Microsoft is pleased to inform you about the next phase of our CTP of SQL Azure Reporting in September 2011, which allows you to try out the completed product prior to its commercial release and provide Microsoft with feedback that is important for you and your business.
This upcoming CTP is the last prior to SQL Azure Reporting’s commercial release and we have completed all the feature work for the first release of SQL Azure Reporting. For the CTP, we have enriched the Windows Azure Portal experience with new features:
- User and report management functions.
- Enabled a self-provisioning function so you can create a SQL Azure Reporting server by yourself.
In addition to the enriched Portal functionalities, we have improved availability and performance of the SQL Azure Reporting service and are rolling out the CTP to all Microsoft datacenters around the world.
Steps to Prepare for the Upcoming CTP:
- You will need a Windows Azure Platform account in order to participate in this CTP. If you already have an account, proceed to step 2.
If you do not already have an account, please visit the "Start Now" section on http://www.microsoft.com/en-us/sqlazure/database.aspx. You will find Free Trial offers which will allow you to participate.- SQL Azure databases remain the supported data sources in the upcoming CTP. If you have already created a SQL Azure server in any Windows Azure subscription, you may create reports against it in this CTP. If you do not have a SQL Azure server, please proceed to the Windows Azure portal (http://windows.azure.com) and create one.
Note:
- This upcoming CTP will be deployed to a different environment than the current Limited CTP. Your reports will NOT be automatically migrated to this CTP environment by Microsoft. We recommend that you republish the reports from your Business Intelligence Development Studio project to the new Web Service endpoint URL that you will get from the new portal once you provision a new “server”.
- After the upcoming CTP is deployed, we will replace the current “Reporting” page on Windows Azure portal (http://windows.azure.com) with a new Reporting portal experience. But from the new portal page, you will not be able to access the information for the current CTP, e.g. old Web Service endpoint URL and Admin user name. If you want to continue using the current CTP, please make sure to copy and keep the information somewhere else.
Sincerely,
The SQL Azure Reporting team
<Return to section navigation list>
MarketPlace DataMarket and OData
Bruce Kyle described an ISV Video: Corelytics Provides Financial Dashboards for Small Businesses Through Windows Azure Marketplace in an 8/17/2011 post to the US ISV Evangelism blog:
Saas provider Coreconnex explain in a new video on Channel 9 why they chose Windows Azure to deploy their product for small businesses. Their product enables non-financial people to understand the financial graphical "picture" of their business.
CEO Frank Coker and Chief Architect Torsten Kablitz talk with Azure Incubation Architect Evangelist Greg Oliver about the business of financial dashboards and the technology decisions involved in moving such a system to Windows Azure.
The team explains the decision to use Windows Azure was based around the integration with Visual Studio for development tools, integration with Intuit in the cloud, their need add customers quickly, and up time across multiple geographies. Their capital investments are also reduced signifiantly by going with Microsoft's cloud solution.
The Corelytics financial dashboard is available at the Windows Azure Marketplace. The product enables small businesses to save time and money by providing unique insight into performance allowing business owners to monitor key indicators, see visuals of trends and forecasts and compare performance to their industry peers and annual goals.
About Coreconnex
Coreconnex helps small businesses save time and money accessing a clearer picture of financial performance using the Corelytics financial dashboard and predictive analytics.
About Windows Azure Marketplace
Windows Azure Marketplace is a global online market for customers and partners to share, buy, and sell finished SaaS applications and premium datasets. Whether you are looking for new customers for your Windows Azure based application or datasets, or are seeking new Windows Azure solutions to power your business, the Windows Azure Marketplace is a one-stop location supported by Microsoft to help you succeed.
Other ISV Videos
For videos on Windows Azure Platform, see:
- Accumulus Makes Subscription Billing Easy for Windows Azure
- Azure Email-Enables Lists, Low-Cost Storage for SharePoint
- Credit Card Processing for Windows Phone 7 on Windows Azure
- Crowd-Sourcing Public Sector App for Windows Phone, Azure
- Food Buster Game Achieves Scalability with Windows Azure
- BI Solutions Join On-Premises To Windows Azure Using Star Analytics Command Center
- StorSimple Integrates On-Premises, Cloud Storage with Windows Azure
- How Quark Promote Hosts Multiple Tenants on Windows Azure
- NewsGator Moves 3 Million Blog Posts Per Day on Azure
<Return to section navigation list>
Windows Azure AppFabric: Apps, Access Control, WIF and Service Bus
<Return to section navigation list>
Windows Azure VM Role, Virtual Network, Connect, RDP and CDN
<Return to section navigation list>
Live Windows Azure Apps, APIs, Tools and Test Harnesses
Avkash Chauhan reported the availability of a Full Startup task based Tomcat/Java Worker Role Application for Windows Azure in an 8/17/2011 post:
This Windows Azure Tomcat Solution is designed in a way that you can update your Tomcat project outside VS2010 project. The VS2010 application will be deployed once however whenever you would need to update, Tomcat, Java Run time or your own Tomcat based solution, you can update without updating your Windows Azure application. This project is based on Startup task in which Tomcat, Java and your project is downloaded from Azure Storage and then installed in Azure VM. Full source code is available for startup task and solution.[*]
Java Run Time:
- Download Java Run Time from Java web site and the zip the runtime from the installed location.
- The JRE zip file should have a root folder as "jre7"
Tomcat Installation Package:
- Please download desired Tomcat installation zip file from http://tomcat.apache.org/download-70.cgi
- Please unzip the ZIP file to a location in your machine and edit \conf\server.xml as below: (MUST)
<Connector port="80" protocol="HTTP/1.1" connectionTimeout="20000" redirectPort="443" />
Your Tomcat application project:
- In most of the cases your Tomcat application will be inside "webapp"s folder in a folder itself. Please package your whole application into a ZIP file.
The above 3 zip files should have the root folder as below:
Step 2: Uploading all the files to Windows Azure Storage
Once you upload all 3 files (or more) to Windows Azure Storage your Windows Azure Storage container will look like as below:
Step 3: Opening Windows Azure Tomcat Solution in VS2010.
After you open Windows Azure Solution in Visual Studio 2010, the solution will look like as below:
Edit startup\startup.bat to have correct Windows Azure Storage credentials:
@REM Setting up Azure Stroage Credentials
set azurestoragename=*********azure_storage_name**************** set azurestoragekey=**********azure_storage_key***************** set storagecontainername=*****azure_storage_container_name******
@REM Download Tomcat ZIP
set filename=apache-tomcat-7.0.19-windows-x64.zip
startup\packagedownloader.exe "%azurestoragename%" "%azurestoragekey%" "%storagecontainername%" "%filename%"
@REM Download JRE Runtime ZIP
set filename=jre7.zip
startup\packagedownloader.exe "%azurestoragename%" "%azurestoragekey%" "%storagecontainername%" "%filename%"
@REM Download Tomcat/Java Applicaiton ZIP
set filename=myproject.zip
startup\packagedownloader.exe "%azurestoragename%" "%azurestoragekey%" "%storagecontainername%" "%filename%"
@REM unzip Tomcat
cscript //B //Nologo startup\unzip.vbs apache-tomcat-7.0.19-windows-x64.zip "%ROLEROOT%\approot"
@REM unzip JRE
cscript //B //Nologo startup\unzip.vbs jre7.zip "%ROLEROOT%\approot\apache-tomcat-7.0.19"
@REM unzip project files to tomcatServer\webapps folder
md "%ROLEROOT%\approot\apache-tomcat-7.0.19\webapps"
cscript //B //Nologo startup\unzip.vbs myproject.zip "%ROLEROOT%\approot\apache-tomcat-7.0.19\webapps"
set JRE_HOME=%ROLEROOT%\approot\apache-tomcat-7.0.19\jre7
set CATALINA_HOME=%ROLEROOT%\approot\apache-tomcat-7.0.19
@REM Edit Server.xml
@REM cd "%ROLEROOT%\approot\apache-tomcat-7.0.19\conf"
@REM copy server.xml server.orig.xml
@REM cscript //nologo startup\editserverxml.vbs server.xml 80
@REM start the server
cd "%ROLEROOT%\approot\apache-tomcat-7.0.19\bin"
copy startup.bat startup.back
set > %RoleRoot%\approot\env.txt
startup.bat > %RoleRoot%\approot\tomcat.txt
exit /b 0The provided solution already has port 80 based TCP endpoint defined. IF you would need to add HTTPS endpoint please look the "Adding SSL to Tomcat Solution" documentation section.
Step 4: Adding RDP access to application and packaging for deployment
It is good to add RDP access to your application when you are ready to publish the application, add RDP access as well. If you dont know how to do, please follow link below:
Now just deploy your application to Windows Azure.
Step 5: Verifying that Zip files are downloaded from Windows Azure Blob storage
After your VM will start and download your zip file the download completion log will be available on Azure Storage as below
Step 6: Running Tomcat Application
After some time your worker role will be ready as below:
* I wasn’t able to find a link to the source code, so left Chauhan a comment.
<Return to section navigation list>
Visual Studio LightSwitch and Entity Framework 4.1+
Robert Green reported Episode 8 of Visual Studio Toolbox (LightSwitch Starter Kits) is now live in an 8/17/2011 post:
This week we look at LightSwitch, which launched on July 26 and is now available for purchase. Of course, if you are an MSDN Subscriber you get it as part of your subscription. LightSwitch is the fastest way to build database applications for the desktop and the cloud. In this episode, you will see how to make developing with LightSwitch even faster with the LightSwitch Starter Kits. These Starter Kits cover common business needs like expense reporting and issue tracking. All you need to do is download and install one of the kits and then create a new LightSwitch project based on it. LightSwitch creates data tables and screens and writes some basic code for you. You can press F5 and run or customize the application to meet your needs. And because you built it with LightSwitch, the application is well architected and scalable.
My previous post talked about episode 6 of the show and now episode 8 is live. What happened to episode 7? I could make up a story about how this is the missing episode or how it was held up due to contractual issues, but the reality is that I was on vacation in Montana the week it went live, so I didn't get a chance to post about it. Episode 7 featured the Power Commands for Visual Studio 2010. This extension adds a number of commands to Visual Studio to help you work with projects, files, references, and code.
Interestingly, although these episodes were taped several weeks apart, I am wearing the same shirt. I tape next week's episode tomorrow, so my burgundy t-shirt wearing streak will just have to end at 2 episodes. :)
Robert is a Technical Evangelist in the Developer Platform and Evangelism (DPE) group at Microsoft.
Beth Massi (@bethmassi) explained How to Allow Adding of Data to an Auto-Complete Drop-down Box in LightSwitch in an 8/16/2011 post:
In my last post I showed you how to create a multi-column auto-complete box in Visual Studio LightSwitch. I also showed you how to use multiple layouts and easily enable editing on the data inside the auto-complete box. In this post I want to address another very common use case in business applications – allowing the user to enter new rows of data directly into the auto-complete box. Our auto-complete box is displaying data from a lookup table and we want to allow users to add data to this table if they don’t see the selection they need. I’ll continue with the same data model I used in the previous post where Category has many Products.
Now I want to allow the user to add new Categories without having to open the Create New Category screen manually. It should be a simple click here on the screen. We could put a button on ribbon at the top but a better idea would be to put a button next to the auto-complete box itself.
Adding Commands to Screens
You can add commands to screens in a variety of places. In fact all grouping controls expose what’s called a “command bar” where you can place buttons or links that execute commands. This allows you to place commands in the right contexts, near the controls that they work with. To add a command next to the auto-complete box, open the screen in the Screen Designer and expand the auto-complete box node in the content tree and select “command bar”. Then you can add a new button.
This will open the “Add Button” dialog that will allow you to create a method. You put code in this method to execute the command. For this example I’ll name the method AddNewCategory.
By default this shows up as a button right under the auto-complete box. You can also choose to display this as a link instead by changing the Control Type to “Link” in the property window. I also like to put an ellipses after the display name of the commands to indicate to the user that another screen will open.
When you run the application you will see the command displayed like so:
Calling New Data Screens Programmatically
Now we need to write some code to execute our command. In the Screen Designer, right-click on the command and select “Edit Execute Code”.
Now we need to call our CreateNewCategory screen. You can access all the screens in your application through the Application object.
Private Sub AddNewCategory_Execute() ' Write your code here. Me.Application.ShowCreateNewCategory() End SubThis will open the new data screen we want, however, LightSwitch by default always opens a default edit screen after the save of a new data screen. If you open the CreateNewCategory screen and click the “Write Code” button at the top of the designer, you will see code like this in there:
Private Sub CreateNewCategory_Saved() ' Write your code here. Me.Close(False) Application.Current.ShowDefaultScreen(Me.CategoryProperty) End SubI don’t want the default edit screen to display if the user is adding to the auto-complete box directly. So what we need to do is create a screen parameter and check that so we can optionally display the default edit screen. In the screen designer for the CreateNewCategory screen select “Add Data Item” on the designer toolbar. Add a local property of type Boolean and uncheck “Is Required”. Name the parameter “ShowDefaultEditScreen”. Then click OK.
This will add a property to the screen. Next in the property window make sure you check “Is Parameter”
Now drop down the “Write Code” button and select the CreateNewCaegory_Saved() method and write this code:
Private Sub CreateNewCategory_Saved() Me.Close(False) 'If the parameter was not passed in, or if it is explicitly True, then show the default edit screen If Not Me.ShowDefaultEditScreen.HasValue OrElse Me.ShowDefaultEditScreen Then Application.Current.ShowDefaultScreen(Me.CategoryProperty) End If End SubNow we just need to pass “False” when we call this screen from our command back on the ProductDetail screen.
Private Sub AddNewCategory_Execute() ' Write your code here. Me.Application.ShowCreateNewCategory(False) End SubGo ahead and run this now and see what you get. When you click the “Add New Category…” command the CreateNewCategory screen is displayed to allow you to enter a new Category. Click Save and the screen just closes to reveal the Product detail screen again. To see the new Category, click the “Refresh” button at the bottom of the auto-complete box.
Refreshing Lists of Data Automatically
One thing you might want to do is automatically refresh this auto-complete box for the user instead of making them click the “Refresh” button. Each collection of data on your screen exposes a .Refresh() method that you can call easily from the same screen. In the case of an auto-complete box (or modal window picker), to get this functionality you need to create a custom query and use that instead of the automatic query (more on that in a second).
However in order to call the Refresh from another screen we need to get into some threading details. LightSwitch applications are always multi-threaded so that the applications are always responsive while loading and working with remote data. Since LightSwitch is all about working with data, when you write code (even in the screen), you are writing code on the data thread. In order to have one screen call a method on another screen you need to marshal the call the the main UI thread. This may sound complicated but it’s actually not too bad.
There’s a couple ways to perform a data refresh across screens depending on your needs. One way is to use the technique described in How to Communicate Across LightSwitch Screens. This uses custom events on the Application object to notify all screens when particular sets of data are added to the system from any other screen. But for our purpose here we can do something simpler. Off of the Application object LightSwitch exposes an ActiveScreens collection which gets you the list of all the screens that are open. We can use this collection to check the type of screen and then call a method to refresh the data we want. This avoids having to deal with events and keeps the scope smaller.
So to make this particular refresh possible we need to do a few things.
- Setup the Category auto-complete box on our ProductDetail screen to use a custom query instead of the automatic query.
- Create a Public method called RefreshCategories on our ProductDetail screen that executes the Category Refresh.
- Marshal the call to RefreshCategories onto the main thread from the NewCategory screen after the data is saved.
So the first thing you need to do if you haven’t done so already is to create a query using the Query Designer for the auto-complete box and use that on the edit detail screen. I’m going to create a query called SortedCategories that sorts by Category.Name. Right-click on the Categories table and select Add Query and define the query like so:
Now open the ProductDetail screen and select “Add Data Item” on the designer toolbar once again. This time select the SortedCategories query.
Next set the “Choices” property of the Category auto-complete box to SortedCategories.
Next click the “Write Code” button at the top of the designer and create a Public method called RefreshCategories that calls Refresh on the SortedProducts query.
Public Sub RefreshCategories() Me.SortedCategories.Refresh() End SubThe last thing we need to do is write the code to call this method back in the CreateNewCategory screen. Recall that we are checking a parameter here to determine whether we should display the default edit screen or not. We could add another parameter to this screen the same way to check whether a refresh is needed so that these checks would be independent. However for this example the ProductDetail screen is the only one sending a “False” for the parameter so I can just write the following for the CreateNewCategory_Saved() method:
Private Sub CreateNewCategory_Saved() Me.Close(False) 'If the parameter was not passed in, or if it is explicitly True then show the default edit screen If Not Me.ShowDefaultEditScreen.HasValue OrElse Me.ShowDefaultEditScreen Then Application.Current.ShowDefaultScreen(Me.CategoryProperty) Else 'ProductDetail is the only screen sending False. Refresh the Category auto-complete boxMicrosoft.LightSwitch.Threading.Dispatchers.Main.BeginInvoke(
Sub() For Each s In Me.Application.ActiveScreens If TypeOf s.Screen Is ProductDetail Then DirectCast(s.Screen, ProductDetail).RefreshCategories() End If Next End Sub) End If End SubNotice that this code will call RefrechCategories on all open ProductDetail screens if there are multiple active. And if this code looks too complicated for your taste, you can simply hit the refresh button on the auto-complete box itself. :-)
I hope this helps you in building better user productivity business applications. And as you can see there are a lot of flexible ways to lay out controls and work with screens in Visual Studio LightSwitch. For videos showing some of these techniques please see:
#21 - How Do I: Add Commands as Buttons and Links to Screens?
#22 - How Do I: Show Multiple Columns in an Auto-Complete Dropdown Box?Enjoy!
Return to section navigation list>
Windows Azure Infrastructure and DevOps
The Windows Azure Team (@WindowsAzure) reported Microsoft Patterns & Practices Books Offer Guidance on Adopting Windows Azure in an 817/2011 post:
If you’re looking for a deeper dive into the technical considerations for making a move to the cloud with Windows Azure you should check out these great books from Microsoft patterns & practices. Click on each title for more information.
A Guide to Claims-based Identity and Access Control
Imagine a world where you don't have to worry about authentication. Imagine instead that all requests to your application already include the information you need to make access control decisions and to personalize the application for the user. This is the utopia of claims-based identity that this book describes. As you'll see, claims provide an innovative approach for building applications that authenticate and authorize users.
Moving Applications to the Cloud on the Microsoft Windows Azure Platform
This book focuses on the migration scenario of a fictitious company called Adatum as it modifies its expense tracking and reimbursement system step-by-step so it can be deployed to Windows Azure. Each chapter explores different considerations: authentication and authorization, data access, session management, deployment, development life cycle and cost analysis.
Developing Applications for the Cloud on the Microsoft Windows Azure Platform
This book demonstrates how you can create from scratch a multi-tenant, Software as-a-Service (SaaS) application to run in the cloud by using the latest versions of the Windows Azure tools and the latest features of Windows Azure.
…
Windows Phone 7 Developer Guide
By combining Windows Phone 7 applications with remote services and applications that run in the cloud (such as those using Windows Azure), developers can create highly scalable, reliable, and powerful applications that extend the functionality beyond the traditional desktop or laptop; and into a truly portable and much more accessible environment.
Nathan Totten (@ntotten) reported the release of Windows Azure Accelerator for Web Roles Version 1.1 in an 8/17/2011 post:
Today we released the first update to the Windows Azure Accelerator for Web Roles. This update includes bug fixes and several new features. This release is compatible with the previous version of the accelerator so you can replace this version with your existing accelerator deployment and your sites will synchronize and continue working. In this post I will show you some of the new features and a few work arounds for some known issues. If you have not used this accelerator before be sure to read my intro post here.
MVC3 and PHP Support
Improved Logging
The second change is that we improved the logging processes for the accelerator. We added additional trace messages, and improved the details in the messages. These changes should help you better understand what is happening with the synchronization process and better diagnose issues if they arise.
Start/Stop Synchronization Process
The next feature we added was the ability to start and stop the synchronization process. Normally, the synchronization process continually runs on each web role instance to check for updated sites, configuration changes, etc. There are a couple reasons why you may want to stop this process.
The first is a work around for a known issue. When deploying large sites, the synchronization tool can actually pick up the site before it has finished deploying. This can result in inconsistent deployments across your instances. Generally this would be temporary as the synchronizer should resynchronize after the large site has finished deploying, but you want to avoid the partial synchronization altogether. For this reason, the recommended practice for deploying large sites is to stop the synchronizer before you deploy and then start it after the deployment has finished.
The second reason you may want to stop the synchronization process is to save on storage transactions. The default configuration for the synchronization is to check blob and table storage for changes every 15 seconds. Additionally, there is a log message written every time a synchronization occurs. This results in 12 storage transactions per instance per minute or about 518,400 transactions per month. This may seem like a lot, but remember that transactions only cost $0.01 per 10,000 so the cost of these synchronizations is only about $0.50 per instance per month. For a company doing lots of deployments to Windows Azure this is probably well worth the cost. However, if you are only deploying occasionally you can save a few dollars per month by disabling the synchronization process until you need to deploy.
To start and stop the synchronization process simply click the “Disable Sync” link at the top of your admin page. When synchronization is disabled, you will see a warning message at the bottom of the admin portal.
Test Site Binding
The next feature in this release makes it easier to test your site without changing your DNS or modifying your host file. In the previous release the only way to view your site was using the host header such as mysite.example.com. In this release we have enabled an option for testing your site from the mysite.cloudapp.net url. If you enable the test site feature you will be able to view your site at: <myhost>.cloudapp.net/test/<sitename>. You can enable that option by editing your site configuration information as shown below.
Windows Azure CDN Support
Similar to the test site binding we also added a CDN binding. This allows you to use your web role accelerator sites as a backing store for the Windows Azure CDN. The Windows Azure CDN will only request content from <myhost>.cloudapp.net/cdn/ so we had to create a binding for this purpose. To allow this to work with multiple sites on the same role each site can have a binding at <myhost>.cloudapp.net/cdn/<sitename>. You can use this in conjunction with the Windows Azure CDN Helpers to easily host your static content from the CDN and improve your sites performance.
To enable support for the Windows Azure CDN in each site simply check the “Enable CDN” box in the site configuration page as shown below.
We hope this version of the accelerator improves your experience and makes it easier to you to deploy your sites to Windows Azure. As always, if you have any feedback please leave a comment here or on the CodePlex site.
Lori MacVittie (@lmacvittie) asserted “The quest for truly stateful failover continues…” in a introduction to her Mission Impossible: Stateful Cloud Failover post of 8/17/2011 to F5’s DevCentral blog:
Lightning was the latest cause of an outage at Amazon, this time in its European zones. Lightning, like tornadoes, volcanoes, and hurricanes are often categorized as “Acts of God” and therefore beyond the sphere of control of, well, anyone other than God. Outages or damages caused by such are rarely reimbursable and it’s very hard to blame an organization for not having a “plan” to react to the loss of both primary and secondary power supplies due to intense lightning strikes. The odds of a lightning strike are pretty high in the first place – 576,000 to 1 – and though the results can be disastrous, such risk is often categorized as low enough to not warrant a specific “plan” to redress.
What’s interesting about the analysis of the outage is the focus on what is, essentially, stateful failover capabilities. The Holy Grail of disaster recovery is to design a set of physically disparate systems in which the secondary system, in the event the primary fails, immediately takes over with no interruption in service or loss of data.
Yes, you read that right: it’s a zero-tolerance policy with respect to service and data loss.
And we’ve never, ever achieved it. Not that we haven’t tried, mind you, as Charles Babcock points out in his analysis of the outage:
Some companies now employ a form of disaster recovery that stores a duplicate set of virtual machines at a separate site; they're started up in the event of failure at the primary site. But Kodukula said such a process takes several minutes to get systems started at an alternative site. It also results in loss of several minutes worth of data.
Another alternative is to set up a data replication system to feed real-time data into the second site. If systems are kept running continuously, they can pick up the work of the failed systems with a minimum of data loss, he said. But companies need to employ their coordination expertise to make such a system work, and some data may still be lost.
-- Amazon Cloud Outage: What Can Be Learned? (Charles Babcock, InformationWeek, August 2011)
Disaster recovery plans are designed and implemented with the intention of minimizing loss. Practitioners are well aware that a zero-tolerance policy toward data loss for disaster recovery architectures is unrealistic. That is in part due to the “weakest link” theory that says a system’s <attribute> is only as good as its weakest component’s <attribute>. No application or network component can perform absolutely zero-tolerance failover, a.k.a. stateful failover, on its own. There is always the possibility that some few connections, transactions or sessions will be lost when a system fails over to a secondary system.
Consider it an immutable axiom of computer science that distributed systems can never be data-level consistent. Period. If you think about it, you’ll see why we can deduce, then, that we’ll likely never see stateful failover of network devices. Hint: it’s because ultimately all state, even in network components, is stored in some form of “database” whether structured, unstructured, or table-based and distributed systems can never be data-level consistent. And if even a single connection|transaction|session is lost from a component’s table, it’s not stateful, because stateful implies zero-tolerance for loss.
WHY ZERO-TOLERANCE LOSS is IMPOSSIBLE
Now consider what we’re trying to do in a failover situation. Generally speaking we’re talking about component-level failure which, in theory and practice, is much easier than a full-scale architectural failover scenario. One device fails, the secondary takes over. As long as data has been synchronized between the two, we should theoretically be able to achieve stateful failover, right?
Except we can’t. One of the realities with respect to high availability architectures is that synchronization is not continuous and neither is heartbeat monitoring (the mechanism by which redundant pairs periodically check to ensure the primary is still active). These processes occur on a period interval as defined by operational requirements, but are generally in the 3-5 second range. Assuming a connection from a client is made at point A, and the primary component fails at point A+1 second, it is unlikely that its session data will be replicated to the secondary before point A+3 seconds, at which time the secondary determines the primary has failed and takes over operation. This “miss” results in data loss. A minute amount, most likely, but it’s still data loss.
Basically the axiom that zero-tolerance loss is impossible is a manifestation in the network infrastructure of Brewer’s CAP theorem at work which says you cannot simultaneously have Consistency, Availability and Partition Tolerance. This is evident before we even consider the slight delay that occurs on the network despite the use of gratuitous ARP to ensure a smooth(er) transition between units in the event of a failover, during which time the service may be (rightfully) perceived as unavailable. But we don’t need to complicate things any more than they already are, methinks.
What is additionally frustrating, perhaps, is that the data loss could potentially be caused by a component other than the one that fails. That is, because of the interconnected and therefore interdependent nature of the network, a sort of cascading effect can occur in the event of a failover based on the topological design of the systems. It’s the architecture, silly, that determines whether data loss will be localized to the failing components or cascade throughout the entire architecture. High availability architectures based on a parallel data path design are subject to higher data loss throughout the network than are those based on cross-connected data path designs. Certainly the latter is more complicated and harder to manage, but it’s less prone to data loss cascade throughout the infrastructure.
Now, add in the possibility that cloud-based disaster recovery systems necessarily leverage a network connection instead of a point-to-point serial connection. Network latency can lengthen the process and, if the failure is in the network connection itself, is obviously going to negatively impact the amount of data lost because synchronization cannot occur at all for that period of time when the failed primary is still “active” and the secondary realizes there’s a problem, Houston, and takes over responsibility. Now take these potential hiccups and multiply them by every redundant component in the system you are trying to assure availability for, and remember to take into consideration that to fail over to a second “site” requires not only data (as in database) replication but also state data replication across the entire infrastructure.
Clearly, this is an unpossible task. A truly stateful cloud-based failover might be able to occur if the stars were aligned just right and the chickens sacrificed and the no-fail dance performed to do so. And even then, I’d bet against it happening. The replication of state residing in infrastructure components, regardless of how necessary they may be, is almost never, ever attempted.
The reality is that we have to count on some data loss and the best strategy we can have is to minimize the loss in part by minimizing the data that must be replicated and components that must be failed over.
Or is it?
<Return to section navigation list>
Windows Azure Platform Appliance (WAPA), Hyper-V and Private/Hybrid Clouds
No significant articles today.
<Return to section navigation list>
Cloud Security and Governance
Liz Macmillan asserted “” as a deck for her Cloud Computing: FireHost and Gazzang Partner post of 8/17/2011 to the Cloud Security Journal:
Gazzang on Wednesday announced that FireHost will sell Gazzang ezNcrypt. A member of the Cloud Security Alliance, FireHost recently announced a PCI 2.0 compliant public cloud. Now, customers who host database servers in FireHost's compliance-ready hosting solution will have access to Gazzang's transparent data encryption solution to help prevent unauthorized access, even from system administrators with full root access.
"Protecting the cloud requires sophisticated solutions that can stay ahead of the ever-growing list of threats, each more damaging than the last," said Larry Warnock, CEO of Gazzang. "FireHost has proven its commitment to security and is taking it one step further by offering customers access to data encryption solutions. Whether patient medical records, student education transcripts or consumer credit card information, Gazzang ezNcrypt offers FireHost customers the assurance that should data get into the wrong hands, it will be encrypted, and protected from harm."
Gazzang's ezNcrypt is a solution for encrypting MySQL database transactions and data. The solution's transparent data encryption (TDE) process auto-encrypts MySQL data at rest with dual authentication keys without making changes to existing applications or databases. Of key importance to hosting providers and their customers, ezNcrypt also protects critical data from being accessed by even root-level administrators as required by many government and regulatory guidelines. The solution is easy to implement and does not burden servers with undue performance degradation.
Other features include:
- Unlimited encryption of MySQL databases, tables and files
- Hosted Key Storage System (KSS)
- Protection against system and local users
- Compliance with PCI DSS, HIPAA, HITECH, FISMA and other industry-specific regulations
- Scalable, pay-as-you-go-per-server subscription licensing
"Database encryption is quickly becoming a last line of defense for websites with compliance and high traffic needs," said Mark McCurley, director of security and compliance for FireHost. "When it comes to protecting our customers, we make no compromises. Gazzang ezNcrypt offers the industry's only easy-to-use, effective solution for MySQL, so customers hosting database servers in the FireHost cloud can rest assured their data is protected by the leading enterprise security tools in the industry."
K. Scott Morrison (@KScottMorrison) reported The Cloud Security Alliance Introduces The Security, Trust and Assurance Registry in an 8/17/2011 post:
As a vendor of security products, I see a lot of Requests for Proposal (RFPs). More often than not these consist of an Excel spreadsheet with dozens—sometimes even hundreds—of questions ranging from how our products address business concerns to security minutia that only a high-geek can understand. RFPs are a lot of work for any vendor to respond to, but they are an important part of the selling process and we always take them seriously. RFPs are also a tremendous amount of work for the customer to prepare, so it’s not surprising that they vary greatly in sophistication.
I’ve always thought it would be nice if the SOA gateway space had a standardized set of basic questions that focused vendors and customers on the things that matter most in Governance, Risk and Compliance (GRC). In the cloud space, such a framework now exists. The Cloud Security Alliance (CSA) has introduced the Security, Trust and Assurance Registry (STAR), which is a series of questions designed to document the security controls a cloud provider has in place. IaaS, PaaS and SaaS cloud providers will self-assess their status and publish the results in the CSA’s centralized registry.
Providers report on their compliance with CSA best practices in two different ways. From the CSA STAR announcement:
1. The Consensus Assessments Initiative Questionnaire (CAIQ), which provides industry-accepted ways to document what security controls exist in IaaS, PaaS, and SaaS offerings. The questionnaire (CAIQ) provides a set of over 140 questions a cloud consumer and cloud auditor may wish to ask of a cloud provider. Providers may opt to submit a completed Consensus Assessments Initiative Questionnaire.
2. The Cloud Controls Matrix (CCM), which provides a controls framework that gives detailed understanding of security concepts and principles that are aligned to the Cloud Security Alliance guidance in 13 domains. As a framework, the CSA CCM provides organizations with the needed structure, detail and clarity relating to information security tailored to the cloud industry. Providers may choose to submit a report documenting compliance with Cloud Controls Matrix.The spreadsheets cover eleven control areas, each subdivided into a number of distinct control specifications. The control areas are:
- Compliance
- Data Governance
- Facility Security
- Human Resources
- Information Security
- Legal
- Operations Management
- Risk Management
- Release Management
- Resiliency
- Security Architecture
The CSA hopes that STAR will help to shorten purchasing cycles for cloud services because the assessment addresses many of the security concerns that users have today with the cloud. As with any benchmark, over time vendors will refine their product to do well against the test—and as with many benchmarks, this may be to the detriment of other important indicators. But this set of controls has been well thought through by the security professionals in the CSA community, so cramming for this test will be a positive step for security in the cloud.
<Return to section navigation list>
Cloud Computing Events
Julie Lerman (@julielerman) listed in an 8/17/2011 post Sessions for Vermont Code Camp III to be held 9/10/2011 in Burlington, VT:
Once again, we have a fabulous array of talks with many technologies covered. We’ll be building the actual schedule in the next week.
In alphabetical order …
Chris Bowen HTML5 - A Practical First Look
Rene Churchill A Holistic view of Website Performance
Christian Cote SSIS Whats new in SQL Server Denali?
Rob Friesel CSS Wrangling with SASS
John Garland What's New in Windows Phone 7.1 Silverlight Development
Vincent Grondin Mocking and mocking frameworks
David Howell Tackling Big Data with Hadoop
Everett McKay Effective Prototyping: A developer's guide to better design through prototyping
Dane Morgridge Testable Entity Framework
Dane Morgridge jQuery & CoffeeScript: Let The Awesomeness Begin
Dennis Perlot Silverlight Performance
Jonathan Phillips Functional Programming on the JVM
Al Ramirez What's New In ASP.NET MVC3
Maxime Rouiller ASP.NET MVC 3 for Web Developers
Josh Sled Dependency Injection
Josh Sled Emacs: Everday, Everyway
Eric Smith What is functional programming?
Kevin Thorley FOSS in the Enterprise
Etienne Tremblay Let’s talk Virtualization
Matt Van Horn Getting High on MEF
Joan Wortman & Maura Wilder Introduction to the Ext JS JavaScript framework "for Rich Apps in Every Browser"
John Zablocki .NET and NoSQL: Relaxing with CouchDBBonus Parlor Chat Session: Matt Van Horn: Software Consulting
More info: http://vtcodecamp.org
Register: http://vtcodecamp.eventbrite.org
<Return to section navigation list>
Other Cloud Computing Platforms and Services
Dina Bass reported VMware Enlists Dell to Help Wage ‘Operating Systems War’ in Cloud Business in an 8/17/2011 article for Bloomberg Technology News:
VMware Inc. (VMW) is working with Dell Inc. (DELL) and the Ubuntu Linux operating system to spur adoption of its cloud-computing software, ratcheting up competition withMicrosoft Corp. (MSFT) for corporate customers.
Dell’s services arm will help install VMware’s Cloud Foundry program, while Ubuntu will begin including parts of the software, said Jerry Chen, a vice president at VMware. EnStratus Networks LLC, which lets companies manage cloud-computing software, also will support Cloud Foundry.
VMware is forging alliances to capitalize on the shift to cloud computing, which lets businesses keep information in data centers. Cloud Foundry can store programs in different data centers, as well as a customer’s own facility, so there’s a backup in case of an outage. While that portion of market is still small, it may become as pivotal as operating systems are to computers, attracting Microsoft, VMware, Google Inc. (GOOG) and Salesforce.com Inc., said Ben Pring, an analyst at Gartner Inc. (IT)
“It’s still a very early-stage marketplace and very unproven,” he said. “They see it as the next strategic chokehold that a cloud vendor can get, the next go-round of the operating systems war.”
Gartner, based in Stamford, Connecticut, expects the market for this kind of product -- called application infrastructure services, or platform as a service -- to grow from $1.4 billion this year to $2.4 billion by 2015. That doesn’t include the management software and developer tools the companies can sell with it. It also doesn’t account for the cloud storage and computing power that companies such as Amazon.com Inc. offer separately, Pring said.
Virtualization Leader
VMware, based in Palo Alto, California, is the biggest seller of virtualization software, which lets companies use fewer server computers in data centers by running different operating systems on a single machine. VMware’s stock has risen 20 percent in the past year, boosted by companies putting more of their information in data centers. The company is majority-owned by EMC Corp., the largest maker of storage computers.
Cloud Foundry is still available only in a test version and includes an open-source project that lets outside developers contribute to the product.
VMware also aims to appeal to corporate customers with software that writes cloud applications using several different programming languages. That contrasts with the approach of Microsoft, which relies on .Net[*], and Google, which uses Java, Chen said.
Bigger Market
Most large customers currently use one technology or the other, so it’s hard to say how much demand there is for an all-in-one feature, Pring said. Still, the flexibility will help VMware target a wider swath of businesses, rather than just one camp, he said.
Microsoft may be the toughest adversary in wooing large corporate customers, Chen said. Because of Microsoft’s long history with business customers and its work on the .Net language, “they’re the one we worry about most in competing for enterprise customers,” he said.
Salesforce.com, which also has a partnership with VMware, is increasingly vying for those buyers as well, Pring said.
Having multiple copies of programs stored in different cloud data centers may be a draw for customers, particularly after Amazon’s Web Services suffered a brief outage this month and one lasting several days in April.
“If Amazon goes down, you are kind of out of luck,” Chen said. “If you put your app on Cloud Foundry and it’s running on VMware and Amazon, you can just move to another instance. You don’t have the problem of being locked into one vendor.”
* Chen appears to have forgotten that the Windows Azure Platform supports Java, PHP, Python and Ruby, in addition to .NET, and that Google Apps Engine’s native programming language is Python.
The Cloud Foundry and Crowbar (see below post) combination is probably what has dampened Dell’s enthusiasm for delivering on the Windows Azure Platform Appliance promised at Microsoft’s 2010 Worldwide Partner Conference. The only reference I can find to WAPA on the Dell site is a token Windows Azure Technology From Dell page.
Barton George (@Barton808) announced Cloud Foundry picks up Crowbar to speed installation in an 8/17/2011 post:
In case you’re not familiar with Cloud Foundry, it’s an open source Platform as a Service project initiated at VMware. More specifically it provides a platform for building, deploying, and running cloud apps using Spring for Java developers, Rails and Sinatra for Ruby developers, Node.js and other JVM frameworks including Grails.
The project began two years ago when VMware’s CEO Paul Maritz recruited Derek Collison and Mark Lucovsky out of Google and set them to working on Cloud Foundry. Collison and Lucovsky, who built and maintained Google’s API services, were brought into leverage their experience of working with hugely scaled out architectures.
The Cloud Foundry project has only been public for a matter of months and one question that I’m sure has popped into your mind is what if I want to pilot Cloud Foundry in my own environment, won’t installation and configuration be a total pain?
Enter the Crowbar
Crowbar is an open source software framework developed at Dell to speed up the installation and configuration of open source cloud software onto bare metal systems. By automating the process, Crowbar can reduce the time needed for installation from days to hours.
The software is modular in design so while the basic functionality is in Crowbar itself, “barclamps” sit on top of it to allow it work with a variety of projects. The first use for crowbar was for OpenStack and the barclamp for that has been donated to the community. Next came The Dell | Cloudera solution for Apache Hadoop and, just recently, Dreamhost announced that they currently working on a Ceph barclamp. And now…
Two great tastes that taste great together
Today’s big news is that VMware is working with Dell to release and maintain a Crowbar barclamp that, in conjunction with Crowbar, will install and configure Cloud Foundry. This capability, which will include multi-node configs over time, will allow organizations and service providers the ability to quickly and easily get pilots of Cloud Foundry up and running.
Once the initial deployment is complete, Crowbar can be used to maintain, expand, and architect the instance, including BIOS configuration, network discovery, status monitoring, performance data gathering, and alerting.
If you’d like to try out Crowbar for yourself, check out: https://github.com/DellCloudEdge
Extra-credit reading
- Collaboration between Dell Crowbar & VMware CloudFoundry – unleashes your inner cloud – Rob’s Blog
- Introducing the Dell | Cloudera solution for Apache Hadoop — Harnessing the power of big data
- DreamHost uses Dell’s Crowbar as a lever to raise automation.
- Dell announces availability of OpenStack solution; Open sources “Crowbar” software framework
- OpenStack Installer Demo at SXSW
- VMware taps Google ethos with Cloud Foundry
- VMware Launches Open Source PaaS Cloud Foundry
Pau for now…
Martin Tantow reported Cloud Cruiser Launches Cloud Cost Management For OpenStack in an 8/17/2011 post to the Cloud Times blog:
Cloud Cruiser launched today its cloud cost management solution for OpenStack in general availability; the solution features workflow tools that provide management and accounting capabilities for optimizing capex and opex costs in heterogeneous cloud computing environments. Cloud Cruiser develops cost management software that is designed specifically for enterprises and server providers to gain transparency, accountability, chargeback and proactive controls of their IT costs across a heterogeneous environment of internal, private and public workloads.
The OpenStack project is a global collaboration of developers which aims to deliver solutions for all types of clouds by being simple to implement, massively scalable, and feature rich.
According to James Staten, Vice President & Principal Analyst at Forrester, “OpenStack might just have the more credible story enterprise infrastructure and operations professionals have been waiting on.”
Jonathan Bryce, Rackspace Cloud Co-Founder and OpenStack Project Policy Board Chairman said “We are excited to have important tools needed for enterprise and service provider cloud deployments, like Cloud Cruiser’s cost management solution built for OpenStack. By adding support for OpenStack, the Cloud Cruiser cost management solution will be compatible with the growing number of public and private cloud deployments based on the emerging open cloud standard.”
Dave Zabrowski, founder and CEO of Cloud Cruiser added “By offering the industry’s first cost management solution for OpenStack, users will now have the capability to gain control of costs and visibility of their cloud-based computing environments with new tools to measure, manage and optimize capex and opex costs.”
Cloud Cruiser will be demonstrating the OpenStack solution in the OpenStack Booth at LinuxCon North America 2011 taking place August 17-19 in Vancouver.
Jeff Barr (@jeffbarr) reported a New - AWS GovCloud (US) Region - ITAR Compliant in an 8/16/2011 post:
A New Region
Our new AWS GovCloud (US) Region was designed to meet the unique regulatory requirements of the United States Government. The US federal government, state and local governments, and the contractors who support their mission now have access to secure, flexible, and cost-effective AWS services running in an environment that complies with US Government regulations for processing of sensitive workloads and storing sensitive data as described below.
The AWS GovCloud (US) Region supports the processing and storage of International Traffic in Arms (ITAR) controlled data and the hosting of ITAR controlled applications. As you may know, ITAR stipulates that all controlled data must be stored in an environment where logical and physical access is limited to US Persons (US citizens and permanent residents). This Region (and all of the AWS Regions) also provides FISMA Moderate controls. This means that we have completed the implementation of a series of controls and have also passed an independent security test and evaluation. Needless to say, it also supports existing security controls and certifications such as PCI DSS Level 1, ISO 27001, and SAS 70.
To demonstrate that GovCloud complies with ITAR, we have commissioned a third-party review of the ITAR compliance program for AWS GovCloud (US) and have received a favorable letter of attestation with respect to the stated ITAR objectives.
The Details
The new Region is located on the west coast of the US.All EC2 instances launched within this Region must reside within a Virtual Private Cloud (VPC). In addition to Amazon EC2, the following services are now available:
- Amazon Simple Storage Service (S3)
- Amazon Elastic Block Store (EBS)
- Amazon CloudWatch
- AWS Identity and Access Management (IAM)
If you are currently using one of the other AWS Regions, I'd like you to take note of one really important aspect of this release:
Other than the restriction to US persons and the requirement that EC2 instances are launched within a VPC, we didn't make any other changes to our usual operational systems or practices. In other words, the security profile of the existing Regions was already up to the task of protecting important processing and data. In effect, we simply put a gateway at the door -- "Please show your passport or green card before entering."
You can read more about our security processes, certifications, and accreditations in the AWS Security Center.
Full pricing information is available on the new GovCloud (US) page.
AWS in Action
I recently learned that more than 100 federal, state, and local government agencies are already using AWS in various ways. Here are some examples:
- The US Treasury's Recovery Accountability and Transparency board hosts recovery.gov on AWS (case study). The newest version of treasury.gov is also hosted on AWS (press release).
- NASA's Jet Propulsion Laboratory processes telemetry data and high resolution images on an array of EC2 cluster compute instances (case study).
The AWS Federal Government page contains a number of additional case studies and use cases.
Getting Access
Agencies with a need to access the AWS GovCloud must sign an AWS GovCloud (US) Enterprise Agreement. We will also make this Region accessible to government contractors, software integrators, and service providers with a demonstrated need for access. Those of you in this category will need to meet the requirements set out in ITAR Regulation 120.15.Help Wanted
The AWS team enjoys taking on large, complex challenges to deliver new services, features, and regions to our customers. A typical release represents the combined efforts of a multitude of developers, testers, writers, program managers, and business leaders.If you would like to work on large, complicated offerings such as AWS GovCloud, we'd love to talk to you. Here's a small sampling of our current job postings (there's a full list on the AWS careers page):
- Software Development Engineer
- IT Security Software Development Engineer
- Application Security Engineer
- Security Engineer
- Program Manager
- Amazon Web Services Public Sector Compliance Manager
-- Jeff;
PS - As you might be able to guess from the name of this Region, we would be interested in talking to other sovereign nations about their cloud computing needs.
Werner Vogels (@werner) posted Expanding the Cloud - The AWS GovCloud (US) Region to his All Things Distributed blog on 8/16/2011:
Today AWS announced the launch of the AWS GovCloud (US) Region. This new region, which is located on the West Coast of the US, helps US government agencies and contractors move more of their workloads to the cloud by implementing a number of US government-specific regulatory requirements.
The concept of regions gives AWS customers control over the placement of their resources and services. Next to GovCloud (US) there are five general purpose regions; two in the US (one on the west coast and one on the east coast), one in the EU (in Ireland) and two in APAC (in Singapore and Tokyo). There are different considerations when deciding where to allocate resources with latency and cost being the two obvious ones, but compliance sometimes plays an important role as well. For example a number of our European customers are subject to data residency requirements when it comes to PII data and they use the EU Region to meet to those requirements.
Our government customers sometimes have an additional layer of regulatory requirements given that they at times deal with highly sensitive information, such as defense-related data. These customers are satisfied with the general security controls and procedures in AWS but in these more sensitive cases they often need assurances that only personnel that meet certain requirements, e.g. citizenship or permanent residency, can access their data. AWS GovCloud (US) implements specific requirements of the US government such that agencies at the federal, state and local levels can use the AWS cloud for their more sensitive workloads.
Cloud First
The US Federal Cloud Computing Strategy lays out a "Cloud First" strategy which compels US federal agencies to consider Cloud Computing first as the target for their IT operations:
To harness the benefits of cloud computing, we have instituted a Cloud First policy. This policy is intended to accelerate the pace at which the government will realize the value of cloud computing by requiring agencies to evaluate safe, secure cloud computing options before making any new investments
By leveraging shared infrastructure and economies of scale, cloud computing presents a compelling business model for Federal leadership Organizations will be able to measure and pay for only the IT resources they consume, increase or decrease their usage to match requirements and budget constraints, and leverage the shared underlying capacity of IT resources via a network Resources needed to support mission critical capabilities can be provisioned more rapidly and with minimal overhead and routine provider interaction.
Given the current economic climate, reducing cost within the US federal government is essential – and an aggressive move to cloud will have a substantial positive impact on the governments IT budget. The move to the cloud is projected by 2015 see a reduction of 30% in IT infrastructure costs, which amounts to $7.2 billion. The application of the Cloud First strategy across all agencies will see many cost savings similar to what the GSA saw when they moved their main portal to the cloud: a savings of $1.7M on a yearly basis while greatly improving uptime and maintainability.
With AWS's strategy of continuous price reduction as additional economies of scale are achieved, many of these cost saving may become even more substantial without the agencies have to do anything.
Many US federal agencies are already migrating existing IT infrastructure onto the cloud using Amazon Web Services. The Cloud First strategy is most visible with new Federal IT programs, which are all designed to be "Cloud Ready"; many of these applications are launching on AWS from the start, and a number can be found on the AWS Federal use case list.
There were however a number of programs that really could benefit from the Cloud but which had unique regulatory requirements, such as ITAR, that blocked migration to AWS. ITAR is the International Traffic in Arms Regulatory framework which stipulates for example that data must be stored in an environment where physical and logical access is restricted to US Persons. There is no formal ITAR certification process, but a review of the ITAR compliance program for AWS GovCloud (US) has been conducted and resulted in a favorable letter of attestation with respect to the stated ITAR objects. This clears the path for agencies that have IT programs that need to be ITAR-compliant to start using AWS GovCloud (US) for these applications.
This new region, like all other AWS regions, provides FISMA Moderate controls and supports existing AWS security controls and certifications such as SAS-70, ISO 27001 and PCI DSS Level 1.
Government and Big Data
One particular early use case for AWS GovCloud (US) will be massive data processing and analytics. Several agencies of very different parts of the government have needs for data analytics that really put the Big in Big-Data, sometimes several orders of magnitude larger than commonly found in industry. Examples here are certain agencies that work on national security and those that work on economic recovery; their incoming data streams are exploding in size and their needs for collecting, storing, organizing, analyzing and sharing are changing rapidly. It is very difficult for an on-premise IT infrastructure to effectively address the needs of these agencies and the time scales at which they need to operate. The scalability, flexibility and the elasticity of AWS makes it an ideal environment for the agencies to run their analytics.
Often the data streams that they operate on are not classified in nature, but the combination and aggregation of these streams using complex new algorithms may fall for example under the controls of ITAR. AWS GovCloud (US) will be used by several of these agencies to help them with their Bigger-than-Big-Data needs.
More information
As with all AWS services and regions, information on GovCloud is publicly available on the AWS website, However, given the restrictive nature of this new AWS Region, customers will need to sign an AWS GovCloud Enterprise agreement that requires a manual step beyond the usual self-service signup process. To make use of the services in this region, customers will use the Amazon Virtual Private Cloud (VPC) to organize their AWS resources.
As the name of the region already suggests, we do not envision that over time GovCloud will address only the needs of the US Government and contractors. We are certainly interested in understanding whether there are opportunities in other governments with respect to their specific regulatory requirements that could be solved by a specialized region.
For more details on the AWS GovCloud (US) visit the detail pages and the Federal Government section of the AWS website and the posting on the AWS developer blog.
James Hamilton reported Hortonworks Taking Hadoop to Next Level in an 8/16/2011 post:
I got a chance to chat with Eric Baldeschwieler while he was visiting Seattle a couple of weeks back and catch up on what’s happening in the Hadoop world at Yahoo and beyond. Eric recently started Hortonworks whose tag line is “architecting the future of big data.” I’ve known Eric for years when he led the Hadoop team at Yahoo! most recently as VP of Hadoop Engineering. It was Eric’s team at Yahoo that contributed much of the code in Hadoop, Pig, and ZooKeeper.
Many of that same group form the core of Hortonworks whose mission is revolutionize and commoditize the storage and processing of big data via open source. Hortonworks continues to supply Hadoop engineering to Yahoo! And Yahoo! Is a key investor in Hortonworks along with Benchmark Capital. Hortonworks intends to continue to leverage the large Yahoo! development, test, and operations team. Yahoo! has over 1,000 Hadoop users and are running Hadoop over many clusters the largest of which was 4,000 nodes back in 2010. Hortonworks will be providing level 3 support for Yahoo! Engineering.
From Eric slides at the 2011 Hadoop summit, Hortonworks objectives:
Make Apache Hadoop projects easier to install, manage & use
Regular sustaining releases
- Compiled code for each project (e.g. RPMs)
- Testing at scale
Make Apache Hadoop more robust
- Performance gains
- High availability
- Administration & monitoring
Make Apache Hadoop easier to integrate & extend
- Open APIs for extension & experimentation
Hortonworks Technology Roadmap:
Phase 1: Making Hadoop Accessible (2011)
- Release the most stable Hadoop version ever
- Release directly usable code via Apache (RPMs, debs,…)
- Frequent sustaining releases off of the stable branches
Phase 2: Next Generation Apache Hadoop (2012)
- Address key product gaps (Hbase support, HA, Management, …)
- Enable community and partner innovation via modular architecture & open APIs
- Work with community to define integrated stack
Core
- HDFS Federation
- Next Gen MapReduce
- New Write Pipeline (HBase support)
- HA (no SPOF) and Wire compatibility
Data - HCatalog 0.3
- Pig, Hive, MapReduce and Streaming as clients
- HDFS and HBase as storage systems
- Performance and storage improvements
Management & Ease of use
- All components fully tested and deployable as a stack
- Stack installation and centralized config management
- REST and GUI for user tasks
Eric’s presentation from Hadoop Summit 2011 where he gave the keynote: Hortonworks: Architecting the Future of Big Data
<Return to section navigation list>
Atom 1.0 site feed
0 comments:
Post a Comment