OakLeaf Systems Windows Azure Table Services Sample Project Updated with Tools v1.4 and Storage Analytics

The Windows Azure Team updated Windows Azure Tools for Visual Studio 2010 v1.4 and released a new Windows Azure Storage Analytics feature on 8/3/2011. The Windows Azure Storage Team added Windows Azure Storage Metrics: Using Metrics to Track Storage Usage and Windows Azure Storage Logging: Using Logs to Track Storage Requests posts on the same date.

Here’s the start of the Storage Team’s Analytics feature description:

Windows Azure Storage Analytics offers you the ability to track, analyze, and debug your usage of storage (Blobs, Tables and Queues). You can use this data to analyze storage usage to improve the design of your applications and their access patterns to Windows Azure Storage. Analytics data consists of:

• Logs, which trace executed requests for your storage accounts
• Metrics, which provide summary of key capacity and request statistics for Blobs, Tables and Queues

I hadn’t worked on the OakLeaf Systems Azure Table Services Sample Project since May 2011, so I updated the project with v1.4 of the Azure SDK and Tools, added code to implement Storage Analytics, and changed from a brown/yellow (Colorful theme) to a more Web 2.0-like custom blue theme:

Note: Use of the “Powered by Windows Azure” logo is authorized by an earlier (v1.3) version having passed the Microsoft Platform Ready Test last November. For more details, see my Old OakLeaf Systems’ Azure Table Services Sample Project Passes New Microsoft Platform Ready Test post of 11/5/2010.

Prior OakLeaf posts have covered load tests and instrumentation for the sample project:

• My Load-Testing the OakLeaf Systems Azure Table Services Sample Project with up to 25 LoadStorm Users post of 11/18/2011 described some simple LoadStorm tests on the site.
• My Adding Trace, Event, Counter and Error Logging to the OakLeaf Systems Azure Table Services Sample Project of 12/5/2010 describes instrumentation added after the LoadStorm tests.

Azure Table Services Logging, Metrics and Analytics

Following is the source code in the Global.asax.cs file that determines the logging and analytic settings for table storage; the sample project doesn’t use blob or queue storage. AnalyticsSettings, AnalyticsSettingsExtensions and SettingsSerializerHelper classes perform the actual work. You can download the code for these classes here.

protected void Session_Start(object sender, EventArgs e)
{
    // AnalyticsSettings code added on 8/21/2011 from the Azure Storage Team's 
    // "Windows Azure Storage Logging: Using Logs to Track Storage Requests" post of 8/3/2011
    // http://blogs.msdn.com/b/windowsazurestorage/archive/2011/08/03/windows-azure-storage-logging-using-logs-to-track-storage-requests.aspx
    // See also AnalyticsSettings.cs and AnalyticsSettingsExtensions.cs

    AnalyticsSettings settings = new AnalyticsSettings()
    {
        LogType = LoggingLevel.Delete | LoggingLevel.Read | LoggingLevel.Write,
        IsLogRetentionPolicyEnabled = true,
        LogRetentionInDays = 1,
        IsMetricsRetentionPolicyEnabled = true,
        MetricsRetentionInDays = 7,
        MetricsType = MetricsType.All
    };

    var account = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    // CloudBlobClient blobClient = account.CreateCloudBlobClient();
    // CloudQueueClient queueClient = account.CreateCloudQueueClient(); 
    CloudTableClient tableClient = account.CreateCloudTableClient();

    // set the settings for each service
    // blobClient.SetServiceSettings(settings);
    // queueClient.SetServiceSettings(account.QueueEndpoint, settings);
    tableClient.SetServiceSettings(settings);

    // get the settings from each service
    // AnalyticsSettings blobSettings = blobClient.GetServiceSettings();
    // AnalyticsSettings queueSettings = queueClient.GetServiceSettings(account.QueueEndpoint);
    AnalyticsSettings tableSettings = tableClient.GetServiceSettings();
}

Here’s a report returned by Cerebrata’s free Windows Azure Storage Analytics Configuration Utility after clicking the Get Current Analytics Configuration button:

You can download it from https://www.cerebrata.com/Downloads/SACU/SACU.application.

Here’s a capture of Steve Marx’s online Windows Azure StorageAnalytics demo application that reads configuration data from Azure cloud storage:

I was surprised to see the following graph of Get Blob usage without having issued settings to the BlobClient:

 

The constant 118/hour blob requests probably are from diagnostic blobs generated by routine uptime tests from mon.itor.us and pingdom.com that I report monthly. August uptime data will show a few minutes of downtime as a result of replacing the old with the new version today:

Note: I wasn’t able to do a VIP Swap because the Windows Azure Portal didn’t detect that the new version had two endpoints, despite the fact that two Web Role instances were running in Staging.

Here’s a recent version of Cerebrata V2 (2011.03.0811.00) displaying summary data from the $MetricsTransactionsTable for today (8/21/2011):

Note: TotalEgress (bytes), Availability (%), AverageE2E Latency (ms), AverageServerLatency (ms), Percent Success (%) and about 20 more columns to the right of TotalIngress are hidden in the above capture. The Windows Azure Storage Metrics: Using Metrics to Track Storage Usage post provides a detailed description of each column’s contents in a very lengthy table. Cerebrata V2 was in private beta testing as of 8/21/2011. The current v2011.08.11.00 of Cerebrata’s Cloud Storage Studio includes identical storage analytics features.

After I collect more analytic data on the Table Services Sample Project, I’ll update this post.

Full disclosure: I have received no-charge licenses for Cerebrata products.

Technorati Tags: Windows Azure,Azure,Windows Azure Storage,Azure Storage Services,Windows Azure Storage Analytics,Windows Azure Storage Logging,Cerebrata,Cerebrata V2,Cerebrata Azure Storage Analytics Configuration Utility