Azure Storage Services Test Harness: Table Services 2 – the Table Services API

Update 1/31/2009: Code is now available for download.

This is the second of a series of articles about a ASP.NET 3.5 C# test harness for Azure Storage Services that is will be available for download from my “Retire Your Data Center” cover story for Visual Studio Magazine’s February 2009 issue (click the Get Code Download link) in the near future.

Note: The Azure Storage Services Test Harness: Table Services 1 – Introduction and Overview describes the test harness for the Customers table and Azure Table Storage services.

The Table Storage API defines structured storage for Table objects that contain Entity objects, both of which you manipulate by Representational State Transfer (REST) methods with Atom feed documents. Atom feeds conform to the Atom Publication (AtomPub) API and ADO.NET Data Services Framework’s Atom Serialization Rules. The API doesn’t support the JavaScript Object Notation (JSON) wire format.

Note: Although Table Storage API documentation mentions ADO.NET Data Services (Astoria) frequently, Table and Entity objects don’t implement the complete Astoria runtime. For example, Table Services doesn’t support $orderby or $skip query string options, but it does respect the $top=n option when you apply the Top(n) operator and $filter=querystring for a Where clause in LINQ to REST queries.

The API supports a $ct=LastPartitionKey/LastRowKey (continuation token) query string option to specify the initial entity for paging query EntitySets, which takes the place of the $skip option.

REST Operations on Tables

The API defines the following REST operations on Tables:

Create Table

The following HTTP POST request creates a CustomerTable if it doesn’t exist. This operation runs from code in the Global.asax.cs file’s Application_BeginRequest event handler*:

POST http://oakleaf.table.core.windows.net/Tables

POST /Tables HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Mon, 10 Nov 2008 16:06:02 GMT
Authorization: SharedKeyLite oakleaf:eRbDw5U7BkeSfZmKj71Zy3WTjrZCWkseVav3NK3tVqA=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
Host: oakleaf.table.core.windows.net
Content-Length: 499
Expect: 100-continue
Proxy-Connection: Keep-Alive

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" 
    xmlns="http://www.w3.org/2005/Atom">
  <title />
  <updated>2008-11-10T16:06:02.1557288Z</updated>
  <author>
    <name />
  </author>
  <id />
  <content type="application/xml">
    <m:properties>
      <d:TableName>CustomerTable</d:TableName>
    </m:properties>
  </content>
</entry>

*Handling the Application_BeginRequest (rather than the Application_Start) event is required because Windows Azure runs IIS 7 in Integrated mode, which throws an exception if attempted in the latter handler. (See Mike Volodarsky’s IIS7 Integrated mode: Request is not available in this context exception in Application_Start post of 11/10/2007.)

Notes: The maximum skew between the x-ms-date value and server UTC is 15 minutes. Blob Storage, Table Storage, and Queue support the SharedKey authentication scheme. The key signature is a Hash Message Authentication Code (HMAC) constructed from the request and computed with the SHA256 algorithm, then encoded using Base64 encoding. The ADO.NET Data Services’ .NET Client library (System.Data.Services.Client) supports a simpler SharedKeyLite authentication scheme only.

If the table doesn’t exist, the project creates an empty table. If the table exists, the POST attempt sends the following error response:

HTTP/1.1 409 The table specified already exists.
Cache-Control: no-cache
Content-Length: 258
Content-Type: application/xml
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 7b66eaa7-68f4-4926-ae43-579074b71c04
Date: Mon, 10 Nov 2008 16:05:50 GMT

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata">
  <code>TableAlreadyExists</code>
  <message xml:lang="en-US">The table specified already exists.</message>
</error>

Query Tables

Here’s the request to return a list of table names as an IEnumerable<string> type:

GET http://myaccount.table.core.windows.net/Tables

GET /Tables() HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Tue, 11 Nov 2008 00:27:35 GMT
Authorization: SharedKeyLite oakleaf:J6YsUkLyrkHc5DW63K/NGQakgaQge+RfMfcIfMwRGf8=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Host: oakleaf.table.core.windows.net

And here’s the response:

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: e7f01114-bd1e-47c3-9c85-26b12080f30c
Date: Tue, 11 Nov 2008 00:28:55 GMT
Content-Length: 1036

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<feed xml:base="http://oakleaf.table.core.windows.net/" 
    xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" 
    xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" 
    xmlns="http://www.w3.org/2005/Atom">
  <title type="text">Tables</title>
  <id>http://oakleaf.table.core.windows.net/Tables</id>
  <updated>2008-11-11T00:28:56Z</updated>
  <link rel="self" title="Tables" href="Tables" />
  <entry>
    <id>http://oakleaf.table.core.windows.net/Tables('CustomerTable')</id>
    <title type="text"></title>
    <updated>2008-11-11T00:28:56Z</updated>
    <author>
      <name />
    </author>
    <link rel="edit" title="Tables" href="Tables('CustomerTable')" />
    <category term="oakleaf.Tables"
       scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
    <content type="application/xml">
      <m:properties>
        <d:TableName>CustomerTable</d:TableName>
      </m:properties>
    </content>
  </entry>
</feed>

Delete Table

Deleting a table is much faster than removing all entities and starting over. Here’s the request:

DELETE http://oakleaf.table.core.windows.net/Tables('CustomerTable')

DELETE /Tables('CustomerTable') HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Tue, 11 Nov 2008 01:04:28 GMT
Authorization: SharedKeyLite oakleaf:Y8hQOQjVqshUUif6E/gKDIB0Ga0rD5P7ENOHsmcAqAs=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
Host: oakleaf.table.core.windows.net
Content-Length: 0

and the response:

HTTP/1.1 204 No Content
Cache-Control: no-cache
Content-Length: 0
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 9e047535-cf33-4bbc-8a83-fa4472e36c65
Date: Tue, 11 Nov 2008 01:05:50 GMT

REST Operations on Entities

The Table API Supports the following REST operations on entities:

Insert Entity (HTTP POST)

Following is the HTTP POST request to insert a single Customer entity:

POST http://oakleaf.table.core.windows.net/CustomerTable

POST /CustomerTable HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Mon, 10 Nov 2008 16:43:18 GMT
Authorization: SharedKeyLite oakleaf:9G5cV5Ad9HVu55GEwIq524OtgK1YI5Tq8IxYn25y8AY=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
Host: oakleaf.table.core.windows.net
Content-Length: 1083
Expect: 100-continue

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
  <title />
  <updated>2008-11-10T16:43:18.802576Z</updated>
  <author>
    <name />
  </author>
  <id />
  <content type="application/xml">
    <m:properties>
      <d:Address>Obere Str. 57</d:Address>
      <d:City>Berlin</d:City>
      <d:CompanyName>Alfreds Futterkiste</d:CompanyName>
      <d:ContactName>Maria Anders</d:ContactName>
      <d:ContactTitle>Sales Representative</d:ContactTitle>
      <d:Country>Germany</d:Country>
      <d:CustomerID>ALFKI</d:CustomerID>
      <d:Fax>030-0076545</d:Fax>
      <d:PartitionKey>Customers</d:PartitionKey>
      <d:Phone>030-0074321</d:Phone>
      <d:PostalCode>12209</d:PostalCode>
      <d:Region m:null="true" />
      <d:RowKey>ALFKI</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">0001-01-01T00:00:00</d:Timestamp>
    </m:properties>
  </content>
</entry>

and the response:

HTTP/1.1 201 Created
Cache-Control: no-cache
Transfer-Encoding: chunked
Content-Type: application/atom+xml;charset=utf-8
ETag: W/"datetime'2008-11-10T16%3A43%3A09.274Z'"
Location: http://oakleaf.table.core.windows.net/CustomerTable(PartitionKey='Customers',RowKey='ALFKI')
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 315d2a3c-32a7-405f-b967-2a6d7c7774a2
Date: Mon, 10 Nov 2008 16:43:08 GMT

5D6
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xml:base="http://oakleaf.table.core.windows.net/" xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" m:etag="W/&quot;datetime'2008-11-10T16%3A43%3A09.274Z'&quot;" xmlns="http://www.w3.org/2005/Atom">
  <id>http://oakleaf.table.core.windows.net/CustomerTable(PartitionKey='Customers',RowKey='ALFKI')</id>
  <title type="text"></title>
  <updated>2008-11-10T16:43:09Z</updated>
  <author>
    <name />
  </author>
  <link rel="edit" title="CustomerTable" href="CustomerTable(PartitionKey='Customers',RowKey='ALFKI')" />
  <category term="oakleaf.CustomerTable" scheme="http://schemas.microsoft.com/ado/2007/08/dataservices/scheme" />
  <content type="application/xml">
    <m:properties>
      <d:PartitionKey>Customers</d:PartitionKey>
      <d:RowKey>ALFKI</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">2008-11-10T16:43:09.274Z</d:Timestamp>
      <d:Address>Obere Str. 57</d:Address>
      <d:City>Berlin</d:City>
      <d:CompanyName>Alfreds Futterkiste</d:CompanyName>
      <d:ContactName>Maria Anders</d:ContactName>
      <d:ContactTitle>Sales Representative</d:ContactTitle>
      <d:Country>Germany</d:Country>
      <d:CustomerID>ALFKI</d:CustomerID>
      <d:Fax>030-0076545</d:Fax>
      <d:Phone>030-0074321</d:Phone>
      <d:PostalCode>12209</d:PostalCode>
    </m:properties>
  </content>
</entry>
0

Query Entities (HTTP GET)

Following is the GET request header for the first eight CustomerTable entities:

GET http://oakleaf.table.core.windows.net/CustomerTable()?$top=8

GET /CustomerTable()?$top=8 HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Mon, 10 Nov 2008 16:06:02 GMT
Authorization: SharedKeyLite oakleaf:lk0wzp7lph5a/CdBBekQnwgFkIz0ZUG0Xr3qsWZEFWs=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Host: oakleaf.table.core.windows.net

and the response (without the AtomPub body)

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Type: application/atom+xml;charset=utf-8
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 6c5e2432-1b60-4d56-8915-37b4b1edd375
x-ms-continuation-NextPartitionKey: Customers
x-ms-continuation-NextRowKey: BONAP
Date: Mon, 10 Nov 2008 16:05:50 GMT
Content-Length: 10752

A query returns a maximum of 1,000 rows. The NextPartitionKey and NextRowKey values are used for paging results.

Update or Merge Entity (HTTP PUT or MERGE)

The HTTP PUT method deletes and recreates the entity. The MERGE method enables replacing individual property values:

MERGE http://oakleaf.table.core.windows.net/CustomerTable(PartitionKey="Customers", RowKey="ALFKI") 

MERGE /CustomerTable(PartitionKey='Customers',RowKey='ALFKI') HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Mon, 10 Nov 2008 17:34:17 GMT
Authorization: SharedKeyLite oakleaf:uKKpW70RiQI0mio90bXkBJ2CxZX5bHhhQQRHuxNXG3Q=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
If-Match: W/"datetime'2008-11-10T16%3A43%3A09.274Z'"
Host: oakleaf.table.core.windows.net
Content-Length: 1194
Expect: 100-continue

<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<entry xmlns:d="http://schemas.microsoft.com/ado/2007/08/dataservices" xmlns:m="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata" xmlns="http://www.w3.org/2005/Atom">
  <title />
  <updated>2008-11-10T17:34:17.1314394Z</updated>
  <author>
    <name />
  </author>
  <id>http://oakleaf.table.core.windows.net/CustomerTable(PartitionKey='Customers',RowKey='ALFKI')</id>
  <content type="application/xml">
    <m:properties>
      <d:Address>Obere Str. 57</d:Address>
      <d:City>Berlin</d:City>
      <d:CompanyName>Alfreds Futterkiste (Updated)</d:CompanyName>
      <d:ContactName>Maria Anders</d:ContactName>
      <d:ContactTitle>Sales Representative</d:ContactTitle>
      <d:Country>Germany</d:Country>
      <d:CustomerID>ALFKI</d:CustomerID>
      <d:Fax>030-0076545</d:Fax>
      <d:PartitionKey>Customers</d:PartitionKey>
      <d:Phone>030-0074321</d:Phone>
      <d:PostalCode>12209</d:PostalCode>
      <d:Region m:null="true" />
      <d:RowKey>ALFKI</d:RowKey>
      <d:Timestamp m:type="Edm.DateTime">2008-11-10T16:43:09.274Z</d:Timestamp>
    </m:properties>
  </content>
</entry>

Here are the response headers with the updated Timestamp value in the ETag header:

HTTP/1.1 204 No Content
Cache-Control: no-cache
Content-Length: 0
ETag: W/"datetime'2008-11-10T17%3A35%3A18.6307782Z'"
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 7687059d-ab33-4c88-bc7b-1aadb6aab331
Date: Mon, 10 Nov 2008 17:34:14 GMT

Delete Entity (HTTP DELETE)

Delete a single entity with the following DELETE request header:

DELETE http://oakleaf.table.core.windows.net/CustomerTable(PartitionKey="Customers", RowKey="ALFKI") 

DELETE /CustomerTable(PartitionKey='Customers',RowKey='ALFKI') HTTP/1.1
User-Agent: Microsoft ADO.NET Data Services
x-ms-date: Mon, 10 Nov 2008 16:35:57 GMT
Authorization: SharedKeyLite oakleaf:PMEUEMoho1GjyedXSeLzfynWUx9OP/oPRad2sO9dgqk=
Accept: application/atom+xml,application/xml
Accept-Charset: UTF-8
DataServiceVersion: 1.0;NetFx
MaxDataServiceVersion: 1.0;NetFx
Content-Type: application/atom+xml
If-Match: W/"datetime'2008-11-10T00%3A53%3A39.2200295Z'"
Host: oakleaf.table.core.windows.net
Content-Length: 0

and receive this response header:

HTTP/1.1 204 No Content
Cache-Control: no-cache
Content-Length: 0
Server: Table Service Version 1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: d1f0cf12-d82c-4e77-b67b-64ce8494a534
Date: Mon, 10 Nov 2008 16:35:30 GMT

Updated 11/18/2008 for a modification that runs the CreateTablesFromModel() method from the Application_BeginRequest event handler instead of for every operation. See Steve Marx’s Try to Create Tables Only Once post of 11/18/2008.

The next episode in this series is Azure Storage Services Test Harness: Table Services 3 –Starting the Test Harness Project of 11/20/2008.

Technorati Tags: Azure Data Services,Azure Table Services,Azure Services,Azure Services Platform,Windows Azure,Cloud Computing,REST APIs,AtomPub