Scalability and Cost Issues with Windows Azure Web and Worker Role Projects – Live Demo
The Thumbnails sample project from the Windows Azure SDK (March 2009 CTP)’s Samples.zip file illustrates combining Web and Worker Roles to upload graphics files from the user’s local file system into Azure Blobs and display thumbnail images from these in a bound ListView control. An AJAX UpdatePanel contains the ListView to minimize UI flashing on postbacks. A Worker Role handles the thumbnail generation process asynchronously by polling the ‘thumbnailmaker’ queue at one-second intervals.
Update 4/21/2009: Comment added by Microsoft’s Steve Marx, who wrote the original Thumbnails.sln sample project; details on the method causing the problem; and hourly network usage data from Windows Azure Metrics. See end of this post.
Update 4/22/2009: Simple modification to eliminate JavaScript exception in Steve Marx’s fix.
Update 4/24/2009: Visit My Thumbnails.sln Original and Enhanced Versions – Flow Diagrams post for flow diagrams of the original and fixed versions of the Thumbnails projects.
Here’s a screen capture of the original Thumbnails.sln’s Web Role with 15 *.jpg thumbnails from *.png images used in previous OakLeaf posts. (Click images to display full-size 1,024 x 768 captures.)
You can run the original Thumbnails.sln project at http://oakleaf6.cloudapp.net/Default.aspx.
Following is the WorkerRole’s HTTP ASYNC Request message with 3,110 bytes of ViewState:
POST /Default.aspx HTTP/1.1 Accept: */* Accept-Language: en-us Referer: http://1f33a544-f6d1-431f-bf6d-42fe54904995.cloudapp.net/Default.aspx x-microsoftajax: Delta=true Content-Type: application/x-www-form-urlencoded; charset=utf-8 Cache-Control: no-cache Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30428; .NET CLR 3.5.30729; .NET CLR 3.0.30618; MS-RTC LM 8; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.1.3) Host: 1f33a544-f6d1-431f-bf6d-42fe54904995.cloudapp.net Content-Length: 3110 Connection: Keep-Alive Pragma: no-cache sm1=up1%7Ctimer1&__EVENTTARGET=timer1&__EVENTARGUMENT=&__VIEWSTATE=%2FwEPDwUJOTA5OTI0ODEyD2QWAgIDDxYCHgdlbmN0eXBlBRNtdWx0aXBhcnQvZm9ybS1kYXRhFgICBw9kFgJmD2QWA ... yM2RkGAEFCnRodW1ibmFpbHMPPCsACgIHPCsADwAIAg9kum72snd50MHlpjvMEsnb17lu6dc%3D&__EVENTVALIDATION=%2FwEWAgL094j7BwKSuuDUC8HaRKdWg3LeOCFwZBwLVxzVxzF2&__ASYNCPOST=true&
and here’s the HTTP Response message with a list of blobs having a ‘thumbnails/’ prefix and ViewState, which total 7,217 bytes:
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/plain; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Mon, 20 Apr 2009 19:39:51 GMT
Content-Length: 7217
3488|updatePanel|up1|
<img id="thumbnails_ctrl0_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633755652469388030_95a72280-fd3e-438a-9624-73b7cfc5e109" style="border-width:0px;" />
<img id="thumbnails_ctrl1_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633756498271797216_1f06998c-d6f9-423c-b8ba-66277e7aebc6" style="border-width:0px;" />
<img id="thumbnails_ctrl2_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757268772274027_d895bb79-8a41-44c4-9514-d391faff878d" style="border-width:0px;" />
<img id="thumbnails_ctrl3_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757440636206298_114f3d2f-4fbc-4d83-9831-954ae0693734" style="border-width:0px;" />
<img id="thumbnails_ctrl4_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758473471680000_549f111f-f1e4-4652-b886-d1b703740797" style="border-width:0px;" />
<img id="thumbnails_ctrl5_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758476929180000_5cc14527-64be-4e19-ba2a-5b19fd1f51dd" style="border-width:0px;" />
<img id="thumbnails_ctrl6_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758477105033750_e25614be-d2a6-46d9-87f3-a0eb63a7b392" style="border-width:0px;" />
<img id="thumbnails_ctrl7_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478020658750_d56f76ad-9a26-46f2-ae0c-c7ce30d228ef" style="border-width:0px;" />
<img id="thumbnails_ctrl8_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478137773750_923ab9f2-3607-4665-8f59-45df1fcc3a77" style="border-width:0px;" />
<img id="thumbnails_ctrl9_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479545430000_74ff03eb-878b-490e-808a-0f6792f726f7" style="border-width:0px;" />
<img id="thumbnails_ctrl10_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479706752500_9a5a7c6f-1dd9-48bc-82b5-aaaa2c431b78" style="border-width:0px;" />
<img id="thumbnails_ctrl11_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758480860273750_e54eee86-8ddb-44cb-82b8-badbaa1b2a8d" style="border-width:0px;" />
<img id="thumbnails_ctrl12_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482608086250_bfbd7e5a-026c-45ce-88f7-b6eb563af89b" style="border-width:0px;" />
<img id="thumbnails_ctrl13_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482981283750_d41a21a7-def6-4688-ae29-74109b6050a7" style="border-width:0px;" />
<img id="thumbnails_ctrl14_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758484541283750_6d03a107-dfc7-4551-8201-8052ddb9ec23" style="border-width:0px;" />
<span id="timer1" style="visibility:hidden;display:none;"></span>
|0|hiddenField|__EVENTTARGET||0|hiddenField|__EVENTARGUMENT||2952|hiddenField|__VIEWSTATE|/wEPDwUJOTA5OTI0ODEyD2QWAgIDDxYCHgdlbmN0eXBlBRNtdWx0aXBhcnQvZm9ybS1kYXRhFgICBw9kFgJmD2QWAg
...
M2RkGAEFCnRodW1ibmFpbHMPPCsACgIHPCsADwAIAg9kum72snd50MHlpjvMEsnb17lu6dc=|48|hiddenField|__EVENTVALIDATION|/wEWAgL094j7BwKSuuDUC8HaRKdWg3LeOCFwZBwLVxzVxzF2|0|asyncPostBackControlIDs|||0|postBackControlIDs|||4|updatePanelIDs||tup1|0|childUpdatePanelIDs|||3|panelsToRefreshIDs||up1|2|asyncPostBackTimeout||90|12|formAction||Default.aspx|13|pageTitle||Photo Gallery|149|scriptBlock|ScriptPath|/ScriptResource.axd?d=9YtLxwU-nmRnm7oJ9nOfEaztduuiriMqe964NLCEARKknzUa7EJGulUSq-QKJyN3_XQG98ij__ElfezvDJkF-OgjZtff28LO1U1_MHbdocg1&t=ffffffff9a77c993|155|scriptStartupBlock|ScriptContentNoTags|Sys.Application.add_init(function() {
$create(Sys.UI._Timer, {"enabled":true,"interval":1000,"uniqueID":"timer1"}, null, null, $get("timer1"));
});
|
Note: Run your own tests with the live demo and Fiddler2 if you’d like to see the unexpurgated viewstate.
Following is a screen capture of an upgraded version of the Thumbnails.sln project from Chapter 8, “Messaging with Azure Queues,” of my forthcoming Cloud Computing with the Microsoft Azure Services Platform book for WROX:
Modifications include the addition of a GridView control to display five attributes of the PhotoGallery blobs having a ‘thumbnails/’ prefix. (The ContentEncoding and ContentLanguage attributes aren’t populated) Another important feature is the ability to delete all but three thumbnails and their source blobs.
You can run the upgraded Thumbnails.sln project at http://oakleaf5.cloudapp.net/Default.aspx.
Here’s the HTTP Request message with 9,179 payload bytes (The GridView increases the ViewState payload):
POST /Default.aspx HTTP/1.1 Accept: */* Accept-Language: en-us Referer: http://oakleaf5.cloudapp.net/Default.aspx x-microsoftajax: Delta=true Content-Type: application/x-www-form-urlencoded; charset=utf-8 Cache-Control: no-cache Accept-Encoding: gzip, deflate User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0; GTB6; SLCC1; .NET CLR 2.0.50727; Media Center PC 5.0; .NET CLR 3.5.21022; .NET CLR 3.5.30428; .NET CLR 3.5.30729; .NET CLR 3.0.30618; MS-RTC LM 8; InfoPath.2; OfficeLiveConnector.1.3; OfficeLivePatch.1.3) Host: oakleaf5.cloudapp.net Content-Length: 9179 Connection: Keep-Alive Pragma: no-cache sm1=up1%7Ctimer1&txtTime=&__EVENTTARGET=timer1&__EVENTARGUMENT=&__VIEWSTATE=WR%2BUOP22CwPyX73JtlDmUmmPrRsHh%2BiBwiRDWCm%2BViU00I%2B4hxdW5gvjpq9nCGyVwHj1CdJ8bwxR3oBOYshQUQi2TSlwFRCsOrbowEk4cyE7Ykxo3FcFv9q0k7COA092NHbEls1 ... 2FxpttnPCr4zm6hlQbOPlbRXR0yso6CJk9acjrOxqRWOw5l15ypXkVJ0jXv7svxpJ34Ob%2BUGRboSIp6t8xMB7n5mB7cnUqBIsqW7bFnn01Z5gvCxUajLJFx1mUs7XkUr4Yfsbj9w0xTzcL7IMJenC%2BriHNm7GSCJnONw%3D&__VIEWSTATEENCRYPTED=&__ASYNCPOST=true&
and the HTTP Response with payload bytes:
HTTP/1.1 200 OK
Cache-Control: private
Content-Type: text/plain; charset=utf-8
Server: Microsoft-IIS/7.0
X-AspNet-Version: 2.0.50727
X-Powered-By: ASP.NET
Date: Mon, 20 Apr 2009 20:51:21 GMT
Content-Length: 18849
9586|updatePanel|up1|
<img id="thumbnails_ctrl0_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633755652469388030_95a72280-fd3e-438a-9624-73b7cfc5e109" style="border-width:0px;" />
<img id="thumbnails_ctrl1_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633756498271797216_1f06998c-d6f9-423c-b8ba-66277e7aebc6" style="border-width:0px;" />
<img id="thumbnails_ctrl2_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757268772274027_d895bb79-8a41-44c4-9514-d391faff878d" style="border-width:0px;" />
<img id="thumbnails_ctrl3_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633757440636206298_114f3d2f-4fbc-4d83-9831-954ae0693734" style="border-width:0px;" />
<img id="thumbnails_ctrl4_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758473471680000_549f111f-f1e4-4652-b886-d1b703740797" style="border-width:0px;" />
<img id="thumbnails_ctrl5_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758476929180000_5cc14527-64be-4e19-ba2a-5b19fd1f51dd" style="border-width:0px;" />
<img id="thumbnails_ctrl6_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758477105033750_e25614be-d2a6-46d9-87f3-a0eb63a7b392" style="border-width:0px;" />
<img id="thumbnails_ctrl7_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478020658750_d56f76ad-9a26-46f2-ae0c-c7ce30d228ef" style="border-width:0px;" />
<img id="thumbnails_ctrl8_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758478137773750_923ab9f2-3607-4665-8f59-45df1fcc3a77" style="border-width:0px;" />
<img id="thumbnails_ctrl9_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479545430000_74ff03eb-878b-490e-808a-0f6792f726f7" style="border-width:0px;" />
<img id="thumbnails_ctrl10_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758479706752500_9a5a7c6f-1dd9-48bc-82b5-aaaa2c431b78" style="border-width:0px;" />
<img id="thumbnails_ctrl11_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758480860273750_e54eee86-8ddb-44cb-82b8-badbaa1b2a8d" style="border-width:0px;" />
<img id="thumbnails_ctrl12_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482608086250_bfbd7e5a-026c-45ce-88f7-b6eb563af89b" style="border-width:0px;" />
<img id="thumbnails_ctrl13_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758482981283750_d41a21a7-def6-4688-ae29-74109b6050a7" style="border-width:0px;" />
<img id="thumbnails_ctrl14_photoImage" src="http://oakleaf3.blob.core.windows.net/photogallery/thumbnails/1633758484541283750_6d03a107-dfc7-4551-8201-8052ddb9ec23" style="border-width:0px;" />
<div id="timer" style="height: 37px">
Create Thumbnail Time, seconds:
<input name="txtTime" type="text" id="txtTime" style="width:67px;" />
</div>
<div id="divGridView"">
<div>
<table title="Displays the current thumbnail blobs." cellspacing="0" cellpadding="4" rules="cols" border="1" id="gvBlobs" style="color:#6B696B;background-color:White;border-color:#DEDFDE;border-width:1px;border-style:None;border-collapse:collapse;">
<tr style="color:White;background-color:#6B696B;font-weight:bold;">
<th scope="col">Blobs</th><th scope="col">Name</th><th scope="col">ContentType</th><th scope="col">ContentLength</th><th scope="col">LastModifiedTime</th><th scope="col">ETag</th>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$0')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633755652469388030_95a72280-fd3e-438a-9624-73b7cfc5e109</td><td>image/jpeg</td><td>3993</td><td>4/17/2009 6:39:39 PM</td><td>0x8CB8DA2FFD51660</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$1')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633756498271797216_1f06998c-d6f9-423c-b8ba-66277e7aebc6</td><td>image/jpeg</td><td>4269</td><td>4/18/2009 6:10:01 PM</td><td>0x8CB8E6806E01B60</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$2')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633757268772274027_d895bb79-8a41-44c4-9514-d391faff878d</td><td>image/jpeg</td><td>4803</td><td>4/19/2009 3:33:40 PM</td><td>0x8CB8F1B598712C0</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$3')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633757440636206298_114f3d2f-4fbc-4d83-9831-954ae0693734</td><td>image/jpeg</td><td>3761</td><td>4/19/2009 8:20:12 PM</td><td>0x8CB8F4360FB7640</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$4')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758473471680000_549f111f-f1e4-4652-b886-d1b703740797</td><td>image/jpeg</td><td>3433</td><td>4/20/2009 6:01:35 PM</td><td>0x8CB8FF92DB10050</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$5')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758476929180000_5cc14527-64be-4e19-ba2a-5b19fd1f51dd</td><td>image/jpeg</td><td>3584</td><td>4/20/2009 6:07:20 PM</td><td>0x8CB8FF9FBC05040</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$6')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758477105033750_e25614be-d2a6-46d9-87f3-a0eb63a7b392</td><td>image/jpeg</td><td>4523</td><td>4/20/2009 6:08:42 PM</td><td>0x8CB8FFA2C29CEA0</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$7')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758478020658750_d56f76ad-9a26-46f2-ae0c-c7ce30d228ef</td><td>image/jpeg</td><td>5346</td><td>4/20/2009 6:10:13 PM</td><td>0x8CB8FFA62786520</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$8')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758478137773750_923ab9f2-3607-4665-8f59-45df1fcc3a77</td><td>image/jpeg</td><td>4205</td><td>4/20/2009 6:09:22 PM</td><td>0x8CB8FFA440486F0</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$9')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758479545430000_74ff03eb-878b-490e-808a-0f6792f726f7</td><td>image/jpeg</td><td>5058</td><td>4/20/2009 6:11:43 PM</td><td>0x8CB8FFA986DB810</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$10')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758479706752500_9a5a7c6f-1dd9-48bc-82b5-aaaa2c431b78</td><td>image/jpeg</td><td>3675</td><td>4/20/2009 6:13:01 PM</td><td>0x8CB8FFAC69E8480</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$11')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758480860273750_e54eee86-8ddb-44cb-82b8-badbaa1b2a8d</td><td>image/jpeg</td><td>3218</td><td>4/20/2009 6:13:54 PM</td><td>0x8CB8FFAE65235B0</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$12')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758482608086250_bfbd7e5a-026c-45ce-88f7-b6eb563af89b</td><td>image/jpeg</td><td>4567</td><td>4/20/2009 6:16:49 PM</td><td>0x8CB8FFB4E5FC870</td>
</tr><tr style="background-color:White;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$13')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758482981283750_d41a21a7-def6-4688-ae29-74109b6050a7</td><td>image/jpeg</td><td>2909</td><td>4/20/2009 6:18:28 PM</td><td>0x8CB8FFB89EB7740</td>
</tr><tr style="background-color:#F7F7DE;">
<td><a href="javascript:__doPostBack('gvBlobs','Delete$14')" style="color:#6B696B;">Delete</a></td><td>thumbnails/1633758484541283750_6d03a107-dfc7-4551-8201-8052ddb9ec23</td><td>image/jpeg</td><td>4005</td><td>4/20/2009 6:21:04 PM</td><td>0x8CB8FFBE6BEBDB0</td>
</tr>
</table>
</div><span id="statusMessage" style="color:Red;"></span>
</div><span id="timer1" style="visibility:hidden;display:none;"></span>
|0|hiddenField|__EVENTTARGET||0|hiddenField|__EVENTARGUMENT||8300|hiddenField|__VIEWSTATE|WR+UOP22CwPyX73JtlDmUmmPrRsHh+iBwiRDWCm+ViU00I+
...
+W3PbzKDhmGYLycE6Aw4kYh4TL842jUgYDkeXmfSwpIUW621XJxMRbpw6x4fJPslo2aJZPiGi6OuPuXXTGS0ozejw6pMwaDuWvI9EPxEnMQ+gojkKoAusxU6q+lA0benFiF9OlukXDn68uEK+oso0hoMV8wfDVBDWqG6bJi2kIUNPCiaNAzVRW1rxLJETlwtbl6d6LA=|0|hiddenField|__VIEWSTATEENCRYPTED||172|hiddenField|__EVENTVALIDATION|Hi0WJQfF/xpttnPCr4zm6hlQbOPlbRXR0yso6CJk9acjrOxqRWOw5l15ypXkVJ0jXv7svxpJ34Ob+UGRboSIp6t8xMB7n5mB7cnUqBIsqW7bFnn01Z5gvCxUajLJFx1mUs7XkUr4Yfsbj9w0xTzcL7IMJenC+riHNm7GSCJnONw=|0|asyncPostBackControlIDs|||0|postBackControlIDs|||4|updatePanelIDs||tup1|0|childUpdatePanelIDs|||3|panelsToRefreshIDs||up1|2|asyncPostBackTimeout||90|12|formAction||Default.aspx|38|pageTitle||Photo Gallery Azure Queue Test Harness|149|scriptBlock|ScriptPath|/ScriptResource.axd?d=9YtLxwU-nmRnm7oJ9nOfEaztduuiriMqe964NLCEARKknzUa7EJGulUSq-QKJyN3_XQG98ij__ElfezvDJkF-OgjZtff28LO1U1_MHbdocg1&t=ffffffff9a77c993|155|scriptStartupBlock|ScriptContentNoTags|Sys.Application.add_init(function() {
$create(Sys.UI._Timer, {"enabled":true,"interval":1000,"uniqueID":"timer1"}, null, null, $get("timer1"));
});
|
The following table compares the Request, Response and total message sizes in bytes of the two Photo Gallery versions:
| Project Version | Request Header | Request Payload | Response Header | Response Payload | Total Message |
| Original |
702 |
3,110 |
221 |
7,217 |
11,250 |
| Enhanced |
646 |
9,179 |
224 |
18,849 |
28,898 |
Estimated Data Egress and Ingress Charges Based on Amazon Web Services EC2
Microsoft hasn’t announced its pricing but says Azure Services will be “competitive.” Therefore, AWS posted prices are used to estimate data transfer charges.
1,000 simultaneous users of the enhanced version with an average of 15 saved thumbnails would generate (646 + 9,179) = 9.825 MB/s of data egress and (224 + 18,849) = 19.073 MB/s of data ingress from polling postbacks alone. This corresponds to (9.825 * 60 * 60 * 24)/1000 = 848.880 GB/day egress and (19.073 * 60 * 60 * 24)/1000 = 1647.9 GB/day ingress.
The AWS price for EC2 or S3 data ingress is US$ 0.10 per GB = $164.79/day.
Pricing for EC2 or S3 data egress is based on a monthly volume of (848.9 + 1647.9) GB * 30 = 74.9 GB/month = < 1 TB/month. The first 10 TB/month is US$ 0.17/GB for 848.9 GB/day = US$ 144.31/day.
Daily data transfer costs are US$ 164.79 + $ 144.31 = US$ 309.10/day = US$ 9273.09/month per 1,000 average simultaneous users.
Increasing the polling interval from 1 to 10 seconds would reduce the cost to US$ 927.31 /month but might cause unacceptable latency.
Estimated Data Storage Charges Based on Amazon Web Services S3
The average size of the *.png images is ~400 KB and of the thumbnails ~4 KB for a total of 15 * 404 KB = 6.060 MB/user * 1,000 users = 6.06 GB average. AWS charges $0.15/GB*month for S3 storage, so the monthly storage cost would be $0.91/month, which is not significant with 1,000 users.
I’ll run the enhanced version for 24 hours or so and report in an update the stats from the Azure Analytics data for Hosted Services and Storage Accounts.
I’d be very grateful if someone would check my calculations.
Updates 4/21/2009
Microsoft developer-evangelist Steve Marx added a comment to this post and a reply to my Scalability and Cost Issues with Windows Azure Web and Worker Role Projects – Live Demo thread in the Windows Azure forum. Steve’s point is:
[T]his has nothing to do with Windows Azure; it's just a question of how best to use AJAX to make a responsive UI. The same optimizations would apply if you're hosting somewhere other than Windows Azure.
My point is that most Web developers don’t write polling apps very often and that Azure Queues with poorly designed code can lead to serious performance and cost problems. Microsoft has been taking a lot of heat lately for sample ASP.NET code and architectural recommendations that don’t meet the .NET Developer community’s definitions of best practices. This is another example.
Here’s the culprit method in the WebRole’s original _Default class:
protected void Page_PreRender(object sender, EventArgs e) { BlobContainer testBlob = GetPhotoGalleryContainer(); thumbnails.DataSource = from o in GetPhotoGalleryContainer().ListBlobs("thumbnails/", false) select new { Url = ((BlobProperties)o).Uri }; thumbnails.DataBind(); }
I tried wrapping the operative statements in an
if (GlobalClass.refreshData || !Page.IsPostBack)
block where GlobalClass.refreshData boolean variable is set true by the gvBlobs.RowDeleting() event handler and the WorkerRole’s GetMessage() loop and is set false after invoking the DataBind() method.
This updates the page immediately after deleting a thumbnail but, due to timing problems, doesn’t solve the problem with displaying additions immediately. In fact, added thumbnails never appear unless you close and reopen the service connection.
Following are the Hourly Network Usage metrics for 18 hours of continuous operation with (presumably) a single user connected to the oakleaf5 host:
The average for AJAX polling postbacks for the 18 hours is about 30 MB/hour * 24 = 720 MB per day, which is off by a factor of about 2.4 from the previously calculated 848.880 GB/day egress and (19.073 * 60 * 60 * 24)/1000 = 1647.9 GB/day ingress for 1,000 users = 2496.8 MB/day for a single user. Thus, I’m taking a second look at my calculations. Watch for an update later today.
Updates 4/22-24/2009
To make Steve Marx’s alternate polling approach work without throwing a JavaScript exception when adding a thumbnail, add EnablePageMethods=”true” to the ScriptManager declaration:
<asp:ScriptManager ID="sm1" runat="server" EnablePageMethods="true" />
To see your deletions occur immediately, you’ll need to set refreshData = true in your deletion method code.
Thanks to Steve Marx for the heads up on the missing EnablePageMethods attribute.
You can run a live demo of this version at http://oakleaf6.cloudapp.net/Default.aspx. Launch Fiddler2 to prove there’s no traffic when the client is quiescent (just connected to the Azure service.)
The 2.4x overcalculation of data ingress and egress load appears to be due to the actual average polling rate being 2.4 seconds, not the expected 1.0 seconds. More investigation as to the reason follows, but it’s probably due to the slow upstream speed of my AT&T DSL connection to the data center.
Atom 1.0 site feed
3 comments:
Er why do you scale up by 1000 concurrent users twice?
Why assume 1000 views per second, are you expecting 1000 unique visitors per second?
Anyhow if the above is true you will need more than 1 web role VM.
Finally do people really code clasic .aspx systems these days with all that view state baggage?
Suggest you investigate a modern RIA framework like extjs.com, that would reduce your active page bandwidth 10 or 100 times.
Sorry ignore my 1000 users twice comment, I am more used to working the numbers for a RIA application and JSON exchanges.
Assume 1k of JSON data to populate your grid x 100 hits per second at peak load = 0.36 million hits in the peak hour.
Assume 20% of your daily traffic in the peak hour = 1.8 million hits per day.
= 1.8 GB of egress data per day.
= a third of a dollar per day.
Roger, I think what you're pointing out is that the sample isn't well optimized... it does some very heavy-weight AJAX polling. I was the one who wrote the first version of this sample, and my goal was just to make it easily understandable... not to write a good model of how to implement polling.
The purpose of the AJAX polling was just so that the client refreshed after the recently-uploaded photo had been processed. If that's the only goal, then we don't need to keep polling after the thumbnail has been generated. We also don't need to do a full UpdatePanel refresh (asynchronous postback) each time... this is a bandwidth intense process.
So in response to this, I wrote some quick and dirty code to use an RPC method to call from AJAX to the web server to ask if the blob that was just uploaded has a thumbnail yet. Only once it has do I trigger an UpdatePanel async postback, and then I stop any polling after that.
The code is simple: I added a hidden field to store the name of the new blob:
<asp:HiddenField ID="uploadedBlobName" runat="server" />
I added code to set the value in submitButton_Click: uploadedBlobName.Value = props.Name.
Then I added a static method to handle the polling:
[WebMethod]
public static bool IsThumbnailReady(string name)
{
return BlobStorage.Create(StorageAccountInfo.GetDefaultBlobStorageAccountFromConfiguration()).GetBlobContainer("photogallery").DoesBlobExist(string.Format("thumbnails/{0}", name));
}
and finally some client-side script to perform the poll:
<script type="text/javascript">
Sys.WebForms.PageRequestManager.getInstance().add_pageLoaded(function(sender, args) {
if (args.get_panelsUpdated() == '') {
var blobname = $get('<%= uploadedBlobName.ClientID %>').value;
if (blobname != '') {
checkReady(blobname);
}
}
});
function checkReady(blobname) {
PageMethods.IsThumbnailReady(blobname, function(result) {
if (result) {
__doPostBack('<%= up1.ClientID %>', '');
}
else {
setTimeout(function() { checkReady(blobname) }, 1000);
}
});
}
</script>
So the new scheme is: after submitting a new photo, poll every second with a lightweight "is the thumbnail ready" ping. This is 67 bytes of posted data and a 10 byte response, and it needs to be done a couple times (depending on how many seconds your service is taking to generate thumbnails... will vary based on load). Once it's done, it does one UpdatePanel refresh, which takes the number of bytes you calculated above. (You can trim that down by throwing out unneeded ViewState.) Then no more polling happens.
To calculate the total usage, we'd need to make some assumptions about how many users are uploading photos, but I would expect dramatically lower bandwidth consumption.
The moral of the story is that if you have high volume, you need to optimize bandwidth usage. ASP.NET AJAX gives you easy-to-use methods like the UpdatePanel (which can be resource-intensive if you're not careful) and more targeted lightweight methods like RPC. Choosing the right strategy to optimize bandwidth and create a responsive user experience can be a real challenge.
Post a Comment