Test-Driving IBM’s SmartCloud Enterprise Infrastructure as a Service: Part 2 - Spring 2011 Promotion Free Trial

Update 5/5/2011: IBM’s Chris Calzetoni reported a workaround for the problem I encountered when attempting to create a failover instance of SQL Server 2008 R2 Express. See the Creating a Failover Instance from the Private Image section at the end of the post and updated Initial Conclusion.

Update 5/4/2011: IBM stonewalled my request for “details about the default durability and availability features of persistent storage?” See my commentary in the High Availability Management Issues section. I have amended my Initial Conclusion as a result.

Update 5/2/2011: Split out a Creating a Side-by-Side Instance from the Private Image and added a new Creating a Failover Instance from the Private Image section at the end of the post. I encountered a fatal error when failing over the SQL Server instance (see end of post). I will retry in mid-May when I have more free time.

Update 4/30/2011: Workaround for known problem with saving private image of my Windows Server R1 instance with SQL Server 2008 R2 succeeded. See Workaround for Private Image Save Failure section marked •• near the end of post.

Update 4/28/2011: Expanded My Initial Conclusion and High Availability Management Issues, and added Installing SQL Server 2008 R2 [Express] on a Persistent Storage Drive and sections marked •. Started Creating a Private Instance Copy of a Windows Server 2008 R1 Instance section.

Note: My Test-Driving IBM’s SmartCloud Enterprise Infrastructure as a Service: Part 1 post of 4/11/2011 (still under construction) compares IBM’s IaaS offering with Amazon Web Services.

I applied for a free SmartCloud trial account on 4/8/2011, my order was confirmed by IBM on the same day, and Sarderr Corr, IBM Inside Specialist (and “SmartCloud Leader”) for Cloud Computing in the Western U.S. and Canada called on 4/11/2011 to advise me that IBM was processing my order for the free trial.

On 4/21 I received a message notifying me that my account was ready. At that time, I was busy preparing a one-hour Linking Access tables to on-premise SQL Server 2008 R2 Express or SQL Azure in the cloud Web cast for release by Que Publishing on 4/26. I delivered the *.MP4 file to Que on 4/25 and began testing IBM’s SmartCard offering on 4/26.

IBM’s Support page offers links to videos that explain the process for creating SmartCloud compute instances and persistent storage, but the videos don’t appear to me to reflect the provisioning process for the Spring 2011 Promotion version, which has many locked down choices. So I decided to blog the process for creating a Windows Server 2008 R1 instance with the available persistent storage option. IBM persistent storage appears to correspond to Amazon’s Elastic Block Store (EBS.)

Online Help: IBM has a detailed Users Guide for the Smart Business Development and Test on the IBM Cloud service v1.4, which is the name of SmartCloud Enterprise’s predecessor. (You must be logged into Development & Test v1.4 to open this link.) You can access developerWorks’ Cloud Computing Developer Community from a link on the Support page.

• My Initial Conclusion

The provisioning process is simple and straightforward. Simplicity is result of a lack of most advanced options offered by Amazon Web Services (AWS). In particular, managing instance availability and scalability (DevOps) is a complex manual process. IBM papers on do-it-yourself high-availability topics apply only to Red Hat Enterprise Unix instances. Unlike AWS with its Elastic Beanstalk and CloudFormation offerings, there appear be no IBM or third party deployment, management or monitoring tools available for SmartCloud Enterprise.

My attempts to fail over the SQL Server instance to a new SmartCloud Enterprise instance with data residing on a persistent storage disk failed. The failover process is cumbersome and needs to be automated. IBM’s Chris Calzetoni was able to workaround the issues I encountered and stated in a 5/5/2011 message: “We are aware of this behavior and are actively looking to improve the user experience with this function.”

As noted in the High Availability Management Issues section below, IBM representative refused to divulge scalability and durability details about their persistent storage architecture upon my request. I would be disinclined to trust my data to a cloud provider that isn’t forthcoming about how they intend to protect it and make it available to my applications.

---

Log in to the SmartCloud Portal and Create Persistent Storage

The Spring 2011 Promotion free trial entitles you to a single Copper or Silver instance of Windows 2003 R2 or Windows 2008 R1 Data Center edition, Red Hat Enterprise Linux 5.5 or SUSE Linux Enterprise Server 11 SP1 for x86 hosted in IBM’s Toronto data center, 60 GB of local storage, and a Small persistent storage account. Setting up the free trial differs from the process for activating a paid account, so I’ll cover that subject first.

Important: You must set up your persistent storage before you create your compute instance, if you want to associate your persistent storage with the instance.

1. Navigate to the SmartCloud Development & Test dashboard.

2. Enter the IBM user name under which you applied for the Spring 2011 Promotion and password in the Sign In form: 

Note: You must have registered on the IBM Web site previously to place the order. If you want to add other users to your account, they must first register on the IBM Web site.

3. Click Submit to open the Development & Test Portal v1.4 page and click the Control Panel tab to display the initial page:

4. Click the Storage Link to load the initial storage page and click Add storage to open the eponymous dialog:

5. Give the persistent storage a name, accept the Small Size and ext3 format defaults and click Next to display the Verify Configuration dialog:

Note: ext3 is Linux’s third extended file system. Click here for more information on ext3 from Wikipedia.

6. Click Next, agree with the Terms of Service, and click Submit to provision the storage and display the Request Submitted Successfully dialog:

 

7. Click Return to Control Panel and click the Instance link to display the page shown in step 3.

---

Create a 32-bit Windows Server 2008 R1 Instance with Persistent Storage

8. Click Add Instance to open the dialog of the same name with Toronto selected as the data center:

9. Select a 32-bit instance of the operating system you want, Microsoft Windows Server 2008 R1 for this example, and click Next to display dialog 2:

 

10. Type a unique instance name in the Request Name text box, accept the Quantity default (1), select Silver as the Server Configuration (the other option is Copper, which is smaller). Accept the VLAN, Select IP, and Virtual Disk defaults, which you can’t change, and click Add Disk to open the Assign Instance Disk dialog:

11. The Mount Disk and Mount Point options don’t apply to Windows disks, so click the Add Disk button to display the confirmation dialog:

12.Click Close to return to the Add Instance dialog:

13. Click Next to open a dialog in which to type a name and password for the Windows Server’s administrative account:

Note: The password must meet current Windows complexity requirements. Failure to do this results in a error message.

14. Confirm your password and click Next to open the Verify Configuration dialog:

15. Click Next to open the Service Agreement dialog, mark the I Agree radio button, and click Submit to display the Request Submitted Successfully dialog:

16. Click the Return to Control Panel link and display the current status: Provisioning:

Connecting to the instance with Windows' Remote Desktop Protocol

17. The system-generated IP Address won’t appear until the instance becomes active, as shown here in Control Panel:

Note: Activating the instance took about 32 minutes in my case. The $0.24/hour charge reflects the Silver compute instance hourly charge from the following table in Test-Driving IBM’s SmartCloud Enterprise Infrastructure as a Service: Part 1:

Note: Windows Server 2008 R1 Silver instances cost US$ 0.07 per hour less than RHEL Silver instances, which extends to a $ 50 per month saving. The saving with a Copper instance is $ 65 per month.

18. Start the Remote Desktop Connection app, click Options to expand the dialog, and type the system-generated IP address from step 17 and the username you entered in step 13:

19. Click Connect to open the Enter Your Credentials dialog, type the password you created in step 13, and mark the Remember My Credentials check box if you’re the only user of the computer:

20. Click OK to connect to your instance with the Remote Desktop Protocol. If you receive a certificate error message because the server’s HTTPS certificate was self signed by the instance, accept the certificate as valid. Optionally you can add the certificate to your Trusted Certificate Publishers list.

---

Enable and Format Persistent Storage

21. When you connect to the instance, set up your persistent storage right-clicking Computer and choosing manage to open Server Manager and select the Storage node to open the Disk Management pane:

 

22. The three disks are RedHat VirtIO SCSI Disk Devices and the two disks added by the Small persistent storage selection are offline and not configured. Right-click Disk 1 and choose Online to bring it online, right-click it again and select MBR (Master Boot Record) and click OK to initialize it as E: (an inaccessible CD-ROM 0 is E:. Quick-Format E: as NTFS.

23. Disk 2 wasn’t associated with your instance, so you must initialize it by bringing it online, right-clicking the image, choosing Delete Volume and confirming deletion.

24. Right-click the image again, choose New Simple Volume to start the New Simple Volume Wizard click Next to open the Specify Volume Size dialog:

25. Accept the 262,141 MB defaults and click Next to Open the Assign Drive Letter or Path dialog:

26. Accept the F: drive letter and click Next to open the Format Partition dialog:

27. Type a Volume Label and click Next and Finish to create and format the volume. Optionally add a volume name to the C: drive. Disk Manager appears as shown here:

28. Click Control Panel’s Storage link to display the properties, including the hourly charge, for the Small Storage option you selected:

350 GB of your local storage is included with the Silver compute instance, in accordance with this table from Test-Driving IBM’s SmartCloud Enterprise Infrastructure as a Service: Part 1:

The $0.0384/hour charge shown in the preceding figure is based on this table’s Small instance:

The pay as you go (PAYG) price for this configuration would be (0.240 + 0.03840)*24*30 = $200.45 per month.

---

Transfer Files from Your Local Computer to SmartCloud Storage

29. Open the Remote Desktop Connection app and click the Local Resources tab to expose Local Devices and Resources you select to the remote instance:

30. Click More to open the dialog and mark the Drives check box to enable the SmartCloud instance to access your local drives:

31. Click OK and Connect to reconnect to the instance, opening this message in the process:

32. Mark the Don’t Ask Me Again …check box and click Details to expand the dialog:

33. Click Connect to reconnect with the resources available and double-click the Computer icon on the remote server to display the four volumes on your local machine:

34. Double-click one of the Other items to open it in Windows Explorer:

---

Check Resources with Task Manager

35. Press Ctrl+Alt+End to open the menu and click Start Task Manager to open it. Click the performance tab to verify the processors and physical memory allocated to your compute instance:

Task Manager confirms allocation of the 4 GB of virtual (physical) memory reported in step 28’s table.

---

Activate the Compute Instance for License Compliance:

36. The User Manual’s Planning for volume activation for your Windows instance topic describes the following steps for activating your compute instance:

1. Connect to the instance via the Remote Desktop Connection client. See Connecting to your Windows instance with RDC if you need help for this step.
2. Turn off the Windows firewall:
   1. Click Start -> Control Panel -> Windows Firewall.
   2. On the General tab, select Off to turn off the firewall.
3. Synchronize time:
   1. From Control Panel, open Date and Time.
   2. From the Internet time tab, [click Change Settings to open the Internet Time Settings dialog] select the times.windows.com server and click Update now.
   3. [Click OK twice to close the dialogs.]
4. Activate the license:
   1. Click Start. Right click Command Prompt and then click Run as Administrator.
   2. Issue the command slmgr.vbs /ato to activate the license for the particular instance. [See the screen capture below.]
5. Turn on the Windows firewall:
   1. Click Start -> Control Panel -> Windows Firewall. [See screen capture below.]
   2. On the General tab, select On to turn on the firewall.
6. Check and make sure the Internet connectivity is available on the instance.

When the Windows Script Host message appears, click OK:

Click OK to acknowledge successful activation:

Select the On option and click OK to restart the Windows firewall:

 

37. By default, your instance runs on U.S. Central time. To change to a different time zone, open Control Panel’s Data and Time tool:

38. Click the Change Date and Time button to open the Time Zone Settings dialog. Select your time zone from the list:

39. Click OK twice to save the new time zone setting and close the two dialogs.

---

Add Application Server and Web Server (IIS) Roles to Windows Server

40. You’ll probably need Windows Server Application Server and IIS roles for .NET applications you upload to and run on your SmartCloud Instance. Open Server Manager, and select the Roles node, which indicates that File Services is the only role installed.

41. Right-click the node and choose Add Roles to start the Add Roles Wizard and click Next to open the Server Roles dialog. Mark the Application Server and Web Server (IIS) check boxes:

42. Click Next to start the Role Addition process and follow the Wizard’s instructions. When the process is complete, Server Manager’s Roles node and Role Services section appears as shown here:

---

• High Availability Management Issues

The Users Guide for the Smart Business Development and Test on the IBM Cloud service v1.4’s FAQ section includes the following Q&A items about availability and accessibility of compute instances and local/persistent storage:

What backup and recovery services are available?

We do not provide a standard backup and recovery service for virtual machine instances. However, users can back up data from ephemeral storage to persistent storage, or implement their own backup solution such as remote backup and download data to their machine.

What happens when there is a system failure? Is there a mechanism to save data?

When a virtual machine instance fails or is unresponsive, the user that owns that instance can try to reboot it through the Cloud Web UI. If there is a recoverable failure of the underlying system, for example, in the event of a temporary power outage or a fan failure, the virtual machine and its file system will be back online with the associated hardware. However, if there a permanent failure (such as the loss of a number of hard drives where instances are stored), the user instance might be lost. The user must create a new virtual machine instance based on a Master Image or on a “snapshot” of his customized virtual machine that was captured previously.

The user who owns the virtual machine instance can capture a snapshot of the boot file system at any time using the Create Image function in the Cloud Web user interface. The Create Image function will stop the virtual machine instance, create a copy of the file system into a new image file and restart the virtual machine instance, using the file system that was active before the Create Image function. When the end user selects the Delete function for the virtual machine instance, the instance is halted and the data in the boot (ephemeral) file system is deleted un-recoverably.

Note: Important data must be kept on persistent storage provided by the IBM Smart Business Development and Test on the IBM Cloud service or on some other storage system provided by the customer.

I’ve not been able discover details of the architecture that IBM uses to provide highly durable and available persistent storage. Amazon Web Services provides a detailed Overview of EBS System topic in its Summary of the Amazon EC2 and Amazon RDS Service Disruption in the US East Region article of 4/29/2011. It’s assumed that SmartCloud Enterprise uses a similar architecture, but the provisioning process described above doesn’t appear to offer an option for choosing IBM’s equivalent of AWS’s multiple availability zones to provide increased durability.

On 4/29/2011, I started the following Where can I find details about the default durability and availability features of persistent storage? thread on IBM’s SmartCloud Message Board:

I assume that SmartCloud Enterprise persistent storage has an architecture similar to AWS's Elastic Block Store but would like to confirm or dispel that assumption. I'd appreciate a link to any detailed information that describes persistent storage architecture.

On 5/4/2011, I received the following response from IBM’s Chris Calzetoni:

We sincerely value your inquiry.  We look forward to help address any technical support issues that you may be having with SmartCloud Enterprise.  Like other leading cloud providers, architecture details represent intellectual property of IBM and cannot be shared at this time. Thank you for your understanding.

I don’t understand IBM’s reluctance to discuss how they provide or intend to implement durability and scalability for their persistent storage. Microsoft provides full architectural details of their Azure blob and table storage services, as well as SQL Azure data storage. I would be disinclined to trust my data to a cloud provider that isn’t forthcoming about how they intend to protect it and make it available to my applications.

Update 5/5/2011: Jon Etkins (jetkins@us.ibm.com), Infrastructure Specialist, IBM, asserted  that you can Quickly gain an understanding of how to assign ephemeral storage to your IBM Cloud instance in a deck for his IBM Cloud tip: A basic understanding of ephemeral storage article of 2/9/2011 (requires developerWorks registration). From the Summary:

Ephemeral disk storage, also known as local disk devices (although the disks aren't always collocated with other VM resources) is an important component in using the IBM® Smart Business Development and Test in the IBM Cloud. In this article, the author details the layout of the virtual disk drives available to the Linux® and Windows™ operating systems for each choice of instance size in the IBM Cloud.

The High availability apps in the IBM Cloud white paper of 1/10/2011 by Dima Rekesh and Alan Robertson is subtitled “How to protect a cloud-enabled, production-grade application
against a failure on any node.” From the introduction (repeated here from Part 1):

The new features of the IBM® Cloud enable application developers and architects to eliminate single points of failures in applications. This article provides a detailed
guide on those features. It includes a discussion of the approach the IBM Cloud
takes (added support for virtual IP addresses); how to prepare your cloud instances to take advantage of this feature; how to set up a highly available website; and how to test that site.

The IBM® Smart Business Development and Test[*] on the IBM Cloud is a dynamically provisioned and scaled elastic environment that provides enterprise customers with everything they need to develop, test, and host applications. It includes a web portal to configure and manage the cloud resources, software images of IBM products that jump-start development and test efforts, and APIs that enable users to control cloud resources and software programmatically. And IBM teams have been adding new features since the debut of the IBM Cloud that are designed to provide additional flexibility and resiliency.

This greater agility and much improved elasticity — which helps adjust your
application topology to the demands of the business in real time — comes with a
tradeoff: A decrease in compatibility with cloud environments of such features as high availability (HA).

High availability, the requirement to protect a production-grade application against a failure of any node, isn't a new concept by any standard; many software products address this challenge. But most of these products are, by and large, not compatible with the cloud; most public cloud providers do not provide the required functionality.[**]

With that in mind, customers need to supplement their cloud deployments with HA
constructs that exist outside the cloud. In this article, you will see what IBM has done with its cloud to address this issue and how you can tap into that feature:

• We'll discuss the approach the IBM Cloud takes (added support for virtual
  IP addresses).
• We'll show you how to prepare your cloud instances to take advantage of
  this feature.
• In an example, we'll show you how to set up a highly available website.
• And we'll show you how to test that site.

The Rekesh and Hamilton paper describes manually setting up HA features for proxies running nginx under Linux HA and Pacemaker software, Apache Web servers and 64-bit Red Hat Enterprise Linux (RHEL) 5.4. Users of Windows instances appear to be on their own.

Boas Betzler and Nitin Koshti posted Create GPFS clusters on multiple IBM Cloud instances to developerWorks on 4/25/2011. According to the Summary, the paper describes:

Learn how to install, configure, and create a GPFS multi-node cluster on four RedHat 5.5 64-bit instances on IBM SmartCloud Enterprise. The authors describe how GPFS and cloud computing work together, set up a GPFS cluster, create network shared disk devices, and more.

For more details about IBM’s General Parallel File System, see this Wikipedia article. Although GPFS is available for use with Windows Servers, there’s no advice for users of Windows instances.

---

• Installing SQL Server 2008 R2 [Express] on a Persistent Storage Drive

As noted earlier, IBM considers the 80 GB of C:\ drive instance storage created for your Windows Server instance to be ephemeral because it’s deleted along with a deleted instance. This means you must install the data folder on one of the two persisted storage volumes. To install SQL Server 2008 R2 [Express] executables on the C:\ drive and data folders on the F:\ drive, so you can replace the compute instance without losing the data files for its database, do the following:

1. Download 32-bit Windows PowerShell 2.0 and WinRM 2.0 for Windows Server 2008 (KB968930) and install it.

2. Start a normal installation of SQL Server 2008 R2 [Express] with management tools.

3. Continue the installation until you reach the the Database Engine Configuration dialog.

4. Select the Mixed Mode Option and verify that your administrative account is a member of the SQL Server Administrators group. If not, click the Add Current User button:

5. Click the Data Directories tab and change the C:\ (ephemeral) drive to the F:\ (Persistent Storage Disk2) drive, which changes the path for all other data folders:

You must use persistent storage that isn’t permanently dedicated an instance to make the data directories available to an new instance created from a private image. (See the Workaround for Private Image Save Failure section below.) When you delete a failed instance, its instance storage (drive E:\) is deleted, too.

6. Optionally, click the FILESTREAM tab to enable FILESTREAM for Transact-SQL Access.

7. Click Next twice to perform the installation and click Close when installation completes.

8. Navigate to F:\Program Files\Microsoft SQL Server\MSSQL10_50[.SQLEXPRESS]\MSSQL\DATA to verify that the system files shown below are present:

9. On completion, I installed from AdventureWorks2008R2_SR1.exe the latest version of the Adventure Works Sample databases (without FILESTREAM) to enable testing database recovery with a new instance created from a saved private image:

Note: AWS uses EBS as the persistent store for its Relational Database Services (RDS). RDS stores use a primary and two secondary replicas to assure durability. The Windows Azure Platform’s SQL Azure database uses a similar triple-replication architecture. I’ve not been able to find any details of how IBM’s SmartCloud persistent storage provides similar durability enhancements. I’m awaiting a reply to this message of 4/29/2011:

---

• Creating a Private Instance Copy of a Windows Server 2008 R1 Instance

The Users Guide for the Smart Business Development and Test on the IBM Cloud service v1.4’s Managing Images section includes a Prerequisites for creating a private image from Windows instance section, which contains the following instructions:

Before you create an image from your Windows instance, perform the following prerequisite activities:

• [Windows Server 2008 only:] Manually activate your Windows® Server 2008 instance. See Planning for volume activation for your Windows instance for more information.
• Run the autolog.bat command from the desktop of the instance you want to capture. For more help information, see the "Enabling autologon for administrator" section in Connecting to your Windows instance with RDC.
• Disable all pop-up windows such as ITCS104 pop-up when the OS loads or shuts down.
• Manually shut down the instance from the desktop of the instance you want to capture. To shut down the instance, connect to the instance via RDP client. Then from the desktop, click Start -> Shutdown.

Here’s Autolog.bat’s contents:

Following are the instructions from the "Enabling autologon for administrator" section:

If you plan to capture a private image from the Windows instance, you must enable autologon for the administrator.

1. Run the RDC client to connect to the Windows instance.
2. Double click the Autolog.bat file on desktop.
3. When prompted, enter the administrator password for three times.
4. To verify that autologon is enabled for administrator, open the Registry and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon, you should see the following information:

 

1. Shut down the instance as described above and wait a couple of minutes for shutdown to complete. (Control Panel’s Compiled Notifications section continues to indicate your instance is active.)

2. Click Control Panel’s Create Private Image button to open the dialog of the same name:

Tip: If you don’t see the above dialog, scroll up.

3. Click Submit to start the process and display the status in Control Panel’s compiled notifications section:

The above entries indicate that shutting the instance down in step 1 might not be required.

After about 45 minutes I received the following notifications:

With no additional details.

•• Workaround for Private Image Save Failure

1. I posted the following message on 4/29/2011 and received a reply on 4/30:

The first link leads to this message of 4/7/2011:

Windows Capture Image Workaround is an illustrated PDF file describing procedures to be run with an instance-within-an-instance created by opening an new instance with the Remote Desktop tool running in the base instance. Using the 64-bit image isn’t an option with Spring 2011 Promotion accounts.

2. My attempts to follow the workaround instruction resulted in the following failures:

1. Connecting with my administrative username and password opens the instance with an Access Is Denied notice on the desktop.
2. Attempting to connect with Administrator as the username fails with the above password.

The FAQ and ALL CURRENT KNOWN PROBLEMS, WORK AROUNDS AND LIMITATIONS document is dated 9/13/2010 and contains the following content regarding Windows private image capture:

Q15: Why doesn't my Windows Capture work?

Please review this link first for current Windows Capture problems.

• During the creation of your Instance did you make sure to create a Username that was not "Administrator" or "Guest"?
• During the creation of your Instance did you make sure to create a Password that was compliant with Windows rules?
• Have you made sure to run the autolog.bat command from the Desktop of the instance you wish to capture? (this has to be run every time you want to capture your Windows instance, you can check if autolog.bat has worked by using regedit and find HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon, then look for updates variables: DefaultPassword, AutoAdminLogin = 1 and DefaultUserName = Administrator)
• Have you made sure the password typed during autolog.bat was correctly typed (also does the password chosen comply with Windows rules?)
• (Only Windows 2008 R1 and R2) Have you made sure to activate Windows on the instance you wish to capture? See Q13 above.
• Have you manually Shutdown the instance from the Desktop of the instance you wish to capture?
  Have you made sure to disable all popups (ex. ITCS104 popup) when the OS loads or shutsdown?
• [W]alk through for above can be found in this document: Windows Capture Image Workaround.

3. I identified three possible problems with my first attempt to capture the private image:

1. I did not run Autolog.bat immediately before attempting the capture.
2. The instance’s time was out of sync with NIST by about 8 hours.
3. The last three Windows Updates repeatedly failed to install.

4. To overcome these problems, I did the following:

1. Ran Autolog.bat and verified the Registry entries
2. Resynchronized time with NIST
3. Turned off Windows Updates with the Control Panel tool and disabled the Windows Update Service with the Services.msc snap-in as described in the Workaround document.

5. With the above workaround, I was able to save the private image:

6. You access the private image from the dashboard’s Asset Catalog’s list. To open the Asset Catalog, on the Control Panel tab, click the View Asset Catalog link to open the Asset Catalog:

7. Click the item to display image details and enable starting an instance from it:

•• Creating a Side-by-Side Instance from the Private Image

The Spring 2011 Promotion provisions only a single instance, but you can start a second, side-by-side instance from a private image. Multiple instances can use load balancing to increase scalability. However, each image would need an independent instance of SQL Server with transactional replication to synchronize data and maintain data consistency with the other image. Transactional replication, which SQL Server Express doesn’t support, would slow data modification (insertions, updates and deletions.) For this reason, a side-by-side example is useful to demonstrate persistent storage visibility between instances but isn’t a real-world scenario.

To create a second Windows Server 2008 R1 instance from the detailed view of its private image in the Asset Catalog (see above), do this:

1. Click the Start an Instance of Your Image link in the detailed view of to open the Step 2 of the Add Instance dialog. Add a suffix to create unique Request Name and accept the remaining default values:

Note: The Minimal Local Disk check box is disabled. You add storage in Control Panel’s Storage page. You can’t add storage in this case, because persistent storage must be created before the instance. When simulating failover (see below), you can attach the SmallStorage disk 2.

2. Click Next to open the Add Instance Step 2B Dialog and enter the same user name and password that you used for the initial instance:

3. Click Next to confirm the configuration:

4. Click Next again to agree to the Service Agreement, and click Submit to submit the instance for provisioning:

 

5. Return to Control panel which displays the new instance in the Requesting and then the Provisioning state:

6. The Development and Test on the IBM Cloud portal was unavailable from 10:00 to 12:00 PM EDT when the second instance was being provisioned, which resulted in this status message:

7. Waiting until after the maintenance outage was over and refreshing the page produced the following result:

8. Logging on to the new instance with your administrative user name produces the same certificate error dialog as the initial login:

Mark the Don’t ask me again … check box and click Yes to connect with your administrative password.

9. You won’t be able to log in to the original or the new instance’s SQL Server 2008 R2 Express with SQL Server Management Studio. The following error results with either the VHOST0021\SQLEXPRESS or VHOST0376\SQLEXPRESS instance names:

10. The error is due invisibility of the original instance’s drive F:\, which contains the data files required by SQL Server to start:

Observe that the CDROM and PersistentDisk1 have interchanged drive letters.

11. Server Manger’s Application Log confirms that missing files are the problem:

•• Creating a Failover Instance from the Private Image

A failover scenario is more practical with the Spring 2011 Promotion’s single instance restrictions. In this case, the situation is that the original instance has failed and a new instance will be created from the private image to replace it.

1. Select the original instance (Win2008R1Ag) in Control Panel’s Instances page and click Delete Instance to open the dialog of the same name:

2. Click Delete to delete the instance. The instance’s Status changes to Deprovisioning and, after a few minutes, disappears from the list.

3. Click storage and verify that Win2008R1SmallStorage has Non-Attached status:

Notice that the Clone Storage button is now enabled and that there is no means of attaching the previously attached storage to the new instance created from the private image.

4. Click Instances and observer that there is no means of attaching storage there either. You must attach the storage when creating the instance from the private image, so Delete Instance and Delete:

5. Create a new Win2008R1Ag instance from the private image in the Assets Catalog and attach the Win2008R1SmallStorage persistent disk:

6. Click Next, create the same administrative account and password as in the initial instance, and click Next to confirm the instance properties:

7. Click Next, agree to the Service Agreement, and click Submit to start the provisioning process.

8. Return to Control Panel and wait about 30 minutes for the instance to activate:

Notice that the IP address is the same as that for the original instance you deleted.

9. Connect to the instance with the credentials you established in step. In this case you see two user icons:

 

10. Select your administrators account, log in and double-click Computer to view drive assignments:

Notice the difference between the original and new drive assignments due to the CDROM being changed from D:\ to F:\.

11. Attempt to log on with SQL Server Management Studio using .\SQLEXPRESS as the server name. You receive an unexpected error message:

SQL Server must load master, mastlog.mdf, model.ldf, MSDBData.mdf, MSDBLog.ldf, tempdb.mdf, and templog.ldf from the E:\Program Files\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS\MSSQL\DATA folder during the startup process. If an error is encountered loading one or more of these files, SQL Server won’t start.

12. Right-click Computer, choose Manage and expand the Diagnostics, Event Viewer, Windows Logs and select Application. examining the Application log shows that access was denied to the master.mdf database:

13. Right-click F:\Program Files\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS\DATA\master.mdf, choose Properties to open its properties sheet, and click the Security tag:

Although the administrative account you used has the same name and password, and also is a member of the administrators group, it is unknown in the original file permissions.

14. Click Edit, Select Add to open the Select Users or Groups dialog, type your administrators user name (rogerj for this example) and click Check Names to verify the account (by an underline):

 

15. Click OK to return to the Permissions for master dialog. With the account you just added, mark the Full Control check box:

16. Click Edit, select the unknown account and click Delete. You receive the following error message:

17. Close the Windows Security dialog, select the Unknown account, and click the Advanced button to open the Advanced Security Settings for Master dialog. Click Edit, clear the Include Inheritable Permissions from This Object’s Parent check box to open another Windows Security dialog:

18. Click Remove to close the dialog and delete the unknown account. The Permissions for master dialog appears as shown here:

19. Click OK to close the dialog, right-click Computer and choose Manage to open Server Manager, expand the Configuration and Services nodes, and scroll to the SQL Server (SQLEXPRESS) item. If the Status of SQL Server (SQLEXPRESS) isn’t Started, right-the entry and choose Start from the context menu. An error message similar to the following usually indicates either an “access is denied” or “file not found” condition.

20. Determine the source of the problem by inspecting the Windows Application log, as described in step 12.

21. If the error is “access denied” with master.mdf, repeat 13 through 15 but substitute NETWORK SERVICE for your account name in step 14.

Note: I had to give NETWORK SERVICE Full Control permissions to get master.mdf to load.

22. If the error is “file not found” with model.dbf, modellog.ldf, MSDBData.mdf or MSDBLog.ldf, start SQL Server in master-only recover mode, by issuing the following instruction at a command prompt:

NET START MSSQL$SQLEXPRESS /f /T3608

For more information about master-only recovery mode, see Moving System Databases and How to: Start an Instance of SQL Server (net Commands) in the MSDN Library.

Note: The problem I encountered after fixing NETWORK SERVICE was all four of the above files not found due to an F:\ instead of E:\ drive designator.

23. Open SQL Server Management Studio (SSMS) and connect to the SQLEXPRESS instance with .\SQLEXPRESS as the Server Name with Windows.

Note: Although the last message in the Windows Application log was “SQL Server is now ready for client connections. This is an informational message; no user action is required,” I was not able to start the SQLEXPRESS instance in SSMS to correct the drive letter. I consider the problem to be fatal to failing over an SQL Server instance with from a SmartCloud Enterprise instance created from a private image.

As noted at the beginning of this post, I will retry in mid-May when I have more free time.

Update 5/5/2011: On 5/2/2011, I started the following Unable to Start SQL Server after Failing Over to a SmartCloud Instance Created from a Private Image thread on IBM’s SmartCloud Enterprise Message board:

The end of my Test-Driving IBM’s SmartCloud Enterprise Infrastructure as a Service: Part 2 - Spring 2011 Promotion Free Trial post (updated 5/2/2011) describes a serious problem I encountered when testing a failover scenario with Windows Server 2008R1 and a saved private image with persistent storage.

Has anyone else tried this scenario and succeeded?

I plan to try again in mid-May.

On 5/5/2011, IBM’s Chris Calzetoni replied as follows:

I spent some time reproducing this behavior and was able to get SQL Server working properly. Rather than changing the data file locations, I changed the CD-ROM drive to a drive letter other than F, then changed my persistent storage drive to be F. After updating the data file permissions, as well as the errorlog, and adding additional account names of Network Service, I was able to log into the SQL Management Studio to manage the instance. We are aware of this behavior and are actively looking to improve the user experience with this function.

Hopefully, Chris will be able to streamline this process.

Technorati Tags: IBM SmartCloud,IBM,SmartCloud,Cloud Computing,Infrastructure as a Services,IaaS,SQL Server 2008 R2,SQL Server 2008 R2 Express