Step-by-step: Migrating ASP.NET Application to Windows Azure (Part 5 – Implementing Bulk Copy from Blob Storage to SQL Azure)

This is an additional post for Migrating ASP.NET Application to Windows Azure series. Please refer to previous related posts if you have not read it before.

Introduction

In the step 6 of the Part 3 post, remembering that I commented some portion of the code in AdminPhoto.aspx and App_CodePhotoManager.cs which unable be run directly on Windows Azure. What actually the feature does is enable us to perform bulk copy from the web server’s folder namely “upload” to the database. However, those code could not simply work on Azure since we have no control over the “upload” folder when our app has been uploaded to Windows Azure.
image_0D77A98A

The idea to implement similar feature

As I promise to answer to this issue, in this post I’ll show you step-by-step how to implement similar feature. We will be do those things in 2 files: AdminPhoto.aspx and AdminPhoto.aspx.cs.
The idea to use blob storage as replacement of “upload” folder. We can upload our photos to blob storage by using tool such as Azure Storage Explorer.
image_78315616
Assuming that I’ve created a container called “photos” and uploaded some photos in that container.
When user is ready to upload the photos inside the container, they will click on Import and what our application do is to store these photos to SQL Azure using the existing class library “PhotoManager.cs”.
*Please note that actually the better practice is to store those file in blob storage rather than SQL Azure since scalability and cost-efficient concern. However, since I’ve used SQL Azure to store the data (including photos) from the beginning of this post, I’ll just continue to do with respect to consistency manner.

Step by Step: how to implement

1. DataConnectionString setting on compute role.

Since we are using Azure Storage, we’ll need to add a setting on intended role (in my sample: PersonalWebRole) as shown below.
image_4D40A439
To do that, double click on PersonalWebRole which is located below CloudServicePersonalRoles folder. Subsequently click on Settings tab.
You may note that in the example I use DevelopmentStorage, of course you could be able to use the “real” Azure Storage. All you need to do is to create a Azure Storage Service and then specify the service name as well as the key as the value.

2. Some codes in WebRole.cs

To tell WebRole that “please refer to the setting to retrieve the storage value”, we’ll need to add something on OnStart method inside WebRole.cs. (the yellow highlighted section).
   1: public override bool OnStart()

 

   2: {

 

   3:     DiagnosticMonitor.Start("DiagnosticsConnectionString");

 

   4:

 

   5:     RoleEnvironment.Changing += RoleEnvironmentChanging;

 

   6:

 

   7:     CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>

 

   8:         {

 

   9:             configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));

 

  10:         });

 

  11:

 

  12:     return base.OnStart();

 

  13: }

3.  In AdminPhoto.aspx file.

I replace the commented section with the following codes.
   1: <div id="sidebar">

 

   2: <asp:ScriptManager ID="ScriptManager1" runat="server">

 

   3: </asp:ScriptManager>            <h4>Bulk Upload    Photos</h4>

 

   4:     <p>The following files were found in your <b><%
   1:  = ddlBlobContainer.SelectedValue
%></b> folder, in your Azure Blobs Storage.
   5:     <br />Click on <b>Import</b> to import these pictures to your photo album.</p>

 

   6:     <asp:UpdatePanel ID="up" runat="server">

 

   7:         <ContentTemplate>

 

   8:             Select Container: <asp:DropDownList AutoPostBack="true" ID="ddlBlobContainer" runat="server"

 

   9:             onselectedindexchanged="ddlBlobContainer_SelectedIndexChanged" />

 

  10:             <br /><br />

 

  11:             <div style="height:300px; overflow:scroll; text-align:center; border:solid 1px #fff" >

 

  12:                 <asp:ListView runat="server" id="UploadList" repeatcolumns="1"

 

  13:                     repeatlayout="table" repeatdirection="horizontal"

 

  14:                     onitemdatabound="UploadList_ItemDataBound" >

 

  15:                     <LayoutTemplate>

 

  16:                         <div style="border:solid 1px #000; padding:5px; margin:5px" runat="server" id="itemPlaceholder">

 

  17:                         </div>

 

  18:                     </LayoutTemplate>

 

  19:                     <ItemTemplate>

 

  20:                         <asp:Image ID="img" runat="server" Width="120px" />

 

  21:                     </itemtemplate>

 

  22:                     <ItemSeparatorTemplate><br /><br /></ItemSeparatorTemplate>

 

  23:                     <EmptyDataTemplate>the container doesn''t contain any photos</EmptyDataTemplate>

 

  24:                 </asp:ListView>

 

  25:             </div>

 

  26:         </ContentTemplate>

 

  27:     </asp:UpdatePanel>

 

  28:     <asp:ImageButton ID="ImageButton1" Runat="server" onclick="Button1_Click" SkinID="import" />

 

  29: </div>

Let me explain you how these things were added
a. I added a DropDownList called ddlBlobContainer. This ddlBlobContainer will be used to list out all of our blob containers that were associated with our storage account.
b. When we select on one of the ddlBlobContainer, we’ll fetch all blob item in that container and display it to user interface (UI).  l use ListView as UI container rather than of using DataList. Inside the ListView, I added Image object to show the thumbnail display, so that use could be able to see it before performing bulk copy.
c. I use UpdatePanel to enable ajax functionality so that the browser won’t be refreshed in full page view. (Remember that we’ll also need to include the ScriptManager in order to use UpdatePanel).

4. In AdminPhoto.aspx.cs file.

a. The next thing to do when using Azure Storage is to initialize the storage account as well as object that we are going to use.
   1: private static bool storageInitialized = false;

 

   2: private static object gate = new Object();

 

   3: private static CloudBlobClient blobStorage;

 

   4:

 

   5: private void InitializeStorage()

 

   6: {

 

   7:     if (storageInitialized) return;

 

   8:     lock (gate)

 

   9:     {

 

  10:         if (storageInitialized) return;

 

  11:         try

 

  12:         {

 

  13:             var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

 

  14:             blobStorage = storageAccount.CreateCloudBlobClient();

 

  15:             CloudBlobContainer container = blobStorage.GetContainerReference("photos");

 

  16:             container.CreateIfNotExist();

 

  17:

 

  18:             var permission = container.GetPermissions();

 

  19:             permission.PublicAccess = BlobContainerPublicAccessType.Container;

 

  20:             container.SetPermissions(permission);

 

  21:         }

 

  22:         catch (Exception ex)

 

  23:         {

 

  24:             throw new Exception("Error : " + ex.Message);

 

  25:         }

 

  26:         storageInitialized = true;

 

  27:     }

 

  28: }

  • The storageInitialized variable is used as flag to indicate whether the storage has been initialized
  • The gate variable is to be used along with “lock” statement, to ensure that other threads won’t enter the critical section before the resource has been released.
  • The blobStorage object serves as CloudBlobClient, to manipulate the blob of container.
  • In the InitializeStorage method, we get the connection value from the settings, create cloud blob client, get / create the specific container (In this example called “photos”) and finally set the permission of the container to public access.
b. Subsequently, I added BindUploadList() method which fetches blob from the container and display it to the listview.
   1: private void BindUploadList()

 

   2: {

 

   3:     CloudBlobContainer container = blobStorage.GetContainerReference(ddlBlobContainer.SelectedValue);

 

   4:     IEnumerable<IListBlobItem> blobItems = container.ListBlobs();

 

   5:

 

   6:     var photoBlobs = from i in blobItems

 

   7:                      where i.Uri.AbsoluteUri.ToLower().EndsWith(".jpg")

 

   8:                      || i.Uri.AbsoluteUri.ToLower().EndsWith(".png") ||

 

   9:                      i.Uri.AbsoluteUri.ToLower().EndsWith(".gif")

 

  10:                      select i;

 

  11:

 

  12:     UploadList.DataSource = photoBlobs;

 

  13:     UploadList.DataBind();

 

  14: }

I also ensure that only file that ends with .jpg, .png, gif will be displayed.
c. The next thing is to implement the UploadList_ItemDataBound method to handle the ItemDataBound event.
   1: protected void UploadList_ItemDataBound(object sender, ListViewItemEventArgs e)

 

   2: {

 

   3:     if (e.Item.ItemType == ListViewItemType.DataItem)

 

   4:     {

 

   5:         IListBlobItem blobItem =  ((ListViewDataItem)e.Item).DataItem as IListBlobItem;

 

   6:         Image img = e.Item.FindControl("img") as Image;

 

   7:         img.ImageUrl = blobItem.Uri.AbsoluteUri;

 

   8:     }

 

   9: }

d. To ensure that each time the photos would be refreshed when container was selected, I implemented the following snippet.
   1: protected void ddlBlobContainer_SelectedIndexChanged(object sender, EventArgs e)

 

   2: {

 

   3:     BindUploadList();

 

   4: }

e. In the Page_Load method, I get the setting first and then dump the blob containers to the dropdownlist, so that user could select the available container.
   1: protected void Page_Load(object sender, EventArgs e)

 

   2: {

 

   3:     if (!Page.IsPostBack)

 

   4:     {

 

   5:         InitializeStorage();

 

   6:

 

   7:         var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

 

   8:         blobStorage = storageAccount.CreateCloudBlobClient();

 

   9:

 

  10:         IEnumerable<CloudBlobContainer> cloudBlobContainers = blobStorage.ListContainers();

 

  11:         ddlBlobContainer.DataSource = cloudBlobContainers;

 

  12:         ddlBlobContainer.DataTextField = ddlBlobContainer.DataValueField = "Name";

 

  13:         ddlBlobContainer.DataBind();

 

  14:

 

  15:         BindUploadList();

 

  16:     }

 

  17: }

f. When the button “Import” was clicked, I loop through all the available photo in the container. For each blob item in the container, I’ll read the blob item, buffer it into array of byte, and finally submit the photos to the SQL Azure.
   1: protected void Button1_Click(object sender, ImageClickEventArgs e) {

 

   2:     CloudBlobContainer container = blobStorage.GetContainerReference("photos");

 

   3:

 

   4:     foreach (IListBlobItem blobItem in container.ListBlobs())

 

   5:     {

 

   6:         CloudBlob obj = container.GetBlobReference(blobItem.Uri.AbsoluteUri);

 

   7:         BlobStream blobStream = obj.OpenRead();

 

   8:

 

   9:         System.Drawing.Image objImage = null;

 

  10:         objImage = System.Drawing.Image.FromStream(blobStream);

 

  11:

 

  12:         string uniqueBlobName = Guid.NewGuid().ToString();

 

  13:         byte[] buffer = new byte[obj.OpenRead().Length];

 

  14:         obj.OpenRead().Read(buffer, 0, (int)obj.OpenRead().Length);

 

  15:

 

  16:         PhotoManager.AddPhoto(Convert.ToInt32(Request.QueryString["AlbumID"]), uniqueBlobName, buffer);

 

  17:     }

 

  18:     GridView1.DataBind();

 

  19: }

5. Verification

Let’s press Run button and see how it works!
a. We’ll need to login as administrator first before we can perform bulk upload.
image_516667FF
b. Having done so, please proceed to Manage link and click Edit on one of the Album.
image_000FBED8
c. You can now see the list of container in the dropdownlist, once you select one of them, the photos are refreshed accordingly.
image_75565152
Now, click on the Import button. In few moments, you can see that, you’ve successfully uploaded the photos.
image_70E3BA5B
Alright, we are now done.
I hope you enjoy this post and would be useful for you. See you at another post.
This entry was posted in Uncategorized. Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

*