Windows Azure Blob Storage

Windows Azure Storage is one of the core components in Windows Azure that offers a scalable, highly available, and competitively priced storage option. Amongst others abstractions in Azure Storage (Table Storage and Queue Storage), Blob Storage is perhaps the most widely-used service. Blob Storage allows us to store any unstructured text and binary data such as video, audio, images, and so many more.

Blob Storage can either be accessed through the API programmatically or explorer tools. This article discusses and reviews several popular explorer tools for Blob Storage.

Reviews and Ratings

Disclaimer

The reviews and ratings are entirely my individual opinion and preference. The reviews and ratings are based on my personal experience when using each product, and what I consider important.

Measurement Criteria

This review will examine these products using the following four dimensions:

• User interface and experience
  I’ll look at how usable the product is. Have the user interface and experience been designed to be comfortable and user friendly?
• Basic features
  This category covers the standard and basic functionality when dealing with Blob Storage. This includes operations such copying / moving files, managing security, and access.
• Advanced settings
  This dimension measures how flexible and configurable the product is. This includes the ability to adjust settings or preferences such as defining block size, retry policy, bandwidth settings, and so on.
• Others notable features
  This metric is about supplementary features that enrich the product, making the product more powerful and beneficial for users. This might include innovative features such as multi-language support, directory comparison, graphical user interface for logging, etc.

For each measurement, I’ll provide a brief description and rating ranging from 1 to 5. 1 means the product provides a poor experience or lacks capability, 5 means the product provides awesome proficiency. Additionally, I would be also giving a N/A (not applicable) for the product that doesn’t have any applicable features.

1. Cloud Storage Studio 2 by Cerebrata

We start the review with Cloud Storage Studio 2 (CSS2) from Cerebrata, a company acquired by Red Gate last Oct 2011. CSS 2 is an exploration tool not only for Blob Storage, but also for Tables and Queue Storage.

The product costs $195 for a Professional License (volume discounts apply). Customers are encouraged to try it out with a 30-day free trial.

a) User interface and experience

CSS2 provides a powerful UI grouping concept and navigation, enabling users to group related storage accounts and subscriptions together – as can be seen in the Figure 1.

Figure 1 – Cloud Storage Studio UI

The Navigation (in red) and Tabs (in yellow) area look good to me. However, I find the Explorer Area (in blue) is tedious. Copying files and directories will prompt a dialog box that only allows us to copy the blobs within the container only as can be seen in Figure 2. I believe there should be more intuitive way to implement this.

Figure 2 – Cloud Storage Studio Copying Blobs

Rating: 3.5

b) Basic features

I would say it satisfies most of the basic needs when dealing with Blob Storage. Starting from managing containers, displaying directories, all the way down to individual blob level are all properly supported.

Rating: 5.0

c) Advanced settings

CSS2 provides powerful settings that enable users to easily define the configuration settings.

Figure 3 – Cloud Storage Studio Configuration Settings

Rating: 4.5

d) Other notable features

One of the features that I like most in CSS2 is the graphical UI for Storage Analytic Logging and Metric. It provides a really expressive experience and has a good look and feel.

Figure 4 – View Storage Analytics Data

Rating: 4.0

2. CloudXplorer by ClumsyLeaf

CloudXplorer is a lightweight yet handy explorer tool from ClumsyLeaf Software. It has been very popular and has been used by many people including Microsoft folks in various events.

CloudXplorer is entirely free-of-charge, downloadable from here.

a) User interface and experience

CloudXplorer comes with Windows Explorer-like user interface, providing a friendly experience, especially for Windows users. Uploading and downloading Blobs are implemented with “Copy / Cut and Paste” experience, and the same when dealing with our local files.

Figure 5 – CloudXplorer User Interface

Rating: 5.0

b) Basic features

I would say it satisfies most of the basic needs.

Rating: 5.0

c) Advanced settings

I don’t find any options for user to define advanced configuration and settings.

Rating: N/A

d) Other notable features

Unfortunately, I also didn’t find any fancy features in CloudXplorer.

Rating: N/A

3. CloudBerry Explorer for Azure Blob Storage by CloudBerry Lab

The last product I’m reviewing is the CloudBerry Explorer. CloudBerry Labs offers many great products focusing for explorer tools and online backup for various cloud providers such asAmazon AWS, Windows Azure, and RackSpace.

Furthermore, CloudBerry Explorer supports multi-languages: English, Chinese, and Japanese. The product comes in two versions:

• Free version
• And PRO version, purchasable at $ 39.99

Check out the following for the comparison between the two.

Figure 6 – CloudBerry Explorer User Interface

Rating: 4.5

b) Basic features

Like the other two tools, I would say it satisfies most needs.

Rating: 5.0

c) Advanced settings

CloudBerry Explorer also provides a powerful and flexible option for user to configure settings such as setting bandwidth, chunk size, encryption, etc. However, I notice that a few of the features such as encryption and compression are only available in PRO version.

Figure 7 – CloudBerry Explorer Options

Rating: 5.0

d) Other notable features

My favorite feature of CloudBerry Explorer is Compare and Sync Folders. This is an extremely useful feature enabling us to compare and sync either cloud or local folders. As seen in the screenshot below, the tool shows the comparison result between the two displays. Then we can finally define to either sync left to right, right to left, or in both directions.

Figure 5 – CloudBerry Explorer User Interface

Rating: 4.5

Conclusion

We have gone through three explorer tools for Windows Azure Blob Storage. I would say all three products are pretty awesome. There are always advantages from one to another. The following table summarizes reviews and ratings that we’ve come across.

In conclusion, if you need a simple and lightweight explorer, CloudXplorer is probably the way to go. However, if you need more flexible settings and innovative features, you should consider Cloud Storage Studio or CloudBerry Explorer.

Windows Azure Blob Storage could be analogized as file-system on the cloud. It enables us to store any unstructured data file such as text, images, video, etc. In this post, I will show how to upload big file into Windows Azure Storage. Please be inform that we will be using Block Blob for this case. For more information about Block Blob and Page Block, please visit here.

I am assume that you know how to upload a file to Windows Azure Storage. If you don’t know, I would recommend you to check out this lab from Windows Azure Training Kit.

Uploading a blob (commonly-used technique)

The following snippet show you how to upload a blob using a commonly-used technique, blob.UploadFromStream() which eventually invoking PutBlob REST-API.

protected void btnUpload_Click(object sender, EventArgs e)
{
    var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    blobClient = storageAccount.CreateCloudBlobClient();

    CloudBlobContainer container = blobClient.GetContainerReference("image2");
    container.CreateIfNotExist();

    var permission = container.GetPermissions();
    permission.PublicAccess = BlobContainerPublicAccessType.Container;
    container.SetPermissions(permission);

    string name = fu.FileName;
    CloudBlob blob = container.GetBlobReference(name);
    blob.UploadFromStream(fu.FileContent);
}

The above code snippet works well in most case. Although you could upload at maximum 64 MB per file (for block blob), it’s more recommended to upload using another technique which I am going to describe more detail.

Uploading a blob by splitting it into chunks and calling PutBlockList

The idea of this technique is to split a block blob into smaller chunk of blocks, uploading them one-by-one or in-parallel and eventually join them all by calling PutBlockList().

protected void btnUpload_Click(object sender, EventArgs e)
{
    CloudBlobClient blobClient;
    var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");
    blobClient = storageAccount.CreateCloudBlobClient();

    CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
    container.CreateIfNotExist();

    var permission = container.GetPermissions();
    permission.PublicAccess = BlobContainerPublicAccessType.Container;
    container.SetPermissions(permission);

    string name = fu.FileName;
    CloudBlockBlob blob = container.GetBlockBlobReference(name);

    blob.UploadFromStream(fu.FileContent);

    int maxSize = 1 * 1024 * 1024; // 4 MB

    if (fu.PostedFile.ContentLength > maxSize)
    {
        byte[] data = fu.FileBytes; 
        int id = 0;
        int byteslength = data.Length;
        int bytesread = 0;
        int index = 0;
        List<string> blocklist = new List<string>();
        int numBytesPerChunk = 250 * 1024; //250KB per block
    
        do
        {
            byte[] buffer = new byte[numBytesPerChunk];
            int limit = index + numBytesPerChunk;
            for (int loops = 0; index < limit; index++)
            {
                buffer[loops] = data[index];
                loops++;
            }
            bytesread = index;
            string blockIdBase64 = Convert.ToBase64String(System.BitConverter.GetBytes(id));

            blob.PutBlock(blockIdBase64, new MemoryStream(buffer, true), null); 
            blocklist.Add(blockIdBase64);
            id++;
        } while (byteslength - bytesread > numBytesPerChunk);

        int final = byteslength - bytesread;
        byte[] finalbuffer = new byte[final];
        for (int loops = 0; index < byteslength; index++)
        {
            finalbuffer[loops] = data[index];
            loops++;
        }
        string blockId = Convert.ToBase64String(System.BitConverter.GetBytes(id));
        blob.PutBlock(blockId, new MemoryStream(finalbuffer, true), null);
        blocklist.Add(blockId);

        blob.PutBlockList(blocklist); 
    }
    else
        blob.UploadFromStream(fu.FileContent);            
}

Explanation about the code snippet

Since the idea is to split the big file into chunks. We would need to define size of each chunk, in this case 250KB. By dividing actual size with size of each chunk, we should be able to know number of chunk we need to split.

We also need to have a list of string (in this case: blocklist variable) to determine the blocks are in one group. Then we will loop to through each chunk and perform and upload by calling blob.PutBlock() and add it (as form of Base64 String) into the blocklist.

Note that there’s actually a left-over block that didn’t uploaded inside the loop. We will need to upload it again. When all blocks are successfully uploaded, finally we call blob.PutBlockList(). Calling PutListBlock() will commit all the blocks that we’ve uploaded previously.

Pros and Cons

The benefits (pros) of the technique

There’re a few benefit of using this technique:

• In the event where uploading one of the block fail due to whatever condition like connection time-out, connection lost, etc. We’ll just need to upload that particular block only, not the entire big file / blob.
• It’s also possible to upload each block in-parallel which might result shorter upload time.
• The first technique will only allow you to upload a block blob at maximum 64MB. With this technique, you can do more almost unlimited.

The drawbacks (cons) of the technique

Despite of the benefits, there’re also a few drawbacks:

• You have more code to write. As you can see from the sample, you can simply call the one line blob.UploadFromStream() in the first technique. But you will need to write 20+ lines of code for the second technique.
• It incurs more storage transaction as may lead to higher cost in some case. Referring to a post by Azure Storage team. The more chuck you have, the more storage transaction is incurred.

Large blob upload that results in 100 requests via PutBlock, and then 1 PutBlockList for commit = 101 transactions

Summary

I’ve shown you how to upload file with simple technique at beginning. Although, it’s easy to use, it has a few limitation. The second technique (using PutListBlock) is more powerful as it could do more than the first one. However, it certainly has some pros and cons as well.

I hope you could be able to use either one of them appropriately in your scenario. Hope this helps!