Friday 27 April 2012

Deploying Web and Worker Roles in a Single Role


When we have multiple applications which are considerably small and those can run with minimum hardware requirements, we can combine those applications together and host in a single server to handle all the applications. This will be the option we mostly will take when we consider on-premise hosting centers as hosting environment.

When the applications are hosted on Azure we don’t have the opportunity to manually manage the IIS and to host them under separate Web sites (considering Web Role and Worker Role and not VM Role). But, Windows Azure provides such capability for hosting multiple sites under a single Web role (means under single IIS) using some configuration changes with the deployment package.

Previously, I have blogged some posts on how to publish multiple web applications, web sites and virtual directories in a single Web Role. This concept helps to reduce the total cost we pay as we use common instances for a multiple applications. But, those post are useful when we consider Web Applications, Web Sites and Virtual Directories which runs under IIS.

In this post, we will consider hosting Web Role and Worker Role under a single Role; call it as Web Role itself as we will be creating a Web Role template and add Worker Role logic into it.

Note:
As this helps to running Web Role and Worker Role under a single Role, it required a common hardware resource allocation for both. Eventhough the total billing will get reduce, we are overloading the Web Role by adding Worker Role logic. So if the Web Role itself considerably medium or large in size and can utilize the instance hardware more than 85% at peak time, better to host both the roles separately or increase the VM size (and/or number of instances) of the Role per analysis.

For better understanding, I am planning to show how to create both the role in a single role with some sample messages in the first example and second example shows by having both the roles logic in a single role.

First Example

This example shows by creating Web Role and Worker Role in a Single Role with some sample messages. The steps follow –

Step 1: Create a new Windows Azure Project with WebWorkerInSingleRole as Project Name.
Step 2: In the New Windows Azure Project window, select any Web Role template that you required to add with Worker Role. In this example, I required an ASP.NET Web Role.
Visual Studio will create a Windows Azure Project by adding Web Role in it.

Step 3: Open the WebRole.cs file and look at the source.

Here, we have OnStart() events which will be called when initializing the role (means starting up the role). But, we can also have some more events by overriding such as Run() (will be called after role initialized), OnStop() (will be called when role is stopping) event.

Note: If you create a separate Windows Azure Project with Worker Role template and open the WorkerRole.cs file, it will have the OnStart() event and also Run() event. We can implement WorkerRole by adding Run() event.

Step 4: To add Worker Role logic in Web Role, add the Run() event in the WebRole.cs file and the coding will be go as we do normally in WorkerRole.
Note: As the Worker Role runs in background with Web Role application, it should not have any direct communication with the output of the Web Role. The Worker Role can use Queue storage for communicating messages. (As we have both the code in a single project, calling methods from one to another role might make issue).

Step 5: Run the project and see the outputs.
The browser shows the defaut.aspx screen.
Open the output windows (Ctrl + W, O) in Visual Studio and note the messages occur.

Second Example

This second example takes the following use case and implement by having Web Role and Worker Role logic in a single role.
  1. Need a Web Page for selecting multiple images at a time and to upload to blob storage.
  2. Once the images are uploaded, the images must be converted in to the following requirements
    • The images must be 300px x 300px in size. When width or height exceeds other (means landscape or portrait), the percentage must be considered while shirking (but either width or height should be 300px).
    • The images must have a message www.dotnettwiter.com in a right bottom corner.
  3. The converted image must get stored in the same blob by adding name final word at the end of the image name.
In this example, the Web Role will get the list of images using the web page and upload to the Blob storage. It also creates a queue for adding the image name for processing in the Worker Role.

The worker Role will take each image name from the queue (as a message) and process it as per the requirement then store in the same blob. It also deletes the message from the queue.

The Web and Worker Roles are totally separated and communication happening thro’ queue storage. The logic is very simple and similar to one of the Azure Training Kit lab example provided.

For implementation, I am taking the same example created before. The further steps follow.

Step 6: As we use storage account for storing images in blob and messages in queue, we required to add the storage account connection string into the configuration.

So open the Web Role Properties (double click the WebRole in the cloud project), and add DataConnectionString under settings.
Step 7: Open the Default.aspx and add the following source.

The ASPX script
.ErrorInfo
{
 color:Red;
}
function ShowNextUpload() {
 if (document.getElementById('tr1').style.display == 'none')
  document.getElementById('tr1').style.display = '';

 else if (document.getElementById('tr2').style.display == 'none')
  document.getElementById('tr2').style.display = '';

 else if (document.getElementById('tr3').style.display == 'none')
  document.getElementById('tr3').style.display = '';

 else if (document.getElementById('tr4').style.display == 'none')
  document.getElementById('tr4').style.display = '';

 else if (document.getElementById('tr5').style.display == 'none')
  document.getElementById('tr5').style.display = '';

 else if (document.getElementById('tr6').style.display == 'none')
  document.getElementById('tr6').style.display = '';

 else if (document.getElementById('tr7').style.display == 'none')
  document.getElementById('tr7').style.display = '';

 else if (document.getElementById('tr8').style.display == 'none')
  document.getElementById('tr8').style.display = '';

 else if (document.getElementById('tr9').style.display == 'none')
  document.getElementById('tr9').style.display = '';
}
<h1>Image Processing</h1>
<br/>
<table>
 <tr>
  <td width="550px">
   <asp:FileUpload ID="FileUpload0" runat="server" Width="100%" />
   <asp:RequiredFieldValidator ID="ImageRequiredValidator" runat="server" ControlToValidate="FileUpload0" Text="*" CssClass="ErrorInfo" />
  </td>
 </tr>
 <tr>
  <td colspan="3">
   <a href="#" onclick="ShowNextUpload()">Add more row</a>
  </td>
 </tr>
 <tr id='tr1' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload1" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr2' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload2" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr3' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload3" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr4' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload4" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr5' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload5" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr6' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload6" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr7' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload7" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr8' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload8" runat="server" Width="100%" />
  </td>
 </tr>
 <tr id='tr9' style='display:none'>
  <td>
   <asp:FileUpload ID="FileUpload9" runat="server" Width="100%" />
  </td>
 </tr>
</table>
<asp:Button ID="btnUpload" runat="server" Text="Upload" onclick="btnUpload_Click" />
The C# code behind
using System.Net;
using System.IO;
using Microsoft.WindowsAzure;
using Microsoft.WindowsAzure.StorageClient;
using Microsoft.WindowsAzure.ServiceRuntime;
public partial class _Default : System.Web.UI.Page
{
    private static bool storageInitialized = false;
    private static object gate = new Object();
    private static CloudBlobClient blobStorage;
    private static CloudQueueClient queueStorage;
 
 protected void btnUpload_Click(object sender, EventArgs e)
    {
        InitializeStorage();

        for (int intIndex = 0; intIndex < 10; intIndex++)
        {
            FileUpload fileUploadControl = ((FileUpload)((Button)sender).Parent.FindControl("FileUpload" + intIndex));
            if (fileUploadControl.HasFile)
            {
                // upload the image to blob storage
                string uniqueBlobName = string.Format("imageprocessing/image_{0}{1}", Guid.NewGuid().ToString(), Path.GetExtension(fileUploadControl.FileName));
                CloudBlockBlob blob = blobStorage.GetBlockBlobReference(uniqueBlobName);
                blob.Properties.ContentType = FileUpload1.PostedFile.ContentType;
                blob.UploadFromStream(fileUploadControl.FileContent);
                System.Diagnostics.Trace.TraceInformation("Uploaded image '{0}' to blob storage as '{1}'", fileUploadControl.FileName, uniqueBlobName);

                // queue a message to process the image
                var queue = queueStorage.GetQueueReference("imageprocessingqueue");
                var message = new CloudQueueMessage(String.Format("{0}", blob.Uri.ToString()));
                queue.AddMessage(message);
                System.Diagnostics.Trace.TraceInformation("Queued message to process blob '{0}'", uniqueBlobName);
            }
        }
    }
    private void InitializeStorage()
    {
        if (storageInitialized)
        {
            return;
        }

        lock (gate)
        {
            if (storageInitialized)
            {
                return;
            }

            try
            {
                // For information on handling configuration changes
                // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
                // read storage account configuration settings
                CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
                {
                    configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
                });

                // read account configuration settings
                var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

                // create blob container for images
                blobStorage = storageAccount.CreateCloudBlobClient();
                CloudBlobContainer container = blobStorage.GetContainerReference("imageprocessing");
                container.CreateIfNotExist();

                // configure container for public access
                var permissions = container.GetPermissions();
                permissions.PublicAccess = BlobContainerPublicAccessType.Container;
                container.SetPermissions(permissions);

                // create queue to communicate with worker role
                queueStorage = storageAccount.CreateCloudQueueClient();
                CloudQueue queue = queueStorage.GetQueueReference("imageprocessingqueue");
                queue.CreateIfNotExist();
            }
            catch (WebException)
            {
                throw new WebException("Storage services initialization failure. "
                    + "Check your storage account configuration settings. If running locally, "
                    + "ensure that the Development Storage service is running.");
            }

            storageInitialized = true;
        }
    }
}

Step 8: Open the WebRole.cs file and add the following code in the WebRole class.
public class WebRole : RoleEntryPoint
{
    private CloudQueue queue;
    private CloudBlobContainer container;

    public override bool OnStart()
    {
        // Set the maximum number of concurrent connections 
        ServicePointManager.DefaultConnectionLimit = 12;

        // For information on handling configuration changes
        // see the MSDN topic at http://go.microsoft.com/fwlink/?LinkId=166357.
        // read storage account configuration settings
        CloudStorageAccount.SetConfigurationSettingPublisher((configName, configSetter) =>
        {
            configSetter(RoleEnvironment.GetConfigurationSettingValue(configName));
        });
        var storageAccount = CloudStorageAccount.FromConfigurationSetting("DataConnectionString");

        // initialize blob storage
        CloudBlobClient blobStorage = storageAccount.CreateCloudBlobClient();
        container = blobStorage.GetContainerReference("imageprocessing");

        // initialize queue storage 
        CloudQueueClient queueStorage = storageAccount.CreateCloudQueueClient();
        queue = queueStorage.GetQueueReference("imageprocessingqueue");

        Trace.TraceInformation("Creating container and queue...");

        bool storageInitialized = false;
        while (!storageInitialized)
        {
            try
            {
                // create the blob container and allow public access
                container.CreateIfNotExist();
                var permissions = container.GetPermissions();
                permissions.PublicAccess = BlobContainerPublicAccessType.Container;
                container.SetPermissions(permissions);

                // create the message queue(s)
                queue.CreateIfNotExist();

                storageInitialized = true;
            }
            catch (StorageClientException e)
            {
                if (e.ErrorCode == StorageErrorCode.TransportError)
                {
                    Trace.TraceError("Storage services initialization failure. "
                        + "Check your storage account configuration settings. If running locally, "
                        + "ensure that the Development Storage service is running. Message: '{0}'", e.Message);
                    System.Threading.Thread.Sleep(5000);
                }
                else
                {
                    throw;
                }
            }
        }

        return base.OnStart();
    }

    public override void Run()
    {
        Trace.TraceInformation("Worker Role started listening for processing messages...");

        while (true)
        {
            try
            {
                // retrieve a new message from the queue
                CloudQueueMessage msg = queue.GetMessage();
                if (msg != null)
                {
                    // parse message retrieved from queue
                    var imageBlobUri = msg.AsString;
                    Trace.TraceInformation("Processing image in blob '{0}'.", imageBlobUri);

                    string targetBlobUri = System.Text.RegularExpressions.Regex.Replace(imageBlobUri, "([^\\.]+)(\\.[^\\.]+)?$", "$1-final$2");

                    CloudBlob inputBlob = container.GetBlobReference(imageBlobUri);
                    CloudBlob outputBlob = container.GetBlobReference(targetBlobUri);

                    using (BlobStream input = inputBlob.OpenRead())
                    using (BlobStream output = outputBlob.OpenWrite())
                    {
                        ProcessImage(input, output, new Size(new Point(400, 400)));

                        // commit the blob and set its properties
                        output.Commit();
                        outputBlob.Properties.ContentType = "image/jpeg";
                        outputBlob.SetProperties();

                        // remove message from queue
                        queue.DeleteMessage(msg);

                        // To delay the background processing time (to understand)
                        System.Threading.Thread.Sleep(5000);
                    }
                }
                else
                {
                    System.Threading.Thread.Sleep(1000);
                }
            }
            catch (StorageClientException e)
            {
                Trace.TraceError("Exception when processing queue item. Message: '{0}'", e.Message);
                System.Threading.Thread.Sleep(5000);
            }
        }
    }

    public void ProcessImage(Stream input, Stream output, Size size)
    {
        var originalImage = new Bitmap(input);

        int sourceWidth = originalImage.Width;
        int sourceHeight = originalImage.Height;

        float nPercent = 0;
        float nPercentW = 0;
        float nPercentH = 0;

        nPercentW = ((float)size.Width / (float)sourceWidth);
        nPercentH = ((float)size.Height / (float)sourceHeight);

        if (nPercentH < nPercentW)
            nPercent = nPercentH;
        else
            nPercent = nPercentW;

        int destWidth = (int)(sourceWidth * nPercent);
        int destHeight = (int)(sourceHeight * nPercent);

        Bitmap targetImage = new Bitmap(destWidth, destHeight);
        Graphics g = Graphics.FromImage((Image)targetImage);
        g.InterpolationMode = InterpolationMode.HighQualityBicubic;

        g.DrawImage(originalImage, 0, 0, destWidth, destHeight);

        Font font = new Font("Calibri", 15, FontStyle.Bold | FontStyle.Underline, System.Drawing.GraphicsUnit.Pixel);
        g.SmoothingMode = SmoothingMode.AntiAlias;
        g.TextRenderingHint = TextRenderingHint.AntiAlias;
        g.DrawString("www.dotnettwitter.com", font, new SolidBrush(Color.Black), targetImage.Width - 170, targetImage.Height - 25);
        g.Flush();

        targetImage.Save(output, ImageFormat.Jpeg);

        g.Dispose();
    }
}
There is nothing difference in the code when doing both Web and Worker Roles as separate Roles. So I am not explaining what each code is doing here.

Step 9: Run the project and verify by adding some images. The images can added as required from the screen.

The Worker Role fetches from the blob and process and store it to the same blob in the background. Below screen shots shows on Windows Azure MMC.






download the working copy of the source code in C# here.

0 Responses to “Deploying Web and Worker Roles in a Single Role”

Post a Comment