Code Library

The Azure Queue service is used to store and retrieve messages. Queue messages can be up to 64 KB in size, and a queue can contain millions of messages—up to the maximum size of a storage account. Queues generally are used to create a list of messages to be processed asynchronously. The Queue service supports best-effort first in, first out (FIFO) queues.

For example, you might have a background process (such as a worker role or Azure WebJob) that continuously checks for messages on a queue. When it finds a message, it processes the message and then removes it from the queue. One of the most common examples is image or video processing.
Let’s say you have a web application that allows a customer to upload images into a container in blob storage. Your application needs to create thumbnails for each image. Rather than making the customer wait while this processing is done, you put a message on a queue with the customer ID and container name. Then, you have a background process that retrieves the message and parses it to get the customer ID and the container name. The background process then retrieves each image, creates a thumbnail, and writes the thumbnail back to the same blob storage container as the original image. After all images are processed, the background process removes the message from the queue.
What if you need the message to exceed 64 KB in size? In that case, you could write a file with the information to a blob in blob storage and put the URL to the file in the queue message. The background process could retrieve the message from the queue and then take the URL and read the file from blob storage to do the required processing.

Azure Queues provide at-least-once semantics in which each message may be read one or more times. This makes it important that all processing of the message be idempotent, which means the outcome of the processing must be the same regardless of how many times the message is processed.
When you retrieve a message from a queue, it is not deleted from the queue—you have to delete it when you’re done with it. When the message is read from the queue, it becomes invisible. The Invisibility Timeout is the amount of time to allow for processing the message—if the message is not deleted from the queue within this amount of time, it becomes visible again for processing. In general, you want to set this property to the largest amount of time that would be needed to process a message so that while one instance of a worker role is processing it, another instance doesn’t find it (visible) on the queue and try to process it at the same time.

You don’t want to read the message from the queue, delete it from the queue, and then start processing it. If the receiver fails, that queue entry will never be processed. Leaving the message on the queue (but invisible) until the processing has completed handles the case of the receiving process failing—eventually, the message will become visible again and will be processed by another instance of the receiver.

You can simulate a workflow by using a different queue for each step. A message can be processed from one queue from which it is deleted on completion, and then that processing can place a new message on a different queue to initiate processing for the next step in the workflow. You can also prioritize messages by using queues and processing the messages in them with different priorities.

The Queue service provides poison message support through the dequeue count. The concern is that an invalid message could cause an application handling it to crash, causing the message to become visible on the queue again only to crash the application again the next time the message is processed. Such a message is referred to as a poison message. You can prevent this by checking the dequeue count for the message. If this exceeds some level, the processing of the message should be stopped, the message deleted from the queue, and a copy inserted in a separate poison message queue for offline review. You could process those entries periodically and send an email when an entry is placed on the queue, or you could just let them accumulate and check them manually.

If you want to process the queue messages in batches, you can retrieve up to 32 messages in one call and then process them individually. Note, however, that when you retrieve a batch of messages, it sets the Invisibility Timeout for all of the messages to the same time. This means you must be able to process all of them within the time allotted.

Source of Information : Microsoft Azure Essentials Fundamentals of Azure Second Edition