Code Library: Windows Azure

Showing posts with label Windows Azure. Show all posts

Azure Service Bus Azure Service Bus is a managed service for building reliable and connected applications (either on-premises or in the cloud) leveraging messaging patterns. Service Bus is often used as a key component in eventually consistent solution architectures—providing asynchronous messaging integrated with additional Azure resources such as SQL Database, Storage, Web Apps for App Service, or applications hosted on Azure Virtual Machines.

Service Bus features four different communication patterns:
 Queues Provide a basic FIFO (first in, first out) messaging pattern. Messages in a queue are stored until they are retrieved and deleted. Service Bus queues are conceptually similar to Azure Storage queues, yet they offer a few more advanced middleware capabilities (dead lettering, auto-forwarding, sessions, duplicate detection, etc.).

 Topics Provide a publish-and-subscribe messaging pattern. A message can be written to a topic, and multiple subscriptions can be attached to that topic, with different subscriptions receiving different messages depending on a filter.

 Relays Provide a bidirectional (two-way) communication pattern. Instead of storing messages like queues and topics, a relay simply proxies calls from the client/producer to the server/receiver. Service Bus Relays is also one of the older services in Azure. It was publicly announced in May 2006 as Live Labs Relay (since incorporated into the Service Bus family).

 Event Hubs Provide a highly scalable event and telemetry ingestion service for supporting scenarios requiring low latency and high reliability. The section below discusses Event Hubs in more detail.

Source of Information : Microsoft Azure Essentials Fundamentals of Azure Second Edition

Handling Blobs in Windows Azure The Windows Azure Blob Service is the Windows Azure Storage Service feature that manages the storage of file-based entities referred to as blobs. In this chapter, we will focus on the Blob service. In the next two chapters, we will look at the other storage services: the Table service and the Queue service. The definitive way to access the storage service is through the Windows Azure Storage Service REST API. The Windows Azure Storage Client library is a high-level managed library, included in the Windows Azure SDK, which abstracts the RESTful interface into a set of classes. The Windows Azure Storage Client Protocol classes contained in the Storage Client library expose functionality not provided in the core library classes. Specifically, the protocol classes provide access to the underlying HttpWebRequest object used to invoke the RESTful operations.

Nearly all the recipes in this chapter use the Storage Client library. Scattered through the recipes are examples of how to use it for basic operations, such as creating a container and uploading blobs. However, we use the RESTful interface in the Copying a blob with the Windows Azure Storage Service REST API recipe. It is worthwhile to have some knowledge of the Storage Service REST API because it is informative to look at the raw REST operations sent over the wire to the storage service when debugging problems with storage operations. We use the Protocol classes in the Leasing a blob using the Protocol classes in the Windows Azure Storage Client Library recipe.

There are two types of blob: block blobs optimized for streaming and page blobs optimized for random access. In the Uploading blocks to a block blob recipe, we see how to increase control over the uploading of a block blob by uploading individual blocks and then committing them to create the block blob. In the Uploading a VHD into a page blob recipe, we see how to improve the upload performance by uploading only non-empty pages. The primary use case for page blobs is the storage of VHDs, which can then be attached as an Azure Drive to a role instance.

Blobs can have both properties and metadata associated with them. The properties comprise standard HTTP header fields that provide additional information to clients downloading the blobs. The metadata comprises a sequence of arbitrary name-value pairs. In the Setting properties and metadata for a blob recipe, we see how to use properties and metadata.

The Blob service supports the creation of a blob snapshot to provide a read-only version of the content of a blob when the snapshot is created. A blob can be rolled-back to an earlier state by copying the snapshot over it. An important use case for a snapshot is when its base blob is a VHD page blob. A writable VHD page blob can be attached as an Azure Drive to only one instance at a time. However, a read-only VHD blob snapshot can be attached simultaneously to an unlimited number of instances. The Creating and using a blob snapshot recipe shows us blob snapshots in action.

The Blob service stores blobs in a simple two-level hierarchy of containers and blobs. In the Using blob directories recipe we use the directory support provided by the Blob service to simulate a more complex hierarchy mimicking directories in a file system. In the Creating and using the root container for blobs recipe we see how to create a root directory that allows blobs to appear to be located at the root of the hierarchy and not inside a container. This allows cross-domain policy files to be stored as blobs.

The Storage Client library provides both synchronous and asynchronous versions of almost all methods that access the storage service. The asynchronous methods adhere to the common language runtime Asynchronous Programming Model (APM). In the Downloading a blob asynchronously recipe, we see how easy it is to use the asynchronous methods to download a blob. In the Optimizing blob uploads and downloads recipe, we see how to use the Task Parallel library provided in .NET Framework 4 to improve the performance of blob uploads and downloads by performing operations in parallel.
A storage operation against the storage service may fail for various reasons. Consequently, the Storage Client library provides built-in support for retries of failed operations. In the Using retry policies with blob operations recipe, we see how to parameterize existing retry behavior and implement custom retry functionality.

The Windows Azure Content Delivery Network (CDN) can be used to cache blobs closer to the user. This significantly improves the download performance of blobs. In the Using the Windows Azure Content-Delivery Network (CDN) recipe, we see how to use the CDN. We also see how to configure custom domains for both the Blob service and the CDN.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Managing access control for containers and blobs The Windows Azure Storage Service authenticates all requests against the Table service and Queue service. However, the storage service allows the possibility of unauthenticated access against the Blob service. The reason is that blobs provide an ideal location for storing large static content for a website. For example, the images in a photo-sharing site could be stored as blobs and downloaded directly from the Blob service without being transferred through a web role.

Public access control for the Blob service is managed at the container level. The Blob service supports the following three types of access control:

» No public read access in which all access must be authenticated
» Public read access which allows blobs in a container to be readable without authentication
» Full public read access in which authentication is not required to read the container data and the blobs contained in it

No public read access is the same access control as for the Queue service and Table service. The other two access control types both allow anonymous access to a blob, so that, for example, the blob can be downloaded into a browser by providing its full URL.

In the Windows Azure Storage Client library, the BlobContainerPublicAccessType enumeration specifies the three types of public access control for a container. The BlobContainerPermissions class exposes two properties: PublicAccess specifying a member of the BlobContainerPublicAccessType enumeration and SharedAccessPolicies specifying a set of shared access policies. The SetPermissions() method of the CloudBlobContainer class is used to associate a BlobContainerPermissions instance with the container. The GetPermissions() method retrieves the access permissions for a container.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Connecting to the storage emulator The Windows Azure SDK provides a compute emulator and a storage emulator that work in a development environment to provide a local emulation of Windows Azure hosted services and storage services. There are some differences in functionality between storage services and the storage emulator. Prior to Windows Azure SDK v1.3, the storage emulator was named development storage.

An immediate difference is that the storage emulator supports only one account name and access key. The account name is hard-coded to be devstoreaccount1. The access key is hard-coded to be:

Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==

Another difference is that the storage endpoints are constructed differently for the storage emulator. The storage service uses URL subdomains to distinguish the endpoints for the various types of storage. For example, the endpoint for the Blob service for a storage account named myaccount is:

myaccount.blob.core.windows.net

The endpoints for the other storage types are constructed similarly by replacing the word blob with either table or queue.

This differentiation by subdomain name is not used in the storage emulator which is hosted on the local host at 127.0.0.1. Instead, the storage emulator distinguishes the endpoints for various types of storage through use of different ports. Furthermore, the account name, rather than being part of the subdomain, is provided as part of the URL. Consequently, the endpoints used by the storage emulator are as follows:

127.0.0.1:10000/devstoreaccount1 Blob
127.0.0.1:10001/devstoreaccount1 Queue
127.0.0.1:10002/devstoreaccount1 Table

The Windows Azure Storage Client library hides much of this complexity but an understanding of it remains important in case something goes wrong. The account name and access key are hard-coded into the Storage Client library, which also provides simple access to an appropriately constructed CloudStorageAccount object.

The Storage Client library also supports a special value for the DataConnectionString in the service configuration file. Instead of specifying the account name and access key, it is sufficient to specify the following:

UseDevelopmentStorage=true

For example, this is specified as follows in the service configuration file:

This value can also be used for the Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString data connection string required for Windows Azure Diagnostics.

The CloudStorageAccount.Parse() and CloudStorageAccount.FromConnectionString() methods handle this value in a special way to create a CloudStorageAccount object that can be used to authenticate against the storage emulator.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Using SetConfigurationSettingPublisher() The CloudStorageAccount class in the Windows Azure Storage Client library encapsulates a StorageCredential instance that can be used to authenticate against the Windows Azure Storage Service. It also exposes a FromConfigurationSetting() factory method that creates a CloudStorageAccount instance from a configuration setting.

This method has caused much confusion since, without additional configuration, it throws an InvalidOperationException with a message of "SetConfigurationSettingPublisher needs to be called before FromConfigurationSetting() can be used." Consequently, before using FromConfigurationSetting(), it is necessary to invoke SetConfigurationSettingPublisher() once. The intent of this method is that it can be used to specify alternate ways of retrieving the data connection string that FromConfigurationSetting() uses to initialize the CloudStorageAccount instance. This setting is process-wide, so is typically done in the OnStart() method of the RoleEntryPoint class for the role.

CloudStorageAccount.SetConfigurationSettingPublisher(
(configName, configSetter) =>
{
configSetter(RoleEnvironment.GetConfigurationSettingValue(
configName));
});

There are several levels of indirection here, but the central feature is the use of a method that takes a String parameter specifying the name of the configuration setting and returns the value of that setting. In the example here, the method used is RoleEnvironment.GetConfigurationSettingValue(). The configuration-setting publisher can be set to retrieve configuration settings from any location including app.config or web.config.

The use of SetConfigurationSettingPublisher() is no longer encouraged. Instead, it is better to use CloudStorageAccount.Parse(), which takes a data connection string in canonical form and creates a CloudStorageAccount instance from it. We see how to do this in the Connecting to the Windows Azure Storage Service recipe.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Connecting to the Windows Azure Storage Service In a Windows Azure hosted service, the storage account name and access key are stored in the service configuration file. By convention, the account name and access key for data access are provided in a setting named DataConnectionString. The account name and access key needed for Windows Azure diagnostics must be provided in a setting named Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString.

The DataConnectionString setting must be declared in the ConfigurationSettings section of the service definition file. However, unlike other settings, the connection string setting for Windows Azure diagnostics is implicitly defined when the diagnostics module is specified in the Imports section of the service definition file. Consequently, it must not be specified in the ConfigurationSettings section.

A best practice is to use different storage accounts for application data and diagnostic data. This reduces the possibility of application data access being throttled by competition for concurrent writes from the diagnostics monitor. It also provides a security boundary between application data and diagnostics data, as diagnostics data may be accessed by individuals who should have no access to application data.

In the Windows Azure Storage Client library, access to the storage service is through one of the client classes. There is one client class for each of Blob service, Queue service, and Table service—CloudBlobClient, CloudQueueClient, and CloudTableClient respectively. Instances of these classes store the pertinent endpoint, as well as the account name and access key.

The CloudBlobClient class provides methods to access containers list their contents and get references to containers and blobs. The CloudQueueClient class provides methods to list queues and to get a reference to the CloudQueue instance used as an entry point to the Queue service functionality. The CloudTableClient class provides methods to manage tables and to get the TableServiceContext instance used to access the WCF Data Services functionality used in accessing the Table service. Note that CloudBlobClient, CloudQueueClient, and CloudTableClient instances are not thread safe so distinct instances should be used when accessing these services concurrently.

The client classes must be initialized with the account name and access key, as well as the appropriate storage service endpoint. The Microsoft.WindowsAzure namespace has several helper classes. The StorageCredentialsAccountAndKey class initializes a StorageCredential instance from an account name and access key while the StorageCredentialsSharedAccessSignature class initializes a StorageCredential instance from a shared access signature. The CloudStorageAccount class provides methods to initialize an encapsulated StorageCredential instance directly from the service configuration file.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Managing Windows Azure Storage Service access keys The data stored by the Windows Azure Storage Service must be secured against unauthorized access. To ensure that security, all storage operations against the table service and the queue service must be authenticated. Similarly, other than inquiry requests against public containers and blobs, all operations against the blob service must also be authenticated. The blob service supports public containers so that, for example, blobs containing images can be downloaded directly into a web page.

Each storage account has a primary access key and a secondary access key that can be used to authenticate operations against the storage service. When creating a request against the storage service, one of the keys is used along with various request headers to generate a 256-bit, hash-based message authentication code (HMAC). This HMAC is added as an Authorization request header to the request. On receiving the request, the storage service recalculates the HMAC and rejects the request if the received and calculated HMAC values differ. The Windows Azure Storage Client library provides methods that manage the creation of the HMAC and attaching it to the storage operation request.

There is no distinction between the primary and secondary access keys. The purpose of the secondary access key is to enable continued use of the storage service while the other access key is being regenerated. While the primary access key is used for authentication against the storage service, the secondary access key can be regenerated without affecting the service—and vice versa. This can be extremely useful in situations where storage access credentials must be rotated regularly.

As possession of the storage account name and access key is sufficient to provide full control over the data managed by the storage account, it is essential that the access keys be kept secure. In particular, access keys should never be downloaded to a client, such as a Smartphone, as that exposes them to potential abuse.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Controlling Access in the Windows Azure Platform The various components of the Windows Azure Platform are exposed using Internet protocols. Consequently, they need to support authentication so that access to them can be controlled. The Windows Azure Storage Service manages the storage of blobs, queues, and tables. It is essential that this data be kept secure, so that there is no unauthorized access to it. Each storage account has an account name and an access key which are used to authenticate access to the storage service. The management of these access keys is important. The storage service provides two access keys for each storage account, so that the access key not being used can be regenerated. We see how to do this in the Managing Windows Azure Storage Service access keys recipe.

The storage service supports hash-based message authentication (HMAC), in which a storage operation request is hashed with the access key. On receiving the request, the storage service validates it and either accepts or denies it. The Windows Azure Storage Client library provides several classes that support various ways of creating an HMAC, and which hide the complexity of creating and using one. We see how to use them in the Connecting to the Windows Azure Storage Service recipe. The SetConfigurationSettingPublisher() method has caused some programmer grief, so we look at it in the Using SetConfigurationSettingPublisher() recipe.

The Windows Azure SDK provides a compute emulator and a storage emulator. The latter uses a hard-coded account name and access key. We see the support provided for this in the Connecting to the storage emulator recipe.

Blobs are ideal for storing static content for web roles, so the storage service provides several authentication methods for access to containers and blobs. Indeed, a container can be configured to allow anonymous access to the blobs in it. Blobs in such a container can be downloaded without any authentication. We see how to configure this in the Managing access control for containers and blobs recipe.

There is a need to provide an intermediate level of authentication for containers and blobs, a level that lies between full authentication and anonymous access. The storage service supports the concept of a shared access signature, which is a pre-calculated authentication token and can be shared in a controlled manner allowing the bearer to access a specific container or blob for up to one hour. We see how to do this in the Creating a shared access signature for a container or blob recipe.

A shared access policy combines access rights with a time for which they are valid. A container-level access policy is a shared access policy that is associated by name with a container. A best practice is to derive a shared access signature from a container-level access policy. Doing this provides greater control over the shared access signature as it becomes possible to revoke it. We see how do this in the Using a container-level access policy recipe.

There is more to the Windows Azure Platform than storage. The Windows Azure Service Management REST API is a RESTful API that provides programmatic access to most of the functionality available on the Windows Azure Portal. This API uses X.509 certificates for authentication. Prior to use, the certificate must be uploaded, as a management certificate, to the Windows Azure Portal. The certificate must then be added as a certificate to each request made against the Service Management API. We see how to do this in the Authenticating against the Windows Azure Service Management REST API recipe.

The Windows Azure AppFabric services use a different authentication scheme, based on a service namespace and authentication token. In practice, these are similar to the account name and access key used to authenticate against the storage service, although the implementation is different. The Windows Azure AppFabric services use the Windows Azure Access Control Service (ACS) to perform authentication. However, this is abstracted away in the various SDKs provided for the services. We see how to authenticate to one of these services in the Authenticating with the Windows Azure AppFabric Caching Service recipe.

Source of Information : MICROSOFT WINDOWS AZURE DEVELOPMENT COOKBOOK

Code Library

Categories