Event Hubs is optimized for high volumes and throughput of events and data, for example, when using hundreds of IoT devices that may continually send high volumes of data.

Event Hubs groups incoming data into partitions that buffer the information, ensuring that subscribers can keep up with the traffic. Each partition can also have its own distinct set of subscribers, which is useful for scaling out work among multiple consumers.

Event Hubs also supports authentication and saves the data stream to Azure Data Lake or Blob storage.

We mentioned earlier that events and messaging are best used together, so next, we will examine message queues in Azure Queue Storage and Service Bus.

Storage queues

Azure Storage contains a simple message queue mechanism called storage queues. Queues act as an intermediary for different system components to communicate with each other. Rather than one component talking directly to another, which would mean both ends need to be free and available, sending data to a queue allows the different components to work on tasks as they become free without impacting the broader system. In other words, using queues helps solution components become independent of each other.

Azure Storage Queues are the most straightforward option and offer a First In First Out (FIFO) queuing system. However, the actual order of messages is not guaranteed.

Azure components needing to send a message write the messages to a queue, and consumers pick up the messages from the queue one at a time to process them, as in the following example:

Figure 11.7 – Azure storage queues

Azure Storage Queues also provide an audit trail, and the queue messages can be up to 64 KB in size, but you can store millions of messages.

Azure Service Bus

Azure Service Bus is an enterprise-scale message queuing mechanism that enables you to use it either as a straightforward queuing mechanism or as a topic and subscriber mechanism. The publish/subscribe pattern allows multiple consumers rather than a single consumer, which is the case with a single queue. Filters can also be set on the subscriptions, enabling different consumers to process various topics.

For example, an order management system may need orders to be processed by different systems based on the product type or location. Orders for the US would be processed by a US-based Azure function, whereas orders bound for Europe would be processed by a European-based function, as in the following diagram:

Figure 11.8 – Service Bus topics and subscriptions

The first question to consider when choosing between these options is whether to use Storage Queues or Service Bus queues. The following table can help with this:

Two of the critical decisions are around ordering guarantee and delivery guarantee. A typical exam question is where you’re asked to ensure the FIFO pattern is used for message delivery. For this, you need to use Service Bus, and also, to enforce this, you need to enable sessions – which is one of the options when creating queues or subscriptions to a topic.

It’s also useful to remember that sessions require the Standard pricing tier or above; Basic does not support them.

You also need to consider that Service Bus only supports message queues of up to 80 GB compared to storage queues, which are only limited by the storage account’s capacity.

Other reasons to use Service Bus queues over Storage Queues are when you want to ensure FIFO delivery and when you need automatic de-duplication of messages.

The next decision to make is whether to use simple queues or topics and subscriptions. You should choose queues when you want to ensure one consumer only consumes each message. Conversely, select a topic when you want multiple consumers to read the messages or when you need to apply filtering.

Through the correct use of Event Grid, Event Hubs, Storage Queues, and Service Bus, you can architect robust applications whose components are independent of each other and therefore more reliable and robust.