Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 14 additions & 8 deletions sdk/eventhub/azure-eventhubs/HISTORY.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,24 @@
# Release History

## 5.0.0b3 (2019-09-10)

**New features**
- `EventProcessor` has a load balancer that balances load among multiple EventProcessors automatically
- A new `PartitionManager` implementation that uses Azure Blob Storage is added
to centrally store the checkpoint data for event processors. It's packaged separately as a plug-in to this package.
Refer to [Azure Blob Storage Partition Manager](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs-checkpointstoreblob-aio) for details.

- Added support for automatic load balancing among multiple `EventProcessor`.
- Added `BlobPartitionManager` which implements `PartitionManager`.
- Azure Blob Storage is applied for storing data used by `EventProcessor`.
- Packaged separately as a plug-in to `EventProcessor`.
- For details, please refer to [Azure Blob Storage Partition Manager](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs-checkpointstoreblob-aio).
- Added property `system_properties` on `EventData`.

**Breaking changes**

- `PartitionProcessor` constructor removed argument "checkpoint_manager". Its methods (initialize, process_events,
process_error, close) added argument "partition_context", which has method update_checkpoint.
- `CheckpointManager` was replaced by `PartitionContext`
- Renamed `Sqlite3PartitionManager` to `SamplePartitionManager`
- Removed constructor method of `PartitionProcessor`. For initialization please implement the method `initialize`.
- Replaced `CheckpointManager` by `PartitionContext`.
- `PartitionContext` has partition context information and method `update_checkpoint`.
- Updated all methods of `PartitionProcessor` to include `PartitionContext` as part of the arguments.
- Updated accessibility of class members in `EventHub/EventHubConsumer/EventHubProducer`to be private.


## 5.0.0b2 (2019-08-06)

Expand Down
6 changes: 1 addition & 5 deletions sdk/eventhub/azure-eventhubs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ The Azure Event Hubs client library allows for publishing and consuming of Azure
- Observe interesting operations and interactions happening within your business or other ecosystem, allowing loosely coupled systems to interact without the need to bind them together.
- Receive events from one or more publishers, transform them to better meet the needs of your ecosystem, then publish the transformed events to a new stream for consumers to observe.

[Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs) | [Package (PyPi)](https://pypi.org/project/azure-eventhub/) | [API reference documentation](https://azure.github.io/azure-sdk-for-python/ref/azure.eventhub) | [Product documentation](https://docs.microsoft.com/en-ca/azure/event-hubs/)
[Source code](https://github.com/Azure/azure-sdk-for-python/tree/master/sdk/eventhub/azure-eventhubs) | [Package (PyPi)](https://pypi.org/project/azure-eventhub/) | [API reference documentation](https://azure.github.io/azure-sdk-for-python/ref/azure.eventhub) | [Product documentation](https://docs.microsoft.com/en-us/azure/event-hubs/)

## Getting started

Expand All @@ -20,10 +20,6 @@ Install the Azure Event Hubs client library for Python with pip:
```
$ pip install --pre azure-eventhub
```
For Python2.7, please install package "typing". This is a workaround for [issue 6767](https://github.com/Azure/azure-sdk-for-python/issues/6767).
```
$ pip install typing
```

**Prerequisites**

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ class EventHubConsumer(ConsumerProducerMixin): # pylint:disable=too-many-instan
group to be actively reading events from the partition. These non-exclusive consumers are
sometimes referred to as "Non-Epoch Consumers."

Please use the method `create_consumer` on `EventHubClient` for creating `EventHubConsumer`.
"""
_timeout = 0
_epoch_symbol = b'com.microsoft:epoch'
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@

class EventProcessor(object): # pylint:disable=too-many-instance-attributes
"""
An EventProcessor constantly receives events from all partitions of the Event Hub in the context of a given
An EventProcessor constantly receives events from multiple partitions of the Event Hub in the context of a given
consumer group. The received data will be sent to PartitionProcessor to be processed.

It provides the user a convenient way to receive events from multiple partitions and save checkpoints.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ class EventHubProducer(ConsumerProducerMixin): # pylint: disable=too-many-insta
be created to allow event data to be automatically routed to an available partition or specific
to a partition.

Please use the method `create_producer` on `EventHubClient` for creating `EventHubProducer`.
"""
_timeout_symbol = b'com.microsoft:timeout'

Expand Down
1 change: 1 addition & 0 deletions sdk/eventhub/azure-eventhubs/azure/eventhub/consumer.py
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ class EventHubConsumer(ConsumerProducerMixin): # pylint:disable=too-many-instan
group to be actively reading events from the partition. These non-exclusive consumers are
sometimes referred to as "Non-Epoch Consumers."

Please use the method `create_consumer` on `EventHubClient` for creating `EventHubConsumer`.
"""
_timeout = 0
_epoch_symbol = b'com.microsoft:epoch'
Expand Down
1 change: 1 addition & 0 deletions sdk/eventhub/azure-eventhubs/azure/eventhub/producer.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ class EventHubProducer(ConsumerProducerMixin): # pylint:disable=too-many-instan
be created to allow event data to be automatically routed to an available partition or specific
to a partition.

Please use the method `create_producer` on `EventHubClient` for creating `EventHubProducer`.
"""
_timeout_symbol = b'com.microsoft:timeout'

Expand Down
6 changes: 3 additions & 3 deletions sdk/eventhub/azure-eventhubs/tests/test_longrunning_send.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,12 +61,12 @@ def send(sender, args):
total += 1
except ValueError:
sender.send(batch, timeout=0)
print("Sent total {} of partition {}".format(total, sender.partition))
print("Sent total {} of partition {}".format(total, sender._partition))
batch = sender.create_batch()
except Exception as err:
print("Partition {} send failed {}".format(sender.partition, err))
print("Partition {} send failed {}".format(sender._partition, err))
raise
print("Sent total {} of partition {}".format(total, sender.partition))
print("Sent total {} of partition {}".format(total, sender._partition))


@pytest.mark.liveTest
Expand Down