Skip to content

Commit

Permalink
Update Readme.md
Browse files Browse the repository at this point in the history
  • Loading branch information
xurui203 committed Aug 10, 2018
1 parent 5a88800 commit 940d6e9
Showing 1 changed file with 8 additions and 8 deletions.
16 changes: 8 additions & 8 deletions samples/e2e/EventHubsCaptureEventGridDemo/Readme.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# How Event Hubs Capture integrates with Event Grid and Azure Function
# Migrate Captured Event Hubs data to a SQL Data Warehouse using Event Grid and Azure Function

One of the key scenarios for modern cloud scale apps is seamless integration and notification amongst other apps and services. In this blog post, we will go over a realistic scenario of capturing Azure Event Hub data into a SQL Database Warehouse, using an [Event Grid](https://docs.microsoft.com/azure/event-grid/overview) triggered Azure Function.
Event Hubs [Capture](https://docs.microsoft.com/en-us/azure/event-hubs/event-hubs-capture-overview) is the easiest way to automatically deliver streamed data in Event Hubs to an Azure Blob storage or Azure Data Lake store. You can subsequently process and deliver the data to any other storage destinations of your choice, such as SQL Data Warehouse or Cosmos DB. In this tutorial, we will demonstrate how you can pipe Captured data from your event hub into a SQL Database Warehouse by using an [Event Grid](https://docs.microsoft.com/azure/event-grid/overview) triggered Azure Function.

### What is covered in this tutorial:

![Visual Studio](./media/EventGridIntegrationOverview.PNG)

* First, we turn on Capture on Azure Event Hub with an Azure blob storage as destination. Data generated by WindTurbineGenerator will be streamed into the Event Hub and automatically Captured into Azure Storage.
* Next, we create an Azure EventGrid subscription with the Azure Event Hub namespace as its source and the Azure Function endpoint and its destination.
* Whenever a new Avro blob file is generated by Azure EventHub Capture, Azure Event Grid notifies the Azure Function with the blob URI. The Function then does the required processing to migrate the data from the Storage blob to a SQL Database data warehouse.
* First, we create an Azure Event Hub with Capture enabled and set an Azure blob storage as the destination. Data generated by WindTurbineGenerator will be streamed into the event hub and automatically Captured into Azure Storage as Avro files.
* Next, we create an Azure Event Grid subscription with the Event Hubs namespace as its source and the Azure Function endpoint as its destination.
* Whenever a new Avro file is delivered to the Storage blob by Event Hubs Capture, Event Grid notifies the Azure Function with the blob URI. The Function then does the required processing to migrate the data from the Storage blob to a SQL Database data warehouse.

There are no worker services involved in polling for these Avro files, which eliminates management overhead and significantly lower COGS, especially in a cloud-scale production environment!

This sample solution contains files that do the following:
1. *WindTurbineDataGenerator* – A simple publisher that sends wind turbine data to a Capture-enable Event Hub
1. *WindTurbineDataGenerator* – A simple publisher that sends wind turbine data to a Capture-enabled event hub
1. *FunctionDWDumper* – An Azure Function that receives an Event Grid notification an Avro file is Captured to the Azure Storage blob. It receives the blob’s URI path, reads its contents and pushes this data to a SQL Data Warehouse.

# Prerequisites
Expand Down Expand Up @@ -80,7 +80,7 @@ Create a table in your Data Warehouse by running the *CreateDataWarehouseTable.s

![Select publish](./media/select-publish.png)

After publishing the function, you're ready to subscribe to the event.
After publishing the function, you are ready to subscribe to the Capture event!


## 4. Create an Event Grid subscription from the Functions app
Expand Down Expand Up @@ -138,6 +138,6 @@ This article shows how to use [Power BI with SQL Data Warehouse](https://docs.mi
Now you are all set to plug in the UI you need to get valuable business insights for your management.

# Conclusion
Looking forward to you trying out this sample and providing us your feedback. We would also love to see pull requests.
We look forward to your feedback after you give this tutorial a try! We also love to see pull requests so if you would like to contribute to our community.

Stay tuned for more samples from the Azure Messaging team, and till next time!

0 comments on commit 940d6e9

Please sign in to comment.