Skip to content

Latest commit

 

History

History
15 lines (11 loc) · 703 Bytes

streaming.md

File metadata and controls

15 lines (11 loc) · 703 Bytes

Streaming Data Import

This section covers importing data for streaming. The streaming example in the previous chapter received data through a single socket - which is not a scalable solution. In a real production system, there are many servers continuously writing logs, and we want to process all of those files. This section contains scalable solutions for data import. Since streaming is now used, there is no longer the need for a nightly batch job to process logs, but instead - this logs processing program can be long-lived - continuously receiving new logs data, processing the data, and computing log stats.

  1. Built In Methods for Streaming Import