Streamlining Streaming System Management with Apache Heron

Josh Fischer

2019-09-12 10:00 #bigdata

Apache Heron (Incubating) is a real-time, distributed, fault-tolerant stream processing engine. In 2018 it was donated to the Apache Foundation and is used for processing data sets in a way that allows systems to react to data as it is created. The big data era is here and it has become more critical than ever that organizations are able to make decisions based on the data they own in an efficient and accurate way. However, making this data available when and where it is needed can be problematic for organizations, product managers, and development teams across the globe. This is where data streaming steps in. The streaming system approach comes with both advantages and disadvantages. Heron assists with managing the pain points of streaming by relieving the developer from needing to manually intervene in many cases. Heron provides a way to build better distributed systems by implementing a process based architecture built in modern programming languages. We will walk through how Heron can assist organizations to build these systems with specific focus on use cases, system architecture, and system observability.