Gearpump：基于 Akka 的大数据实时处理引擎

Gearpump is a real-time big data streaming engine.

It is inspired by recent advances in the Akka framework and a desire to improve on existing streaming frameworks.

Gearpump is event/message based and featured as low latency handling, high performance, exactly once semantics,

dynamic topology update, Apache Storm compatibility, etc.

The name Gearpump is a reference to the engineering term “gear pump,” which is a super simplepump that consists of only two gears, but is very powerful at streaming water.

Gearpump Technical Highlights

Gearpump is a modern big data real-time streaming engine which has many unique features:

Extremely high performance

Low latency

Configurable message delivery guarantee (at least once, exactly once).

Highly extensible

Dynamic DAG

Storm compatibility

Samoa compatibility

widely IoT connectivity

Both high level and low level API

Gearpump Performance

Per initial benchmarks we are able to process 18 million messages/second (100 bytes per message) with a 8ms latency on a 4-node cluster.

Gearpump and Akka

Gearpump is a 100% Akka based platform. We model big data streaming within the Akka actor hierarchy.

Gearpump