Highlights:

  • Redpanda claims its platform can transfer data between systems with less latency than Apache Kafka, a well-known open-source alternative. One reason for the software’s superior performance is that it’s written in the C++ programming language.
  • Benthos is a London-based startup known for its open-source data streaming engine of the same name. The platform boasts over 200 connectors tailored for popular databases, cloud services, and other products.

Redpanda Data Inc. has acquired Benthos, an open-source platform developer that assists organizations in transferring information between their applications.

The Redpanda Benthos deal was announced recently, though the financial terms were not disclosed.

San Francisco-based Redpanda has received over USD 160 million in funding from investors, including Alphabet Inc.’s GV fund. The company offers a Redpanda platform that facilitates near-real-time data transfer between applications. For instance, a manufacturer could use the software to stream malfunction alerts from factory equipment to a cloud-based troubleshooting application.

Redpanda claims its platform can transfer data between systems with less latency than Apache Kafka, a well-known open-source alternative. One reason for the software’s superior performance is that it’s written in the C++ programming language. C++ is significantly faster than Java, the language used for most of Kafka’s components.

Another factor contributing to Redpanda’s speed is its use of a processing approach known as the thread-per-core design. The platform utilizes this technique with the help of an open-source C++ library named Seastar. As per the company, this technology enables its software to use hardware resources more efficiently.

Rather than running multiple threads on each CPU core as is typical, Redpanda distributes the computations involved in processing data across a set of central processing unit threads, with only one thread per core using Seastar. This approach prevents the threads from interrupting one another’s calculations, avoiding the processing delays associated with such interruptions and improving performance.

Redpanda also promises to offer additional advantages over Kafka. On the other hand, Kafka utilizes an open-source tool named ZooKeeper for various configuration tasks, necessitating administrators to install it separately, thereby complicating the setup process. In contrast, Redpanda does not rely on ZooKeeper, and it can be deployed in minutes.

Benthos is a London-based startup known for its open-source data streaming engine of the same name. The platform boasts over 200 connectors tailored for popular databases, cloud services, and other products. Benthos’ connectors facilitate data streaming between the integrated products, Redpanda and the reverse.

Benthos offers various additional functionalities beyond data streaming. Besides streaming data into Redpanda, the platform can modify data to streamline processing. Benthos offers various additional functionalities beyond data streaming. Apart from streaming data into Redpanda, the platform can modify data to simplify processing.

Redpanda leveraged Benthos to develop a new commercial product called Redpanda Connect, launched recently alongside the acquisition announcement. The company claims its offering requires only one-third of the compute capacity needed to operate Kafka Connect, a comparable product designed for Kafka environments.

“Our largest Redpanda Connect user is pushing 50Gbps via 2X 25Gbps NICs to consume market data,” Alex Gallego, the founder and CEO of Redpanda, provided specifics in a recent blog post. “Our smallest Redpanda Connect user uses 100 millicores and 10 MiB of memory, processing hundreds of messages a second deployed on a heavily resource-constrained environment.”

Redpanda plans to maintain Benthos’ core code base and most of its connectors under an open-source license. The company intends to expand the software with additional features to facilitate connections to artificial intelligence platforms.