Introduction to Message Broker Performance Testing
When evaluating message brokers like NATS, RabbitMQ, and Kafka, performance is a key factor. While vendors often provide benchmarks, these are sometimes accused of being overly curated. To truly understand how these brokers perform under different conditions, it is essential to conduct independent testing. This analysis focuses on two specific messaging patterns: async producer-consumer queues and request-reply interactions. By examining how these brokers handle various workloads, we can identify their strengths and weaknesses.
Testing was conducted using default configurations to simulate real-world scenarios. All brokers ran inside Docker containers on the same host system, ensuring consistency. Metrics such as memory usage, CPU consumption, and throughput were recorded to evaluate performance under varied message counts and payload sizes. The results offer valuable insights into how each broker handles diverse operational demands.
Test Environment and Setup
The testing was performed on an AMD Ryzen 7 8845HS CPU with 8 cores and 16 threads, running at 3.80 GHz. Each broker was deployed in its respective Docker container without any custom tuning to ensure fairness. The brokers included RabbitMQ with AMQP 0.9.1, Kafka in KRaft mode, and NATS with JetStream enabled. These configurations were chosen to reflect their most commonly used setups.
Memory usage was a notable metric, observed using Docker stats. On a cold start, Kafka showed significantly higher memory consumption, requiring 54 times more memory than NATS and 27 times more than RabbitMQ. This disparity highlights the resource efficiency of NATS, which consumed only 6 MiB of memory at startup. Such differences can impact the choice of broker, depending on the hardware constraints of the deployment environment.
Messaging Patterns and Metrics
The two messaging patterns tested were async producer-consumer and request-reply. In the former, a producer sends messages to a broker, which then delivers them to one or more consumers. In the latter, a client sends a request and waits for a reply, a less common scenario for message brokers but worth exploring given NATS native support for it. These patterns were tested across various combinations of message counts and payload sizes.
The test scenarios included combinations of 50000 messages at 256 bytes, 25000 messages at 1 KB, and other similar configurations. These combinations ensured a balanced load on the brokers while keeping the benchmark runtime manageable. Metrics like P95 latency were used instead of the mean, as they better represent the worst-case performance by filtering out noise and capturing tail-end delays.
Key Observations on Throughput
The results revealed some critical differences in how each broker handles throughput. NATS consistently demonstrated the lowest latency across most scenarios, aligning with its reputation for high performance. RabbitMQ also performed well but tended to show higher memory usage under heavier loads. Kafka, while powerful, exhibited the highest resource consumption due to its JVM-based architecture.
These findings suggest that NATS is particularly suited for applications requiring low-latency communication with minimal resource overhead. RabbitMQ offers a balanced performance profile, making it a solid choice for general-purpose messaging. Kafka, with its high throughput capabilities, is better suited for scenarios involving massive data streams, albeit at the cost of higher resource consumption.
Conclusion and Recommendations
Selecting the right message broker depends on the specific requirements of your application. If you need a lightweight solution with minimal resource usage, NATS is an excellent option. For more traditional workloads with moderate resource availability, RabbitMQ is a reliable choice. Finally, for applications demanding extremely high throughput, Kafka is a suitable candidate despite its higher resource demands.
Understanding the trade-offs between these brokers is essential for making an informed decision. By testing them in real-world scenarios, you can identify which solution aligns best with your operational needs. Always consider metrics like latency, memory usage, and throughput to ensure optimal performance.