Performance Testing
- Upstream UKafka, downstream is UHadoop’s HDFS
| Number of Connector Nodes | Number of Partitions | Number of brokers | Message Size/bytes | Kafka Machine Type | Hadoop Master | Hadoop Core | Speed (message count/s) |
|---|---|---|---|---|---|---|---|
| 1 | 3 | 3 | 1024 | 2-core 8G | C1-large | F1-large | 11658 |
| 1 | 3 | 3 | 1024 | 2-core 8G | C1-large | F1-large | 11830 |
| 3 | 3 | 3 | 1024 | 2-core 8G | C1-large | F1-large | 31618 |
| 3 | 3 | 3 | 1024 | 2-core 8G | C1-large | F1-large | 31562 |
| 1 | 3 | 3 | 512 | 2-core 8G | C1-large | F1-large | 18773 |
| 1 | 3 | 3 | 512 | 2-core 8G | C1-large | F1-large | 17156 |
| 3 | 3 | 3 | 512 | 2-core 8G | C1-large | F1-large | 46183 |
- Upstream UKafka, downstream is UES
| Number of Connector Nodes | Number of Partitions | Number of brokers | Message Size/bytes | Kafka Machine Type | ES version | ES Machine Type | Number of ES Nodes | Number of Shards | Number of Copies | Speed (message count/s) |
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 3 | 3 | 1024 | 2-core 8G | 6.5.4 | M1ES-large | 3 | 5 | 1 | 6429 |
| 1 | 3 | 3 | 1024 | 2-core 8G | 6.2.1 | M1ES-large | 3 | 5 | 1 | 6349 |
| 1 | 3 | 3 | 1024 | 2-core 8G | 5.5.1 | M1ES-large | 3 | 5 | 1 | 4995 |
| 1 | 3 | 3 | 1024 | 2-core 8G | 5.5.1 | M1ES-large | 3 | 5 | 1 | 4902 |
| 3 | 3 | 3 | 1024 | 2-core 8G | 6.5.4 | M1ES-large | 3 | 5 | 1 | 11801 |
| 3 | 3 | 3 | 1024 | 2-core 8G | 6.2.1 | M1ES-large | 3 | 5 | 1 | 12129 |
| 3 | 3 | 3 | 1024 | 2-core 8G | 5.5.1 | M1ES-large | 3 | 5 | 1 | 7540 |
| 1 | 3 | 3 | 512 | 2-core 8G | 6.5.4 | M1ES-large | 3 | 5 | 1 | 8902 |
| 1 | 3 | 3 | 512 | 2-core 8G | 6.2.1 | M1ES-large | 3 | 5 | 1 | 9800 |
| 1 | 3 | 3 | 512 | 2-core 8G | 5.5.1 | M1ES-large | 3 | 5 | 1 | 7952 |
| 3 | 3 | 3 | 512 | 2-core 8G | 6.5.4 | M1ES-large | 3 | 5 | 1 | 17183 |
| 3 | 3 | 3 | 512 | 2-core 8G | 6.2.1 | M1ES-large | 3 | 5 | 1 | 18216 |
| 3 | 3 | 3 | 512 | 2-core 8G | 5.5.1 | M1ES-large | 3 | 5 | 1 | 11538 |