There are a couple of factors that can affect Kafka consumption speed:
- The most obvious one is network speed of the server.
- The amount of brokers associated to the Kafka Topic configured in the producer. The more brokers there are, the higher throughput allowed.
- The amount of partitions in the Kafka Topic. You are able to create multiple Consumers to retrieve data from the same topic using the same Group ID and that will load balance the consumption process. But the limit of consumers allowed to have concurrent connections is limited by the number of partitions in the topic configured by the producer.
On a current Kafka Consumer setup we are consuming 7MB/s. This is on one Consumer and the Kafka Topic has one Broker.