> Another idea is applying query engine optimizations like predicate pushdown. This makes a massive impact on query performance and efficiency. Unfortunately, standard Kafka storage is not designed to support this. That’s why I’m very bullish on LakeHouse support in Kafka like Confluent Tableflow and Iceberg Topics in Redpanda.
Could you explain this better? We're pretty far away from having first-class schema support and the ability to query these things in a predicate manner from Kafka
Yep, so my point is: you can shift these types of queries to Iceberg. You'll at least be able to leverage partitioning, which helps a lot with query time.
> Another idea is applying query engine optimizations like predicate pushdown. This makes a massive impact on query performance and efficiency. Unfortunately, standard Kafka storage is not designed to support this. That’s why I’m very bullish on LakeHouse support in Kafka like Confluent Tableflow and Iceberg Topics in Redpanda.
Could you explain this better? We're pretty far away from having first-class schema support and the ability to query these things in a predicate manner from Kafka
Yep, so my point is: you can shift these types of queries to Iceberg. You'll at least be able to leverage partitioning, which helps a lot with query time.