测试环境压缩测试

获取topic列表

kafka-topics.sh --zookeeper cdh-worker-dc-1:2181,cdh-worker-dc-2:2181,cdh-worker-dc-3:2181/kafka --list

获取topic配置信息

kafka-configs.sh --zookeeper cdh-worker-dc-1:2181,cdh-worker-dc-2:2181,cdh-worker-dc-3:2181/kafka --entity-type topics --entity-name test_dis-upper-lower-limits-all_gzip --describe

获取kafka日志信息

du -h /data/kafkalogs/test_*

修改topic配置信息

kafka-configs.sh --zookeeper cdh-worker-dc-1:2181,cdh-worker-dc-2:2181,cdh-worker-dc-3:2181/kafka  --entity-type topics --entity-name test_dis-upper-lower-limits-all_gzip --alter --add-config compression.type=gzip

kafka_2.11-1.1.0]# kafka-configs.sh --zookeeper cdh-worker-dc-1:2181,cdh-worker-dc-2:2181,cdh-worker-dc-3:2181/kafka  --entity-type topics --entity-name test_dis-upper-lower-limits-all_snappy --alter --add-config compression.type=snappy

kafka-configs.sh --zookeeper cdh-worker-dc-1:2181,cdh-worker-dc-2:2181,cdh-worker-dc-3:2181/kafka  --entity-type topics --entity-name test_dis-upper-lower-limits-all_lz4 --alter --add-config compression.type=lz4

查看偏移量

kafka-run-class.sh kafka.tools.GetOffsetShell --topic test_dis-upper-lower-limits-all  --time -1 --broker-list 10.65.6.19:9092 --partitions 0

消费数据

打赏