A Distributed Streaming Platform

文件	最后提交记录	最后更新时间
0001-adopt-huaweimaven.patch	update 0001-adopt-huaweimaven.patch. Signed-off-by: xiexing <xiexing4@hisilicon.com>	4 年前
0002-CVE-2022-41881.patch	升级netty到4.1.86.Final，修复 2022-41881	3 年前
0003-CVE-2023-34455.patch	snappy升级到1.1.10.1，修复CVE-2023-34455	3 年前
0004-CVE-2022-42004.patch	jackson升级到2.13.4，修复CVE-2022-42004	3 年前
0005-CVE-2016-3189.patch	rocksdbjni 升级到6.19.3，修复CVE-2016-3189	2 年前
0006-NPE-subscriptionState.patch	Fix NPE in SubscriptionState	2 年前
0007-fix-payload-incorrectly.patch	Fix using random payload in ProducerPerformance incorrectly	2 年前
0008-Cast-SMT-allow-null.patch	Cast SMT should allow null value records to pass through	2 年前
0009-format-RocksDBConfigSetter.patch	Fix the formatting of example RocksDBConfigSetter	2 年前
0010-not-update-connection.patch	Don't update connection idle time for muted connections	2 年前
0011-ConfigEntry.patch	ConfigEntry#equal does not compare other fields when value is NOT null	2 年前
0012-incorrectly-LeaderElectionCommand.patch	Fix noop set is incorrectly replaced with succeeded set from LeaderElectionCommand	2 年前
0013-AlterIsr.patch	AlterIsr and LeaderAndIsr race condition	2 年前
0014-override-toString.patch	override toString method to show correct value in doc	2 年前
0015-SessionWindows-closed-early.patch	SessionWindows are closed too early	2 年前
0016-non-existent-URL.patch	Javadocs search sends you to a non-existent URL	2 年前
0017-fix-log-clean.patch	log clean relative index range check of group consider empty log segment to avoid too many empty log segment left	2 年前
0018-Backport-fix-for-CVE-2026-35554.patch	Backport fix for CVE-2026-35554 and CVE-2026-33558 CVE-2026-35554 (KAFKA-19012): don't reuse producer batch buffers on the client in certain error cases. Upstream commit 1df2ac5b (PR #21065), adapted to 2.8.2 Sender/RecordAccumulator/ProducerBatch. CVE-2026-33558: redact sensitive request/response content (SCRAM salt, delegation token HMAC, SASL authBytes, config values) from DEBUG logs. Upstream commits 7e9d50a0 (#19655) + c2f90235 (#20776), following the duplicate-and-mask approach of #20776. Co-Authored-By: Claude Code <noreply@anthropic.com>	25 天前
0019-Backport-fix-for-CVE-2026-33558.patch	Backport fix for CVE-2026-35554 and CVE-2026-33558 CVE-2026-35554 (KAFKA-19012): don't reuse producer batch buffers on the client in certain error cases. Upstream commit 1df2ac5b (PR #21065), adapted to 2.8.2 Sender/RecordAccumulator/ProducerBatch. CVE-2026-33558: redact sensitive request/response content (SCRAM salt, delegation token HMAC, SASL authBytes, config values) from DEBUG logs. Upstream commits 7e9d50a0 (#19655) + c2f90235 (#20776), following the duplicate-and-mask approach of #20776. Co-Authored-By: Claude Code <noreply@anthropic.com>	25 天前
0020-CVE-2022-23302-23307.patch	Backport fixes for CVE-2022-23302, CVE-2022-23307, CVE-2024-31141, CVE-2024-56128, CVE-2025-27818, CVE-2025-27819 - CVE-2022-23302 & CVE-2022-23307: replace bundled log4j 1.2.17 with reload4j 1.2.19 (KAFKA-13660), bump slf4j to 1.7.36 - CVE-2024-31141: filter automatic config providers by class name - CVE-2024-56128: validate client nonce in SCRAM final message - CVE-2025-27818 & CVE-2025-27819: disallow JNDI/LDAP login modules in SASL JAAS by default Consolidates infra_team PRs #76-#81 (which conflict with each other and with master after PR #82) into a single PR. Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	21 天前
0021-CVE-2024-31141.patch	Backport fixes for CVE-2022-23302, CVE-2022-23307, CVE-2024-31141, CVE-2024-56128, CVE-2025-27818, CVE-2025-27819 - CVE-2022-23302 & CVE-2022-23307: replace bundled log4j 1.2.17 with reload4j 1.2.19 (KAFKA-13660), bump slf4j to 1.7.36 - CVE-2024-31141: filter automatic config providers by class name - CVE-2024-56128: validate client nonce in SCRAM final message - CVE-2025-27818 & CVE-2025-27819: disallow JNDI/LDAP login modules in SASL JAAS by default Consolidates infra_team PRs #76-#81 (which conflict with each other and with master after PR #82) into a single PR. Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	21 天前
0022-CVE-2024-56128.patch	Backport fixes for CVE-2022-23302, CVE-2022-23307, CVE-2024-31141, CVE-2024-56128, CVE-2025-27818, CVE-2025-27819 - CVE-2022-23302 & CVE-2022-23307: replace bundled log4j 1.2.17 with reload4j 1.2.19 (KAFKA-13660), bump slf4j to 1.7.36 - CVE-2024-31141: filter automatic config providers by class name - CVE-2024-56128: validate client nonce in SCRAM final message - CVE-2025-27818 & CVE-2025-27819: disallow JNDI/LDAP login modules in SASL JAAS by default Consolidates infra_team PRs #76-#81 (which conflict with each other and with master after PR #82) into a single PR. Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	21 天前
0023-CVE-2025-27818-27819.patch	Backport fixes for CVE-2022-23302, CVE-2022-23307, CVE-2024-31141, CVE-2024-56128, CVE-2025-27818, CVE-2025-27819 - CVE-2022-23302 & CVE-2022-23307: replace bundled log4j 1.2.17 with reload4j 1.2.19 (KAFKA-13660), bump slf4j to 1.7.36 - CVE-2024-31141: filter automatic config providers by class name - CVE-2024-56128: validate client nonce in SCRAM final message - CVE-2025-27818 & CVE-2025-27819: disallow JNDI/LDAP login modules in SASL JAAS by default Consolidates infra_team PRs #76-#81 (which conflict with each other and with master after PR #82) into a single PR. Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	21 天前
0024-Backport-fix-for-CVE-2023-44981.patch	Backport fixes for CVE-2023-44981 and CVE-2024-23944 - 0024: upgrade ZooKeeper 3.5.9 to 3.8.3 (KAFKA-15596, CVE-2023-44981) - 0025: upgrade ZooKeeper 3.8.3 to 3.8.4 (CVE-2024-23944) - Release 21 -> 22 Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	18 天前
0025-Backport-fix-for-CVE-2024-23944.patch	Backport fixes for CVE-2023-44981 and CVE-2024-23944 - 0024: upgrade ZooKeeper 3.5.9 to 3.8.3 (KAFKA-15596, CVE-2023-44981) - 0025: upgrade ZooKeeper 3.8.3 to 3.8.4 (CVE-2024-23944) - Release 21 -> 22 Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	18 天前
LICENSE	add LICENSE	6 年前
README.md	update README.md.	4 年前
gradle-6.8.1-all.zip	update version 2.8.1	4 年前
gradle-wrapper.jar	update version 2.8.1	4 年前
kafka-2.8.2-src.tgz	update version 2.8.2	4 年前
kafka.service	add service	5 年前
kafka.spec	Backport fixes for CVE-2023-44981 and CVE-2024-23944 - 0024: upgrade ZooKeeper 3.5.9 to 3.8.3 (KAFKA-15596, CVE-2023-44981) - 0025: upgrade ZooKeeper 3.8.3 to 3.8.4 (CVE-2024-23944) - Release 21 -> 22 Co-Authored-By: DeepSeek V4 Pro <noreply@anthropic.com>	18 天前

kafka

介绍

Apache Kafka是分布式发布-订阅消息系统。它最初由LinkedIn公司开发，之后成为Apache项目的一部分。Kafka是一种快速、可扩展的、设计内在就是分布式的，分区的和可复制的提交日志服务。Apache Kafka与传统消息系统相比，有以下不同：

它被设计为一个分布式系统，易于向外扩展；
它同时为发布和订阅提供高吞吐量；
它支持多订阅者，当失败时能自动平衡消费者；
它将消息持久化到磁盘，因此可用于批量消费，例如ETL，以及实时应用程序。

软件架构

软件架构说明 Apache kafka组件介绍

Broker：和RabbitMQ概念相似，Broker是一台独立的Kafka服务节点，负责消息的存储和转发。默认端口9092，集群可以包含多个Kafka Broker。生产者和消费者需要跟Broker建立连接才可以收发消息。
Producer：消息发送方。在Kafka中，生产者不是逐条发送消息给Broker，而是批量发送的，这样可以提升消息发送速率。通过batch.size参数可以控制批次的大小
Consumer：消息接收方，消费者。Kafka只支持pull模式，通过pull拉取信息。
Topic：消息队列，生产者发送消息给Topic，消费者从Topic获取消息。生产者和Topic，Topic和消费者，都是多对多关系。生产者发送消息时，如果Topic不存在，会自动创Topic。
Partition：一个Topic可以被划分成多个分区，每个分区可以对外单独提供消息服务能力，这也是分片思想的落地。Partition类似于MySQL的分库分表，为了达到横向扩展和负载均衡的目的。
Replication: 分区副本，是Partition的冗余备份分区，当Partition不可用时，ZooKeeper会自动将Replication（Follower）分区升级为Partition（Leader）分区。
Offset: 分区中的Record的位置标示，每一个消费者都会记录自己的消费位置（offset）。