文件最后提交记录最后更新时间
fix(stream): 深度审计整改——并发安全、类型安全、错误处理、测试覆盖、模块边界、import排序 Phase 5B 错误处理: - 09-01: 核心数据路径裸RuntimeException替换为StreamRuntimeException - 09-02: GraphModelCheckpointExecutor统一使用StreamException - 09-05: JdbcCheckpointStorage TaskLocation解析失败添加WARN日志 Phase 6 类型安全: - 15-01/15-02: CepOperator/SharedBuffer raw cast添加@SuppressWarnings和注释 - 15-05: StreamReduceOperator/WindowAggregationOperator添加key类型防御检查 - 15-08: MemoryKeyedStateBackend.TypedNamespaceAndKey添加防御性类型检查 Phase 7 测试质量+模块边界: - 16-02: 新增TestNFACompilerExtended覆盖times/oneOrMore/optional/GroupPattern - 16-06: TestConnectorConsistencyCapability修正tautological断言 - 16-08: TestCepOperatorStateRecovery验证独立实例恢复 - 16-10: TestStateSnapshotRoundTrip新增snapshot/restore往返测试 - 16-11: TestWindowOperatorBasic添加Javadoc说明 - 16-12: 新增TestCepSkipStrategyE2E覆盖skip strategy端到端 - 01-02: nop-stream-runtime pom.xml nop-message-core改为test scope - 03-02: connector零引用接口标记@Internal - 03-09: RPC接口标记@Internal - 03-01: nop-stream-api pom.xml添加注释 Phase 8 import排序: - 251个非生成源文件按AGENTS.md规范重排import(java.*→jakarta.*→third-party→io.nop.stream.*) - 替换FQN引用为import+短名 9 天前
fix(stream-cep): 修复CEP引擎溢出、状态清理和模式诊断问题 - DeweyNumber.increase增加整数溢出检测 - SharedBuffer.registerEvent增加eventId计数器溢出保护和缓存一致性修复 - CepOperator修复advanceTime中非超时partial match被误清理的问题 - CepOperator注册per-state windowTime的定时器 - Pattern所有MalformedPatternException添加诊断参数 - NFA暴露getWindowTimes用于per-state超时判断 - 新增回归测试覆盖上述修复 19 小时前
fix(stream): 修复审计发现的全部P0-P3缺陷(41项) - N17: MemoryInternalAppendingState累加器每次add前重置 - N1: WindowOperator存储累加器引用而非getLocalValue() - N2: mergeWindowContents ClassCastException改为WARN日志 - K15: windowNamespace使用类名前缀避免namespace碰撞 - N7: emitWindowContents签名改为显式key参数 - N12+N16: snapshot/restore简化为super调用修复key分区和非Serializable ACC - N3: MergingWindowSet.persist()重新接入ListState - N19: instantiateOperators处理TimestampsAndWatermarksTransformation - N22: ValueStateDescriptor传递typeInfo - N23: StreamSourceOperator移除正常完成后的cancel() - N29: KeySelectorPartitioner处理null key和Integer.MIN_VALUE - K10: ChainingOutput side output改为WARN日志 - K4+K6: WindowedStreamImpl标注@Deprecated, 清理注释残留 - N8: getSimpleAccumulator抛UnsupportedOperationException - N20: checkpointExecutorFactory改为实例字段 - N28: SimpleStreamOperatorFactory通过序列化创建新实例 - K11: WindowOperatorTimerService timer从Supplier获取key - N25: MemoryMapState保存descriptor引用 - N26: accumulator标记transient + rebind重建 - N18: KeyedStreamImpl从parentStream获取environment - N9: BatchLoaderSourceFunction loader在finally中关闭 - N10: DebeziumCdcSourceFunction用CountDownLatch替代Thread.sleep - N27/N33: WindowAssignerContext可空标注, watermarkInterval可配置 - K14: JdbcCheckpointStorage标注MySQL only - N41: StreamSinkOperator条件性rollback - K17/K20/K24: TimerService/Configuration/累加器接口标注@Internal/@Deprecated - N21: DataStreamImpl增加TypeInformation重载 - N34: UnknownTypeInformation实现Serializable - N40: DataStreamImpl移除Serializable - K19: 空壳模块pom.xml添加placeholder注释 - N4/N5/N6/N14/N11: 示例代码修复(CEP条件/事件类型/demo stub/用户检查/统一数据源) 14 天前
feat(stream): SinkFunction契约和RuntimeContext API填充 - BatchConsumerSinkFunction: 添加finish()方法覆盖,flush后标记flushed防止close()重复flush - RuntimeContext接口: 添加3个核心方法签名(getIndexOfThisSubtask/getNumberOfParallelSubtasks/getTaskName) - IterationRuntimeContext接口: 添加getIterationCount方法签名 - StreamingRuntimeContext: 实现RuntimeContext新方法 - CepRuntimeContext: 委托模式实现RuntimeContext新方法 - MockRuntimeContext/匿名实现: 同步更新 2 天前
fix(stream-core): 修复流处理引擎状态管理和执行计划问题 - MemoryInternalAppendingState增加routeKey溢出保护 - MemoryKeyedStateBackend修复getOrCreateState线程安全隐患 - MemoryStateSerDe增加序列化异常处理和类型安全 - SimpleKeyedStateStore重构状态管理逻辑 - DataStreamImpl/SingleOutputStreamOperatorImpl修复allowedLateness传播 - WindowedStreamImpl增加forceNonParallel支持 - InputGate增加barrier转发机制 - RebalancePartitionRouter修复整数溢出 - OperatorChain增强算子链管理 - WindowAggregationOperator增加merge回调 - GraphExecutionPlan/TaskExecutor增强执行计划容错 - PartitionedPlanGenerator增加分区策略感知 - JobGraphGenerator简化生成逻辑 - 新增PartitionPolicyAware接口 - 新增回归测试覆盖上述修复 19 小时前
fix(stream): 修复审计发现的全部P0-P3缺陷(41项) - N17: MemoryInternalAppendingState累加器每次add前重置 - N1: WindowOperator存储累加器引用而非getLocalValue() - N2: mergeWindowContents ClassCastException改为WARN日志 - K15: windowNamespace使用类名前缀避免namespace碰撞 - N7: emitWindowContents签名改为显式key参数 - N12+N16: snapshot/restore简化为super调用修复key分区和非Serializable ACC - N3: MergingWindowSet.persist()重新接入ListState - N19: instantiateOperators处理TimestampsAndWatermarksTransformation - N22: ValueStateDescriptor传递typeInfo - N23: StreamSourceOperator移除正常完成后的cancel() - N29: KeySelectorPartitioner处理null key和Integer.MIN_VALUE - K10: ChainingOutput side output改为WARN日志 - K4+K6: WindowedStreamImpl标注@Deprecated, 清理注释残留 - N8: getSimpleAccumulator抛UnsupportedOperationException - N20: checkpointExecutorFactory改为实例字段 - N28: SimpleStreamOperatorFactory通过序列化创建新实例 - K11: WindowOperatorTimerService timer从Supplier获取key - N25: MemoryMapState保存descriptor引用 - N26: accumulator标记transient + rebind重建 - N18: KeyedStreamImpl从parentStream获取environment - N9: BatchLoaderSourceFunction loader在finally中关闭 - N10: DebeziumCdcSourceFunction用CountDownLatch替代Thread.sleep - N27/N33: WindowAssignerContext可空标注, watermarkInterval可配置 - K14: JdbcCheckpointStorage标注MySQL only - N41: StreamSinkOperator条件性rollback - K17/K20/K24: TimerService/Configuration/累加器接口标注@Internal/@Deprecated - N21: DataStreamImpl增加TypeInformation重载 - N34: UnknownTypeInformation实现Serializable - N40: DataStreamImpl移除Serializable - K19: 空壳模块pom.xml添加placeholder注释 - N4/N5/N6/N14/N11: 示例代码修复(CEP条件/事件类型/demo stub/用户检查/统一数据源) 14 天前
fix(stream): 修复审计发现的全部P0-P3缺陷(41项) - N17: MemoryInternalAppendingState累加器每次add前重置 - N1: WindowOperator存储累加器引用而非getLocalValue() - N2: mergeWindowContents ClassCastException改为WARN日志 - K15: windowNamespace使用类名前缀避免namespace碰撞 - N7: emitWindowContents签名改为显式key参数 - N12+N16: snapshot/restore简化为super调用修复key分区和非Serializable ACC - N3: MergingWindowSet.persist()重新接入ListState - N19: instantiateOperators处理TimestampsAndWatermarksTransformation - N22: ValueStateDescriptor传递typeInfo - N23: StreamSourceOperator移除正常完成后的cancel() - N29: KeySelectorPartitioner处理null key和Integer.MIN_VALUE - K10: ChainingOutput side output改为WARN日志 - K4+K6: WindowedStreamImpl标注@Deprecated, 清理注释残留 - N8: getSimpleAccumulator抛UnsupportedOperationException - N20: checkpointExecutorFactory改为实例字段 - N28: SimpleStreamOperatorFactory通过序列化创建新实例 - K11: WindowOperatorTimerService timer从Supplier获取key - N25: MemoryMapState保存descriptor引用 - N26: accumulator标记transient + rebind重建 - N18: KeyedStreamImpl从parentStream获取environment - N9: BatchLoaderSourceFunction loader在finally中关闭 - N10: DebeziumCdcSourceFunction用CountDownLatch替代Thread.sleep - N27/N33: WindowAssignerContext可空标注, watermarkInterval可配置 - K14: JdbcCheckpointStorage标注MySQL only - N41: StreamSinkOperator条件性rollback - K17/K20/K24: TimerService/Configuration/累加器接口标注@Internal/@Deprecated - N21: DataStreamImpl增加TypeInformation重载 - N34: UnknownTypeInformation实现Serializable - N40: DataStreamImpl移除Serializable - K19: 空壳模块pom.xml添加placeholder注释 - N4/N5/N6/N14/N11: 示例代码修复(CEP条件/事件类型/demo stub/用户检查/统一数据源) 14 天前
docs(stream): 添加依赖说明文档和修复import分组 - 新增connector/package-info.java说明optional依赖 - JdbcCheckpointStorage添加nop-dao provided依赖Javadoc - 6个源文件import分组修复为java.*→jakarta.*→third-party→io.nop.* 2 天前
fix(stream-runtime): 修复检查点清理和窗口算子问题 - CheckpointCoordinator简化成功快照清理逻辑 - PendingCheckpoint修复状态标记 - WindowOperator修复time evictor集成问题 - 新增回归测试覆盖上述修复 19 小时前
fix(nop-stream): Phase 1 - BOM+POM governance, IDE debris cleanup (Plan 76) - R2-01-03: Add nop-stream-connector and nop-stream-runtime to nop-bom - R1-01-04: Remove explicit ${project.version} from fraud-example pom - R1-01-05: Remove redundant maven.compiler properties from runtime pom - R1-02-02: Remove IDE debris (.classpath/.project/.settings) from checkpoint/flink/flow - R1-01-03: Add spec drift comment in fraud-example pom - Add .gitignore for IDE debris patterns 2 天前
重构代码。拆分出nop-dataset, nop-orm-model, nop-orm-eql模块 3 年前
fix(stream): P3审计Phase3 - 依赖优化+文档对齐 - nop-stream-cep/pom.xml中nop-xlang改为optional - fraud-example移除对core内部SimpleKeyedStateStore的依赖,内联DemoKeyedStateStore - README.md标注RuntimeTopology为规划中 - 5个未使用连接器接口添加@apiNote标注为API预留 2 天前
feat(nop-stream): 新增connector模块,实现基于nop-batch统一抽象的Source/Sink适配器 - 新建nop-stream-connector模块,提供BatchLoader/Consumer、Message、Debezium CDC适配器 - StreamConnectors工具类统一入口 - StreamExecutionEnvironment增加算子关闭逻辑 - StreamSinkOperator支持AutoCloseable关闭 - nop-bom新增nop-message-debezium依赖 14 天前
README.md

nop-stream

Nop 平台的流处理引擎,定位为声明式图模型驱动的可分布式执行引擎

核心模型是 StreamModel(可序列化的算子图及其组件注册表),可由 XDSL 声明式定义、Java DataStream API 编程构造、或 Delta 定制合成。三种入口最终生成同一套 canonical StreamModel,经统一的五层执行管线(StreamGraph → JobGraph → PartitionedPlan → DeploymentPlan → RuntimeTopology)编译执行。

注: RuntimeTopology 处于规划阶段,LOCAL 模式当前通过 GraphExecutionPlan 直接执行,DISTRIBUTED 模式规划通过 IStreamExecutionDispatcher 调度。

模块

模块 状态 说明
nop-stream-core 活跃 StreamModel、StreamGraph/JobGraph、执行引擎、算子、状态管理
nop-stream-runtime 活跃 窗口算子、Checkpoint 协调器、存储实现、分布式执行框架
nop-stream-cep 活跃 CEP 引擎(NFA + Pattern API + 声明式模型)
nop-stream-connector 活跃 Source/Sink 连接器(nop-batch 桥接、CDC、消息队列)
nop-stream-fraud-example 活跃 欺诈检测示例
nop-stream-api 规划中 公共 API 提取
nop-stream-checkpoint 规划中 Checkpoint 独立模块
nop-stream-flink 规划中 Flink 执行引擎后端
nop-stream-flow 规划中 XDSL 声明式流编排

设计文档

  • ai-dev/design/nop-stream/ — 完整设计文档目录(架构、核心子系统、关键设计决策)

快速开始(Java DataStream API 方式)

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.fromElements(1, 2, 3, 4, 5)
   .map(x -> x * 2)
   .filter(x -> x > 4)
   .print();
env.execute("simple-pipeline");  // 统一走图模型路径:StreamGraph → JobGraph → TaskExecutor

DataStream API 是 StreamModel 的编程构造器,不是最终用户的主入口。主入口是 XDSL 声明式图模型定义(规划中,见 nop-stream-flow 模块)。

构建

./mvnw clean install -pl nop-stream -am -DskipTests -T 1C