已合并
feat(gaussdb): add GaussDB/openGauss async SQLAlchemy dialect and handler #101
潘银创建于 4月17日
feat(gaussdb): add GaussDB/openGauss async SQLAlchemy dialect and handler #101
已合并
共 28 个文件变更+912-52
| @@ -92,10 +92,16 @@ cp .env.example .env | |||
| 92 | 92 | ||
| 93 | 然后按需修改 `server/.env`,重点包括: | 93 | 然后按需修改 `server/.env`,重点包括: |
| 94 | 94 | ||
| 95 | -- `DB_TYPE`:支持 `sqlite` / `mysql` | 95 | +- `DB_TYPE`:支持 `sqlite` / `mysql` / `gaussdb` / `opengauss` |
| 96 | - `IP`:Runtime 服务地址 | 96 | - `IP`:Runtime 服务地址 |
| 97 | - `LOWCODE_IMAGE`:低码 Agent 相关镜像配置 | 97 | - `LOWCODE_IMAGE`:低码 Agent 相关镜像配置 |
| 98 | 98 | ||
| 99 | +说明: | ||
| 100 | + | ||
| 101 | +- 默认安装仅包含 `sqlite` / `mysql` 所需依赖。 | ||
| 102 | +- 当 `DB_TYPE=gaussdb` 或 `DB_TYPE=opengauss` 时,需要额外安装 `async-gaussdb`。 | ||
| 103 | +- 使用仓库自带启动脚本时,脚本会根据 `server/.env` 中的 `DB_TYPE` 自动安装 `foundation[gaussdb]` 可选依赖;手工安装时可执行 `uv pip install -e "./foundation[gaussdb]"`。 | ||
| 104 | + | ||
| 99 | 完整配置项说明请参考 `docs/zh/2. 配置说明.md`。 | 105 | 完整配置项说明请参考 `docs/zh/2. 配置说明.md`。 |
| 100 | 106 | ||
| 101 | ### 3) 一键启动 Runtime 服务 | 107 | ### 3) 一键启动 Runtime 服务 |
| @@ -144,7 +150,7 @@ Windows(PowerShell): | |||
| 144 | 当前支持的关键配置能力: | 150 | 当前支持的关键配置能力: |
| 145 | 151 | ||
| 146 | - 运行端口与服务地址配置 | 152 | - 运行端口与服务地址配置 |
| 147 | -- 数据库类型配置(SQLite / MySQL) | 153 | +- 数据库类型配置(SQLite / MySQL / GaussDB / openGauss) |
| 148 | - 部署模式选择(进程;容器与 K8s 持续完善) | 154 | - 部署模式选择(进程;容器与 K8s 持续完善) |
| 149 | - 租户上下文透传与隔离策略 | 155 | - 租户上下文透传与隔离策略 |
| 150 | 156 | ||
| @@ -92,10 +92,16 @@ cp .env.example .env | |||
| 92 | 92 | ||
| 93 | Edit `server/.env` as needed. Important fields include: | 93 | Edit `server/.env` as needed. Important fields include: |
| 94 | 94 | ||
| 95 | -- **`DB_TYPE`**: `sqlite` or `mysql` | 95 | +- **`DB_TYPE`**: `sqlite`, `mysql`, `gaussdb`, or `opengauss` |
| 96 | - **`IP`**: reachable address for Runtime / agents | 96 | - **`IP`**: reachable address for Runtime / agents |
| 97 | - **`LOWCODE_IMAGE`**: low-code agent container image (required when using Docker/K8s-style flows) | 97 | - **`LOWCODE_IMAGE`**: low-code agent container image (required when using Docker/K8s-style flows) |
| 98 | 98 | ||
| 99 | +Notes: | ||
| 100 | + | ||
| 101 | +- The default install only includes dependencies for `sqlite` and `mysql`. | ||
| 102 | +- When `DB_TYPE=gaussdb` or `DB_TYPE=opengauss`, Runtime additionally needs `async-gaussdb`. | ||
| 103 | +- If you use the repository startup scripts, they detect `DB_TYPE` from `server/.env` and install `foundation[gaussdb]` automatically. For a manual setup, run `uv pip install -e "./foundation[gaussdb]"`. | ||
| 104 | + | ||
| 99 | For a full reference, see **`docs/en/2. Configuration.md`** (Chinese: `docs/zh/2. 配置说明.md`). | 105 | For a full reference, see **`docs/en/2. Configuration.md`** (Chinese: `docs/zh/2. 配置说明.md`). |
| 100 | 106 | ||
| 101 | ### 3) Start Runtime (recommended) | 107 | ### 3) Start Runtime (recommended) |
| @@ -146,7 +152,7 @@ Configure Runtime host and port in **agent-studio**’s backend environment, the | |||
| 146 | ## Configuration topics | 152 | ## Configuration topics |
| 147 | 153 | ||
| 148 | - Listen port and service address | 154 | - Listen port and service address |
| 149 | -- Database type (SQLite / MySQL) | 155 | +- Database type (SQLite / MySQL / GaussDB / openGauss) |
| 150 | - Deployment mode (process; container and K8s evolving) | 156 | - Deployment mode (process; container and K8s evolving) |
| 151 | - Tenant context propagation and isolation | 157 | - Tenant context propagation and isolation |
| 152 | 158 | ||
| @@ -110,7 +110,7 @@ MILVUS_PASSWORD=123456 | |||
| 110 | # ----------------------------------------------------------------------------- | 110 | # ----------------------------------------------------------------------------- |
| 111 | # 十、业务关系型数据库(与 openjiuwen_studio.ops.config.Settings 同名) | 111 | # 十、业务关系型数据库(与 openjiuwen_studio.ops.config.Settings 同名) |
| 112 | # ----------------------------------------------------------------------------- | 112 | # ----------------------------------------------------------------------------- |
| 113 | -# DB_TYPE:sqlite 或 mysql。 | 113 | +# DB_TYPE:sqlite、mysql、gaussdb 或 opengauss。 |
| 114 | DB_TYPE=mysql | 114 | DB_TYPE=mysql |
| 115 | 115 | ||
| 116 | 116 | ||
| @@ -139,6 +139,23 @@ OPS_DB_NAME=openjiuwen_ops | |||
| 139 | AGENT_DB_NAME=openjiuwen_runtime | 139 | AGENT_DB_NAME=openjiuwen_runtime |
| 140 | 140 | ||
| 141 | 141 | ||
| 142 | +# ----------------------------------------------------------------------------- | ||
| 143 | +# 十二点一、GaussDB/openGauss(仅当 DB_TYPE=gaussdb 或 opengauss) | ||
| 144 | +# ----------------------------------------------------------------------------- | ||
| 145 | +# 若独立启动 IR 执行服务且使用 GaussDB/openGauss,请额外安装 async-gaussdb>=0.30.4 | ||
| 146 | +# 或使用带 gaussdb optional dependency 的安装方式。 | ||
| 147 | +# 复用 DB_HOST / DB_PORT / DB_USER / DB_PASSWORD / OPS_DB_NAME / AGENT_DB_NAME。 | ||
| 148 | +# 默认端口通常为 5432。 | ||
| 149 | +# 验证 POC 时,建议将以下值替换为真实库信息: | ||
| 150 | +# DB_TYPE=gaussdb # 或 opengauss | ||
| 151 | +# DB_HOST=127.0.0.1 | ||
| 152 | +# DB_PORT=5432 | ||
| 153 | +# DB_USER=gaussdb_user | ||
| 154 | +# DB_PASSWORD=gaussdb_password | ||
| 155 | +# OPS_DB_NAME=gaussdb_runtime | ||
| 156 | +# AGENT_DB_NAME=gaussdb_runtime | ||
| 157 | + | ||
| 158 | + | ||
| 142 | # ----------------------------------------------------------------------------- | 159 | # ----------------------------------------------------------------------------- |
| 143 | # 十三、KV 存储(记忆引擎中间状态等) | 160 | # 十三、KV 存储(记忆引擎中间状态等) |
| 144 | # ----------------------------------------------------------------------------- | 161 | # ----------------------------------------------------------------------------- |
| @@ -22,6 +22,11 @@ dependencies = [ | |||
| 22 | "python-dotenv>=1.2.1", | 22 | "python-dotenv>=1.2.1", |
| 23 | ] | 23 | ] |
| 24 | 24 | ||
| 25 | +[project.optional-dependencies] | ||
| 26 | +gaussdb = [ | ||
| 27 | + "async-gaussdb>=0.30.4", | ||
| 28 | +] | ||
| 29 | + | ||
| 25 | [tool.uv.sources] | 30 | [tool.uv.sources] |
| 26 | openjiuwen-runtime-service = { path = "../../service", editable = true } | 31 | openjiuwen-runtime-service = { path = "../../service", editable = true } |
| 27 | 32 | ||
| @@ -0,0 +1,26 @@ | |||
| 1 | +# coding: utf-8 | ||
| 2 | +# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | ||
| 3 | + | ||
| 4 | +"""IR 内部兼容别名:实际实现位于 foundation.db.dialects.gaussdb_asyncgaussdb。 | ||
| 5 | + | ||
| 6 | +保留本模块是为了 IR 作为独立部署单元时的包内 import 路径稳定 | ||
| 7 | +(`from .gaussdb_sqlalchemy_dialect import ensure_gaussdb_dialect_registered`), | ||
| 8 | +同时避免与 foundation 维护两份几乎相同的方言实现造成代码漂移。 | ||
| 9 | +""" | ||
| 10 | +from __future__ import annotations | ||
| 11 | + | ||
| 12 | +from openjiuwen_runtime.foundation.db.dialects.gaussdb_asyncgaussdb import ( | ||
| 13 | + AsyncAdapt_async_gaussdb_dbapi, | ||
| 14 | + PGDialect_async_gaussdb, | ||
| 15 | + dialect, | ||
| 16 | + ensure_async_gaussdb_installed, | ||
| 17 | + ensure_gaussdb_dialect_registered, | ||
| 18 | +) | ||
| 19 | + | ||
| 20 | +__all__ = [ | ||
| 21 | + "AsyncAdapt_async_gaussdb_dbapi", | ||
| 22 | + "PGDialect_async_gaussdb", | ||
| 23 | + "dialect", | ||
| 24 | + "ensure_async_gaussdb_installed", | ||
| 25 | + "ensure_gaussdb_dialect_registered", | ||
| 26 | +] | ||
| @@ -1,12 +1,12 @@ | |||
| 1 | # coding: utf-8 | 1 | # coding: utf-8 |
| 2 | # Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | 2 | # Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved |
| 3 | 3 | ||
| 4 | -# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved. | ||
| 5 | - | ||
| 6 | """LongTermMemory 单例:向量存储、Embedding、作用域配置,从进程环境变量读取。 | 4 | """LongTermMemory 单例:向量存储、Embedding、作用域配置,从进程环境变量读取。 |
| 7 | 5 | ||
| 8 | 关系型库连接与 openjiuwen_studio.ops.config.Settings 一致:DB_TYPE、DB_HOST、DB_PORT、DB_USER、DB_PASSWORD, | 6 | 关系型库连接与 openjiuwen_studio.ops.config.Settings 一致:DB_TYPE、DB_HOST、DB_PORT、DB_USER、DB_PASSWORD, |
| 9 | 库名使用 AGENT_DB_NAME(MySQL 连接串中的 database 段即该库名)。表名由 ORM/迁移在库内创建,不会出现在 URL 里。 | 7 | 库名使用 AGENT_DB_NAME(MySQL 连接串中的 database 段即该库名)。表名由 ORM/迁移在库内创建,不会出现在 URL 里。 |
| 8 | +当 DB_TYPE 为 gaussdb/opengauss 时,会先生成同步 DSN,再切换为自定义 SQLAlchemy 方言 | ||
| 9 | +gaussdb+async_gaussdb://...,底层驱动使用 async-gaussdb。 | ||
| 10 | """ | 10 | """ |
| 11 | 11 | ||
| 12 | from __future__ import annotations | 12 | from __future__ import annotations |
| @@ -51,6 +51,13 @@ def get_database_url() -> str: | |||
| 51 | f"mysql+pymysql://{user}:{password}@" | 51 | f"mysql+pymysql://{user}:{password}@" |
| 52 | f"{host}:{port}/{database}?charset=utf8mb4" | 52 | f"{host}:{port}/{database}?charset=utf8mb4" |
| 53 | ) | 53 | ) |
| 54 | + if db_type in {"gaussdb", "opengauss"}: | ||
| 55 | + user = quote(get_env("DB_USER", "root"), safe="") | ||
| 56 | + password = quote(get_env("DB_PASSWORD", ""), safe="") | ||
| 57 | + host = get_env("DB_HOST", "localhost") | ||
| 58 | + port = get_int_env("DB_PORT", 5432) | ||
| 59 | + database = get_env("AGENT_DB_NAME", "openjiuwen_agent") | ||
| 60 | + return f"gaussdb://{user}:{password}@{host}:{port}/{database}" | ||
| 54 | if db_type == "sqlite": | 61 | if db_type == "sqlite": |
| 55 | db_path = Path(get_env("SQLITE_DB_PATH", "data/databases")) | 62 | db_path = Path(get_env("SQLITE_DB_PATH", "data/databases")) |
| 56 | db_path.mkdir(parents=True, exist_ok=True) | 63 | db_path.mkdir(parents=True, exist_ok=True) |
| @@ -61,6 +68,12 @@ def get_database_url() -> str: | |||
| 61 | def get_async_database_url(sync_db_url: str) -> str: | 68 | def get_async_database_url(sync_db_url: str) -> str: |
| 62 | if "mysql+pymysql" in sync_db_url: | 69 | if "mysql+pymysql" in sync_db_url: |
| 63 | return sync_db_url.replace("pymysql", "aiomysql") | 70 | return sync_db_url.replace("pymysql", "aiomysql") |
| 71 | + if sync_db_url.startswith("gaussdb://"): | ||
| 72 | + from .gaussdb_sqlalchemy_dialect import ensure_async_gaussdb_installed, ensure_gaussdb_dialect_registered | ||
| 73 | + | ||
| 74 | + ensure_async_gaussdb_installed() | ||
| 75 | + ensure_gaussdb_dialect_registered() | ||
| 76 | + return sync_db_url.replace("gaussdb://", "gaussdb+async_gaussdb://", 1) | ||
| 64 | if sync_db_url.startswith("sqlite:///"): | 77 | if sync_db_url.startswith("sqlite:///"): |
| 65 | return sync_db_url.replace("sqlite:///", "sqlite+aiosqlite:///") | 78 | return sync_db_url.replace("sqlite:///", "sqlite+aiosqlite:///") |
| 66 | raise ValueError(f"Unsupported database URL for async engine: {sync_db_url}") | 79 | raise ValueError(f"Unsupported database URL for async engine: {sync_db_url}") |
| @@ -1,8 +1,6 @@ | |||
| 1 | # coding: utf-8 | 1 | # coding: utf-8 |
| 2 | # Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | 2 | # Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved |
| 3 | 3 | ||
| 4 | -# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved. | ||
| 5 | - | ||
| 6 | """启动前环境准备:不为进程加载 .env 文件,仅根据类型键写入默认并校验必填项。 | 4 | """启动前环境准备:不为进程加载 .env 文件,仅根据类型键写入默认并校验必填项。 |
| 7 | 5 | ||
| 8 | 外部部署请自行 export 或注入环境变量;仓库内 .env 仅作样例参考。 | 6 | 外部部署请自行 export 或注入环境变量;仓库内 .env 仅作样例参考。 |
| @@ -55,6 +53,8 @@ def apply_runtime_type_and_optional_defaults() -> None: | |||
| 55 | db_type = _strip("DB_TYPE", "mysql").lower() | 53 | db_type = _strip("DB_TYPE", "mysql").lower() |
| 56 | if db_type == "mysql": | 54 | if db_type == "mysql": |
| 57 | _setdefault_env("DB_PORT", "3306") | 55 | _setdefault_env("DB_PORT", "3306") |
| 56 | + elif db_type in {"gaussdb", "opengauss"}: | ||
| 57 | + _setdefault_env("DB_PORT", "5432") | ||
| 58 | elif db_type == "sqlite": | 58 | elif db_type == "sqlite": |
| 59 | _setdefault_env("SQLITE_DB_PATH", "data/databases") | 59 | _setdefault_env("SQLITE_DB_PATH", "data/databases") |
| 60 | _setdefault_env("OPS_SQLITE_DB", "ops.db") | 60 | _setdefault_env("OPS_SQLITE_DB", "ops.db") |
| @@ -113,7 +113,7 @@ def _collect_code_sandbox_missing(missing: list[str]) -> None: | |||
| 113 | def _collect_db_missing(missing: list[str]) -> None: | 113 | def _collect_db_missing(missing: list[str]) -> None: |
| 114 | """与 openjiuwen_studio.ops.config.Settings 一致:DB 连接项、OPS_DB_NAME、AGENT_DB_NAME、SQLite 路径与文件名。""" | 114 | """与 openjiuwen_studio.ops.config.Settings 一致:DB 连接项、OPS_DB_NAME、AGENT_DB_NAME、SQLite 路径与文件名。""" |
| 115 | db_type = _strip("DB_TYPE").lower() | 115 | db_type = _strip("DB_TYPE").lower() |
| 116 | - if db_type == "mysql": | 116 | + if db_type in {"mysql", "gaussdb", "opengauss"}: |
| 117 | for key in ("DB_HOST", "DB_PORT", "DB_USER", "OPS_DB_NAME", "AGENT_DB_NAME"): | 117 | for key in ("DB_HOST", "DB_PORT", "DB_USER", "OPS_DB_NAME", "AGENT_DB_NAME"): |
| 118 | if not _strip(key): | 118 | if not _strip(key): |
| 119 | missing.append(key) | 119 | missing.append(key) |
| @@ -124,7 +124,7 @@ def _collect_db_missing(missing: list[str]) -> None: | |||
| 124 | if not _strip(key): | 124 | if not _strip(key): |
| 125 | missing.append(key) | 125 | missing.append(key) |
| 126 | else: | 126 | else: |
| 127 | - missing.append(f"DB_TYPE 非法: {db_type!r},应为 mysql 或 sqlite") | 127 | + missing.append(f"DB_TYPE 非法: {db_type!r},应为 mysql、sqlite、gaussdb 或 opengauss") |
| 128 | 128 | ||
| 129 | 129 | ||
| 130 | def _collect_kv_missing(missing: list[str]) -> None: | 130 | def _collect_kv_missing(missing: list[str]) -> None: |
| @@ -0,0 +1,47 @@ | |||
| 1 | +# coding: utf-8 | ||
| 2 | +# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | ||
| 3 | + | ||
| 4 | +import os | ||
| 5 | +import unittest | ||
| 6 | +from unittest import mock | ||
| 7 | + | ||
| 8 | +from runtime_support import runtime_env_prepare | ||
| 9 | + | ||
| 10 | + | ||
| 11 | +class TestRuntimeEnvPrepareGaussDB(unittest.TestCase): | ||
| 12 | + def setUp(self): | ||
| 13 | + self._original_env = os.environ.copy() | ||
| 14 | + | ||
| 15 | + def tearDown(self): | ||
| 16 | + os.environ.clear() | ||
| 17 | + os.environ.update(self._original_env) | ||
| 18 | + | ||
| 19 | + def test_apply_defaults_sets_5432_for_gaussdb(self): | ||
| 20 | + with mock.patch.dict(os.environ, {"DB_TYPE": "gaussdb"}, clear=True): | ||
| 21 | + runtime_env_prepare.apply_runtime_type_and_optional_defaults() | ||
| 22 | + self.assertEqual(os.environ["DB_PORT"], "5432") | ||
| 23 | + | ||
| 24 | + def test_collect_db_missing_accepts_gaussdb_with_required_fields(self): | ||
| 25 | + env = { | ||
| 26 | + "DB_TYPE": "gaussdb", | ||
| 27 | + "DB_HOST": "127.0.0.1", | ||
| 28 | + "DB_PORT": "5432", | ||
| 29 | + "DB_USER": "gauss_user", | ||
| 30 | + "DB_PASSWORD": "secret", | ||
| 31 | + "OPS_DB_NAME": "ops_db", | ||
| 32 | + "AGENT_DB_NAME": "agent_db", | ||
| 33 | + } | ||
| 34 | + with mock.patch.dict(os.environ, env, clear=True): | ||
| 35 | + missing = [] | ||
| 36 | + runtime_env_prepare._collect_db_missing(missing) | ||
| 37 | + self.assertEqual(missing, []) | ||
| 38 | + | ||
| 39 | + def test_collect_db_missing_rejects_invalid_db_type(self): | ||
| 40 | + with mock.patch.dict(os.environ, {"DB_TYPE": "postgres"}, clear=True): | ||
| 41 | + missing = [] | ||
| 42 | + runtime_env_prepare._collect_db_missing(missing) | ||
| 43 | + | ||
| 44 | + self.assertEqual(len(missing), 1) | ||
| 45 | + self.assertIn("DB_TYPE 非法", missing[0]) | ||
| 46 | + self.assertIn("gaussdb", missing[0]) | ||
| 47 | + self.assertIn("opengauss", missing[0]) | ||
| @@ -3,6 +3,9 @@ | |||
| 3 | # Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | 3 | # Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved |
| 4 | 4 | ||
| 5 | # -*- coding: UTF-8 -*- | 5 | # -*- coding: UTF-8 -*- |
| 6 | +# ruff: noqa: E402 | ||
| 7 | +# 说明:本模块需要在导入业务包之前完成若干环境变量的注入(DB_TYPE、沙箱 URL、 | ||
| 8 | +# 工作流超时等),顺序敏感。故整体豁免 E402 模块级导入顺序检查。 | ||
| 6 | """ | 9 | """ |
| 7 | Lowcode Agent App | 10 | Lowcode Agent App |
| 8 | 11 | ||
| @@ -37,9 +40,18 @@ import os | |||
| 37 | import sys | 40 | import sys |
| 38 | from typing import AsyncIterator, Tuple | 41 | from typing import AsyncIterator, Tuple |
| 39 | 42 | ||
| 40 | -# 设置 DB_TYPE=none,避免数据库配置检查 | 43 | +# 运行时数据库类型:优先保留外部注入(例如 gaussdb),缺失/非法值回退 sqlite。 |
| 41 | -# 注意:使用直接赋值而不是 setdefault,确保覆盖从 runtime 服务继承的 DB_TYPE | 44 | +_VALID_DB_TYPES = {"mysql", "sqlite", "gaussdb", "opengauss"} |
| 42 | -os.environ["DB_TYPE"] = "none" | 45 | +_runtime_db_type = (os.getenv("DB_TYPE") or "").strip().lower() |
| 46 | +if _runtime_db_type not in _VALID_DB_TYPES: | ||
| 47 | + _runtime_db_type = "sqlite" | ||
| 48 | +os.environ["DB_TYPE"] = _runtime_db_type | ||
| 49 | + | ||
| 50 | +# lowcode 运行链路会间接导入 openjiuwen_studio,当前其数据库层仅支持 mysql/sqlite/none。 | ||
| 51 | +# 当 runtime 以 gaussdb/opengauss 启动时,这里切换 studio 侧 DB_TYPE,避免因不支持而阻塞低码启动。 | ||
| 52 | +_studio_db_type = (os.getenv("LOWCODE_STUDIO_DB_TYPE") or "sqlite").strip().lower() | ||
| 53 | +if _studio_db_type not in {"mysql", "sqlite", "none"}: | ||
| 54 | + _studio_db_type = "sqlite" | ||
| 43 | 55 | ||
| 44 | 56 | ||
| 45 | def _parse_userdata_env_vars(): | 57 | def _parse_userdata_env_vars(): |
| @@ -51,7 +63,7 @@ def _parse_userdata_env_vars(): | |||
| 51 | 2. userdata.env_vars | 63 | 2. userdata.env_vars |
| 52 | 3. 默认值 (最低) | 64 | 3. 默认值 (最低) |
| 53 | 65 | ||
| 54 | - 注意:DB_TYPE 环境变量由 lowcode_agent_runner 控制,不应该被 userdata 覆盖 | 66 | + 注意:DB_TYPE 环境变量由启动器或系统环境控制,不应该被 userdata 覆盖 |
| 55 | """ | 67 | """ |
| 56 | userdata_str = os.getenv("RUNTIME_USERDATA", "") | 68 | userdata_str = os.getenv("RUNTIME_USERDATA", "") |
| 57 | env_vars = {} | 69 | env_vars = {} |
| @@ -90,7 +102,6 @@ from openjiuwen.core.runner import Runner | |||
| 90 | from openjiuwen.core.single_agent.legacy import WorkflowAgentConfig as LegacyWorkflowAgentConfig | 102 | from openjiuwen.core.single_agent.legacy import WorkflowAgentConfig as LegacyWorkflowAgentConfig |
| 91 | 103 | ||
| 92 | from openjiuwen_runtime.examples.lowcode_agent.agui_converter import ( | 104 | from openjiuwen_runtime.examples.lowcode_agent.agui_converter import ( |
| 93 | - agui_append_text_and_finish_events, | ||
| 94 | agui_assistant_text_as_answer_events, | 105 | agui_assistant_text_as_answer_events, |
| 95 | agui_error_events, | 106 | agui_error_events, |
| 96 | agui_trace_context, | 107 | agui_trace_context, |
| @@ -104,6 +115,9 @@ from openjiuwen_runtime.examples.lowcode_agent.workflow_registration import ( | |||
| 104 | ) | 115 | ) |
| 105 | from openjiuwen_runtime.service.app.agent_app import AgentApp | 116 | from openjiuwen_runtime.service.app.agent_app import AgentApp |
| 106 | 117 | ||
| 118 | +if _runtime_db_type in {"gaussdb", "opengauss"}: | ||
| 119 | + os.environ["DB_TYPE"] = _studio_db_type | ||
| 120 | + | ||
| 107 | from openjiuwen_studio.core.executor.component.code_runner.remote import remote_code_runner | 121 | from openjiuwen_studio.core.executor.component.code_runner.remote import remote_code_runner |
| 108 | 122 | ||
| 109 | remote_code_runner.code_sandbox_url = _CODE_SANDBOX_URL | 123 | remote_code_runner.code_sandbox_url = _CODE_SANDBOX_URL |
| @@ -0,0 +1,165 @@ | |||
| 1 | +# coding: utf-8 | ||
| 2 | +# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | ||
| 3 | + | ||
| 4 | +import subprocess | ||
| 5 | +import sys | ||
| 6 | +import textwrap | ||
| 7 | +import unittest | ||
| 8 | +from pathlib import Path | ||
| 9 | + | ||
| 10 | + | ||
| 11 | +def _run_runner_and_capture_db_type(runtime_db_type: str, studio_db_type: str) -> dict[str, str]: | ||
| 12 | + repo_root = Path(__file__).resolve().parents[4] | ||
| 13 | + runner_path = ( | ||
| 14 | + repo_root | ||
| 15 | + / "applications" | ||
| 16 | + / "lowcode_agent" | ||
| 17 | + / "openjiuwen_runtime" | ||
| 18 | + / "examples" | ||
| 19 | + / "lowcode_agent" | ||
| 20 | + / "lowcode_agent_runner.py" | ||
| 21 | + ) | ||
| 22 | + | ||
| 23 | + script = textwrap.dedent( | ||
| 24 | + f""" | ||
| 25 | + import os | ||
| 26 | + import runpy | ||
| 27 | + import sys | ||
| 28 | + import types | ||
| 29 | + | ||
| 30 | + class DummyAgentApp: | ||
| 31 | + def __init__(self, *args, **kwargs): | ||
| 32 | + self.args = args | ||
| 33 | + self.kwargs = kwargs | ||
| 34 | + | ||
| 35 | + def __getattr__(self, _name): | ||
| 36 | + def _decorator(func): | ||
| 37 | + return func | ||
| 38 | + return _decorator | ||
| 39 | + | ||
| 40 | + def install_package(name): | ||
| 41 | + mod = types.ModuleType(name) | ||
| 42 | + mod.__path__ = [] | ||
| 43 | + sys.modules[name] = mod | ||
| 44 | + return mod | ||
| 45 | + | ||
| 46 | + def install_module(name, **attrs): | ||
| 47 | + mod = types.ModuleType(name) | ||
| 48 | + for k, v in attrs.items(): | ||
| 49 | + setattr(mod, k, v) | ||
| 50 | + sys.modules[name] = mod | ||
| 51 | + return mod | ||
| 52 | + | ||
| 53 | + install_package("openjiuwen") | ||
| 54 | + install_package("openjiuwen.core") | ||
| 55 | + install_package("openjiuwen.core.application") | ||
| 56 | + install_module( | ||
| 57 | + "openjiuwen.core.application.llm_agent", | ||
| 58 | + LLMAgent=type("LLMAgent", (), {{}}), | ||
| 59 | + ReActAgentConfig=type("ReActAgentConfig", (), {{}}), | ||
| 60 | + ) | ||
| 61 | + install_module( | ||
| 62 | + "openjiuwen.core.application.workflow_agent", | ||
| 63 | + WorkflowAgent=type("WorkflowAgent", (), {{}}), | ||
| 64 | + ) | ||
| 65 | + install_module("openjiuwen.core.runner", Runner=type("Runner", (), {{}})) | ||
| 66 | + install_package("openjiuwen.core.single_agent") | ||
| 67 | + install_module( | ||
| 68 | + "openjiuwen.core.single_agent.legacy", | ||
| 69 | + WorkflowAgentConfig=type("WorkflowAgentConfig", (), {{}}), | ||
| 70 | + ) | ||
| 71 | + | ||
| 72 | + install_package("openjiuwen_runtime") | ||
| 73 | + install_package("openjiuwen_runtime.examples") | ||
| 74 | + install_package("openjiuwen_runtime.examples.lowcode_agent") | ||
| 75 | + install_module( | ||
| 76 | + "openjiuwen_runtime.examples.lowcode_agent.agui_converter", | ||
| 77 | + agui_append_text_and_finish_events=lambda *a, **k: [], | ||
| 78 | + agui_assistant_text_as_answer_events=lambda *a, **k: [], | ||
| 79 | + agui_error_events=lambda *a, **k: [], | ||
| 80 | + agui_trace_context=lambda *a, **k: [], | ||
| 81 | + convert_chunk_to_agui_events=lambda *a, **k: [], | ||
| 82 | + finalize_agui_stream=lambda *a, **k: [], | ||
| 83 | + flush_buffered_agui_text_events=lambda *a, **k: [], | ||
| 84 | + merge_agui_events_for_stream=lambda *a, **k: [], | ||
| 85 | + ) | ||
| 86 | + install_module( | ||
| 87 | + "openjiuwen_runtime.examples.lowcode_agent.workflow_registration", | ||
| 88 | + normalize_workflow_providers_for_agent=lambda *a, **k: None, | ||
| 89 | + ) | ||
| 90 | + | ||
| 91 | + install_package("openjiuwen_runtime.service") | ||
| 92 | + install_package("openjiuwen_runtime.service.app") | ||
| 93 | + install_module( | ||
| 94 | + "openjiuwen_runtime.service.app.agent_app", | ||
| 95 | + AgentApp=DummyAgentApp, | ||
| 96 | + ) | ||
| 97 | + | ||
| 98 | + install_package("openjiuwen_studio") | ||
| 99 | + install_package("openjiuwen_studio.core") | ||
| 100 | + install_package("openjiuwen_studio.core.executor") | ||
| 101 | + install_package("openjiuwen_studio.core.executor.component") | ||
| 102 | + install_package("openjiuwen_studio.core.executor.component.code_runner") | ||
| 103 | + install_module( | ||
| 104 | + "openjiuwen_studio.core.executor.component.code_runner.remote", | ||
| 105 | + remote_code_runner=types.SimpleNamespace(code_sandbox_url=""), | ||
| 106 | + ) | ||
| 107 | + | ||
| 108 | + lowcode_pkg = install_package("openjiuwen_studio.lowcode") | ||
| 109 | + lowcode_pkg.AgentCompiler = type("AgentCompiler", (), {{}}) | ||
| 110 | + install_module( | ||
| 111 | + "openjiuwen_studio.lowcode.config_adapter", | ||
| 112 | + ConfigAdapter=type("ConfigAdapter", (), {{}}), | ||
| 113 | + ) | ||
| 114 | + install_module( | ||
| 115 | + "openjiuwen_studio.lowcode.runtime_workflow_runner", | ||
| 116 | + RuntimeWorkflowRunner=type("RuntimeWorkflowRunner", (), {{}}), | ||
| 117 | + ) | ||
| 118 | + | ||
| 119 | + os.environ["DB_TYPE"] = {runtime_db_type!r} | ||
| 120 | + os.environ["LOWCODE_STUDIO_DB_TYPE"] = {studio_db_type!r} | ||
| 121 | + | ||
| 122 | + g = runpy.run_path({str(runner_path)!r}, run_name="__lowcode_test__") | ||
| 123 | + | ||
| 124 | + print("RUNTIME_DB_TYPE=" + g.get("_runtime_db_type", "")) | ||
| 125 | + print("STUDIO_DB_TYPE=" + g.get("_studio_db_type", "")) | ||
| 126 | + print("FINAL_DB_TYPE=" + os.environ.get("DB_TYPE", "")) | ||
| 127 | + """ | ||
| 128 | + ) | ||
| 129 | + | ||
| 130 | + result = subprocess.run( | ||
| 131 | + [sys.executable, "-c", script], | ||
| 132 | + text=True, | ||
| 133 | + capture_output=True, | ||
| 134 | + ) | ||
| 135 | + | ||
| 136 | + if result.returncode != 0: | ||
| 137 | + raise AssertionError( | ||
| 138 | + "lowcode runner subprocess failed\n" | ||
| 139 | + f"returncode={result.returncode}\n" | ||
| 140 | + f"stdout:\n{result.stdout}\n" | ||
| 141 | + f"stderr:\n{result.stderr}" | ||
| 142 | + ) | ||
| 143 | + | ||
| 144 | + output = {} | ||
| 145 | + for line in result.stdout.splitlines(): | ||
| 146 | + if "=" in line: | ||
| 147 | + key, value = line.split("=", 1) | ||
| 148 | + output[key.strip()] = value.strip() | ||
| 149 | + return output | ||
| 150 | + | ||
| 151 | + | ||
| 152 | +class TestLowcodeRunnerGaussdbBridge(unittest.TestCase): | ||
| 153 | + def test_gaussdb_runtime_overrides_to_studio_supported_db_type(self): | ||
| 154 | + data = _run_runner_and_capture_db_type("gaussdb", "mysql") | ||
| 155 | + | ||
| 156 | + self.assertEqual(data["RUNTIME_DB_TYPE"], "gaussdb") | ||
| 157 | + self.assertEqual(data["STUDIO_DB_TYPE"], "mysql") | ||
| 158 | + self.assertEqual(data["FINAL_DB_TYPE"], "mysql") | ||
| 159 | + | ||
| 160 | + def test_non_gauss_runtime_keeps_runtime_db_type(self): | ||
| 161 | + data = _run_runner_and_capture_db_type("mysql", "none") | ||
| 162 | + | ||
| 163 | + self.assertEqual(data["RUNTIME_DB_TYPE"], "mysql") | ||
| 164 | + self.assertEqual(data["STUDIO_DB_TYPE"], "none") | ||
| 165 | + self.assertEqual(data["FINAL_DB_TYPE"], "mysql") | ||
| @@ -27,9 +27,16 @@ cp .env.example .env | |||
| 27 | 27 | ||
| 28 | Edit `server/.env` as needed. Important fields include: | 28 | Edit `server/.env` as needed. Important fields include: |
| 29 | 29 | ||
| 30 | -- `DB_TYPE`: `mysql` or `sqlite` | 30 | +- `DB_TYPE`: `mysql`, `sqlite`, `gaussdb`, or `opengauss` |
| 31 | - Other runtime settings (e.g. IP address, port) | 31 | - Other runtime settings (e.g. IP address, port) |
| 32 | 32 | ||
| 33 | +Additional notes: | ||
| 34 | + | ||
| 35 | +- The default install only includes dependencies for `sqlite` and `mysql`. | ||
| 36 | +- When `DB_TYPE=gaussdb` or `DB_TYPE=opengauss`, Runtime also needs `async-gaussdb`. | ||
| 37 | +- If you use `scripts/run-server.sh` or `scripts/run-server.ps1`, the script reads `DB_TYPE` from `server/.env` and installs `foundation[gaussdb]` automatically. | ||
| 38 | +- For a manual setup, run `uv pip install -e "./foundation[gaussdb]"`. | ||
| 39 | + | ||
| 33 | ### 3) Run the startup script | 40 | ### 3) Run the startup script |
| 34 | 41 | ||
| 35 | From the **repository root** (`agent-runtime/`): | 42 | From the **repository root** (`agent-runtime/`): |
| @@ -8,37 +8,44 @@ All variables in this file are part of how Runtime starts and runs. Follow confi | |||
| 8 | 8 | ||
| 9 | ### DB_TYPE (database engine) | 9 | ### DB_TYPE (database engine) |
| 10 | 10 | ||
| 11 | -**Purpose:** Selects the database backend for Runtime. Only **`mysql`** and **`sqlite`** are supported. Do not use other values. Default is **`sqlite`**. | 11 | +**Purpose:** Selects the database backend for Runtime. Supported values are **`mysql`**, **`sqlite`**, **`gaussdb`**, and **`opengauss`**. Default is **`sqlite`**. |
| 12 | 12 | ||
| 13 | -**Guidance:** Set to `sqlite` or `mysql` to match your deployment. For MySQL, ensure the database server is running and that the MySQL-specific variables below match your environment. | 13 | +**Guidance:** Set the value to match your deployment. For `mysql`, ensure the MySQL server is reachable and that the relational-database settings below match your environment. For `gaussdb` / `opengauss`, ensure the database is reachable and that the optional driver dependency is installed. |
| 14 | 14 | ||
| 15 | -## MySQL-only settings | 15 | +**Optional dependency note:** |
| 16 | 16 | ||
| 17 | -### DB_HOST (MySQL host) | 17 | +- The default install only includes dependencies for `sqlite` and `mysql`. |
| 18 | +- When `DB_TYPE=gaussdb` or `DB_TYPE=opengauss`, Runtime additionally needs `async-gaussdb`. | ||
| 19 | +- If you use the repository startup scripts, they detect `DB_TYPE` from `server/.env` and install `foundation[gaussdb]` automatically. | ||
| 20 | +- For a manual setup, run `uv pip install -e "./foundation[gaussdb]"`. | ||
| 18 | 21 | ||
| 19 | -**Purpose:** Hostname or IP of the MySQL server Runtime connects to. | 22 | +## MySQL / GaussDB / openGauss settings |
| 23 | + | ||
| 24 | +### DB_HOST (database host) | ||
| 25 | + | ||
| 26 | +**Purpose:** Hostname or IP of the database server Runtime connects to. | ||
| 20 | 27 | ||
| 21 | **Guidance:** Use the remote host’s public IP if MySQL is remote; use `127.0.0.1` or `localhost` if MySQL is on the same machine as Runtime. Ensure the host is reachable from the Runtime host (firewall / security groups / ports). | 28 | **Guidance:** Use the remote host’s public IP if MySQL is remote; use `127.0.0.1` or `localhost` if MySQL is on the same machine as Runtime. Ensure the host is reachable from the Runtime host (firewall / security groups / ports). |
| 22 | 29 | ||
| 23 | -### DB_PORT (MySQL port) | 30 | +### DB_PORT (database port) |
| 24 | 31 | ||
| 25 | -**Purpose:** MySQL listen port. | 32 | +**Purpose:** Database listen port. |
| 26 | 33 | ||
| 27 | -**Guidance:** Set to your MySQL server’s actual port. If you use **SQLite**, this value is ignored but you may comment it out to avoid confusion. | 34 | +**Guidance:** Set this to the actual database port. Typical defaults are **3306** for MySQL and **5432** for GaussDB / openGauss. If you use **SQLite**, this value is ignored. |
| 28 | 35 | ||
| 29 | -### DB_USER (MySQL username) | 36 | +### DB_USER (database username) |
| 30 | 37 | ||
| 31 | -**Purpose:** MySQL user Runtime uses. The account needs sufficient privileges (read/write, DDL such as table creation) for Runtime to operate the schema. | 38 | +**Purpose:** Database user Runtime uses. The account needs sufficient privileges (read/write, DDL such as table creation) for Runtime to operate the schema. |
| 32 | 39 | ||
| 33 | **Guidance:** Avoid `root` in production. Prefer a dedicated account (e.g. `jiuwen_runtime_user`) with least privilege on the database named in `DB_NAME` only. | 40 | **Guidance:** Avoid `root` in production. Prefer a dedicated account (e.g. `jiuwen_runtime_user`) with least privilege on the database named in `DB_NAME` only. |
| 34 | 41 | ||
| 35 | -### DB_PASSWORD (MySQL password) | 42 | +### DB_PASSWORD (database password) |
| 36 | 43 | ||
| 37 | **Purpose:** Password for `DB_USER`. | 44 | **Purpose:** Password for `DB_USER`. |
| 38 | 45 | ||
| 39 | **Guidance:** Always use a strong password in production (mixed case, digits, symbols, length ≥ 8). Avoid trivial passwords. Store and rotate secrets securely. | 46 | **Guidance:** Always use a strong password in production (mixed case, digits, symbols, length ≥ 8). Avoid trivial passwords. Store and rotate secrets securely. |
| 40 | 47 | ||
| 41 | -### DB_NAME (MySQL database name) | 48 | +### DB_NAME (database name) |
| 42 | 49 | ||
| 43 | **Purpose:** Database name Runtime uses. On startup, Runtime validates that the database exists; it **does not** auto-create the database — create it yourself in advance. | 50 | **Purpose:** Database name Runtime uses. On startup, Runtime validates that the database exists; it **does not** auto-create the database — create it yourself in advance. |
| 44 | 51 | ||
| @@ -26,9 +26,15 @@ cp .env.example .env | |||
| 26 | ``` | 26 | ``` |
| 27 | 27 | ||
| 28 | 按需修改 `.env` 中的配置项,重点包括: | 28 | 按需修改 `.env` 中的配置项,重点包括: |
| 29 | -* `DB_TYPE`:支持 `mysql` 和 `sqlite` | 29 | +* `DB_TYPE`:支持 `mysql`、`sqlite`、`gaussdb` 和 `opengauss` |
| 30 | * 其他运行时相关参数(如ip地址、端口号等) | 30 | * 其他运行时相关参数(如ip地址、端口号等) |
| 31 | 31 | ||
| 32 | +补充说明: | ||
| 33 | +* 默认安装仅包含 `sqlite` / `mysql` 所需依赖; | ||
| 34 | +* 当 `DB_TYPE=gaussdb` 或 `DB_TYPE=opengauss` 时,还需要 `async-gaussdb`; | ||
| 35 | +* 使用仓库自带 `scripts/run-server.sh` 或 `scripts/run-server.ps1` 启动时,脚本会根据 `server/.env` 中的 `DB_TYPE` 自动安装 `foundation[gaussdb]`; | ||
| 36 | +* 若手工安装,请执行 `uv pip install -e "./foundation[gaussdb]"`。 | ||
| 37 | + | ||
| 32 | ### 3) 执行启动脚本: | 38 | ### 3) 执行启动脚本: |
| 33 | 39 | ||
| 34 | * Linux/MacOS: | 40 | * Linux/MacOS: |
| @@ -8,37 +8,44 @@ | |||
| 8 | 8 | ||
| 9 | ## DB_TYPE(数据库类型) | 9 | ## DB_TYPE(数据库类型) |
| 10 | 10 | ||
| 11 | -**作用**:定义 Runtime 服务所使用的数据库类型,仅支持 mysql、sqlite 两个可选值,不可填写其他类型,默认配置为 sqlite。 | 11 | +**作用**:定义 Runtime 服务所使用的数据库类型,支持 mysql、sqlite、gaussdb、opengauss 四个可选值,默认配置为 sqlite。 |
| 12 | 12 | ||
| 13 | -**修改建议**:根据实际部署的数据库类型修改,若使用 SQLite 数据库,将值改为 sqlite;若使用 MySQL 数据库,将值改为 mysql, 并确保确保数据库服务已正常启动,且与后续数据库配置参数匹配。 | 13 | +**修改建议**:根据实际部署的数据库类型修改,若使用 SQLite 数据库,将值改为 sqlite;若使用 MySQL 数据库,将值改为 mysql;若使用 GaussDB / openGauss,将值改为 gaussdb 或 opengauss,并确保数据库服务已正常启动,且与后续数据库配置参数匹配。 |
| 14 | 14 | ||
| 15 | -## MySQL数据库专用配置 | 15 | +**可选依赖说明**: |
| 16 | 16 | ||
| 17 | -### DB_HOST(MySQL数据库主机地址) | 17 | +- 默认安装仅包含 `sqlite` / `mysql` 所需依赖; |
| 18 | +- 当 `DB_TYPE=gaussdb` 或 `DB_TYPE=opengauss` 时,需要额外安装 `async-gaussdb`; | ||
| 19 | +- 使用仓库自带启动脚本时,脚本会根据 `server/.env` 中的 `DB_TYPE` 自动安装 `foundation[gaussdb]`; | ||
| 20 | +- 若手工安装,请执行 `uv pip install -e "./foundation[gaussdb]"`。 | ||
| 18 | 21 | ||
| 19 | -**作用**:指定MySQL数据库服务所在的主机IP地址或域名,Runtime 服务通过该地址连接数据库。 | 22 | +## MySQL / GaussDB / openGauss 数据库专用配置 |
| 23 | + | ||
| 24 | +### DB_HOST(数据库主机地址) | ||
| 25 | + | ||
| 26 | +**作用**:指定数据库服务所在的主机IP地址或域名,Runtime 服务通过该地址连接数据库。 | ||
| 20 | 27 | ||
| 21 | **修改建议**:若MySQL数据库部署在远程服务器,需将值改为远程数据库的公网IP地址;若数据库与 Runtime 服务部署在同一服务器,可改为 127.0.0.1或localhost。修改后需确保该地址可被 Runtime 服务所在服务器访问,无防火墙、端口限制。 | 28 | **修改建议**:若MySQL数据库部署在远程服务器,需将值改为远程数据库的公网IP地址;若数据库与 Runtime 服务部署在同一服务器,可改为 127.0.0.1或localhost。修改后需确保该地址可被 Runtime 服务所在服务器访问,无防火墙、端口限制。 |
| 22 | 29 | ||
| 23 | -### DB_PORT(MySQL数据库端口) | 30 | +### DB_PORT(数据库端口) |
| 24 | 31 | ||
| 25 | -**作用**:指定MySQL数据库服务的监听端口。 | 32 | +**作用**:指定数据库服务的监听端口。 |
| 26 | 33 | ||
| 27 | -**修改建议**:修改其为MySQL数据库服务的实际监听端口;若使用 SQLite 数据库,该配置项不生效,但也不影响服务运行,但建议注释说明,避免混淆。 | 34 | +**修改建议**:MySQL 默认端口通常为 3306,GaussDB / openGauss 默认端口通常为 5432。若使用 SQLite 数据库,该配置项不生效,但也不影响服务运行,建议保留注释说明以避免混淆。 |
| 28 | 35 | ||
| 29 | -### DB_USER(MySQL数据库登录用户名) | 36 | +### DB_USER(数据库登录用户名) |
| 30 | 37 | ||
| 31 | **作用**:指定连接MySQL数据库的用户名,需具备该数据库的读写、创建表等权限,确保 Runtime 服务能正常操作数据库。 | 38 | **作用**:指定连接MySQL数据库的用户名,需具备该数据库的读写、创建表等权限,确保 Runtime 服务能正常操作数据库。 |
| 32 | 39 | ||
| 33 | **修改建议**:生产环境中不建议使用 root 账号,建议创建专用数据库账号(如 jiuwen_runtime_user),并分配最小必要权限(仅对 DB_NAME 指定的数据库有操作权限),修改该值为创建的专用账号,提升安全性。 | 40 | **修改建议**:生产环境中不建议使用 root 账号,建议创建专用数据库账号(如 jiuwen_runtime_user),并分配最小必要权限(仅对 DB_NAME 指定的数据库有操作权限),修改该值为创建的专用账号,提升安全性。 |
| 34 | 41 | ||
| 35 | -### DB_PASSWORD(MySQL数据库登录密码) | 42 | +### DB_PASSWORD(数据库登录密码) |
| 36 | 43 | ||
| 37 | **作用**:对应 DB_USER 的登录密码,用于验证数据库连接权限,是数据库安全的核心参数。 | 44 | **作用**:对应 DB_USER 的登录密码,用于验证数据库连接权限,是数据库安全的核心参数。 |
| 38 | 45 | ||
| 39 | **修改建议**:生产环境必须修改,建议设置复杂密码(包含大小写字母、数字、特殊符号,长度不小于8位),避免使用简单密码(如 123456、admin);密码修改后需同步告知相关运维人员,妥善保管,避免泄露。 | 46 | **修改建议**:生产环境必须修改,建议设置复杂密码(包含大小写字母、数字、特殊符号,长度不小于8位),避免使用简单密码(如 123456、admin);密码修改后需同步告知相关运维人员,妥善保管,避免泄露。 |
| 40 | 47 | ||
| 41 | -### DB_NAME(MySQL目标数据库名称) | 48 | +### DB_NAME(目标数据库名称) |
| 42 | 49 | ||
| 43 | **作用**:指定 Runtime 服务需要连接、操作的数据库名称,服务启动时会自动校验该数据库是否存在(不会自动创建, 请自行提前创建)。 | 50 | **作用**:指定 Runtime 服务需要连接、操作的数据库名称,服务启动时会自动校验该数据库是否存在(不会自动创建, 请自行提前创建)。 |
| 44 | 51 | ||
| @@ -16,7 +16,7 @@ class Settings(BaseSettings): | |||
| 16 | # -------------------------- | 16 | # -------------------------- |
| 17 | # 【基础配置】 | 17 | # 【基础配置】 |
| 18 | # -------------------------- | 18 | # -------------------------- |
| 19 | - DB_TYPE: Literal["mysql", "sqlite"] = Field(default="sqlite", env="DB_TYPE") | 19 | + DB_TYPE: Literal["mysql", "sqlite", "gaussdb", "opengauss"] = Field(default="sqlite", env="DB_TYPE") |
| 20 | 20 | ||
| 21 | 21 | ||
| 22 | # -------------------------- | 22 | # -------------------------- |
| @@ -31,9 +31,8 @@ class Settings(BaseSettings): | |||
| 31 | # -------------------------- | 31 | # -------------------------- |
| 32 | # 【服务配置】 | 32 | # 【服务配置】 |
| 33 | # -------------------------- | 33 | # -------------------------- |
| 34 | - # ✅【必选配置】 | 34 | + IP: Optional[str] = Field(default=None, env="IP") |
| 35 | - IP: str = Field(env="IP") | 35 | + LOWCODE_IMAGE: Optional[str] = Field(default=None, env="LOWCODE_IMAGE") |
| 36 | - LOWCODE_IMAGE: str = Field(env="LOWCODE_IMAGE") | ||
| 37 | 36 | ||
| 38 | # 可选配置 | 37 | # 可选配置 |
| 39 | DEPLOY_DIR: str = Field(default="/tmp/deploys", env="DEPLOY_DIR") | 38 | DEPLOY_DIR: str = Field(default="/tmp/deploys", env="DEPLOY_DIR") |
| @@ -54,7 +53,7 @@ class Settings(BaseSettings): | |||
| 54 | # ======================== | 53 | # ======================== |
| 55 | 54 | ||
| 56 | def check_mysql_required(self) -> "Settings": | 55 | def check_mysql_required(self) -> "Settings": |
| 57 | - if self.DB_TYPE == "mysql": | 56 | + if self.DB_TYPE in {"mysql", "gaussdb", "opengauss"}: |
| 58 | missing = [] | 57 | missing = [] |
| 59 | if not self.DB_HOST: | 58 | if not self.DB_HOST: |
| 60 | missing.append("DB_HOST") | 59 | missing.append("DB_HOST") |
| @@ -68,7 +67,26 @@ class Settings(BaseSettings): | |||
| 68 | missing.append("DB_NAME") | 67 | missing.append("DB_NAME") |
| 69 | 68 | ||
| 70 | if missing: | 69 | if missing: |
| 71 | - raise ValueError(f"When DB_TYPE=mysql, the following fields are required: {', '.join(missing)}") | 70 | + raise ValueError( |
| 71 | + f"When DB_TYPE is mysql/gaussdb/opengauss, the following fields are required: {', '.join(missing)}" | ||
| 72 | + ) | ||
| 73 | + return self | ||
| 74 | + | ||
| 75 | + | ||
| 76 | + def check_runtime_required(self) -> "Settings": | ||
| 77 | + missing = [] | ||
| 78 | + if not self.IP: | ||
| 79 | + missing.append("IP") | ||
| 80 | + | ||
| 81 | + if self.DEPLOY_TYPE in {"docker", "k8s"} and not self.LOWCODE_IMAGE: | ||
| 82 | + missing.append("LOWCODE_IMAGE") | ||
| 83 | + | ||
| 84 | + if missing: | ||
| 85 | + deploy_type = self.DEPLOY_TYPE | ||
| 86 | + raise ValueError( | ||
| 87 | + f"Missing required runtime settings for DEPLOY_TYPE={deploy_type}: {', '.join(missing)}" | ||
| 88 | + ) | ||
| 89 | + | ||
| 72 | return self | 90 | return self |
| 73 | 91 | ||
| 74 | # ======================== | 92 | # ======================== |
| @@ -4,6 +4,7 @@ | |||
| 4 | from .handler import DBHandler | 4 | from .handler import DBHandler |
| 5 | from .sqlalchemy_handler import SQLAlchemyHandler | 5 | from .sqlalchemy_handler import SQLAlchemyHandler |
| 6 | from .mysql_handler import MySQLHandler | 6 | from .mysql_handler import MySQLHandler |
| 7 | +from .gaussdb_handler import GaussDBHandler | ||
| 7 | from .sqlite_handler import SQLiteHandler | 8 | from .sqlite_handler import SQLiteHandler |
| 8 | from .redis_handler import RedisHandler | 9 | from .redis_handler import RedisHandler |
| 9 | 10 | ||
| @@ -11,6 +12,7 @@ __all__ = [ | |||
| 11 | "DBHandler", | 12 | "DBHandler", |
| 12 | "SQLAlchemyHandler", | 13 | "SQLAlchemyHandler", |
| 13 | "MySQLHandler", | 14 | "MySQLHandler", |
| 15 | + "GaussDBHandler", | ||
| 14 | "SQLiteHandler", | 16 | "SQLiteHandler", |
| 15 | "RedisHandler", | 17 | "RedisHandler", |
| 16 | ] | 18 | ] |
| @@ -0,0 +1,12 @@ | |||
| 1 | +def ensure_async_gaussdb_installed() -> None: | ||
| 2 | + from .gaussdb_asyncgaussdb import ensure_async_gaussdb_installed as _ensure_async_gaussdb_installed | ||
| 3 | + | ||
| 4 | + _ensure_async_gaussdb_installed() | ||
| 5 | + | ||
| 6 | + | ||
| 7 | +def ensure_gaussdb_dialect_registered() -> None: | ||
| 8 | + from .gaussdb_asyncgaussdb import ensure_gaussdb_dialect_registered as _ensure_gaussdb_dialect_registered | ||
| 9 | + | ||
| 10 | + _ensure_gaussdb_dialect_registered() | ||
| 11 | + | ||
| 12 | +__all__ = ["ensure_async_gaussdb_installed", "ensure_gaussdb_dialect_registered"] | ||
| @@ -0,0 +1,83 @@ | |||
| 1 | +from __future__ import annotations | ||
| 2 | + | ||
| 3 | +from importlib import import_module | ||
| 4 | + | ||
| 5 | +from sqlalchemy.dialects import registry | ||
| 6 | +from sqlalchemy.dialects.postgresql.asyncpg import AsyncAdapt_asyncpg_dbapi | ||
| 7 | +from sqlalchemy.dialects.postgresql.asyncpg import PGDialect_asyncpg | ||
| 8 | +from sqlalchemy.util import memoized_property | ||
| 9 | + | ||
| 10 | +_MODULE_PATH = "openjiuwen_runtime.foundation.db.dialects.gaussdb_asyncgaussdb" | ||
| 11 | +_DRIVER_INSTALL_HINT = ( | ||
| 12 | + "DB_TYPE is set to gaussdb/opengauss, but optional dependency async-gaussdb is not installed. " | ||
| 13 | + "Install openjiuwen-runtime-foundation[gaussdb] or async-gaussdb>=0.30.4." | ||
| 14 | +) | ||
| 15 | + | ||
| 16 | + | ||
| 17 | +def _import_async_gaussdb(): | ||
| 18 | + try: | ||
| 19 | + return import_module("async_gaussdb") | ||
| 20 | + except ModuleNotFoundError as exc: | ||
| 21 | + if exc.name != "async_gaussdb": | ||
| 22 | + raise | ||
| 23 | + raise ModuleNotFoundError(_DRIVER_INSTALL_HINT) from exc | ||
| 24 | + | ||
| 25 | + | ||
| 26 | +def _import_split_server_version_string(): | ||
| 27 | + try: | ||
| 28 | + module = import_module("async_gaussdb.serverversion") | ||
| 29 | + except ModuleNotFoundError as exc: | ||
| 30 | + if exc.name not in {"async_gaussdb", "async_gaussdb.serverversion"}: | ||
| 31 | + raise | ||
| 32 | + raise ModuleNotFoundError(_DRIVER_INSTALL_HINT) from exc | ||
| 33 | + return module.split_server_version_string | ||
| 34 | + | ||
| 35 | + | ||
| 36 | +def ensure_async_gaussdb_installed() -> None: | ||
| 37 | + _import_async_gaussdb() | ||
| 38 | + | ||
| 39 | + | ||
| 40 | +class AsyncAdapt_async_gaussdb_dbapi(AsyncAdapt_asyncpg_dbapi): | ||
| 41 | + | ||
| 42 | + def _asyncpg_error_translate(self): | ||
| 43 | + exceptions = self.asyncpg.exceptions | ||
| 44 | + mappings = { | ||
| 45 | + getattr(exceptions, "IntegrityConstraintViolationError", None): self.IntegrityError, | ||
| 46 | + getattr(exceptions, "PostgresError", None): self.Error, | ||
| 47 | + getattr(exceptions, "SyntaxOrAccessError", None): self.ProgrammingError, | ||
| 48 | + getattr(exceptions, "InterfaceError", None): self.InterfaceError, | ||
| 49 | + getattr(exceptions, "InvalidCachedStatementError", None): self.InvalidCachedStatementError, | ||
| 50 | + getattr(exceptions, "InternalServerError", None): self.InternalServerError, | ||
| 51 | + } | ||
| 52 | + return {source: target for source, target in mappings.items() if source is not None} | ||
| 53 | + | ||
| 54 | + | ||
| 55 | +class PGDialect_async_gaussdb(PGDialect_asyncpg): | ||
| 56 | + driver = "async_gaussdb" | ||
| 57 | + supports_statement_cache = False | ||
| 58 | + | ||
| 59 | + | ||
| 60 | + def import_dbapi(cls): | ||
| 61 | + return AsyncAdapt_async_gaussdb_dbapi(_import_async_gaussdb()) | ||
| 62 | + | ||
| 63 | + def _get_server_version_info(self, connection): | ||
| 64 | + version_string = connection.exec_driver_sql("select pg_catalog.version()").scalar() | ||
| 65 | + split_server_version_string = _import_split_server_version_string() | ||
| 66 | + version = split_server_version_string(version_string) | ||
| 67 | + major = version.major | ||
| 68 | + # GaussDB kernel versions are not PostgreSQL major versions. If we expose | ||
| 69 | + # values like 505.x to SQLAlchemy, it enables reflection SQL that expects | ||
| 70 | + # pg_catalog columns (e.g. pg_attribute.attgenerated) unavailable on some | ||
| 71 | + # GaussDB/openGauss variants. Advertise a conservative PG compatibility level. | ||
| 72 | + if major is not None and major >= 100: | ||
| 73 | + return (11, 0) | ||
| 74 | + return tuple(part for part in (version.major, version.minor, version.micro) if part is not None) | ||
| 75 | + | ||
| 76 | + | ||
| 77 | +dialect = PGDialect_async_gaussdb | ||
| 78 | + | ||
| 79 | + | ||
| 80 | +def ensure_gaussdb_dialect_registered() -> None: | ||
| 81 | + registry.register("gaussdb.async_gaussdb", _MODULE_PATH, "PGDialect_async_gaussdb") | ||
| 82 | + registry.register("opengauss.async_gaussdb", _MODULE_PATH, "PGDialect_async_gaussdb") | ||
| 83 | + registry.register("postgresql.async_gaussdb", _MODULE_PATH, "PGDialect_async_gaussdb") | ||
| @@ -0,0 +1,24 @@ | |||
| 1 | +from urllib.parse import quote | ||
| 2 | + | ||
| 3 | +from .dialects import ensure_async_gaussdb_installed, ensure_gaussdb_dialect_registered | ||
| 4 | +from .sqlalchemy_handler import SQLAlchemyHandler | ||
| 5 | +from ..config import settings | ||
| 6 | + | ||
| 7 | + | ||
| 8 | +class GaussDBHandler(SQLAlchemyHandler): | ||
| 9 | + """GaussDB/openGauss 数据库句柄。 | ||
| 10 | + | ||
| 11 | + 连接串使用自定义的 gaussdb+async_gaussdb 方言名,方言层复用 SQLAlchemy | ||
| 12 | + PostgreSQL asyncpg 适配器,并将底层 DB-API 替换为 async-gaussdb。 | ||
| 13 | + """ | ||
| 14 | + | ||
| 15 | + def __init__(self): | ||
| 16 | + ensure_async_gaussdb_installed() | ||
| 17 | + ensure_gaussdb_dialect_registered() | ||
| 18 | + user = quote(settings.DB_USER or "", safe="") | ||
| 19 | + password = quote(settings.DB_PASSWORD or "", safe="") | ||
| 20 | + database_url = ( | ||
| 21 | + f"gaussdb+async_gaussdb://{user}:{password}@" | ||
| 22 | + f"{settings.DB_HOST}:{settings.DB_PORT}/{settings.DB_NAME}" | ||
| 23 | + ) | ||
| 24 | + super().__init__(database_url) | ||
| @@ -27,6 +27,11 @@ dependencies = [ | |||
| 27 | "ruff==0.9.10", | 27 | "ruff==0.9.10", |
| 28 | ] | 28 | ] |
| 29 | 29 | ||
| 30 | +[project.optional-dependencies] | ||
| 31 | +gaussdb = [ | ||
| 32 | + "async-gaussdb>=0.30.4", | ||
| 33 | +] | ||
| 34 | + | ||
| 30 | [tool.uv] | 35 | [tool.uv] |
| 31 | default-groups = ['dev'] | 36 | default-groups = ['dev'] |
| 32 | 37 | ||
| @@ -0,0 +1,2 @@ | |||
| 1 | +# coding: utf-8 | ||
| 2 | +# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | ||
| @@ -0,0 +1,140 @@ | |||
| 1 | +# coding: utf-8 | ||
| 2 | +# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | ||
| 3 | + | ||
| 4 | +"""Real database system tests for GaussDB/openGauss. | ||
| 5 | + | ||
| 6 | +This suite is opt-in and skipped by default. It is intended for environments | ||
| 7 | +with a reachable real database. Enable with: | ||
| 8 | + | ||
| 9 | + GAUSSDB_REAL_ST_ENABLED=1 | ||
| 10 | +""" | ||
| 11 | + | ||
| 12 | +import os | ||
| 13 | +import socket | ||
| 14 | +import unittest | ||
| 15 | +from urllib.parse import quote | ||
| 16 | + | ||
| 17 | +from sqlalchemy import text | ||
| 18 | +from sqlalchemy.ext.asyncio import create_async_engine | ||
| 19 | + | ||
| 20 | +# foundation settings require IP at import time. | ||
| 21 | +os.environ.setdefault("IP", "127.0.0.1") | ||
| 22 | + | ||
| 23 | +from openjiuwen_runtime.foundation.db.dialects import ensure_gaussdb_dialect_registered | ||
| 24 | + | ||
| 25 | + | ||
| 26 | +def _pick_env(*names: str, default: str = "") -> str: | ||
| 27 | + for name in names: | ||
| 28 | + value = (os.environ.get(name) or "").strip() | ||
| 29 | + if value: | ||
| 30 | + return value | ||
| 31 | + return default | ||
| 32 | + | ||
| 33 | + | ||
| 34 | +def _is_enabled(value: str) -> bool: | ||
| 35 | + return value.strip().lower() in {"1", "true", "yes", "on"} | ||
| 36 | + | ||
| 37 | + | ||
| 38 | +def _check_tcp_connectivity(host: str, port: int, timeout_seconds: float = 5.0) -> tuple[bool, str]: | ||
| 39 | + try: | ||
| 40 | + with socket.create_connection((host, port), timeout=timeout_seconds): | ||
| 41 | + return True, "" | ||
| 42 | + except OSError as exc: | ||
| 43 | + return False, str(exc) | ||
| 44 | + | ||
| 45 | + | ||
| 46 | +class TestGaussDBRealDatabaseSystem(unittest.IsolatedAsyncioTestCase): | ||
| 47 | + | ||
| 48 | + def setUpClass(cls): | ||
| 49 | + super().setUpClass() | ||
| 50 | + | ||
| 51 | + enabled = _pick_env("GAUSSDB_REAL_ST_ENABLED", default="0") | ||
| 52 | + if not _is_enabled(enabled): | ||
| 53 | + raise unittest.SkipTest("real-db ST is disabled; set GAUSSDB_REAL_ST_ENABLED=1 to enable") | ||
| 54 | + | ||
| 55 | + db_type = _pick_env("GAUSSDB_REAL_DB_TYPE", "DB_TYPE", default="gaussdb").lower() | ||
| 56 | + if db_type not in {"gaussdb", "opengauss"}: | ||
| 57 | + raise unittest.SkipTest(f"unsupported db type for this ST: {db_type!r}") | ||
| 58 | + | ||
| 59 | + host = _pick_env("GAUSSDB_REAL_DB_HOST", "DB_HOST") | ||
| 60 | + port = _pick_env("GAUSSDB_REAL_DB_PORT", "DB_PORT", default="5432") | ||
| 61 | + user = _pick_env("GAUSSDB_REAL_DB_USER", "DB_USER") | ||
| 62 | + password = _pick_env("GAUSSDB_REAL_DB_PASSWORD", "DB_PASSWORD") | ||
| 63 | + database = _pick_env("GAUSSDB_REAL_DB_NAME", "DB_NAME", "AGENT_DB_NAME") | ||
| 64 | + strict_network = _is_enabled(_pick_env("GAUSSDB_REAL_ST_STRICT_NETWORK", default="0")) | ||
| 65 | + | ||
| 66 | + missing = [] | ||
| 67 | + for key, value in ( | ||
| 68 | + ("HOST", host), | ||
| 69 | + ("PORT", port), | ||
| 70 | + ("USER", user), | ||
| 71 | + ("PASSWORD", password), | ||
| 72 | + ("DATABASE", database), | ||
| 73 | + ): | ||
| 74 | + if not value: | ||
| 75 | + missing.append(key) | ||
| 76 | + | ||
| 77 | + if missing: | ||
| 78 | + raise unittest.SkipTest( | ||
| 79 | + "real-db ST missing required env vars: " + ", ".join(missing) | ||
| 80 | + ) | ||
| 81 | + | ||
| 82 | + try: | ||
| 83 | + port_int = int(port) | ||
| 84 | + except ValueError as exc: | ||
| 85 | + raise unittest.SkipTest(f"invalid database port: {port!r}") from exc | ||
| 86 | + | ||
| 87 | + reachable, error_message = _check_tcp_connectivity(host, port_int) | ||
| 88 | + if not reachable: | ||
| 89 | + detail = f"real-db ST cannot reach database endpoint {host}:{port_int}: {error_message}" | ||
| 90 | + if strict_network: | ||
| 91 | + raise RuntimeError(detail) | ||
| 92 | + raise unittest.SkipTest(detail) | ||
| 93 | + | ||
| 94 | + ensure_gaussdb_dialect_registered() | ||
| 95 | + | ||
| 96 | + user_enc = quote(user, safe="") | ||
| 97 | + password_enc = quote(password, safe="") | ||
| 98 | + cls._database = database | ||
| 99 | + cls._async_url = f"{db_type}+async_gaussdb://{user_enc}:{password_enc}@{host}:{port}/{database}" | ||
| 100 | + | ||
| 101 | + async def asyncSetUp(self): | ||
| 102 | + self.engine = create_async_engine(self._async_url, pool_pre_ping=True) | ||
| 103 | + | ||
| 104 | + async def asyncTearDown(self): | ||
| 105 | + await self.engine.dispose() | ||
| 106 | + | ||
| 107 | + async def test_can_connect_and_select_one(self): | ||
| 108 | + async with self.engine.connect() as conn: | ||
| 109 | + value = (await conn.execute(text("SELECT 1"))).scalar_one() | ||
| 110 | + self.assertEqual(value, 1) | ||
| 111 | + | ||
| 112 | + async def test_can_query_version_and_current_database(self): | ||
| 113 | + async with self.engine.connect() as conn: | ||
| 114 | + version = (await conn.execute(text("SELECT version()"))).scalar_one() | ||
| 115 | + current_db = (await conn.execute(text("SELECT current_database()"))).scalar_one() | ||
| 116 | + | ||
| 117 | + self.assertTrue(isinstance(version, str) and version) | ||
| 118 | + self.assertEqual(str(current_db), self._database) | ||
| 119 | + | ||
| 120 | + async def test_reflection_smoke_pg_catalog(self): | ||
| 121 | + async with self.engine.connect() as conn: | ||
| 122 | + rows = await conn.execute( | ||
| 123 | + text( | ||
| 124 | + """ | ||
| 125 | + SELECT a.attname | ||
| 126 | + FROM pg_catalog.pg_attribute AS a | ||
| 127 | + JOIN pg_catalog.pg_class AS c ON c.oid = a.attrelid | ||
| 128 | + JOIN pg_catalog.pg_namespace AS n ON n.oid = c.relnamespace | ||
| 129 | + WHERE n.nspname = 'pg_catalog' | ||
| 130 | + AND c.relname = 'pg_class' | ||
| 131 | + AND a.attnum > 0 | ||
| 132 | + AND NOT a.attisdropped | ||
| 133 | + ORDER BY a.attnum | ||
| 134 | + """ | ||
| 135 | + ) | ||
| 136 | + ) | ||
| 137 | + column_names = [str(name) for name in rows.scalars().all()] | ||
| 138 | + | ||
| 139 | + self.assertTrue(column_names) | ||
| 140 | + self.assertIn("relname", column_names) | ||
| @@ -0,0 +1,214 @@ | |||
| 1 | +# coding: utf-8 | ||
| 2 | +# Copyright (c) Huawei Technologies Co., Ltd. 2026-2026. All rights reserved | ||
| 3 | + | ||
| 4 | +import builtins | ||
| 5 | +import importlib | ||
| 6 | +import os | ||
| 7 | +import sys | ||
| 8 | +from pathlib import Path | ||
| 9 | +from types import SimpleNamespace | ||
| 10 | +from unittest import TestCase, mock | ||
| 11 | + | ||
| 12 | +from sqlalchemy.ext.asyncio import create_async_engine | ||
| 13 | + | ||
| 14 | + | ||
| 15 | +class TestGaussDBSupport(TestCase): | ||
| 16 | + def setUp(self): | ||
| 17 | + self.repo_root = Path(__file__).resolve().parents[3] | ||
| 18 | + self.foundation_root = self.repo_root / "foundation" | ||
| 19 | + self.server_root = self.repo_root / "server" | ||
| 20 | + self.management_root = self.repo_root / "management" | ||
| 21 | + self.ir_execution_root = self.repo_root / "applications" / "ir_execution_service" | ||
| 22 | + self._original_env = os.environ.copy() | ||
| 23 | + | ||
| 24 | + os.environ.update( | ||
| 25 | + { | ||
| 26 | + "DB_TYPE": "gaussdb", | ||
| 27 | + "DB_HOST": "127.0.0.1", | ||
| 28 | + "DB_PORT": "5432", | ||
| 29 | + "DB_USER": "gauss_user", | ||
| 30 | + "DB_PASSWORD": "p@ss", | ||
| 31 | + "DB_NAME": "runtime_db", | ||
| 32 | + "AGENT_DB_NAME": "agent_db", | ||
| 33 | + "OPS_DB_NAME": "ops_db", | ||
| 34 | + "IP": "127.0.0.1", | ||
| 35 | + } | ||
| 36 | + ) | ||
| 37 | + | ||
| 38 | + for extra_path in ( | ||
| 39 | + self.foundation_root, | ||
| 40 | + self.server_root, | ||
| 41 | + self.management_root, | ||
| 42 | + self.ir_execution_root, | ||
| 43 | + ): | ||
| 44 | + extra_path_str = str(extra_path) | ||
| 45 | + if extra_path_str not in sys.path: | ||
| 46 | + sys.path.insert(0, extra_path_str) | ||
| 47 | + | ||
| 48 | + def tearDown(self): | ||
| 49 | + os.environ.clear() | ||
| 50 | + os.environ.update(self._original_env) | ||
| 51 | + | ||
| 52 | + for module_name in [ | ||
| 53 | + "openjiuwen_runtime.foundation.config", | ||
| 54 | + "openjiuwen_runtime.foundation.db", | ||
| 55 | + "openjiuwen_runtime.foundation.db.dialects", | ||
| 56 | + "openjiuwen_runtime.foundation.db.dialects.gaussdb_asyncgaussdb", | ||
| 57 | + "openjiuwen_runtime.foundation.db.gaussdb_handler", | ||
| 58 | + "openjiuwen_runtime.server.main", | ||
| 59 | + "openjiuwen_runtime.management", | ||
| 60 | + "runtime_support.gaussdb_sqlalchemy_dialect", | ||
| 61 | + "runtime_support.memory_engine_start", | ||
| 62 | + "runtime_support.runtime_env_prepare", | ||
| 63 | + ]: | ||
| 64 | + sys.modules.pop(module_name, None) | ||
| 65 | + | ||
| 66 | + def _patch_async_gaussdb_missing(self): | ||
| 67 | + original_import = builtins.__import__ | ||
| 68 | + | ||
| 69 | + def guarded_import(name, globals=None, locals=None, fromlist=(), level=0): | ||
| 70 | + if name == "async_gaussdb" or name.startswith("async_gaussdb."): | ||
| 71 | + raise ModuleNotFoundError("No module named 'async_gaussdb'", name="async_gaussdb") | ||
| 72 | + return original_import(name, globals, locals, fromlist, level) | ||
| 73 | + | ||
| 74 | + return mock.patch("builtins.__import__", side_effect=guarded_import) | ||
| 75 | + | ||
| 76 | + def _evict_async_gaussdb_modules(self): | ||
| 77 | + removed_modules = {} | ||
| 78 | + for module_name in list(sys.modules): | ||
| 79 | + if module_name == "async_gaussdb" or module_name.startswith("async_gaussdb."): | ||
| 80 | + removed_modules[module_name] = sys.modules.pop(module_name) | ||
| 81 | + return removed_modules | ||
| 82 | + | ||
| 83 | + def test_management_exports_gaussdb_handler(self): | ||
| 84 | + management_module = importlib.import_module("openjiuwen_runtime.management") | ||
| 85 | + | ||
| 86 | + self.assertEqual(management_module.GaussDBHandler.__name__, "GaussDBHandler") | ||
| 87 | + | ||
| 88 | + def test_server_selects_gaussdb_handler(self): | ||
| 89 | + server_main = importlib.import_module("openjiuwen_runtime.server.main") | ||
| 90 | + | ||
| 91 | + self.assertEqual(type(server_main.db_handler).__name__, "GaussDBHandler") | ||
| 92 | + self.assertEqual( | ||
| 93 | + server_main.db_handler.database_url, | ||
| 94 | + "gaussdb+async_gaussdb://gauss_user:p%40ss@127.0.0.1:5432/runtime_db", | ||
| 95 | + ) | ||
| 96 | + | ||
| 97 | + def test_server_selects_gaussdb_handler_for_opengauss(self): | ||
| 98 | + os.environ["DB_TYPE"] = "opengauss" | ||
| 99 | + for module_name in [ | ||
| 100 | + "openjiuwen_runtime.foundation.config", | ||
| 101 | + "openjiuwen_runtime.foundation.db.gaussdb_handler", | ||
| 102 | + "openjiuwen_runtime.server.main", | ||
| 103 | + ]: | ||
| 104 | + sys.modules.pop(module_name, None) | ||
| 105 | + | ||
| 106 | + server_main = importlib.import_module("openjiuwen_runtime.server.main") | ||
| 107 | + | ||
| 108 | + self.assertEqual(type(server_main.db_handler).__name__, "GaussDBHandler") | ||
| 109 | + | ||
| 110 | + def test_runtime_support_generates_gauss_urls_for_gaussdb_and_opengauss(self): | ||
| 111 | + memory_engine_start = importlib.import_module("runtime_support.memory_engine_start") | ||
| 112 | + | ||
| 113 | + sync_url = memory_engine_start.get_database_url() | ||
| 114 | + async_url = memory_engine_start.get_async_database_url(sync_url) | ||
| 115 | + self.assertEqual(sync_url, "gaussdb://gauss_user:p%40ss@127.0.0.1:5432/agent_db") | ||
| 116 | + self.assertEqual(async_url, "gaussdb+async_gaussdb://gauss_user:p%40ss@127.0.0.1:5432/agent_db") | ||
| 117 | + | ||
| 118 | + os.environ["DB_TYPE"] = "opengauss" | ||
| 119 | + sync_url = memory_engine_start.get_database_url() | ||
| 120 | + async_url = memory_engine_start.get_async_database_url(sync_url) | ||
| 121 | + self.assertEqual(sync_url, "gaussdb://gauss_user:p%40ss@127.0.0.1:5432/agent_db") | ||
| 122 | + self.assertEqual(async_url, "gaussdb+async_gaussdb://gauss_user:p%40ss@127.0.0.1:5432/agent_db") | ||
| 123 | + | ||
| 124 | + def test_runtime_env_prepare_sets_default_port_for_opengauss(self): | ||
| 125 | + os.environ["DB_TYPE"] = "opengauss" | ||
| 126 | + | ||
| 127 | + runtime_env_prepare = importlib.import_module("runtime_support.runtime_env_prepare") | ||
| 128 | + | ||
| 129 | + # Some dependency imports may materialize DB_PORT from external env/.env. | ||
| 130 | + # Remove it immediately before applying defaults to verify opengauss fallback. | ||
| 131 | + os.environ.pop("DB_PORT", None) | ||
| 132 | + runtime_env_prepare.apply_runtime_type_and_optional_defaults() | ||
| 133 | + | ||
| 134 | + self.assertEqual(os.environ["DB_PORT"], "5432") | ||
| 135 | + | ||
| 136 | + def test_custom_dialect_supports_opengauss_alias(self): | ||
| 137 | + gauss_dialect = importlib.import_module("openjiuwen_runtime.foundation.db.dialects.gaussdb_asyncgaussdb") | ||
| 138 | + gauss_dialect.ensure_gaussdb_dialect_registered() | ||
| 139 | + | ||
| 140 | + engine = create_async_engine("opengauss+async_gaussdb://gauss_user:p%40ss@127.0.0.1:5432/runtime_db") | ||
| 141 | + try: | ||
| 142 | + self.assertEqual(type(engine.dialect).__name__, "PGDialect_async_gaussdb") | ||
| 143 | + self.assertEqual(engine.dialect.driver, "async_gaussdb") | ||
| 144 | + finally: | ||
| 145 | + import asyncio | ||
| 146 | + | ||
| 147 | + asyncio.run(engine.dispose()) | ||
| 148 | + | ||
| 149 | + def test_custom_dialect_parses_real_gaussdb_version_string(self): | ||
| 150 | + gauss_dialect = importlib.import_module("openjiuwen_runtime.foundation.db.dialects.gaussdb_asyncgaussdb") | ||
| 151 | + | ||
| 152 | + class FakeConnection: | ||
| 153 | + def exec_driver_sql(self, _sql: str): | ||
| 154 | + return SimpleNamespace( | ||
| 155 | + scalar=lambda: "gaussdb (GaussDB Kernel 505.2.1.SPC0600 build 2aa20d4e) compiled at 2025-05-31 22:48:04 commit 10460 last mr 23863 release" | ||
| 156 | + ) | ||
| 157 | + | ||
| 158 | + dialect = gauss_dialect.PGDialect_async_gaussdb() | ||
| 159 | + version = dialect._get_server_version_info(FakeConnection()) | ||
| 160 | + | ||
| 161 | + self.assertEqual(version, (11, 0)) | ||
| 162 | + | ||
| 163 | + def test_server_mysql_import_does_not_require_async_gaussdb(self): | ||
| 164 | + os.environ["DB_TYPE"] = "mysql" | ||
| 165 | + removed_modules = self._evict_async_gaussdb_modules() | ||
| 166 | + for module_name in [ | ||
| 167 | + "openjiuwen_runtime.foundation.config", | ||
| 168 | + "openjiuwen_runtime.foundation.db.gaussdb_handler", | ||
| 169 | + "openjiuwen_runtime.server.main", | ||
| 170 | + ]: | ||
| 171 | + sys.modules.pop(module_name, None) | ||
| 172 | + | ||
| 173 | + try: | ||
| 174 | + with self._patch_async_gaussdb_missing(): | ||
| 175 | + server_main = importlib.import_module("openjiuwen_runtime.server.main") | ||
| 176 | + finally: | ||
| 177 | + sys.modules.update(removed_modules) | ||
| 178 | + | ||
| 179 | + self.assertEqual(type(server_main.db_handler).__name__, "MySQLHandler") | ||
| 180 | + | ||
| 181 | + def test_memory_engine_start_mysql_path_does_not_require_async_gaussdb(self): | ||
| 182 | + os.environ["DB_TYPE"] = "mysql" | ||
| 183 | + removed_modules = self._evict_async_gaussdb_modules() | ||
| 184 | + for module_name in [ | ||
| 185 | + "runtime_support.gaussdb_sqlalchemy_dialect", | ||
| 186 | + "runtime_support.memory_engine_start", | ||
| 187 | + ]: | ||
| 188 | + sys.modules.pop(module_name, None) | ||
| 189 | + | ||
| 190 | + try: | ||
| 191 | + with self._patch_async_gaussdb_missing(): | ||
| 192 | + memory_engine_start = importlib.import_module("runtime_support.memory_engine_start") | ||
| 193 | + sync_url = memory_engine_start.get_database_url() | ||
| 194 | + async_url = memory_engine_start.get_async_database_url(sync_url) | ||
| 195 | + finally: | ||
| 196 | + sys.modules.update(removed_modules) | ||
| 197 | + | ||
| 198 | + self.assertEqual(sync_url, "mysql+pymysql://gauss_user:p%40ss@127.0.0.1:5432/agent_db?charset=utf8mb4") | ||
| 199 | + self.assertEqual(async_url, "mysql+aiomysql://gauss_user:p%40ss@127.0.0.1:5432/agent_db?charset=utf8mb4") | ||
| 200 | + | ||
| 201 | + def test_gaussdb_handler_raises_helpful_error_when_driver_missing(self): | ||
| 202 | + removed_modules = self._evict_async_gaussdb_modules() | ||
| 203 | + sys.modules.pop("openjiuwen_runtime.foundation.db.gaussdb_handler", None) | ||
| 204 | + | ||
| 205 | + try: | ||
| 206 | + with self._patch_async_gaussdb_missing(): | ||
| 207 | + gaussdb_handler = importlib.import_module("openjiuwen_runtime.foundation.db.gaussdb_handler") | ||
| 208 | + with self.assertRaises(ModuleNotFoundError) as exc: | ||
| 209 | + gaussdb_handler.GaussDBHandler() | ||
| 210 | + finally: | ||
| 211 | + sys.modules.update(removed_modules) | ||
| 212 | + | ||
| 213 | + self.assertIn("openjiuwen-runtime-foundation[gaussdb]", str(exc.exception)) | ||
| 214 | + self.assertIn("async-gaussdb>=0.30.4", str(exc.exception)) | ||
| @@ -4,6 +4,7 @@ | |||
| 4 | """OpenJiuwen Runtime Management SDK""" | 4 | """OpenJiuwen Runtime Management SDK""" |
| 5 | 5 | ||
| 6 | from openjiuwen_runtime.foundation.db.handler import DBHandler | 6 | from openjiuwen_runtime.foundation.db.handler import DBHandler |
| 7 | +from openjiuwen_runtime.foundation.db.gaussdb_handler import GaussDBHandler | ||
| 7 | from openjiuwen_runtime.foundation.db.mysql_handler import MySQLHandler | 8 | from openjiuwen_runtime.foundation.db.mysql_handler import MySQLHandler |
| 8 | from openjiuwen_runtime.foundation.db.sqlite_handler import SQLiteHandler | 9 | from openjiuwen_runtime.foundation.db.sqlite_handler import SQLiteHandler |
| 9 | 10 | ||
| @@ -60,6 +61,7 @@ __all__ = [ | |||
| 60 | "DBHandler", | 61 | "DBHandler", |
| 61 | "SQLiteHandler", | 62 | "SQLiteHandler", |
| 62 | "MySQLHandler", | 63 | "MySQLHandler", |
| 64 | + "GaussDBHandler", | ||
| 63 | # Base | 65 | # Base |
| 64 | "CommonParams", | 66 | "CommonParams", |
| 65 | "DeployContext", | 67 | "DeployContext", |
| @@ -91,6 +91,12 @@ $EnvFile = Join-Path $ServerDir ".env" | |||
| 91 | 91 | ||
| 92 | Import-DotEnv -EnvFile $EnvFile | 92 | Import-DotEnv -EnvFile $EnvFile |
| 93 | 93 | ||
| 94 | +$FoundationInstallTarget = "..\foundation" | ||
| 95 | +if (($env:DB_TYPE ?? "").Trim().ToLower() -in @("gaussdb", "opengauss")) { | ||
| 96 | + $FoundationInstallTarget = "..\foundation[gaussdb]" | ||
| 97 | + Write-Info "Detected DB_TYPE=$($env:DB_TYPE); install foundation with [gaussdb] optional dependency." | ||
| 98 | +} | ||
| 99 | + | ||
| 94 | Push-Location $ProjectDir | 100 | Push-Location $ProjectDir |
| 95 | try { | 101 | try { |
| 96 | Invoke-CheckedCommand -FilePath "git" -Arguments @("submodule", "update", "--init", "--recursive") -ErrorMessage "git submodule update --init failed" | 102 | Invoke-CheckedCommand -FilePath "git" -Arguments @("submodule", "update", "--init", "--recursive") -ErrorMessage "git submodule update --init failed" |
| @@ -162,7 +168,7 @@ try { | |||
| 162 | throw "Runtime python not found: $VenvPython" | 168 | throw "Runtime python not found: $VenvPython" |
| 163 | } | 169 | } |
| 164 | Invoke-CheckedCommand -FilePath "uv" -Arguments (@("pip", "install", "-e", "..\management") + $UvExtraArgs) -ErrorMessage "uv pip install management failed" | 170 | Invoke-CheckedCommand -FilePath "uv" -Arguments (@("pip", "install", "-e", "..\management") + $UvExtraArgs) -ErrorMessage "uv pip install management failed" |
| 165 | - Invoke-CheckedCommand -FilePath "uv" -Arguments (@("pip", "install", "-e", "..\foundation") + $UvExtraArgs) -ErrorMessage "uv pip install foundation failed" | 171 | + Invoke-CheckedCommand -FilePath "uv" -Arguments (@("pip", "install", "-e", $FoundationInstallTarget) + $UvExtraArgs) -ErrorMessage "uv pip install foundation failed" |
| 166 | 172 | ||
| 167 | & $VenvPython -m openjiuwen_runtime.server.main | 173 | & $VenvPython -m openjiuwen_runtime.server.main |
| 168 | if ($LASTEXITCODE -ne 0) { | 174 | if ($LASTEXITCODE -ne 0) { |
| @@ -37,6 +37,13 @@ set -a # Automatically export all variables | |||
| 37 | source ${ENV_FILE} # Load environment variables from file | 37 | source ${ENV_FILE} # Load environment variables from file |
| 38 | set +a # Disable automatic export | 38 | set +a # Disable automatic export |
| 39 | 39 | ||
| 40 | +FOUNDATION_INSTALL_TARGET="../foundation" | ||
| 41 | +DB_TYPE_NORMALIZED="$(echo "${DB_TYPE:-}" | tr '[:upper:]' '[:lower:]')" | ||
| 42 | +if [[ "${DB_TYPE_NORMALIZED}" == "gaussdb" || "${DB_TYPE_NORMALIZED}" == "opengauss" ]]; then | ||
| 43 | + FOUNDATION_INSTALL_TARGET="../foundation[gaussdb]" | ||
| 44 | + echo "Detected DB_TYPE=${DB_TYPE}; install foundation with [gaussdb] optional dependency." | ||
| 45 | +fi | ||
| 46 | + | ||
| 40 | if [[ "$OSTYPE" == "darwin"* ]]; then | 47 | if [[ "$OSTYPE" == "darwin"* ]]; then |
| 41 | SED_I_FLAG="-i ''" | 48 | SED_I_FLAG="-i ''" |
| 42 | else | 49 | else |
| @@ -108,6 +115,6 @@ else | |||
| 108 | source .venv/bin/activate | 115 | source .venv/bin/activate |
| 109 | fi | 116 | fi |
| 110 | uv pip install -e ../management ${UV_EXTRA_ARGS} | 117 | uv pip install -e ../management ${UV_EXTRA_ARGS} |
| 111 | -uv pip install -e ../foundation ${UV_EXTRA_ARGS} | 118 | +uv pip install -e "${FOUNDATION_INSTALL_TARGET}" ${UV_EXTRA_ARGS} |
| 112 | 119 | ||
| 113 | python -m openjiuwen_runtime.server.main 2>&1 | tee server.log | 120 | python -m openjiuwen_runtime.server.main 2>&1 | tee server.log |
| @@ -1,10 +1,10 @@ | |||
| 1 | -# 数据库类型:支持mysql、sqlite | 1 | +# 数据库类型:支持 mysql、sqlite、gaussdb、opengauss |
| 2 | DB_TYPE=mysql | 2 | DB_TYPE=mysql |
| 3 | 3 | ||
| 4 | # 数据库主机地址 | 4 | # 数据库主机地址 |
| 5 | DB_HOST=localhost | 5 | DB_HOST=localhost |
| 6 | 6 | ||
| 7 | -# 数据库端口:MySQL 服务监听的端口号 | 7 | +# 数据库端口:MySQL 默认 3306,GaussDB/openGauss 默认 5432 |
| 8 | DB_PORT=3306 | 8 | DB_PORT=3306 |
| 9 | 9 | ||
| 10 | # 数据库登录用户名 | 10 | # 数据库登录用户名 |
| @@ -16,6 +16,19 @@ DB_PASSWORD=root | |||
| 16 | # 要连接的数据库名称 | 16 | # 要连接的数据库名称 |
| 17 | DB_NAME=jiuwen_runtime | 17 | DB_NAME=jiuwen_runtime |
| 18 | 18 | ||
| 19 | +# ----------------------------------------------------------------------------- | ||
| 20 | +# GaussDB/openGauss POC 验证建议值 | ||
| 21 | +# ----------------------------------------------------------------------------- | ||
| 22 | +# 使用仓库自带 scripts/run-server.sh / run-server.ps1 启动时,脚本会在 DB_TYPE=gaussdb/opengauss | ||
| 23 | +# 场景下自动安装 foundation[gaussdb];若手工安装,请额外安装 async-gaussdb>=0.30.4。 | ||
| 24 | +# 验证 GaussDB/openGauss POC 时,建议至少替换为以下值: | ||
| 25 | +# DB_TYPE=gaussdb # 或 opengauss | ||
| 26 | +# DB_HOST=127.0.0.1 | ||
| 27 | +# DB_PORT=5432 | ||
| 28 | +# DB_USER=gaussdb_user | ||
| 29 | +# DB_PASSWORD=gaussdb_password | ||
| 30 | +# DB_NAME=gaussdb_runtime | ||
| 31 | + | ||
| 19 | # 低码agent 和 agent-runtime-server 运行所在的主机IP地址 | 32 | # 低码agent 和 agent-runtime-server 运行所在的主机IP地址 |
| 20 | IP=127.0.0.1 | 33 | IP=127.0.0.1 |
| 21 | 34 | ||
| @@ -45,8 +45,14 @@ if settings.DB_TYPE == "sqlite": | |||
| 45 | db_handler = SQLiteHandler("deployments.db") | 45 | db_handler = SQLiteHandler("deployments.db") |
| 46 | elif settings.DB_TYPE == "mysql": | 46 | elif settings.DB_TYPE == "mysql": |
| 47 | db_handler = MySQLHandler() | 47 | db_handler = MySQLHandler() |
| 48 | +elif settings.DB_TYPE in {"gaussdb", "opengauss"}: | ||
| 49 | + from openjiuwen_runtime.foundation.db.gaussdb_handler import GaussDBHandler | ||
| 50 | + | ||
| 51 | + db_handler = GaussDBHandler() | ||
| 48 | else: | 52 | else: |
| 49 | - raise ValueError(f"Unsupported DB_TYPE: {settings.DB_TYPE}. Use 'sqlite' or 'mysql'.") | 53 | + raise ValueError( |
| 54 | + f"Unsupported DB_TYPE: {settings.DB_TYPE}. Use 'sqlite', 'mysql', 'gaussdb', or 'opengauss'." | ||
| 55 | + ) | ||
| 50 | 56 | ||
| 51 | manager = DeploymentManager(db_handler) | 57 | manager = DeploymentManager(db_handler) |
| 52 | 58 | ||