已开启
feat(harmonyos): add HAR packaging for HarmonyOS SDK #144
hechengjun创建于 26 天前
feat(harmonyos): add HAR packaging for HarmonyOS SDK #144
已开启
共 13 个文件变更+1902-23
| @@ -8,6 +8,10 @@ | |||
| 8 | /deps/build/ | 8 | /deps/build/ |
| 9 | /deps/openssl/ | 9 | /deps/openssl/ |
| 10 | 10 | ||
| 11 | +# HarmonyOS NAPI HAR module: CMake intermediates and packaged native libs | ||
| 12 | +/harmonyos_har_napi/build/ | ||
| 13 | +/harmonyos_har_napi/libs/ | ||
| 14 | + | ||
| 11 | /smartserve_logs/ | 15 | /smartserve_logs/ |
| 12 | 16 | ||
| 13 | .DS_Store | 17 | .DS_Store |
| @@ -1,23 +1,24 @@ | |||
| 1 | # SmartServe 跨平台 SDK 构建指南 | 1 | # SmartServe 跨平台 SDK 构建指南 |
| 2 | 2 | ||
| 3 | -本文档介绍 SmartServe 在 **Android** 和 **iOS** 平台上的 SDK 构建与接入方式。 | 3 | +本文档介绍 SmartServe 在 **HarmonyOS**、**Android** 和 **iOS** 平台上的 SDK 构建与接入方式。 |
| 4 | 4 | ||
| 5 | SmartServe 跨平台目前提供两类产物: | 5 | SmartServe 跨平台目前提供两类产物: |
| 6 | 6 | ||
| 7 | -- **SDK 产物**:面向移动 App 集成,Android 产出 AAR,iOS 产出 XCFramework。本文档描述这条路径。 | 7 | +- **SDK 产物**:面向移动 App 集成,HarmonyOS 产出 HAR,Android 产出 AAR,iOS 产出 XCFramework。本文档描述这条路径。 |
| 8 | - **可执行文件产物**:面向 macOS / Linux / Android 上直接运行 `smartserve_chatbox_cli`、`smartserve_http_server` 等程序,见 [跨平台二进制构建指南](./cross_platform_binary_build_guide.md)。 | 8 | - **可执行文件产物**:面向 macOS / Linux / Android 上直接运行 `smartserve_chatbox_cli`、`smartserve_http_server` 等程序,见 [跨平台二进制构建指南](./cross_platform_binary_build_guide.md)。 |
| 9 | 9 | ||
| 10 | ---- | 10 | +*** |
| 11 | 11 | ||
| 12 | ## 1. 通用准备 | 12 | ## 1. 通用准备 |
| 13 | 13 | ||
| 14 | ### 1.1 系统要求 | 14 | ### 1.1 系统要求 |
| 15 | 15 | ||
| 16 | -- **支持平台**:Android / iOS | 16 | +- **支持平台**:HarmonyOS / Android / iOS |
| 17 | - **C++ 标准**:C++17 及以上 | 17 | - **C++ 标准**:C++17 及以上 |
| 18 | - **编译工具链**: | 18 | - **编译工具链**: |
| 19 | - `cmake` >= 3.10 | 19 | - `cmake` >= 3.10 |
| 20 | - `ninja` 或 `make` | 20 | - `ninja` 或 `make` |
| 21 | + - HarmonyOS:HarmonyOS Command Line Tools、`zip` | ||
| 21 | - Android:Android SDK、Android NDK r23+、`zip` | 22 | - Android:Android SDK、Android NDK r23+、`zip` |
| 22 | - iOS:完整 Xcode、CocoaPods | 23 | - iOS:完整 Xcode、CocoaPods |
| 23 | 24 | ||
| @@ -58,9 +59,511 @@ cd smartserve | |||
| 58 | 59 | ||
| 59 | 详细的依赖模块、版本信息及相关说明,请参阅项目主文档 [README](../../README.md) 中的「环境要求」一节。 | 60 | 详细的依赖模块、版本信息及相关说明,请参阅项目主文档 [README](../../README.md) 中的「环境要求」一节。 |
| 60 | 61 | ||
| 61 | ---- | 62 | +*** |
| 62 | 63 | ||
| 63 | -## 2. Android SDK | 64 | +## 2. HarmonyOS SDK |
| 65 | + | ||
| 66 | +HarmonyOS SDK 产物是 HAR(HarmonyOS Archive),可以是静态库格式或带 NAPI 桥接的自包含格式。 | ||
| 67 | + | ||
| 68 | +### 2.1 下载 HarmonyOS Command Line Tools | ||
| 69 | + | ||
| 70 | +从华为官方下载 HarmonyOS Command Line Tools: | ||
| 71 | + | ||
| 72 | +**下载地址**:<https://developer.huawei.com/consumer/en/download/command-line-tools-for-hmos> | ||
| 73 | + | ||
| 74 | +1. 下载对应操作系统的 Command Line Tools 压缩包 | ||
| 75 | +2. 解压到目标目录,例如 `~/Library/OpenHarmony/Sdk/` | ||
| 76 | +3. 验证安装: | ||
| 77 | + | ||
| 78 | +```bash | ||
| 79 | +# 检查 SDK 目录结构 | ||
| 80 | +ls /payh/to/Sdk/20/native/build/cmake/ohos.toolchain.cmake | ||
| 81 | +ls /path/to/Sdk/20/native/llvm/bin/clang | ||
| 82 | +``` | ||
| 83 | + | ||
| 84 | +如果看到 toolchain 文件和编译器,说明 SDK 安装成功。 | ||
| 85 | + | ||
| 86 | +### 2.2 初始化依赖 | ||
| 87 | + | ||
| 88 | +初始化 SmartServe 基础依赖和 MNN 引擎子模块: | ||
| 89 | + | ||
| 90 | +```bash | ||
| 91 | +cd smartserve | ||
| 92 | +./scripts/init_submodules.sh --base --engine mnn | ||
| 93 | +``` | ||
| 94 | + | ||
| 95 | +这会拉取: | ||
| 96 | + | ||
| 97 | +- 基础依赖(curl、json 等) | ||
| 98 | +- MNN 推理引擎源码(位于 `third-party/mnn/`) | ||
| 99 | + | ||
| 100 | +### 2.3 配置环境变量 | ||
| 101 | + | ||
| 102 | +设置 HarmonyOS SDK 路径环境变量: | ||
| 103 | + | ||
| 104 | +```bash | ||
| 105 | +export OHOS_SDK_HOME=~/Library/OpenHarmony/Sdk/20 | ||
| 106 | +export HARMONYOS_SDK_HOME=$OHOS_SDK_HOME | ||
| 107 | +export DEVECO_SDK_HOME=$OHOS_SDK_HOME | ||
| 108 | +export OHOS_SDK_NATIVE=$OHOS_SDK_HOME/native | ||
| 109 | +``` | ||
| 110 | + | ||
| 111 | +**说明**: | ||
| 112 | + | ||
| 113 | +- `OHOS_SDK_HOME` — HarmonyOS SDK 根目录(主要变量) | ||
| 114 | +- `HARMONYOS_SDK_HOME` — 别名,部分脚本支持 | ||
| 115 | +- `DEVECO_SDK_HOME` — DevEco Studio 使用的变量名 | ||
| 116 | +- `OHOS_SDK_NATIVE` — SDK 的 native 工具链目录(自动派生) | ||
| 117 | + | ||
| 118 | +可将这些变量添加到 `~/.bashrc` 或 `~/.zshrc` 以便持久化。 | ||
| 119 | + | ||
| 120 | +### 2.4 编译 OpenSSL | ||
| 121 | + | ||
| 122 | +HarmonyOS SDK 依赖 OpenSSL,需要先交叉编译: | ||
| 123 | + | ||
| 124 | +```bash | ||
| 125 | +./scripts/harmonyos/build_openssl.sh \ | ||
| 126 | + --sdk $OHOS_SDK_HOME \ | ||
| 127 | + --arch arm64-v8a \ | ||
| 128 | + --clean | ||
| 129 | +``` | ||
| 130 | + | ||
| 131 | +**参数说明**: | ||
| 132 | + | ||
| 133 | +- `--sdk` — HarmonyOS SDK 根目录 | ||
| 134 | +- `--arch` — 目标架构(`arm64-v8a` 或 `armeabi-v7a`) | ||
| 135 | +- `--clean` — 清理之前的构建产物 | ||
| 136 | + | ||
| 137 | +构建产物位于 `deps/build/harmony/install/`: | ||
| 138 | + | ||
| 139 | +- `lib/libssl.a` | ||
| 140 | +- `lib/libcrypto.a` | ||
| 141 | +- `include/openssl/` | ||
| 142 | + | ||
| 143 | +### 2.5 编译 MNN | ||
| 144 | + | ||
| 145 | +为 HarmonyOS 编译 MNN 推理引擎: | ||
| 146 | + | ||
| 147 | +```bash | ||
| 148 | +# 设置环境变量(替换为实际 SDK 路径) | ||
| 149 | +export OHOS_SDK_HOME=/path/to/OpenHarmony/Sdk/20 | ||
| 150 | +export OHOS_ARCH=arm64-v8a | ||
| 151 | + | ||
| 152 | +# 定义路径 | ||
| 153 | +export MNN_SRC=${PROJECT_ROOT}/third-party/mnn | ||
| 154 | +export MNN_BUILD_DIR=${MNN_SRC}/build_harmonyos_${OHOS_ARCH} | ||
| 155 | +export OHOS_CMAKE=${OHOS_SDK_HOME}/native/build-tools/cmake/bin/cmake | ||
| 156 | +export OHOS_NINJA=${OHOS_SDK_HOME}/native/build-tools/cmake/bin/ninja | ||
| 157 | + | ||
| 158 | +# 配置 CMake | ||
| 159 | +mkdir -p "${MNN_BUILD_DIR}" | ||
| 160 | +cd "${MNN_BUILD_DIR}" | ||
| 161 | + | ||
| 162 | +"${OHOS_CMAKE}" "${MNN_SRC}" \ | ||
| 163 | + -DCMAKE_TOOLCHAIN_FILE="${OHOS_SDK_HOME}/native/build/cmake/ohos.toolchain.cmake" \ | ||
| 164 | + -DCMAKE_BUILD_TYPE=Release \ | ||
| 165 | + -DOHOS_ARCH="${OHOS_ARCH}" \ | ||
| 166 | + -DOHOS_STL=c++_static \ | ||
| 167 | + -DMNN_USE_LOGCAT=true \ | ||
| 168 | + -DMNN_BUILD_BENCHMARK=ON \ | ||
| 169 | + -DMNN_USE_SSE=OFF \ | ||
| 170 | + -DMNN_SUPPORT_BF16=OFF \ | ||
| 171 | + -DMNN_BUILD_TEST=ON \ | ||
| 172 | + -DMNN_BUILD_LLM=ON \ | ||
| 173 | + -DMNN_SUPPORT_TRANSFORMER_FUSE=ON \ | ||
| 174 | + -DOHOS_PLATFORM_LEVEL=20 \ | ||
| 175 | + -DNATIVE_LIBRARY_OUTPUT=. \ | ||
| 176 | + -DNATIVE_INCLUDE_OUTPUT=. \ | ||
| 177 | + -G Ninja | ||
| 178 | + | ||
| 179 | +# 编译(构建 MNN 和 llm 目标) | ||
| 180 | +"${OHOS_CMAKE}" --build "${MNN_BUILD_DIR}" --target MNN llm -j8 | ||
| 181 | +``` | ||
| 182 | + | ||
| 183 | +**说明**: | ||
| 184 | + | ||
| 185 | +- 使用 HarmonyOS SDK 自带的 CMake 和 Ninja 工具 | ||
| 186 | +- `OHOS_ARCH` 可选值:`arm64-v8a`、`armeabi-v7a`、`x86_64` | ||
| 187 | +- `MNN_BUILD_LLM=ON` 启用 LLM 支持(SmartServe 需要) | ||
| 188 | +- `OHOS_STL=c++_static` 使用静态 C++ 标准库 | ||
| 189 | +- `--target MNN llm` 只构建必要的 MNN 核心和 LLM 模块 | ||
| 190 | + | ||
| 191 | +构建产物位于 `${MNN_BUILD_DIR}/`: | ||
| 192 | + | ||
| 193 | +- `libMNN.so` — MNN 核心库 | ||
| 194 | +- `libllm.so` — LLM 扩展 | ||
| 195 | +- `libMNN_Express.so` — Express API | ||
| 196 | + | ||
| 197 | +### 2.6 编译 SmartServe SDK | ||
| 198 | + | ||
| 199 | +编译 SmartServe SDK 主库: | ||
| 200 | + | ||
| 201 | +```bash | ||
| 202 | +./scripts/harmonyos/build_sdk.sh \ | ||
| 203 | + --sdk $OHOS_SDK_HOME \ | ||
| 204 | + --arch arm64-v8a \ | ||
| 205 | + --release \ | ||
| 206 | + --with-mnn \ | ||
| 207 | + --with-gewu \ | ||
| 208 | + --clean | ||
| 209 | +``` | ||
| 210 | + | ||
| 211 | +**参数说明**: | ||
| 212 | + | ||
| 213 | +- `--sdk` — HarmonyOS SDK 根目录 | ||
| 214 | +- `--arch` — 目标架构(`arm64-v8a`、`armeabi-v7a` 或 `x86_64`) | ||
| 215 | +- `--release` — Release 模式编译(默认,可选 `--debug`) | ||
| 216 | +- `--with-mnn` — 启用 MNN 插件(默认开启) | ||
| 217 | +- `--with-gewu` — 启用 HarmonyOS Gewu 插件(默认开启) | ||
| 218 | +- `--clean` — 清理构建目录 | ||
| 219 | + | ||
| 220 | +构建产物位于 `out/harmonyos/arm64-v8a/`: | ||
| 221 | + | ||
| 222 | +``` | ||
| 223 | +out/harmonyos/arm64-v8a/ | ||
| 224 | +├── lib/ | ||
| 225 | +│ └── libgewu_smartserve.a # SDK 静态库 | ||
| 226 | +└── include/ | ||
| 227 | + ├── gewu_smartserve/ | ||
| 228 | + │ ├── core.h # C ABI 核心接口 | ||
| 229 | + │ └── engine.h # 引擎扩展接口 | ||
| 230 | + └── smartserve/ | ||
| 231 | + └── smartserve.h # C++ API(可选) | ||
| 232 | +``` | ||
| 233 | + | ||
| 234 | +### 2.7 打包 HAR | ||
| 235 | + | ||
| 236 | +SmartServe 提供两种 HAR 打包方式,适合不同的使用场景: | ||
| 237 | + | ||
| 238 | +#### 方式 1:静态库 HAR(轻量级,需手动配置) | ||
| 239 | + | ||
| 240 | +**适用场景**: | ||
| 241 | + | ||
| 242 | +- 高级开发者需要完全控制链接过程 | ||
| 243 | +- 项目已有复杂的原生构建系统 | ||
| 244 | +- 需要最小化 HAR 包体积 | ||
| 245 | + | ||
| 246 | +**打包命令**: | ||
| 247 | + | ||
| 248 | +```bash | ||
| 249 | +./scripts/harmonyos/pack_har.sh \ | ||
| 250 | + --sdk-dir out/harmonyos/arm64-v8a \ | ||
| 251 | + --output out/gewu-smartserve-1.0.0-ohos.har \ | ||
| 252 | + --arch arm64-v8a | ||
| 253 | +``` | ||
| 254 | + | ||
| 255 | +**HAR 包结构**: | ||
| 256 | + | ||
| 257 | +``` | ||
| 258 | +gewu-smartserve-1.0.0-ohos.har | ||
| 259 | +├── module.json5 # HAR 模块元数据 | ||
| 260 | +├── oh-package.json5 # 包描述符 | ||
| 261 | +├── libs/ | ||
| 262 | +│ └── arm64-v8a/ | ||
| 263 | +│ └── libgewu_smartserve.a # 静态库(~280KB,薄包装) | ||
| 264 | +└── headers/ | ||
| 265 | + ├── gewu_smartserve/ | ||
| 266 | + │ ├── core.h # C ABI 核心接口 | ||
| 267 | + │ └── engine.h # 引擎扩展接口 | ||
| 268 | + └── smartserve/ | ||
| 269 | + └── smartserve.h # C++ API | ||
| 270 | +``` | ||
| 271 | + | ||
| 272 | +**集成步骤**: | ||
| 273 | + | ||
| 274 | +1. 复制 HAR 到项目: | ||
| 275 | + | ||
| 276 | +```bash | ||
| 277 | +cp out/gewu-smartserve-1.0.0-ohos.har ~/MyHarmonyApp/entry/libs/ | ||
| 278 | +``` | ||
| 279 | + | ||
| 280 | +1. 在 `oh-package.json5` 中声明依赖: | ||
| 281 | + | ||
| 282 | +```json5 | ||
| 283 | +{ | ||
| 284 | + "dependencies": { | ||
| 285 | + "@huawei/gewu-smartserve": "file:./libs/gewu-smartserve-1.0.0-ohos.har" | ||
| 286 | + } | ||
| 287 | +} | ||
| 288 | +``` | ||
| 289 | + | ||
| 290 | +1. 在应用的 `CMakeLists.txt` 中手动链接所有依赖: | ||
| 291 | + | ||
| 292 | +```cmake | ||
| 293 | +# 引入 HAR 头文件 | ||
| 294 | +include_directories( | ||
| 295 | + ${CMAKE_CURRENT_SOURCE_DIR}/../oh_modules/@huawei/gewu-smartserve/headers | ||
| 296 | +) | ||
| 297 | + | ||
| 298 | +add_library(myapp_native SHARED src/main.cpp) | ||
| 299 | + | ||
| 300 | +# 链接 SmartServe 静态库 | ||
| 301 | +target_link_libraries(myapp_native | ||
| 302 | + # HAR 中的主库 | ||
| 303 | + ${CMAKE_CURRENT_SOURCE_DIR}/../oh_modules/@huawei/gewu-smartserve/libs/arm64-v8a/libgewu_smartserve.a | ||
| 304 | + | ||
| 305 | + # 需要手动添加的依赖 | ||
| 306 | + # - SmartServe core 和 plugins(已包含在 .a 中) | ||
| 307 | + # - MNN 共享库(需单独提供) | ||
| 308 | + # - OpenSSL(需单独提供) | ||
| 309 | + # - curl(需单独提供) | ||
| 310 | + | ||
| 311 | + # 系统库 | ||
| 312 | + log | ||
| 313 | + z | ||
| 314 | +) | ||
| 315 | +``` | ||
| 316 | + | ||
| 317 | +1. 在 C++ 代码中使用: | ||
| 318 | + | ||
| 319 | +```cpp | ||
| 320 | +#include "gewu_smartserve/core.h" | ||
| 321 | + | ||
| 322 | +// 初始化 | ||
| 323 | +GewuSmartServeInitialize(); | ||
| 324 | + | ||
| 325 | +// 聊天请求 | ||
| 326 | +const char* request_json = "{\"model\":\"qwen3_5\",\"messages\":[{\"role\":\"user\",\"content\":\"你好\"}]}"; | ||
| 327 | +GewuSmartServeRequest request = NULL; | ||
| 328 | +GewuSmartServeChatCompletions(request_json, &request); | ||
| 329 | + | ||
| 330 | +char* response_json = NULL; | ||
| 331 | +GewuSmartServeGetResponse(request, &response_json); | ||
| 332 | +// 使用 response_json... | ||
| 333 | +GewuSmartServeFree(response_json); | ||
| 334 | +GewuSmartServeDestroyRequest(request); | ||
| 335 | +``` | ||
| 336 | + | ||
| 337 | +**注意**:此方式的 `.a` 文件是薄包装,不包含所有依赖。App 需要自行提供 MNN、OpenSSL、curl 等依赖库。 | ||
| 338 | + | ||
| 339 | +*** | ||
| 340 | + | ||
| 341 | +#### 方式 2:NAPI HAR(推荐,开箱即用) | ||
| 342 | + | ||
| 343 | +**适用场景**: | ||
| 344 | + | ||
| 345 | +- 希望像 Android AAR 一样开箱即用 | ||
| 346 | +- 使用 ArkTS/TypeScript 开发,不想直接调用 C API | ||
| 347 | +- 需要 Promise/async-await 风格的异步 API | ||
| 348 | + | ||
| 349 | +**打包命令**: | ||
| 350 | + | ||
| 351 | +```bash | ||
| 352 | +./scripts/harmonyos/build_har_napi.sh \ | ||
| 353 | + --sdk-build-dir build/harmonyos/arm64-v8a \ | ||
| 354 | + --sdk-install-dir out/harmonyos/arm64-v8a \ | ||
| 355 | + --mnn-root third-party/mnn \ | ||
| 356 | + --output out/gewu-smartserve-1.0.0-napi.har \ | ||
| 357 | + --clean \ | ||
| 358 | + -j 8 | ||
| 359 | +``` | ||
| 360 | + | ||
| 361 | +**参数说明**: | ||
| 362 | + | ||
| 363 | +- `--sdk-build-dir` — SDK CMake 构建目录(包含静态库 .a) | ||
| 364 | +- `--sdk-install-dir` — SDK 安装目录(包含头文件) | ||
| 365 | +- `--mnn-root` — MNN 源码根目录 | ||
| 366 | +- `--output` — 输出 HAR 文件路径 | ||
| 367 | +- `--clean` — 清理 NAPI 构建目录 | ||
| 368 | +- `-j` — 并行编译任务数 | ||
| 369 | + | ||
| 370 | +**HAR 包结构**: | ||
| 371 | + | ||
| 372 | +``` | ||
| 373 | +gewu-smartserve-1.0.0-napi.har | ||
| 374 | +├── oh-package.json5 # 包元数据 | ||
| 375 | +├── build-profile.json5 # 构建配置 | ||
| 376 | +├── hvigorfile.ts # hvigor 构建脚本 | ||
| 377 | +├── src/main/ | ||
| 378 | +│ ├── module.json5 # 模块元数据 | ||
| 379 | +│ └── ets/ | ||
| 380 | +│ ├── index.d.ts # TypeScript 类型声明 | ||
| 381 | +│ └── Index.ets # ArkTS 实现 | ||
| 382 | +└── libs/ | ||
| 383 | + └── arm64-v8a/ | ||
| 384 | + ├── libsmartserve.so # NAPI 桥接层(自包含,~7.8MB) | ||
| 385 | + ├── libMNN.so # MNN 核心 | ||
| 386 | + ├── libllm.so # MNN LLM 扩展 | ||
| 387 | + └── libMNN_Express.so # MNN Express API | ||
| 388 | +``` | ||
| 389 | + | ||
| 390 | +**关键特性**: | ||
| 391 | + | ||
| 392 | +- `libsmartserve.so` 通过 `-Wl,--whole-archive` 包含所有 SmartServe 静态依赖(core、plugins、curl、OpenSSL) | ||
| 393 | +- 提供完整的 TypeScript 类型定义和 Promise API | ||
| 394 | +- DevEco Studio 自动处理 .so 加载,无需手动配置 CMakeLists.txt | ||
| 395 | + | ||
| 396 | +**集成步骤**: | ||
| 397 | + | ||
| 398 | +1. 复制 HAR 到项目: | ||
| 399 | + | ||
| 400 | +```bash | ||
| 401 | +cp out/gewu-smartserve-1.0.0-napi.har ~/MyHarmonyApp/entry/libs/ | ||
| 402 | +``` | ||
| 403 | + | ||
| 404 | +1. 在 `oh-package.json5` 中声明依赖: | ||
| 405 | + | ||
| 406 | +```json5 | ||
| 407 | +{ | ||
| 408 | + "dependencies": { | ||
| 409 | + "@huawei/gewu-smartserve": "file:./libs/gewu-smartserve-1.0.0-napi.har" | ||
| 410 | + } | ||
| 411 | +} | ||
| 412 | +``` | ||
| 413 | + | ||
| 414 | +1. 在 ArkTS 代码中使用: | ||
| 415 | + | ||
| 416 | +**初始化 SDK**: | ||
| 417 | + | ||
| 418 | +```typescript | ||
| 419 | +import { initialize, setModelsDirectory, setModelsConfigJson } from '@huawei/gewu-smartserve'; | ||
| 420 | + | ||
| 421 | +// 应用启动时初始化 | ||
| 422 | +initialize(); | ||
| 423 | + | ||
| 424 | +// 配置模型目录 | ||
| 425 | +const modelsDir = getContext().filesDir + '/models'; | ||
| 426 | +setModelsDirectory(modelsDir); | ||
| 427 | + | ||
| 428 | +// 配置模型列表 | ||
| 429 | +const modelsConfig = JSON.stringify({ | ||
| 430 | + models: [ | ||
| 431 | + { | ||
| 432 | + id: "qwen3_5", | ||
| 433 | + name: "Qwen 3.5 1B", | ||
| 434 | + engine: "mnn", | ||
| 435 | + model_path: modelsDir + "/qwen3_5" | ||
| 436 | + } | ||
| 437 | + ] | ||
| 438 | +}); | ||
| 439 | +setModelsConfigJson(modelsConfig); | ||
| 440 | +``` | ||
| 441 | + | ||
| 442 | +**发送聊天请求**: | ||
| 443 | + | ||
| 444 | +```typescript | ||
| 445 | +import { chat, ChatCompletionRequest } from '@huawei/gewu-smartserve'; | ||
| 446 | + | ||
| 447 | +async function sendMessage(userInput: string): Promise<string> { | ||
| 448 | + const request: ChatCompletionRequest = { | ||
| 449 | + model: "qwen3_5", | ||
| 450 | + messages: [ | ||
| 451 | + { role: "user", content: userInput } | ||
| 452 | + ], | ||
| 453 | + stream: false | ||
| 454 | + }; | ||
| 455 | + | ||
| 456 | + try { | ||
| 457 | + const response = await chat(request); | ||
| 458 | + return response.choices[0].message.content; | ||
| 459 | + } catch (error) { | ||
| 460 | + console.error("Chat failed:", error); | ||
| 461 | + throw error; | ||
| 462 | + } | ||
| 463 | +} | ||
| 464 | + | ||
| 465 | +// 使用示例 | ||
| 466 | +sendMessage("你好,介绍一下华为").then(reply => { | ||
| 467 | + console.log("AI 回复:", reply); | ||
| 468 | +}); | ||
| 469 | +``` | ||
| 470 | + | ||
| 471 | +**列出可用模型**: | ||
| 472 | + | ||
| 473 | +```typescript | ||
| 474 | +import { listModels, ModelInfo } from '@huawei/gewu-smartserve'; | ||
| 475 | + | ||
| 476 | +async function showAvailableModels() { | ||
| 477 | + const models: ModelInfo[] = await listModels(); | ||
| 478 | + models.forEach(model => { | ||
| 479 | + console.log(`模型: ${model.name} (${model.id})`); | ||
| 480 | + console.log(`引擎: ${model.engine}`); | ||
| 481 | + console.log(`状态: ${model.status || 'unknown'}`); | ||
| 482 | + }); | ||
| 483 | +} | ||
| 484 | +``` | ||
| 485 | + | ||
| 486 | +**下载模型**: | ||
| 487 | + | ||
| 488 | +```typescript | ||
| 489 | +import { downloadModel } from '@huawei/gewu-smartserve'; | ||
| 490 | + | ||
| 491 | +async function downloadQwen() { | ||
| 492 | + try { | ||
| 493 | + await downloadModel("qwen3_5"); | ||
| 494 | + console.log("模型下载完成"); | ||
| 495 | + } catch (error) { | ||
| 496 | + console.error("下载失败:", error); | ||
| 497 | + } | ||
| 498 | +} | ||
| 499 | +``` | ||
| 500 | + | ||
| 501 | +1. 在 DevEco Studio 中构建和运行: | ||
| 502 | + - **Sync** — 同步依赖(File → Sync Project) | ||
| 503 | + - **Build** — 构建应用(Build → Make Project) | ||
| 504 | + - **Run** — 在 HarmonyOS 设备或模拟器上运行 | ||
| 505 | + | ||
| 506 | +DevEco 会自动将 `libs/arm64-v8a/*.so` 打包到 HAP,并在运行时加载 NAPI 模块。 | ||
| 507 | + | ||
| 508 | +**ArkTS API 参考**: | ||
| 509 | + | ||
| 510 | +```typescript | ||
| 511 | +// 初始化与配置 | ||
| 512 | +function initialize(): void; | ||
| 513 | +function finalize(): void; | ||
| 514 | +function setModelsDirectory(modelsDir: string): void; | ||
| 515 | +function setModelsConfigPath(configPath: string): void; | ||
| 516 | +function setModelsConfigJson(jsonContent: string): void; | ||
| 517 | + | ||
| 518 | +// 聊天推理 | ||
| 519 | +function chat(request: ChatCompletionRequest | string): Promise<ChatCompletionResponse>; | ||
| 520 | + | ||
| 521 | +interface ChatCompletionRequest { | ||
| 522 | + model: string; // 模型 ID | ||
| 523 | + messages: ChatMessage[]; // 消息历史 | ||
| 524 | + stream?: boolean; // 是否流式输出 | ||
| 525 | + temperature?: number; // 温度参数(0-2) | ||
| 526 | + top_p?: number; // 核采样参数 | ||
| 527 | + max_tokens?: number; // 最大生成 token 数 | ||
| 528 | +} | ||
| 529 | + | ||
| 530 | +interface ChatMessage { | ||
| 531 | + role: 'system' | 'user' | 'assistant'; | ||
| 532 | + content: string; | ||
| 533 | +} | ||
| 534 | + | ||
| 535 | +// 模型管理 | ||
| 536 | +function listModels(): Promise<ModelInfo[]>; | ||
| 537 | +function downloadModel(modelId: string): Promise<void>; | ||
| 538 | +function pauseDownload(modelId: string): Promise<void>; | ||
| 539 | +function unloadModel(modelId: string): Promise<void>; | ||
| 540 | +function deleteModel(modelId: string): Promise<void>; | ||
| 541 | + | ||
| 542 | +// 引擎管理 | ||
| 543 | +function listEngines(): Promise<EngineInfo[]>; | ||
| 544 | +``` | ||
| 545 | + | ||
| 546 | +*** | ||
| 547 | + | ||
| 548 | +#### 两种方式对比 | ||
| 549 | + | ||
| 550 | +| 特性 | 静态库 HAR | NAPI HAR | | ||
| 551 | +| ---------- | ----------------------- | -------------------------- | | ||
| 552 | +| **包体积** | 小(\~280KB) | 大(\~6.9MB,包含所有依赖) | | ||
| 553 | +| **集成复杂度** | 高(需手动配置 CMakeLists.txt) | 低(自动处理) | | ||
| 554 | +| **API 类型** | C ABI | ArkTS/TypeScript + Promise | | ||
| 555 | +| **依赖管理** | 手动提供 MNN、OpenSSL 等 | 自动包含所有依赖 | | ||
| 556 | +| **适用场景** | 高级开发者、自定义构建 | 快速集成、ArkTS 项目 | | ||
| 557 | +| **使用体验** | 类似传统 C 库 | 类似 Android AAR | | ||
| 558 | + | ||
| 559 | +**推荐选择**: | ||
| 560 | + | ||
| 561 | +- 如果你的项目使用 ArkTS 开发,优先选择 **NAPI HAR** | ||
| 562 | +- 如果你需要完全控制链接过程或最小化包体积,选择 **静态库 HAR** | ||
| 563 | + | ||
| 564 | +*** | ||
| 565 | + | ||
| 566 | +## 3. Android SDK | ||
| 64 | 567 | ||
| 65 | Android SDK 当前推荐产物是 AAR,内部包含 `jni/<abi>/*.so` 与公开头文件。 | 568 | Android SDK 当前推荐产物是 AAR,内部包含 `jni/<abi>/*.so` 与公开头文件。 |
| 66 | 569 | ||
| @@ -68,7 +571,7 @@ Android SDK 当前推荐产物是 AAR,内部包含 `jni/<abi>/*.so` 与公开 | |||
| 68 | 571 | ||
| 69 | `headers/gewu_smartserve/engine.h` 会随 AAR 一起提供,它服务的是外部 C ABI 引擎扩展,例如 App 或三方 `.so` 注册 `"androidgenai"`、`"applefm"`、`"harmonygenai"` 或自研引擎。若 engine name 已由内置插件注册,外部注册会失败。 | 572 | `headers/gewu_smartserve/engine.h` 会随 AAR 一起提供,它服务的是外部 C ABI 引擎扩展,例如 App 或三方 `.so` 注册 `"androidgenai"`、`"applefm"`、`"harmonygenai"` 或自研引擎。若 engine name 已由内置插件注册,外部注册会失败。 |
| 70 | 573 | ||
| 71 | -### 2.1 编译 MNN 与 SDK | 574 | +### 3.1 编译 MNN 与 SDK |
| 72 | 575 | ||
| 73 | `scripts/android/build_sdk.sh` 通过 `--abi` 参数选择目标架构,**一次只构建一个 ABI**(默认 `arm64-v8a`)。当前支持 `arm64-v8a` 与 `armeabi-v7a`。SDK 依赖的 MNN 也要按同一 ABI 交叉编译,请按「编该 ABI 的 MNN → 编该 ABI 的 SDK」逐个架构完成;要同时得到 arm64 与 arm32,就把整轮流程各跑一遍。 | 576 | `scripts/android/build_sdk.sh` 通过 `--abi` 参数选择目标架构,**一次只构建一个 ABI**(默认 `arm64-v8a`)。当前支持 `arm64-v8a` 与 `armeabi-v7a`。SDK 依赖的 MNN 也要按同一 ABI 交叉编译,请按「编该 ABI 的 MNN → 编该 ABI 的 SDK」逐个架构完成;要同时得到 arm64 与 arm32,就把整轮流程各跑一遍。 |
| 74 | 577 | ||
| @@ -82,7 +585,7 @@ Android SDK 当前推荐产物是 AAR,内部包含 `jni/<abi>/*.so` 与公开 | |||
| 82 | 关键约束: | 585 | 关键约束: |
| 83 | 586 | ||
| 84 | - **`MNN_ARM82`**:`arm64-v8a` 用 `ON`(启用 fp16 加速);`armeabi-v7a` 必须用 `OFF`(ARMv7 无 fp16/dotprod 指令,开启会编译失败)。 | 587 | - **`MNN_ARM82`**:`arm64-v8a` 用 `ON`(启用 fp16 加速);`armeabi-v7a` 必须用 `OFF`(ARMv7 无 fp16/dotprod 指令,开启会编译失败)。 |
| 85 | -- **切换 ABI 前必须 `rm -rf build-android`**:MNN 的 CMake 缓存只保存一个 ABI,不清空会把上个 ABI 的编译参数(如 `-mfloat-abi`/`-mfpu`)串入,导致 `unsupported option` 报错。`build-android` 一次只保存一个 ABI 的产物,因此不要指望同时保留两个 ABI 的 MNN 构建目录。 | 588 | +- **切换 ABI 前必须** **`rm -rf build-android`**:MNN 的 CMake 缓存只保存一个 ABI,不清空会把上个 ABI 的编译参数(如 `-mfloat-abi`/`-mfpu`)串入,导致 `unsupported option` 报错。`build-android` 一次只保存一个 ABI 的产物,因此不要指望同时保留两个 ABI 的 MNN 构建目录。 |
| 86 | 589 | ||
| 87 | #### arm64-v8a | 590 | #### arm64-v8a |
| 88 | 591 | ||
| @@ -112,7 +615,7 @@ out/android/gewu_sdk_arm64/include/gewu_smartserve/core.h | |||
| 112 | out/android/gewu_sdk_arm64/include/gewu_smartserve/engine.h | 615 | out/android/gewu_sdk_arm64/include/gewu_smartserve/engine.h |
| 113 | ``` | 616 | ``` |
| 114 | 617 | ||
| 115 | -### 2.2 打包 AAR | 618 | +### 3.2 打包 AAR |
| 116 | 619 | ||
| 117 | ```bash | 620 | ```bash |
| 118 | ./scripts/android/pack_aar.sh \ | 621 | ./scripts/android/pack_aar.sh \ |
| @@ -129,7 +632,7 @@ out/android/gewu_sdk_arm64/include/gewu_smartserve/engine.h | |||
| 129 | --copy-to /path/to/gewuai/platforms/android/example/gewu_smartserve/app/libs | 632 | --copy-to /path/to/gewuai/platforms/android/example/gewu_smartserve/app/libs |
| 130 | ``` | 633 | ``` |
| 131 | 634 | ||
| 132 | -### 2.3 App 侧操作 | 635 | +### 3.3 App 侧操作 |
| 133 | 636 | ||
| 134 | Android App 侧只需要消费 AAR: | 637 | Android App 侧只需要消费 AAR: |
| 135 | 638 | ||
| @@ -147,9 +650,9 @@ cd /path/to/gewuai/platforms/android/example/gewu_smartserve | |||
| 147 | 650 | ||
| 148 | 运行时如果出现 `libMNN_CL.so not found`,说明 AAR 中缺少 MNN OpenCL 后端库;请确认使用的是当前 `scripts/android/build_sdk.sh` 重新生成的 SDK 和 AAR。 | 651 | 运行时如果出现 `libMNN_CL.so not found`,说明 AAR 中缺少 MNN OpenCL 后端库;请确认使用的是当前 `scripts/android/build_sdk.sh` 重新生成的 SDK 和 AAR。 |
| 149 | 652 | ||
| 150 | ---- | 653 | +*** |
| 151 | 654 | ||
| 152 | -## 3. 系统自带 GenAI 引擎注册 | 655 | +## 4. 系统自带 GenAI 引擎注册 |
| 153 | 656 | ||
| 154 | 系统自带 GenAI 能力不要让 App 直接调用平台模型 API。推荐把平台能力封装成 SmartServe 外部 C ABI engine,然后 App 只调用 `SmartServe*` API: | 657 | 系统自带 GenAI 能力不要让 App 直接调用平台模型 API。推荐把平台能力封装成 SmartServe 外部 C ABI engine,然后 App 只调用 `SmartServe*` API: |
| 155 | 658 | ||
| @@ -159,11 +662,11 @@ App -> SmartServe API -> registered system engine -> platform GenAI runtime | |||
| 159 | 662 | ||
| 160 | 常用 engine name 约定: | 663 | 常用 engine name 约定: |
| 161 | 664 | ||
| 162 | -| 平台能力 | 推荐 engine name | 说明 | | 665 | +| 平台能力 | 推荐 engine name | 说明 | |
| 163 | -|------|------|------| | 666 | +| ----------------------------- | -------------- | -------------------------------------- | |
| 164 | -| Apple Foundation Models (AFM) | `applefm` | iOS / macOS 系统模型适配器 | | 667 | +| Apple Foundation Models (AFM) | `applefm` | iOS / macOS 系统模型适配器 | |
| 165 | -| Android Gen AI / ML Kit GenAI | `androidgenai` | Android 系统或 Google GenAI 能力适配器 | | 668 | +| Android Gen AI / ML Kit GenAI | `androidgenai` | Android 系统或 Google GenAI 能力适配器 | |
| 166 | -| Harmony Gen AI | `harmonygenai` | HarmonyOS / OpenHarmony 系统 GenAI 能力适配器 | | 669 | +| Harmony Gen AI | `harmonygenai` | HarmonyOS / OpenHarmony 系统 GenAI 能力适配器 | |
| 167 | 670 | ||
| 168 | engine name 必须和 `models.json` 中的 `engine` 字段一致。系统模型通常没有下载文件,只需要配置模型 ID 和 engine: | 671 | engine name 必须和 `models.json` 中的 `engine` 字段一致。系统模型通常没有下载文件,只需要配置模型 ID 和 engine: |
| 169 | 672 | ||
| @@ -249,13 +752,13 @@ SmartServeDestroyRequest(request); | |||
| 249 | - `chatCompletions()` 和 `getResponse()` 由 engine adapter 内部负责调用 AFM、Android Gen AI 或 Harmony Gen AI 的平台 API,并把结果转换成 SmartServe engine ABI 的响应 JSON。 | 752 | - `chatCompletions()` 和 `getResponse()` 由 engine adapter 内部负责调用 AFM、Android Gen AI 或 Harmony Gen AI 的平台 API,并把结果转换成 SmartServe engine ABI 的响应 JSON。 |
| 250 | - 如果 App 需要查看已注册 engine,可调用 `SmartServeListEngines()`;返回字符串用 `SmartServeFree()` 释放。 | 753 | - 如果 App 需要查看已注册 engine,可调用 `SmartServeListEngines()`;返回字符串用 `SmartServeFree()` 释放。 |
| 251 | 754 | ||
| 252 | ---- | 755 | +*** |
| 253 | 756 | ||
| 254 | -## 4. iOS SDK | 757 | +## 5. iOS SDK |
| 255 | 758 | ||
| 256 | iOS SDK 产物是 CocoaPods 可消费的 `GewuSmartServeSDK.xcframework`。 | 759 | iOS SDK 产物是 CocoaPods 可消费的 `GewuSmartServeSDK.xcframework`。 |
| 257 | 760 | ||
| 258 | -### 4.1 编译 XCFramework | 761 | +### 5.1 编译 XCFramework |
| 259 | 762 | ||
| 260 | 构建设备和模拟器双切片: | 763 | 构建设备和模拟器双切片: |
| 261 | 764 | ||
| @@ -287,7 +790,7 @@ build/ios/install/MNN.xcframework # 仅 --with-mnn 且找到 MNN.framework | |||
| 287 | - `--with-downloads` 需要 iOS OpenSSL;缺少时先执行 `./deps/download_openssl.sh` 和 `./deps/build_openssl.sh --target ios --clean`。 | 790 | - `--with-downloads` 需要 iOS OpenSSL;缺少时先执行 `./deps/download_openssl.sh` 和 `./deps/build_openssl.sh --target ios --clean`。 |
| 288 | - 如果 App 只使用 Apple Foundation Models / 系统引擎,可使用 `--without-mnn`。 | 791 | - 如果 App 只使用 Apple Foundation Models / 系统引擎,可使用 `--without-mnn`。 |
| 289 | 792 | ||
| 290 | -### 4.2 App 侧操作 | 793 | +### 5.2 App 侧操作 |
| 291 | 794 | ||
| 292 | iOS App 通过本仓库根目录的 `GewuSmartServeSDK.podspec` 接入: | 795 | iOS App 通过本仓库根目录的 `GewuSmartServeSDK.podspec` 接入: |
| 293 | 796 | ||
| @@ -311,9 +814,9 @@ open GewuSmartServe.xcworkspace | |||
| 311 | - 真机运行需要本机 Xcode 安装对应 iOS 版本的 platform support;否则即使 SDK 编译成功,Xcode 也可能无法选择该设备。 | 814 | - 真机运行需要本机 Xcode 安装对应 iOS 版本的 platform support;否则即使 SDK 编译成功,Xcode 也可能无法选择该设备。 |
| 312 | - App 侧系统引擎通过 `SmartServeRegisterEngine` 注册,C API 请求/响应保持 OpenAI Chat Completions 兼容格式;不要注册与 SDK 内置插件同名的 engine。 | 815 | - App 侧系统引擎通过 `SmartServeRegisterEngine` 注册,C API 请求/响应保持 OpenAI Chat Completions 兼容格式;不要注册与 SDK 内置插件同名的 engine。 |
| 313 | 816 | ||
| 314 | ---- | 817 | +*** |
| 315 | 818 | ||
| 316 | -## 5. 测试 | 819 | +## 6. 测试 |
| 317 | 820 | ||
| 318 | SDK 相关本地验证推荐使用 `scripts/build_and_test.sh`。 | 821 | SDK 相关本地验证推荐使用 `scripts/build_and_test.sh`。 |
| 319 | 822 | ||
| @@ -0,0 +1,156 @@ | |||
| 1 | +# SmartServe HarmonyOS HAR with NAPI Bridge | ||
| 2 | + | ||
| 3 | +本目录包含 SmartServe SDK 的 HarmonyOS HAR 模块,使用 NAPI 桥接层提供 ArkTS 接口。 | ||
| 4 | + | ||
| 5 | +## 目录结构 | ||
| 6 | + | ||
| 7 | +``` | ||
| 8 | +harmonyos_har_napi/ | ||
| 9 | +├── oh-package.json5 # 包元数据 | ||
| 10 | +├── build-profile.json5 # DevEco 构建配置 | ||
| 11 | +├── hvigorfile.ts # hvigor 构建脚本 | ||
| 12 | +├── src/main/ | ||
| 13 | +│ ├── module.json5 # HAR 模块元数据 | ||
| 14 | +│ ├── cpp/ # NAPI C++ 桥接层 | ||
| 15 | +│ │ ├── CMakeLists.txt # 原生库构建配置 | ||
| 16 | +│ │ └── smartserve_napi.cpp # N-API 绑定实现 | ||
| 17 | +│ └── ets/ # ArkTS 接口层 | ||
| 18 | +│ ├── index.d.ts # TypeScript 类型声明 | ||
| 19 | +│ └── Index.ets # ArkTS 实现 | ||
| 20 | +└── libs/ # 构建产物(自动生成) | ||
| 21 | + └── arm64-v8a/ | ||
| 22 | + ├── libsmartserve.so # NAPI 模块(自包含所有静态依赖) | ||
| 23 | + ├── libMNN.so # MNN 核心 | ||
| 24 | + ├── libllm.so # MNN LLM 扩展 | ||
| 25 | + └── libMNN_Express.so # MNN Express API | ||
| 26 | +``` | ||
| 27 | + | ||
| 28 | +## 构建 | ||
| 29 | + | ||
| 30 | +### 前置条件 | ||
| 31 | + | ||
| 32 | +1. 已编译 HarmonyOS SDK(静态库): | ||
| 33 | + ```bash | ||
| 34 | + ./scripts/harmonyos/build_sdk.sh | ||
| 35 | + ``` | ||
| 36 | + | ||
| 37 | +2. 已编译 MNN 共享库: | ||
| 38 | + ```bash | ||
| 39 | + cd third-party/mnn | ||
| 40 | + ./project/harmony/build.sh | ||
| 41 | + ``` | ||
| 42 | + | ||
| 43 | +3. 设置 HarmonyOS SDK 路径: | ||
| 44 | + ```bash | ||
| 45 | + export OHOS_SDK_NATIVE=~/Library/OpenHarmony/Sdk/20/native | ||
| 46 | + ``` | ||
| 47 | + | ||
| 48 | +### 执行构建 | ||
| 49 | + | ||
| 50 | +```bash | ||
| 51 | +cd /path/to/smartserve | ||
| 52 | +./scripts/harmonyos/build_har_napi.sh | ||
| 53 | +``` | ||
| 54 | + | ||
| 55 | +构建产物:`out/gewu-smartserve-<version>-napi.har` | ||
| 56 | + | ||
| 57 | +## 集成到 HarmonyOS 应用 | ||
| 58 | + | ||
| 59 | +参考 [HarmonyOS HAR with NAPI 集成指南](../../docs/build/harmonyos_har_napi_guide.md)。 | ||
| 60 | + | ||
| 61 | +简要步骤: | ||
| 62 | + | ||
| 63 | +1. 复制 HAR 到项目 `entry/libs/` | ||
| 64 | +2. 在 `oh-package.json5` 中添加依赖 | ||
| 65 | +3. 在 ArkTS 代码中导入并使用: | ||
| 66 | + | ||
| 67 | +```typescript | ||
| 68 | +import { initialize, chat } from '@huawei/gewu-smartserve'; | ||
| 69 | + | ||
| 70 | +initialize(); | ||
| 71 | + | ||
| 72 | +const response = await chat({ | ||
| 73 | + model: "qwen3_5", | ||
| 74 | + messages: [{ role: "user", content: "你好" }] | ||
| 75 | +}); | ||
| 76 | + | ||
| 77 | +console.log(response.choices[0].message.content); | ||
| 78 | +``` | ||
| 79 | + | ||
| 80 | +## 技术细节 | ||
| 81 | + | ||
| 82 | +### NAPI 桥接架构 | ||
| 83 | + | ||
| 84 | +``` | ||
| 85 | +ArkTS App | ||
| 86 | + ↓ | ||
| 87 | +Index.ets (Promise 封装) | ||
| 88 | + ↓ | ||
| 89 | +smartserve_napi.cpp (N-API 绑定) | ||
| 90 | + ↓ | ||
| 91 | +gewu_smartserve/core.h (C ABI) | ||
| 92 | + ↓ | ||
| 93 | +libgewu_smartserve.a (静态库,通过 --whole-archive 链接) | ||
| 94 | + ↓ | ||
| 95 | +smartserve_core + plugins + MNN + curl + OpenSSL | ||
| 96 | +``` | ||
| 97 | + | ||
| 98 | +### 与 Android AAR 的对比 | ||
| 99 | + | ||
| 100 | +| 层次 | Android | HarmonyOS | | ||
| 101 | +|------|---------|-----------| | ||
| 102 | +| 上层 API | Java/Kotlin | ArkTS/TypeScript | | ||
| 103 | +| 桥接层 | JNI | NAPI | | ||
| 104 | +| 原生库 | libgewu_smartserve.so (SHARED) | libsmartserve.so (NAPI, SHARED) | | ||
| 105 | +| 依赖打包 | 动态链接到 .so | --whole-archive 静态链接 | | ||
| 106 | +| 构建系统 | Gradle + AGP | hvigor + DevEco | | ||
| 107 | + | ||
| 108 | +## 开发 | ||
| 109 | + | ||
| 110 | +### 修改 NAPI 绑定 | ||
| 111 | + | ||
| 112 | +编辑 `src/main/cpp/smartserve_napi.cpp`,添加新的 N-API 函数: | ||
| 113 | + | ||
| 114 | +```cpp | ||
| 115 | +static napi_value MyNewFunction(napi_env env, napi_callback_info info) { | ||
| 116 | + // ... 实现 | ||
| 117 | +} | ||
| 118 | + | ||
| 119 | +// 在 Init() 中注册 | ||
| 120 | +napi_property_descriptor desc[] = { | ||
| 121 | + // ... | ||
| 122 | + {"myNewFunction", nullptr, MyNewFunction, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 123 | +}; | ||
| 124 | +``` | ||
| 125 | + | ||
| 126 | +### 修改 ArkTS 接口 | ||
| 127 | + | ||
| 128 | +1. 更新 `src/main/ets/index.d.ts`(类型声明) | ||
| 129 | +2. 更新 `src/main/ets/Index.ets`(实现) | ||
| 130 | + | ||
| 131 | +### 重新构建 | ||
| 132 | + | ||
| 133 | +```bash | ||
| 134 | +./scripts/harmonyos/build_har_napi.sh --clean | ||
| 135 | +``` | ||
| 136 | + | ||
| 137 | +## 故障排除 | ||
| 138 | + | ||
| 139 | +### undefined symbol 错误 | ||
| 140 | + | ||
| 141 | +确保 `CMakeLists.txt` 中使用了 `-Wl,--whole-archive` 链接所有静态库。 | ||
| 142 | + | ||
| 143 | +### libMNN.so not found | ||
| 144 | + | ||
| 145 | +检查 MNN_ROOT 环境变量,确认 MNN 共享库已编译: | ||
| 146 | +```bash | ||
| 147 | +ls third-party/mnn/project/harmony/build_64/libMNN.so | ||
| 148 | +``` | ||
| 149 | + | ||
| 150 | +### NAPI 模块加载失败 | ||
| 151 | + | ||
| 152 | +检查 `oh-package.json5` 中的 `main` 字段指向正确的入口文件。 | ||
| 153 | + | ||
| 154 | +## 许可证 | ||
| 155 | + | ||
| 156 | +Apache-2.0 — 参考根目录 LICENSE 文件。 | ||
| @@ -0,0 +1,10 @@ | |||
| 1 | +{ | ||
| 2 | + "apiType": "stageMode", | ||
| 3 | + "buildOption": { | ||
| 4 | + }, | ||
| 5 | + "targets": [ | ||
| 6 | + { | ||
| 7 | + "name": "default" | ||
| 8 | + } | ||
| 9 | + ] | ||
| 10 | +} | ||
| @@ -0,0 +1,19 @@ | |||
| 1 | +// Copyright 2024 Huawei Technologies Co., Ltd | ||
| 2 | +// | ||
| 3 | +// Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 4 | +// you may not use this file except in compliance with the License. | ||
| 5 | +// You may obtain a copy of the License at | ||
| 6 | +// | ||
| 7 | +// http://www.apache.org/licenses/LICENSE-2.0 | ||
| 8 | +// | ||
| 9 | +// Unless required by applicable law or agreed to in writing, software | ||
| 10 | +// distributed under the License is distributed on an "AS IS" BASIS, | ||
| 11 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 12 | +// See the License for the specific language governing permissions and | ||
| 13 | +// limitations under the License. | ||
| 14 | + | ||
| 15 | +export default { | ||
| 16 | + system: '@ohos/hvigor', | ||
| 17 | + name: 'gewu-smartserve', | ||
| 18 | + version: '1.0.0', | ||
| 19 | +} | ||
| @@ -0,0 +1,11 @@ | |||
| 1 | +{ | ||
| 2 | + "name": "@huawei/gewu-smartserve", | ||
| 3 | + "version": "1.0.0", | ||
| 4 | + "description": "SmartServe SDK for HarmonyOS - AI inference engine with NAPI bridge", | ||
| 5 | + "main": "Index.ets", | ||
| 6 | + "types": "./ets/index.d.ts", | ||
| 7 | + "author": "Huawei", | ||
| 8 | + "license": "Apache-2.0", | ||
| 9 | + "dependencies": {}, | ||
| 10 | + "devDependencies": {} | ||
| 11 | +} | ||
| @@ -0,0 +1,85 @@ | |||
| 1 | +# Copyright 2024 Huawei Technologies Co., Ltd | ||
| 2 | +# | ||
| 3 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 4 | +# you may not use this file except in compliance with the License. | ||
| 5 | +# You may obtain a copy of the License at | ||
| 6 | +# | ||
| 7 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 8 | +# | ||
| 9 | +# Unless required by applicable law or agreed to in writing, software | ||
| 10 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 11 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 12 | +# See the License for the specific language governing permissions and | ||
| 13 | +# limitations under the License. | ||
| 14 | + | ||
| 15 | +cmake_minimum_required(VERSION 3.16) | ||
| 16 | +project(smartserve_napi) | ||
| 17 | + | ||
| 18 | +set(CMAKE_CXX_STANDARD 17) | ||
| 19 | +set(CMAKE_CXX_STANDARD_REQUIRED ON) | ||
| 20 | + | ||
| 21 | +# ── SDK paths (adjust these based on your build layout) ────────────────────── | ||
| 22 | +set(SDK_ROOT "${CMAKE_CURRENT_SOURCE_DIR}/../../../..") | ||
| 23 | +set(SDK_BUILD_DIR "${SDK_ROOT}/build/harmonyos/arm64-v8a") | ||
| 24 | +set(SDK_INSTALL_DIR "${SDK_ROOT}/out/harmonyos/arm64-v8a") | ||
| 25 | +set(MNN_ROOT "${SDK_ROOT}/third-party/mnn") | ||
| 26 | +set(MNN_BUILD_DIR "${MNN_ROOT}/build_harmonyos_arm64-v8a") | ||
| 27 | +set(OPENSSL_ROOT "${SDK_ROOT}/deps/build/harmony/install") | ||
| 28 | + | ||
| 29 | +# ── NAPI shared library ─────────────────────────────────────────────────────── | ||
| 30 | +add_library(smartserve SHARED | ||
| 31 | + smartserve_napi.cpp | ||
| 32 | +) | ||
| 33 | + | ||
| 34 | +# OH_QoS_Gewu* live in the device-private libQoS_gewu.z.so, absent from the public SDK. | ||
| 35 | +# The OHOS toolchain injects --no-undefined; --allow-shlib-undefined does not cover | ||
| 36 | +# archive-sourced references, so -z undefs is required to defer them to runtime. | ||
| 37 | +target_link_options(smartserve PRIVATE | ||
| 38 | + -Wl,-z,undefs | ||
| 39 | + -Wl,--allow-shlib-undefined | ||
| 40 | +) | ||
| 41 | + | ||
| 42 | +# ── Include paths ───────────────────────────────────────────────────────────── | ||
| 43 | +target_include_directories(smartserve PRIVATE | ||
| 44 | + ${SDK_INSTALL_DIR}/include | ||
| 45 | + ${MNN_ROOT}/include | ||
| 46 | +) | ||
| 47 | + | ||
| 48 | +# ── Link all static libs with whole-archive ─────────────────────────────────── | ||
| 49 | +# This ensures all symbols from static libs are bundled into the NAPI .so | ||
| 50 | +# Note: Gewu plugin included but symbols unresolved (requires internal libQoS_gewu.z.so) | ||
| 51 | +target_link_libraries(smartserve PRIVATE | ||
| 52 | + -Wl,--whole-archive | ||
| 53 | + ${SDK_BUILD_DIR}/sdk/libgewu_smartserve.a | ||
| 54 | + ${SDK_BUILD_DIR}/libsmartserve_core.a | ||
| 55 | + ${SDK_BUILD_DIR}/libsmartserve_utils.a | ||
| 56 | + ${SDK_BUILD_DIR}/plugin/gewu/libsmartserve_gewu_plugin.a | ||
| 57 | + ${SDK_BUILD_DIR}/plugin/mnn/libsmartserve_mnn_plugin.a | ||
| 58 | + ${SDK_BUILD_DIR}/plugin/engine_cabi/libsmartserve_engine_cabi.a | ||
| 59 | + ${SDK_BUILD_DIR}/third_party/curl/lib/libcurl.a | ||
| 60 | + ${OPENSSL_ROOT}/lib/libssl.a | ||
| 61 | + ${OPENSSL_ROOT}/lib/libcrypto.a | ||
| 62 | + -Wl,--no-whole-archive | ||
| 63 | +) | ||
| 64 | + | ||
| 65 | +# ── Link MNN shared libs ────────────────────────────────────────────────────── | ||
| 66 | +target_link_libraries(smartserve PRIVATE | ||
| 67 | + ${MNN_BUILD_DIR}/libMNN.so | ||
| 68 | + ${MNN_BUILD_DIR}/libllm.so | ||
| 69 | + ${MNN_BUILD_DIR}/express/libMNN_Express.so | ||
| 70 | +) | ||
| 71 | + | ||
| 72 | +# ── System libs ─────────────────────────────────────────────────────────────── | ||
| 73 | +target_link_libraries(smartserve PRIVATE | ||
| 74 | + ace_napi.z | ||
| 75 | + hilog_ndk.z | ||
| 76 | + z | ||
| 77 | +) | ||
| 78 | + | ||
| 79 | +# ── Strip symbols in Release mode ───────────────────────────────────────────── | ||
| 80 | +if(CMAKE_BUILD_TYPE STREQUAL "Release") | ||
| 81 | + add_custom_command(TARGET smartserve POST_BUILD | ||
| 82 | + COMMAND ${CMAKE_STRIP} --strip-unneeded $<TARGET_FILE:smartserve> | ||
| 83 | + COMMENT "Stripping debug symbols from libsmartserve.so" | ||
| 84 | + ) | ||
| 85 | +endif() | ||
| @@ -0,0 +1,468 @@ | |||
| 1 | +// Copyright 2024 Huawei Technologies Co., Ltd | ||
| 2 | +// | ||
| 3 | +// Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 4 | +// you may not use this file except in compliance with the License. | ||
| 5 | +// You may obtain a copy of the License at | ||
| 6 | +// | ||
| 7 | +// http://www.apache.org/licenses/LICENSE-2.0 | ||
| 8 | +// | ||
| 9 | +// Unless required by applicable law or agreed to in writing, software | ||
| 10 | +// distributed under the License is distributed on an "AS IS" BASIS, | ||
| 11 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 12 | +// See the License for the specific language governing permissions and | ||
| 13 | +// limitations under the License. | ||
| 14 | + | ||
| 15 | + | ||
| 16 | + | ||
| 17 | + | ||
| 18 | + | ||
| 19 | + | ||
| 20 | + | ||
| 21 | +// Request handle registry for ArkTS -> C mapping | ||
| 22 | +static std::mutex g_request_mutex; | ||
| 23 | +static std::map<uint64_t, GewuSmartServeRequest> g_requests; | ||
| 24 | +static uint64_t g_next_request_id = 1; | ||
| 25 | + | ||
| 26 | +static uint64_t RegisterRequest(GewuSmartServeRequest req) { | ||
| 27 | + std::lock_guard<std::mutex> lock(g_request_mutex); | ||
| 28 | + uint64_t id = g_next_request_id++; | ||
| 29 | + g_requests[id] = req; | ||
| 30 | + return id; | ||
| 31 | +} | ||
| 32 | + | ||
| 33 | +static GewuSmartServeRequest GetRequest(uint64_t id) { | ||
| 34 | + std::lock_guard<std::mutex> lock(g_request_mutex); | ||
| 35 | + auto it = g_requests.find(id); | ||
| 36 | + return (it != g_requests.end()) ? it->second : nullptr; | ||
| 37 | +} | ||
| 38 | + | ||
| 39 | +static void UnregisterRequest(uint64_t id) { | ||
| 40 | + std::lock_guard<std::mutex> lock(g_request_mutex); | ||
| 41 | + g_requests.erase(id); | ||
| 42 | +} | ||
| 43 | + | ||
| 44 | +// ── Utility: Create JS Error from GewuSmartServeError ──────────────────────── | ||
| 45 | +static napi_value CreateError(napi_env env, GewuSmartServeError code, const char* context) { | ||
| 46 | + napi_value error; | ||
| 47 | + std::string msg = std::string(context) + " (code: " + std::to_string(code) + ")"; | ||
| 48 | + napi_create_error(env, nullptr, nullptr, &error); | ||
| 49 | + napi_value msg_val; | ||
| 50 | + napi_create_string_utf8(env, msg.c_str(), NAPI_AUTO_LENGTH, &msg_val); | ||
| 51 | + napi_set_named_property(env, error, "message", msg_val); | ||
| 52 | + napi_value code_val; | ||
| 53 | + napi_create_int32(env, code, &code_val); | ||
| 54 | + napi_set_named_property(env, error, "code", code_val); | ||
| 55 | + return error; | ||
| 56 | +} | ||
| 57 | + | ||
| 58 | +// ── N-API: Initialize ───────────────────────────────────────────────────────── | ||
| 59 | +static napi_value Initialize(napi_env env, napi_callback_info info) { | ||
| 60 | + GewuSmartServeError err = GewuSmartServeInitialize(); | ||
| 61 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 62 | + napi_throw(env, CreateError(env, err, "GewuSmartServeInitialize failed")); | ||
| 63 | + return nullptr; | ||
| 64 | + } | ||
| 65 | + napi_value result; | ||
| 66 | + napi_get_undefined(env, &result); | ||
| 67 | + return result; | ||
| 68 | +} | ||
| 69 | + | ||
| 70 | +// ── N-API: Finalize ─────────────────────────────────────────────────────────── | ||
| 71 | +static napi_value Finalize(napi_env env, napi_callback_info info) { | ||
| 72 | + GewuSmartServeError err = GewuSmartServeFinalize(); | ||
| 73 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 74 | + napi_throw(env, CreateError(env, err, "GewuSmartServeFinalize failed")); | ||
| 75 | + return nullptr; | ||
| 76 | + } | ||
| 77 | + napi_value result; | ||
| 78 | + napi_get_undefined(env, &result); | ||
| 79 | + return result; | ||
| 80 | +} | ||
| 81 | + | ||
| 82 | +// ── N-API: ChatCompletions ──────────────────────────────────────────────────── | ||
| 83 | +static napi_value ChatCompletions(napi_env env, napi_callback_info info) { | ||
| 84 | + size_t argc = 1; | ||
| 85 | + napi_value args[1]; | ||
| 86 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 87 | + | ||
| 88 | + if (argc < 1) { | ||
| 89 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: requestJson"); | ||
| 90 | + return nullptr; | ||
| 91 | + } | ||
| 92 | + | ||
| 93 | + size_t str_len; | ||
| 94 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 95 | + std::string request_json(str_len, '\0'); | ||
| 96 | + napi_get_value_string_utf8(env, args[0], &request_json[0], str_len + 1, &str_len); | ||
| 97 | + | ||
| 98 | + GewuSmartServeRequest request = nullptr; | ||
| 99 | + GewuSmartServeError err = GewuSmartServeChatCompletions(request_json.c_str(), &request); | ||
| 100 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 101 | + napi_throw(env, CreateError(env, err, "GewuSmartServeChatCompletions failed")); | ||
| 102 | + return nullptr; | ||
| 103 | + } | ||
| 104 | + | ||
| 105 | + uint64_t request_id = RegisterRequest(request); | ||
| 106 | + napi_value result; | ||
| 107 | + napi_create_bigint_uint64(env, request_id, &result); | ||
| 108 | + return result; | ||
| 109 | +} | ||
| 110 | + | ||
| 111 | +// ── N-API: GetResponse ──────────────────────────────────────────────────────── | ||
| 112 | +static napi_value GetResponse(napi_env env, napi_callback_info info) { | ||
| 113 | + size_t argc = 1; | ||
| 114 | + napi_value args[1]; | ||
| 115 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 116 | + | ||
| 117 | + if (argc < 1) { | ||
| 118 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: requestId"); | ||
| 119 | + return nullptr; | ||
| 120 | + } | ||
| 121 | + | ||
| 122 | + bool lossless; | ||
| 123 | + uint64_t request_id; | ||
| 124 | + napi_get_value_bigint_uint64(env, args[0], &request_id, &lossless); | ||
| 125 | + | ||
| 126 | + GewuSmartServeRequest request = GetRequest(request_id); | ||
| 127 | + if (!request) { | ||
| 128 | + napi_throw_type_error(env, nullptr, "Invalid request ID"); | ||
| 129 | + return nullptr; | ||
| 130 | + } | ||
| 131 | + | ||
| 132 | + char* response_json = nullptr; | ||
| 133 | + GewuSmartServeError err = GewuSmartServeGetResponse(request, &response_json); | ||
| 134 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 135 | + napi_throw(env, CreateError(env, err, "GewuSmartServeGetResponse failed")); | ||
| 136 | + return nullptr; | ||
| 137 | + } | ||
| 138 | + | ||
| 139 | + napi_value result; | ||
| 140 | + napi_create_string_utf8(env, response_json ? response_json : "", NAPI_AUTO_LENGTH, &result); | ||
| 141 | + free(response_json); | ||
| 142 | + return result; | ||
| 143 | +} | ||
| 144 | + | ||
| 145 | +// ── N-API: DestroyRequest ───────────────────────────────────────────────────── | ||
| 146 | +static napi_value DestroyRequest(napi_env env, napi_callback_info info) { | ||
| 147 | + size_t argc = 1; | ||
| 148 | + napi_value args[1]; | ||
| 149 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 150 | + | ||
| 151 | + if (argc < 1) { | ||
| 152 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: requestId"); | ||
| 153 | + return nullptr; | ||
| 154 | + } | ||
| 155 | + | ||
| 156 | + bool lossless; | ||
| 157 | + uint64_t request_id; | ||
| 158 | + napi_get_value_bigint_uint64(env, args[0], &request_id, &lossless); | ||
| 159 | + | ||
| 160 | + GewuSmartServeRequest request = GetRequest(request_id); | ||
| 161 | + if (!request) { | ||
| 162 | + napi_throw_type_error(env, nullptr, "Invalid request ID"); | ||
| 163 | + return nullptr; | ||
| 164 | + } | ||
| 165 | + | ||
| 166 | + GewuSmartServeError err = GewuSmartServeDestroyRequest(request); | ||
| 167 | + UnregisterRequest(request_id); | ||
| 168 | + | ||
| 169 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 170 | + napi_throw(env, CreateError(env, err, "GewuSmartServeDestroyRequest failed")); | ||
| 171 | + return nullptr; | ||
| 172 | + } | ||
| 173 | + | ||
| 174 | + napi_value result; | ||
| 175 | + napi_get_undefined(env, &result); | ||
| 176 | + return result; | ||
| 177 | +} | ||
| 178 | + | ||
| 179 | +// ── N-API: CancelRequest ────────────────────────────────────────────────────── | ||
| 180 | +static napi_value CancelRequest(napi_env env, napi_callback_info info) { | ||
| 181 | + size_t argc = 1; | ||
| 182 | + napi_value args[1]; | ||
| 183 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 184 | + | ||
| 185 | + if (argc < 1) { | ||
| 186 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: requestId"); | ||
| 187 | + return nullptr; | ||
| 188 | + } | ||
| 189 | + | ||
| 190 | + bool lossless; | ||
| 191 | + uint64_t request_id; | ||
| 192 | + napi_get_value_bigint_uint64(env, args[0], &request_id, &lossless); | ||
| 193 | + | ||
| 194 | + GewuSmartServeRequest request = GetRequest(request_id); | ||
| 195 | + if (!request) { | ||
| 196 | + napi_throw_type_error(env, nullptr, "Invalid request ID"); | ||
| 197 | + return nullptr; | ||
| 198 | + } | ||
| 199 | + | ||
| 200 | + GewuSmartServeError err = GewuSmartServeCancelRequest(request); | ||
| 201 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 202 | + napi_throw(env, CreateError(env, err, "GewuSmartServeCancelRequest failed")); | ||
| 203 | + return nullptr; | ||
| 204 | + } | ||
| 205 | + | ||
| 206 | + napi_value result; | ||
| 207 | + napi_get_undefined(env, &result); | ||
| 208 | + return result; | ||
| 209 | +} | ||
| 210 | + | ||
| 211 | +// ── N-API: SetModelsDirectory ───────────────────────────────────────────────── | ||
| 212 | +static napi_value SetModelsDirectory(napi_env env, napi_callback_info info) { | ||
| 213 | + size_t argc = 1; | ||
| 214 | + napi_value args[1]; | ||
| 215 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 216 | + | ||
| 217 | + if (argc < 1) { | ||
| 218 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: modelsDir"); | ||
| 219 | + return nullptr; | ||
| 220 | + } | ||
| 221 | + | ||
| 222 | + size_t str_len; | ||
| 223 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 224 | + std::string models_dir(str_len, '\0'); | ||
| 225 | + napi_get_value_string_utf8(env, args[0], &models_dir[0], str_len + 1, &str_len); | ||
| 226 | + | ||
| 227 | + GewuSmartServeError err = GewuSmartServeSetModelsDirectory(models_dir.c_str()); | ||
| 228 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 229 | + napi_throw(env, CreateError(env, err, "GewuSmartServeSetModelsDirectory failed")); | ||
| 230 | + return nullptr; | ||
| 231 | + } | ||
| 232 | + | ||
| 233 | + napi_value result; | ||
| 234 | + napi_get_undefined(env, &result); | ||
| 235 | + return result; | ||
| 236 | +} | ||
| 237 | + | ||
| 238 | +// ── N-API: SetModelsConfigPath ──────────────────────────────────────────────── | ||
| 239 | +static napi_value SetModelsConfigPath(napi_env env, napi_callback_info info) { | ||
| 240 | + size_t argc = 1; | ||
| 241 | + napi_value args[1]; | ||
| 242 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 243 | + | ||
| 244 | + if (argc < 1) { | ||
| 245 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: configPath"); | ||
| 246 | + return nullptr; | ||
| 247 | + } | ||
| 248 | + | ||
| 249 | + size_t str_len; | ||
| 250 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 251 | + std::string config_path(str_len, '\0'); | ||
| 252 | + napi_get_value_string_utf8(env, args[0], &config_path[0], str_len + 1, &str_len); | ||
| 253 | + | ||
| 254 | + GewuSmartServeError err = GewuSmartServeSetModelsConfigPath(config_path.c_str()); | ||
| 255 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 256 | + napi_throw(env, CreateError(env, err, "GewuSmartServeSetModelsConfigPath failed")); | ||
| 257 | + return nullptr; | ||
| 258 | + } | ||
| 259 | + | ||
| 260 | + napi_value result; | ||
| 261 | + napi_get_undefined(env, &result); | ||
| 262 | + return result; | ||
| 263 | +} | ||
| 264 | + | ||
| 265 | +// ── N-API: SetModelsConfigJson ──────────────────────────────────────────────── | ||
| 266 | +static napi_value SetModelsConfigJson(napi_env env, napi_callback_info info) { | ||
| 267 | + size_t argc = 1; | ||
| 268 | + napi_value args[1]; | ||
| 269 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 270 | + | ||
| 271 | + if (argc < 1) { | ||
| 272 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: jsonContent"); | ||
| 273 | + return nullptr; | ||
| 274 | + } | ||
| 275 | + | ||
| 276 | + size_t str_len; | ||
| 277 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 278 | + std::string json_content(str_len, '\0'); | ||
| 279 | + napi_get_value_string_utf8(env, args[0], &json_content[0], str_len + 1, &str_len); | ||
| 280 | + | ||
| 281 | + GewuSmartServeError err = GewuSmartServeSetModelsConfigJson(json_content.c_str()); | ||
| 282 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 283 | + napi_throw(env, CreateError(env, err, "GewuSmartServeSetModelsConfigJson failed")); | ||
| 284 | + return nullptr; | ||
| 285 | + } | ||
| 286 | + | ||
| 287 | + napi_value result; | ||
| 288 | + napi_get_undefined(env, &result); | ||
| 289 | + return result; | ||
| 290 | +} | ||
| 291 | + | ||
| 292 | +// ── N-API: ListModels ───────────────────────────────────────────────────────── | ||
| 293 | +static napi_value ListModels(napi_env env, napi_callback_info info) { | ||
| 294 | + char* models_json = nullptr; | ||
| 295 | + GewuSmartServeError err = GewuSmartServeListModels(&models_json); | ||
| 296 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 297 | + napi_throw(env, CreateError(env, err, "GewuSmartServeListModels failed")); | ||
| 298 | + return nullptr; | ||
| 299 | + } | ||
| 300 | + | ||
| 301 | + napi_value result; | ||
| 302 | + napi_create_string_utf8(env, models_json ? models_json : "[]", NAPI_AUTO_LENGTH, &result); | ||
| 303 | + free(models_json); | ||
| 304 | + return result; | ||
| 305 | +} | ||
| 306 | + | ||
| 307 | +// ── N-API: ListEngines ──────────────────────────────────────────────────────── | ||
| 308 | +static napi_value ListEngines(napi_env env, napi_callback_info info) { | ||
| 309 | + char* engines_json = nullptr; | ||
| 310 | + GewuSmartServeError err = GewuSmartServeListEngines(&engines_json); | ||
| 311 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 312 | + napi_throw(env, CreateError(env, err, "GewuSmartServeListEngines failed")); | ||
| 313 | + return nullptr; | ||
| 314 | + } | ||
| 315 | + | ||
| 316 | + napi_value result; | ||
| 317 | + napi_create_string_utf8(env, engines_json ? engines_json : "[]", NAPI_AUTO_LENGTH, &result); | ||
| 318 | + free(engines_json); | ||
| 319 | + return result; | ||
| 320 | +} | ||
| 321 | + | ||
| 322 | +// ── N-API: DownloadModel ────────────────────────────────────────────────────── | ||
| 323 | +static napi_value DownloadModel(napi_env env, napi_callback_info info) { | ||
| 324 | + size_t argc = 1; | ||
| 325 | + napi_value args[1]; | ||
| 326 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 327 | + | ||
| 328 | + if (argc < 1) { | ||
| 329 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: modelId"); | ||
| 330 | + return nullptr; | ||
| 331 | + } | ||
| 332 | + | ||
| 333 | + size_t str_len; | ||
| 334 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 335 | + std::string model_id(str_len, '\0'); | ||
| 336 | + napi_get_value_string_utf8(env, args[0], &model_id[0], str_len + 1, &str_len); | ||
| 337 | + | ||
| 338 | + // Note: Progress callback not yet implemented in this version | ||
| 339 | + GewuSmartServeError err = GewuSmartServeDownloadModel(model_id.c_str(), nullptr, nullptr); | ||
| 340 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 341 | + napi_throw(env, CreateError(env, err, "GewuSmartServeDownloadModel failed")); | ||
| 342 | + return nullptr; | ||
| 343 | + } | ||
| 344 | + | ||
| 345 | + napi_value result; | ||
| 346 | + napi_get_undefined(env, &result); | ||
| 347 | + return result; | ||
| 348 | +} | ||
| 349 | + | ||
| 350 | +// ── N-API: PauseDownload ────────────────────────────────────────────────────── | ||
| 351 | +static napi_value PauseDownload(napi_env env, napi_callback_info info) { | ||
| 352 | + size_t argc = 1; | ||
| 353 | + napi_value args[1]; | ||
| 354 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 355 | + | ||
| 356 | + if (argc < 1) { | ||
| 357 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: modelId"); | ||
| 358 | + return nullptr; | ||
| 359 | + } | ||
| 360 | + | ||
| 361 | + size_t str_len; | ||
| 362 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 363 | + std::string model_id(str_len, '\0'); | ||
| 364 | + napi_get_value_string_utf8(env, args[0], &model_id[0], str_len + 1, &str_len); | ||
| 365 | + | ||
| 366 | + GewuSmartServeError err = GewuSmartServePauseDownload(model_id.c_str()); | ||
| 367 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 368 | + napi_throw(env, CreateError(env, err, "GewuSmartServePauseDownload failed")); | ||
| 369 | + return nullptr; | ||
| 370 | + } | ||
| 371 | + | ||
| 372 | + napi_value result; | ||
| 373 | + napi_get_undefined(env, &result); | ||
| 374 | + return result; | ||
| 375 | +} | ||
| 376 | + | ||
| 377 | +// ── N-API: UnloadModel ──────────────────────────────────────────────────────── | ||
| 378 | +static napi_value UnloadModel(napi_env env, napi_callback_info info) { | ||
| 379 | + size_t argc = 1; | ||
| 380 | + napi_value args[1]; | ||
| 381 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 382 | + | ||
| 383 | + if (argc < 1) { | ||
| 384 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: modelId"); | ||
| 385 | + return nullptr; | ||
| 386 | + } | ||
| 387 | + | ||
| 388 | + size_t str_len; | ||
| 389 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 390 | + std::string model_id(str_len, '\0'); | ||
| 391 | + napi_get_value_string_utf8(env, args[0], &model_id[0], str_len + 1, &str_len); | ||
| 392 | + | ||
| 393 | + GewuSmartServeError err = GewuSmartServeUnloadModel(model_id.c_str()); | ||
| 394 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 395 | + napi_throw(env, CreateError(env, err, "GewuSmartServeUnloadModel failed")); | ||
| 396 | + return nullptr; | ||
| 397 | + } | ||
| 398 | + | ||
| 399 | + napi_value result; | ||
| 400 | + napi_get_undefined(env, &result); | ||
| 401 | + return result; | ||
| 402 | +} | ||
| 403 | + | ||
| 404 | +// ── N-API: DeleteModel ──────────────────────────────────────────────────────── | ||
| 405 | +static napi_value DeleteModel(napi_env env, napi_callback_info info) { | ||
| 406 | + size_t argc = 1; | ||
| 407 | + napi_value args[1]; | ||
| 408 | + napi_get_cb_info(env, info, &argc, args, nullptr, nullptr); | ||
| 409 | + | ||
| 410 | + if (argc < 1) { | ||
| 411 | + napi_throw_type_error(env, nullptr, "Expected 1 argument: modelId"); | ||
| 412 | + return nullptr; | ||
| 413 | + } | ||
| 414 | + | ||
| 415 | + size_t str_len; | ||
| 416 | + napi_get_value_string_utf8(env, args[0], nullptr, 0, &str_len); | ||
| 417 | + std::string model_id(str_len, '\0'); | ||
| 418 | + napi_get_value_string_utf8(env, args[0], &model_id[0], str_len + 1, &str_len); | ||
| 419 | + | ||
| 420 | + GewuSmartServeError err = GewuSmartServeDeleteModel(model_id.c_str()); | ||
| 421 | + if (err != GEWU_SMARTSERVE_OK) { | ||
| 422 | + napi_throw(env, CreateError(env, err, "GewuSmartServeDeleteModel failed")); | ||
| 423 | + return nullptr; | ||
| 424 | + } | ||
| 425 | + | ||
| 426 | + napi_value result; | ||
| 427 | + napi_get_undefined(env, &result); | ||
| 428 | + return result; | ||
| 429 | +} | ||
| 430 | + | ||
| 431 | +// ── Module Init ─────────────────────────────────────────────────────────────── | ||
| 432 | +EXTERN_C_START | ||
| 433 | +static napi_value Init(napi_env env, napi_value exports) { | ||
| 434 | + napi_property_descriptor desc[] = { | ||
| 435 | + {"initialize", nullptr, Initialize, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 436 | + {"finalize", nullptr, Finalize, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 437 | + {"chatCompletions", nullptr, ChatCompletions, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 438 | + {"getResponse", nullptr, GetResponse, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 439 | + {"destroyRequest", nullptr, DestroyRequest, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 440 | + {"cancelRequest", nullptr, CancelRequest, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 441 | + {"setModelsDirectory", nullptr, SetModelsDirectory, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 442 | + {"setModelsConfigPath", nullptr, SetModelsConfigPath, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 443 | + {"setModelsConfigJson", nullptr, SetModelsConfigJson, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 444 | + {"listModels", nullptr, ListModels, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 445 | + {"listEngines", nullptr, ListEngines, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 446 | + {"downloadModel", nullptr, DownloadModel, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 447 | + {"pauseDownload", nullptr, PauseDownload, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 448 | + {"unloadModel", nullptr, UnloadModel, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 449 | + {"deleteModel", nullptr, DeleteModel, nullptr, nullptr, nullptr, napi_default, nullptr}, | ||
| 450 | + }; | ||
| 451 | + napi_define_properties(env, exports, sizeof(desc) / sizeof(desc[0]), desc); | ||
| 452 | + return exports; | ||
| 453 | +} | ||
| 454 | +EXTERN_C_END | ||
| 455 | + | ||
| 456 | +static napi_module napi_module_def = { | ||
| 457 | + .nm_version = 1, | ||
| 458 | + .nm_flags = 0, | ||
| 459 | + .nm_filename = nullptr, | ||
| 460 | + .nm_register_func = Init, | ||
| 461 | + .nm_modname = "smartserve", | ||
| 462 | + .nm_priv = nullptr, | ||
| 463 | + .reserved = {nullptr}, | ||
| 464 | +}; | ||
| 465 | + | ||
| 466 | +extern "C" __attribute__((constructor)) void RegisterModule() { | ||
| 467 | + napi_module_register(&napi_module_def); | ||
| 468 | +} | ||
| @@ -0,0 +1,134 @@ | |||
| 1 | +// Copyright 2024 Huawei Technologies Co., Ltd | ||
| 2 | +// | ||
| 3 | +// Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 4 | +// you may not use this file except in compliance with the License. | ||
| 5 | +// You may obtain a copy of the License at | ||
| 6 | +// | ||
| 7 | +// http://www.apache.org/licenses/LICENSE-2.0 | ||
| 8 | +// | ||
| 9 | +// Unless required by applicable law or agreed to in writing, software | ||
| 10 | +// distributed under the License is distributed on an "AS IS" BASIS, | ||
| 11 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 12 | +// See the License for the specific language governing permissions and | ||
| 13 | +// limitations under the License. | ||
| 14 | + | ||
| 15 | +import smartserve from 'libsmartserve.so'; | ||
| 16 | + | ||
| 17 | +export { SmartServeError, ChatMessage, ChatCompletionRequest, ChatCompletionResponse, ModelInfo, EngineInfo } from './index'; | ||
| 18 | + | ||
| 19 | +let initialized = false; | ||
| 20 | + | ||
| 21 | +export function initialize(): void { | ||
| 22 | + if (initialized) { | ||
| 23 | + return; | ||
| 24 | + } | ||
| 25 | + smartserve.initialize(); | ||
| 26 | + initialized = true; | ||
| 27 | +} | ||
| 28 | + | ||
| 29 | +export function finalize(): void { | ||
| 30 | + if (!initialized) { | ||
| 31 | + return; | ||
| 32 | + } | ||
| 33 | + smartserve.finalize(); | ||
| 34 | + initialized = false; | ||
| 35 | +} | ||
| 36 | + | ||
| 37 | +export function setModelsDirectory(modelsDir: string): void { | ||
| 38 | + smartserve.setModelsDirectory(modelsDir); | ||
| 39 | +} | ||
| 40 | + | ||
| 41 | +export function setModelsConfigPath(configPath: string): void { | ||
| 42 | + smartserve.setModelsConfigPath(configPath); | ||
| 43 | +} | ||
| 44 | + | ||
| 45 | +export function setModelsConfigJson(jsonContent: string): void { | ||
| 46 | + smartserve.setModelsConfigJson(jsonContent); | ||
| 47 | +} | ||
| 48 | + | ||
| 49 | +export async function chat(request: any): Promise<any> { | ||
| 50 | + const requestJson = typeof request === 'string' ? request : JSON.stringify(request); | ||
| 51 | + | ||
| 52 | + return new Promise((resolve, reject) => { | ||
| 53 | + try { | ||
| 54 | + const requestId = smartserve.chatCompletions(requestJson); | ||
| 55 | + | ||
| 56 | + // Poll for response (blocking getResponse in real implementation) | ||
| 57 | + const responseJson = smartserve.getResponse(requestId); | ||
| 58 | + smartserve.destroyRequest(requestId); | ||
| 59 | + | ||
| 60 | + const response = JSON.parse(responseJson); | ||
| 61 | + resolve(response); | ||
| 62 | + } catch (error) { | ||
| 63 | + reject(error); | ||
| 64 | + } | ||
| 65 | + }); | ||
| 66 | +} | ||
| 67 | + | ||
| 68 | +export async function listModels(): Promise<any[]> { | ||
| 69 | + return new Promise((resolve, reject) => { | ||
| 70 | + try { | ||
| 71 | + const modelsJson = smartserve.listModels(); | ||
| 72 | + const models = JSON.parse(modelsJson); | ||
| 73 | + resolve(models); | ||
| 74 | + } catch (error) { | ||
| 75 | + reject(error); | ||
| 76 | + } | ||
| 77 | + }); | ||
| 78 | +} | ||
| 79 | + | ||
| 80 | +export async function listEngines(): Promise<any[]> { | ||
| 81 | + return new Promise((resolve, reject) => { | ||
| 82 | + try { | ||
| 83 | + const enginesJson = smartserve.listEngines(); | ||
| 84 | + const engines = JSON.parse(enginesJson); | ||
| 85 | + resolve(engines); | ||
| 86 | + } catch (error) { | ||
| 87 | + reject(error); | ||
| 88 | + } | ||
| 89 | + }); | ||
| 90 | +} | ||
| 91 | + | ||
| 92 | +export async function downloadModel(modelId: string): Promise<void> { | ||
| 93 | + return new Promise((resolve, reject) => { | ||
| 94 | + try { | ||
| 95 | + smartserve.downloadModel(modelId); | ||
| 96 | + resolve(); | ||
| 97 | + } catch (error) { | ||
| 98 | + reject(error); | ||
| 99 | + } | ||
| 100 | + }); | ||
| 101 | +} | ||
| 102 | + | ||
| 103 | +export async function pauseDownload(modelId: string): Promise<void> { | ||
| 104 | + return new Promise((resolve, reject) => { | ||
| 105 | + try { | ||
| 106 | + smartserve.pauseDownload(modelId); | ||
| 107 | + resolve(); | ||
| 108 | + } catch (error) { | ||
| 109 | + reject(error); | ||
| 110 | + } | ||
| 111 | + }); | ||
| 112 | +} | ||
| 113 | + | ||
| 114 | +export async function unloadModel(modelId: string): Promise<void> { | ||
| 115 | + return new Promise((resolve, reject) => { | ||
| 116 | + try { | ||
| 117 | + smartserve.unloadModel(modelId); | ||
| 118 | + resolve(); | ||
| 119 | + } catch (error) { | ||
| 120 | + reject(error); | ||
| 121 | + } | ||
| 122 | + }); | ||
| 123 | +} | ||
| 124 | + | ||
| 125 | +export async function deleteModel(modelId: string): Promise<void> { | ||
| 126 | + return new Promise((resolve, reject) => { | ||
| 127 | + try { | ||
| 128 | + smartserve.deleteModel(modelId); | ||
| 129 | + resolve(); | ||
| 130 | + } catch (error) { | ||
| 131 | + reject(error); | ||
| 132 | + } | ||
| 133 | + }); | ||
| 134 | +} | ||
| @@ -0,0 +1,133 @@ | |||
| 1 | +// Copyright 2024 Huawei Technologies Co., Ltd | ||
| 2 | +// | ||
| 3 | +// Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 4 | +// you may not use this file except in compliance with the License. | ||
| 5 | +// You may obtain a copy of the License at | ||
| 6 | +// | ||
| 7 | +// http://www.apache.org/licenses/LICENSE-2.0 | ||
| 8 | +// | ||
| 9 | +// Unless required by applicable law or agreed to in writing, software | ||
| 10 | +// distributed under the License is distributed on an "AS IS" BASIS, | ||
| 11 | +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 12 | +// See the License for the specific language governing permissions and | ||
| 13 | +// limitations under the License. | ||
| 14 | + | ||
| 15 | +export interface SmartServeError extends Error { | ||
| 16 | + code: number; | ||
| 17 | +} | ||
| 18 | + | ||
| 19 | +export interface ChatMessage { | ||
| 20 | + role: 'system' | 'user' | 'assistant'; | ||
| 21 | + content: string; | ||
| 22 | +} | ||
| 23 | + | ||
| 24 | +export interface ChatCompletionRequest { | ||
| 25 | + model: string; | ||
| 26 | + messages: ChatMessage[]; | ||
| 27 | + stream?: boolean; | ||
| 28 | + temperature?: number; | ||
| 29 | + top_p?: number; | ||
| 30 | + max_tokens?: number; | ||
| 31 | +} | ||
| 32 | + | ||
| 33 | +export interface ChatCompletionResponse { | ||
| 34 | + id: string; | ||
| 35 | + object: string; | ||
| 36 | + created: number; | ||
| 37 | + model: string; | ||
| 38 | + choices: Array<{ | ||
| 39 | + index: number; | ||
| 40 | + message: ChatMessage; | ||
| 41 | + finish_reason: string; | ||
| 42 | + }>; | ||
| 43 | + usage?: { | ||
| 44 | + prompt_tokens: number; | ||
| 45 | + completion_tokens: number; | ||
| 46 | + total_tokens: number; | ||
| 47 | + }; | ||
| 48 | +} | ||
| 49 | + | ||
| 50 | +export interface ModelInfo { | ||
| 51 | + id: string; | ||
| 52 | + name: string; | ||
| 53 | + engine: string; | ||
| 54 | + model_path?: string; | ||
| 55 | + status?: string; | ||
| 56 | +} | ||
| 57 | + | ||
| 58 | +export interface EngineInfo { | ||
| 59 | + name: string; | ||
| 60 | + version: string; | ||
| 61 | +} | ||
| 62 | + | ||
| 63 | +/** | ||
| 64 | + * Initialize the SmartServe SDK | ||
| 65 | + * Must be called before any other SDK functions | ||
| 66 | + */ | ||
| 67 | +export function initialize(): void; | ||
| 68 | + | ||
| 69 | +/** | ||
| 70 | + * Finalize the SmartServe SDK and release resources | ||
| 71 | + */ | ||
| 72 | +export function finalize(): void; | ||
| 73 | + | ||
| 74 | +/** | ||
| 75 | + * Set the directory where models are stored | ||
| 76 | + * modelsDir - Absolute path to models directory | ||
| 77 | + */ | ||
| 78 | +export function setModelsDirectory(modelsDir: string): void; | ||
| 79 | + | ||
| 80 | +/** | ||
| 81 | + * Set the path to models.json configuration file | ||
| 82 | + * configPath - Absolute path to models.json | ||
| 83 | + */ | ||
| 84 | +export function setModelsConfigPath(configPath: string): void; | ||
| 85 | + | ||
| 86 | +/** | ||
| 87 | + * Set models configuration from JSON string | ||
| 88 | + * jsonContent - JSON string containing models configuration | ||
| 89 | + */ | ||
| 90 | +export function setModelsConfigJson(jsonContent: string): void; | ||
| 91 | + | ||
| 92 | +/** | ||
| 93 | + * Send a chat completion request | ||
| 94 | + * request - Chat completion request (as JSON string or object) | ||
| 95 | + * Promise resolving to chat completion response | ||
| 96 | + */ | ||
| 97 | +export function chat(request: ChatCompletionRequest | string): Promise<ChatCompletionResponse>; | ||
| 98 | + | ||
| 99 | +/** | ||
| 100 | + * List all available models | ||
| 101 | + * Promise resolving to array of model information | ||
| 102 | + */ | ||
| 103 | +export function listModels(): Promise<ModelInfo[]>; | ||
| 104 | + | ||
| 105 | +/** | ||
| 106 | + * List all registered engines | ||
| 107 | + * Promise resolving to array of engine information | ||
| 108 | + */ | ||
| 109 | +export function listEngines(): Promise<EngineInfo[]>; | ||
| 110 | + | ||
| 111 | +/** | ||
| 112 | + * Download a model by ID | ||
| 113 | + * modelId - Model identifier | ||
| 114 | + */ | ||
| 115 | +export function downloadModel(modelId: string): Promise<void>; | ||
| 116 | + | ||
| 117 | +/** | ||
| 118 | + * Pause an ongoing model download | ||
| 119 | + * modelId - Model identifier | ||
| 120 | + */ | ||
| 121 | +export function pauseDownload(modelId: string): Promise<void>; | ||
| 122 | + | ||
| 123 | +/** | ||
| 124 | + * Unload a model from memory | ||
| 125 | + * modelId - Model identifier | ||
| 126 | + */ | ||
| 127 | +export function unloadModel(modelId: string): Promise<void>; | ||
| 128 | + | ||
| 129 | +/** | ||
| 130 | + * Delete a model from disk | ||
| 131 | + * modelId - Model identifier | ||
| 132 | + */ | ||
| 133 | +export function deleteModel(modelId: string): Promise<void>; | ||
| @@ -0,0 +1,14 @@ | |||
| 1 | +{ | ||
| 2 | + "module": { | ||
| 3 | + "name": "gewu_smartserve", | ||
| 4 | + "type": "har", | ||
| 5 | + "description": "SmartServe SDK for HarmonyOS - AI inference engine", | ||
| 6 | + "deviceTypes": [ | ||
| 7 | + "default", | ||
| 8 | + "tablet", | ||
| 9 | + "2in1" | ||
| 10 | + ], | ||
| 11 | + "deliveryWithInstall": true, | ||
| 12 | + "pages": "$profile:main_pages" | ||
| 13 | + } | ||
| 14 | +} | ||
| @@ -0,0 +1,159 @@ | |||
| 1 | +#!/usr/bin/env bash | ||
| 2 | +# ============================================================================= | ||
| 3 | +# SmartServe SDK — HarmonyOS HAR with NAPI bridge build script | ||
| 4 | +# | ||
| 5 | +# Creates a DevEco-compatible HAR package with NAPI .so that bundles all | ||
| 6 | +# static dependencies, making it as convenient as Android AAR. | ||
| 7 | +# | ||
| 8 | +# Prerequisites | ||
| 9 | +# • HarmonyOS SDK (OHOS_SDK_NATIVE environment variable) | ||
| 10 | +# • Completed SmartServe SDK build for HarmonyOS (./scripts/harmonyos/build_sdk.sh) | ||
| 11 | +# • MNN shared libraries built for HarmonyOS | ||
| 12 | +# • DevEco Studio hvigor toolchain | ||
| 13 | +# • Node.js (for hvigor) | ||
| 14 | +# | ||
| 15 | +# Usage | ||
| 16 | +# ./scripts/harmonyos/build_har_napi.sh [OPTIONS] | ||
| 17 | +# | ||
| 18 | +# Options | ||
| 19 | +# --sdk-build-dir <dir> SDK build directory (default: build/harmonyos/arm64-v8a) | ||
| 20 | +# --sdk-install-dir <dir> SDK install directory (default: out/harmonyos/arm64-v8a) | ||
| 21 | +# --mnn-root <dir> MNN source root (default: third-party/mnn) | ||
| 22 | +# --output <file> Output HAR path (default: out/gewu-smartserve-<ver>-napi.har) | ||
| 23 | +# --clean Clean before building | ||
| 24 | +# -j <N> Parallel jobs (default: nproc) | ||
| 25 | +# ============================================================================= | ||
| 26 | +set -euo pipefail | ||
| 27 | + | ||
| 28 | +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| 29 | +PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)" | ||
| 30 | + | ||
| 31 | +# ── Defaults ────────────────────────────────────────────────────────────────── | ||
| 32 | +SDK_BUILD_DIR="${PROJECT_ROOT}/build/harmonyos/arm64-v8a" | ||
| 33 | +SDK_INSTALL_DIR="${PROJECT_ROOT}/out/harmonyos/arm64-v8a" | ||
| 34 | +MNN_ROOT="${PROJECT_ROOT}/third-party/mnn" | ||
| 35 | +OUTPUT_FILE="" | ||
| 36 | +CLEAN=0 | ||
| 37 | +VERSION="$(tr -d '[:space:]' < "${PROJECT_ROOT}/version.txt" 2>/dev/null || echo "1.0.0")" | ||
| 38 | +JOBS=$(nproc 2>/dev/null || sysctl -n hw.ncpu 2>/dev/null || echo 4) | ||
| 39 | + | ||
| 40 | +# ── Parse args ──────────────────────────────────────────────────────────────── | ||
| 41 | +while [[ $# -gt 0 ]]; do | ||
| 42 | + case "$1" in | ||
| 43 | + --sdk-build-dir) SDK_BUILD_DIR="$2"; shift 2 ;; | ||
| 44 | + --sdk-install-dir) SDK_INSTALL_DIR="$2"; shift 2 ;; | ||
| 45 | + --mnn-root) MNN_ROOT="$2"; shift 2 ;; | ||
| 46 | + --output) OUTPUT_FILE="$2"; shift 2 ;; | ||
| 47 | + --clean) CLEAN=1; shift ;; | ||
| 48 | + -j) JOBS="$2"; shift 2 ;; | ||
| 49 | + -h|--help) | ||
| 50 | + sed -n '2,22p' "$0" | grep '^#' | sed 's/^# \{0,1\}//' | ||
| 51 | + exit 0 ;; | ||
| 52 | + *) echo "Unknown arg: $1 (use --help)"; exit 1 ;; | ||
| 53 | + esac | ||
| 54 | +done | ||
| 55 | + | ||
| 56 | +[[ -z "${OUTPUT_FILE}" ]] && OUTPUT_FILE="${PROJECT_ROOT}/out/gewu-smartserve-${VERSION}-napi.har" | ||
| 57 | + | ||
| 58 | +# ── Verify dependencies ─────────────────────────────────────────────────────── | ||
| 59 | +if [[ ! -f "${SDK_BUILD_DIR}/sdk/libgewu_smartserve.a" ]]; then | ||
| 60 | + echo "ERROR: SDK not built. Run ./scripts/harmonyos/build_sdk.sh first" | ||
| 61 | + exit 1 | ||
| 62 | +fi | ||
| 63 | + | ||
| 64 | +if [[ -z "${OHOS_SDK_NATIVE:-}" ]]; then | ||
| 65 | + OHOS_SDK_NATIVE="${HOME}/Library/OpenHarmony/Sdk/20/native" | ||
| 66 | + if [[ ! -d "${OHOS_SDK_NATIVE}" ]]; then | ||
| 67 | + echo "ERROR: OHOS_SDK_NATIVE not set and default not found" | ||
| 68 | + echo " Set: export OHOS_SDK_NATIVE=/path/to/ohos/sdk/native" | ||
| 69 | + exit 1 | ||
| 70 | + fi | ||
| 71 | +fi | ||
| 72 | + | ||
| 73 | +TOOLCHAIN="${OHOS_SDK_NATIVE}/build/cmake/ohos.toolchain.cmake" | ||
| 74 | +if [[ ! -f "${TOOLCHAIN}" ]]; then | ||
| 75 | + echo "ERROR: HarmonyOS toolchain not found at ${TOOLCHAIN}" | ||
| 76 | + exit 1 | ||
| 77 | +fi | ||
| 78 | + | ||
| 79 | +echo "╔══════════════════════════════════════════════════════════════╗" | ||
| 80 | +echo "║ SmartServe SDK · HarmonyOS NAPI HAR Build ║" | ||
| 81 | +echo "╚══════════════════════════════════════════════════════════════╝" | ||
| 82 | +echo " SDK build : ${SDK_BUILD_DIR}" | ||
| 83 | +echo " SDK install : ${SDK_INSTALL_DIR}" | ||
| 84 | +echo " MNN root : ${MNN_ROOT}" | ||
| 85 | +echo " Version : ${VERSION}" | ||
| 86 | +echo " Output : ${OUTPUT_FILE}" | ||
| 87 | +echo "" | ||
| 88 | + | ||
| 89 | +HAR_MODULE_DIR="${PROJECT_ROOT}/harmonyos_har_napi" | ||
| 90 | +NAPI_BUILD_DIR="${HAR_MODULE_DIR}/build" | ||
| 91 | + | ||
| 92 | +if [[ "${CLEAN}" == "1" && -d "${NAPI_BUILD_DIR}" ]]; then | ||
| 93 | + echo "-- Cleaning ${NAPI_BUILD_DIR}" | ||
| 94 | + rm -rf "${NAPI_BUILD_DIR}" | ||
| 95 | +fi | ||
| 96 | + | ||
| 97 | +# ── Build NAPI .so ──────────────────────────────────────────────────────────── | ||
| 98 | +mkdir -p "${NAPI_BUILD_DIR}" | ||
| 99 | +cd "${NAPI_BUILD_DIR}" | ||
| 100 | + | ||
| 101 | +echo "-- Configuring NAPI module..." | ||
| 102 | +cmake "${HAR_MODULE_DIR}/src/main/cpp" \ | ||
| 103 | + -DCMAKE_TOOLCHAIN_FILE="${TOOLCHAIN}" \ | ||
| 104 | + -DOHOS_ARCH=arm64-v8a \ | ||
| 105 | + -DCMAKE_BUILD_TYPE=Release \ | ||
| 106 | + -G Ninja | ||
| 107 | + | ||
| 108 | +echo "-- Building NAPI module (${JOBS} jobs)..." | ||
| 109 | +cmake --build . -- -j"${JOBS}" | ||
| 110 | + | ||
| 111 | +# ── Copy NAPI .so + MNN .so to HAR libs/ ────────────────────────────────────── | ||
| 112 | +LIBS_DIR="${HAR_MODULE_DIR}/libs/arm64-v8a" | ||
| 113 | +mkdir -p "${LIBS_DIR}" | ||
| 114 | + | ||
| 115 | +echo "-- Packaging native libraries..." | ||
| 116 | +cp "${NAPI_BUILD_DIR}/libsmartserve.so" "${LIBS_DIR}/" | ||
| 117 | +echo " libsmartserve.so (NAPI)" | ||
| 118 | + | ||
| 119 | +# Must match MNN_BUILD_DIR in harmonyos_har_napi/src/main/cpp/CMakeLists.txt, otherwise | ||
| 120 | +# the .so linked against and the .so shipped in the HAR are different builds. | ||
| 121 | +MNN_BUILD_DIR="${MNN_ROOT}/build_harmonyos_arm64-v8a" | ||
| 122 | +for mnn_so in libMNN.so libllm.so libMNN_Express.so; do | ||
| 123 | + if [[ -f "${MNN_BUILD_DIR}/${mnn_so}" || -f "${MNN_BUILD_DIR}/express/${mnn_so}" ]]; then | ||
| 124 | + src="${MNN_BUILD_DIR}/${mnn_so}" | ||
| 125 | + [[ ! -f "${src}" ]] && src="${MNN_BUILD_DIR}/express/${mnn_so}" | ||
| 126 | + cp "${src}" "${LIBS_DIR}/" | ||
| 127 | + echo " ${mnn_so}" | ||
| 128 | + fi | ||
| 129 | +done | ||
| 130 | + | ||
| 131 | +# ── Package HAR (DevEco will do this, but we can simulate with zip) ─────────── | ||
| 132 | +echo "" | ||
| 133 | +echo "-- Creating HAR archive..." | ||
| 134 | +mkdir -p "$(dirname "${OUTPUT_FILE}")" | ||
| 135 | +cd "${HAR_MODULE_DIR}" | ||
| 136 | +zip -qr "${OUTPUT_FILE}" \ | ||
| 137 | + oh-package.json5 \ | ||
| 138 | + build-profile.json5 \ | ||
| 139 | + hvigorfile.ts \ | ||
| 140 | + src/main/module.json5 \ | ||
| 141 | + src/main/ets/ \ | ||
| 142 | + libs/ | ||
| 143 | + | ||
| 144 | +HAR_SIZE="$(du -sh "${OUTPUT_FILE}" | cut -f1)" | ||
| 145 | +echo "" | ||
| 146 | +echo "╔══════════════════════════════════════════════════════════════╗" | ||
| 147 | +echo "║ NAPI HAR ready ║" | ||
| 148 | +echo "╚══════════════════════════════════════════════════════════════╝" | ||
| 149 | +echo " ${OUTPUT_FILE} (${HAR_SIZE})" | ||
| 150 | +echo "" | ||
| 151 | +echo " Usage in HarmonyOS project:" | ||
| 152 | +echo " 1. Copy HAR to your project's oh_modules/ or libs/" | ||
| 153 | +echo " 2. Add dependency in oh-package.json5:" | ||
| 154 | +echo " \"dependencies\": {" | ||
| 155 | +echo " \"@huawei/gewu-smartserve\": \"file:./libs/$(basename "${OUTPUT_FILE}")\"" | ||
| 156 | +echo " }" | ||
| 157 | +echo " 3. Import in ArkTS:" | ||
| 158 | +echo " import { initialize, chat } from '@huawei/gewu-smartserve';" | ||
| 159 | +echo "" | ||
| @@ -0,0 +1,183 @@ | |||
| 1 | +#!/usr/bin/env bash | ||
| 2 | +# ============================================================================= | ||
| 3 | +# SmartServe SDK — HarmonyOS HAR packaging (libs/ + headers/ layout) | ||
| 4 | +# | ||
| 5 | +# Creates a HAR package compatible with HarmonyOS/OpenHarmony native development. | ||
| 6 | +# The HAR contains native libraries (.a/.so) and public SDK headers. | ||
| 7 | +# | ||
| 8 | +# Prerequisites | ||
| 9 | +# • zip | ||
| 10 | +# • A completed build_sdk.sh run | ||
| 11 | +# | ||
| 12 | +# Usage | ||
| 13 | +# ./scripts/harmonyos/pack_har.sh [OPTIONS] | ||
| 14 | +# | ||
| 15 | +# Options | ||
| 16 | +# --sdk-dir <dir> SDK install dir (default: <repo>/out/harmonyos/arm64-v8a) | ||
| 17 | +# --output <file> Output .har path (default: <repo>/out/gewu-smartserve-<ver>-ohos.har) | ||
| 18 | +# --version <ver> Library version (default: <repo>/version.txt) | ||
| 19 | +# --arch <arch> Target ABI (default: arm64-v8a) | ||
| 20 | +# --api <N> Min OHOS API (default: 12) | ||
| 21 | +# --copy-to <dir> Also copy HAR to this directory (e.g. app libs) | ||
| 22 | +# ============================================================================= | ||
| 23 | +set -euo pipefail | ||
| 24 | + | ||
| 25 | +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" | ||
| 26 | +PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)" | ||
| 27 | + | ||
| 28 | +_relpath() { | ||
| 29 | + local base="${1%/}" file="$2" | ||
| 30 | + case "${file}" in | ||
| 31 | + "${base}/"*) printf '%s\n' "${file#"${base}/"}" ;; | ||
| 32 | + *) printf '%s\n' "${file}" ;; | ||
| 33 | + esac | ||
| 34 | +} | ||
| 35 | + | ||
| 36 | +# ── Defaults ────────────────────────────────────────────────────────────────── | ||
| 37 | +SDK_DIR="${PROJECT_ROOT}/out/harmonyos/arm64-v8a" | ||
| 38 | +OUTPUT_FILE="" | ||
| 39 | +COPY_TO="" | ||
| 40 | +VERSION="$(tr -d '[:space:]' < "${PROJECT_ROOT}/version.txt" 2>/dev/null || echo "1.0.0")" | ||
| 41 | +ARCH="arm64-v8a" | ||
| 42 | +OHOS_API="12" | ||
| 43 | + | ||
| 44 | +# ── Parse args ──────────────────────────────────────────────────────────────── | ||
| 45 | +while [[ $# -gt 0 ]]; do | ||
| 46 | + case "$1" in | ||
| 47 | + --sdk-dir) SDK_DIR="$2"; shift 2 ;; | ||
| 48 | + --output) OUTPUT_FILE="$2"; shift 2 ;; | ||
| 49 | + --copy-to) COPY_TO="$2"; shift 2 ;; | ||
| 50 | + --version) VERSION="$2"; shift 2 ;; | ||
| 51 | + --arch) ARCH="$2"; shift 2 ;; | ||
| 52 | + --api) OHOS_API="$2"; shift 2 ;; | ||
| 53 | + -h|--help) | ||
| 54 | + sed -n '2,28p' "$0" | grep '^#' | sed 's/^# \{0,1\}//' | ||
| 55 | + exit 0 ;; | ||
| 56 | + *) echo "Unknown arg: $1 (use --help)"; exit 1 ;; | ||
| 57 | + esac | ||
| 58 | +done | ||
| 59 | + | ||
| 60 | +[[ -z "${OUTPUT_FILE}" ]] && OUTPUT_FILE="${PROJECT_ROOT}/out/gewu-smartserve-${VERSION}-ohos.har" | ||
| 61 | +if [[ "${OUTPUT_FILE}" != /* ]]; then | ||
| 62 | + OUTPUT_FILE="${PROJECT_ROOT}/${OUTPUT_FILE}" | ||
| 63 | +fi | ||
| 64 | + | ||
| 65 | +MAIN_LIB="${SDK_DIR}/lib/libgewu_smartserve.a" | ||
| 66 | +if [[ ! -f "${MAIN_LIB}" ]]; then | ||
| 67 | + echo "ERROR: ${MAIN_LIB} not found. Run build_sdk.sh first." | ||
| 68 | + exit 1 | ||
| 69 | +fi | ||
| 70 | + | ||
| 71 | +if ! command -v zip &>/dev/null; then | ||
| 72 | + echo "ERROR: 'zip' not found. Install: brew install zip" | ||
| 73 | + exit 1 | ||
| 74 | +fi | ||
| 75 | + | ||
| 76 | +echo "╔══════════════════════════════════════════════════════════════╗" | ||
| 77 | +echo "║ SmartServe SDK · HAR Packaging ║" | ||
| 78 | +echo "╚══════════════════════════════════════════════════════════════╝" | ||
| 79 | +echo " SDK dir : ${SDK_DIR}" | ||
| 80 | +echo " Arch : ${ARCH}" | ||
| 81 | +echo " Version : ${VERSION}" | ||
| 82 | +echo " Output : ${OUTPUT_FILE}" | ||
| 83 | +echo "" | ||
| 84 | + | ||
| 85 | +WORK_DIR="$(mktemp -d)" | ||
| 86 | +trap 'rm -rf "${WORK_DIR}"' EXIT | ||
| 87 | + | ||
| 88 | +LIBS_OUT="${WORK_DIR}/libs/${ARCH}" | ||
| 89 | +HEADERS_OUT="${WORK_DIR}/headers" | ||
| 90 | +mkdir -p "${LIBS_OUT}" "${HEADERS_OUT}" | ||
| 91 | + | ||
| 92 | +# ── module.json5 (HarmonyOS HAR metadata) ──────────────────────────────────── | ||
| 93 | +cat > "${WORK_DIR}/module.json5" << EOF | ||
| 94 | +{ | ||
| 95 | + "module": { | ||
| 96 | + "name": "gewu_smartserve", | ||
| 97 | + "type": "har", | ||
| 98 | + "deviceTypes": [ | ||
| 99 | + "default" | ||
| 100 | + ] | ||
| 101 | + } | ||
| 102 | +} | ||
| 103 | +EOF | ||
| 104 | + | ||
| 105 | +# ── oh-package.json5 (HarmonyOS package descriptor) ────────────────────────── | ||
| 106 | +cat > "${WORK_DIR}/oh-package.json5" << EOF | ||
| 107 | +{ | ||
| 108 | + "name": "@huawei/gewu-smartserve", | ||
| 109 | + "version": "${VERSION}", | ||
| 110 | + "description": "SmartServe SDK for HarmonyOS - AI inference engine", | ||
| 111 | + "main": "", | ||
| 112 | + "author": "Huawei", | ||
| 113 | + "license": "Apache-2.0", | ||
| 114 | + "dependencies": {} | ||
| 115 | +} | ||
| 116 | +EOF | ||
| 117 | + | ||
| 118 | +# ── Native libs ─────────────────────────────────────────────────────────────── | ||
| 119 | +echo "Packaging native libraries..." | ||
| 120 | +while IFS= read -r -d '' _lib; do | ||
| 121 | + _name="$(basename "${_lib}")" | ||
| 122 | + cp "${_lib}" "${LIBS_OUT}/${_name}" | ||
| 123 | + _size="$(du -sh "${_lib}" | cut -f1)" | ||
| 124 | + echo " [lib] ${_name} (${_size})" | ||
| 125 | +done < <(find "${SDK_DIR}/lib" -maxdepth 1 \( -name "*.a" -o -name "*.so" \) -print0 2>/dev/null) | ||
| 126 | + | ||
| 127 | +# ── Headers (public SDK surface) ────────────────────────────────────────────── | ||
| 128 | +echo "Packaging headers..." | ||
| 129 | +GEWU_HDR_DIR="${SDK_DIR}/include/gewu_smartserve" | ||
| 130 | +if [[ -d "${GEWU_HDR_DIR}" ]]; then | ||
| 131 | + mkdir -p "${HEADERS_OUT}/gewu_smartserve" | ||
| 132 | + for _h in core.h engine.h; do | ||
| 133 | + if [[ -f "${GEWU_HDR_DIR}/${_h}" ]]; then | ||
| 134 | + cp "${GEWU_HDR_DIR}/${_h}" "${HEADERS_OUT}/gewu_smartserve/${_h}" | ||
| 135 | + echo " [h] gewu_smartserve/${_h}" | ||
| 136 | + else | ||
| 137 | + echo "WARN: missing ${GEWU_HDR_DIR}/${_h}" | ||
| 138 | + fi | ||
| 139 | + done | ||
| 140 | +else | ||
| 141 | + echo "WARN: ${GEWU_HDR_DIR} not found" | ||
| 142 | +fi | ||
| 143 | + | ||
| 144 | +SMARTSERVE_HDR_DIR="${SDK_DIR}/include/smartserve" | ||
| 145 | +if [[ -d "${SMARTSERVE_HDR_DIR}" ]]; then | ||
| 146 | + mkdir -p "${HEADERS_OUT}/smartserve" | ||
| 147 | + for _h in smartserve.h; do | ||
| 148 | + if [[ -f "${SMARTSERVE_HDR_DIR}/${_h}" ]]; then | ||
| 149 | + cp "${SMARTSERVE_HDR_DIR}/${_h}" "${HEADERS_OUT}/smartserve/${_h}" | ||
| 150 | + echo " [h] smartserve/${_h}" | ||
| 151 | + fi | ||
| 152 | + done | ||
| 153 | +fi | ||
| 154 | + | ||
| 155 | +# ── Create HAR archive ──────────────────────────────────────────────────────── | ||
| 156 | +mkdir -p "$(dirname "${OUTPUT_FILE}")" | ||
| 157 | +(cd "${WORK_DIR}" && zip -qr "${OUTPUT_FILE}" .) | ||
| 158 | + | ||
| 159 | +if [[ -n "${COPY_TO}" ]]; then | ||
| 160 | + mkdir -p "${COPY_TO}" | ||
| 161 | + cp "${OUTPUT_FILE}" "${COPY_TO}/" | ||
| 162 | + echo "" | ||
| 163 | + echo " Copied to: ${COPY_TO}/$(basename "${OUTPUT_FILE}")" | ||
| 164 | +fi | ||
| 165 | + | ||
| 166 | +HAR_SIZE="$(du -sh "${OUTPUT_FILE}" | cut -f1)" | ||
| 167 | +echo "" | ||
| 168 | +echo "╔══════════════════════════════════════════════════════════════╗" | ||
| 169 | +echo "║ HAR ready ║" | ||
| 170 | +echo "╚══════════════════════════════════════════════════════════════╝" | ||
| 171 | +echo " ${OUTPUT_FILE} (${HAR_SIZE})" | ||
| 172 | +echo "" | ||
| 173 | +echo " Usage in HarmonyOS project:" | ||
| 174 | +echo " 1. Copy HAR to your project's oh_modules/ or libs/" | ||
| 175 | +echo " 2. Add dependency in oh-package.json5:" | ||
| 176 | +echo " \"dependencies\": {" | ||
| 177 | +echo " \"@huawei/gewu-smartserve\": \"file:./libs/$(basename "${OUTPUT_FILE}")\"" | ||
| 178 | +echo " }" | ||
| 179 | +echo " 3. Reference native libraries in CMakeLists.txt:" | ||
| 180 | +echo " target_link_libraries(your_target" | ||
| 181 | +echo " \${CMAKE_CURRENT_SOURCE_DIR}/../oh_modules/@huawei/gewu-smartserve/libs/${ARCH}/libgewu_smartserve.a" | ||
| 182 | +echo " )" | ||
| 183 | +echo "" | ||
copyright 的格式的年份问题依旧存在,需要确认下