已合并
Performance：refine heuristic and kernel launch #9
wangzitao创建于 3月9日
Performance：refine heuristic and kernel launch #9
已合并
共 13 个文件变更+3112-1403
| @@ -1,69 +1,42 @@ | |||
| 1 | # ops-blas | 1 | # ops-blas |
| 2 | 2 | ||
| 3 | ## 🔥Latest News | 3 | ## 🔥Latest News |
| 4 | -- [2026/03] ops-blas项目上线，提供BLAS计算的API以及现代灵活接口aclBLASLt，支持混合精度、融合后处理、启发式算法选择等高级特性。 | 4 | +- [2026/03] ops-blas项目上线，提供BLAS计算的API以及现代灵活接口aclBLASLt。 |
| 5 | 5 | ||
| 6 | ## 🚀概述 | 6 | ## 🚀概述 |
| 7 | ops-blas是[CANN](https://hiascend.com/software/cann) （Compute Architecture for Neural Networks）算子库中提供高性能线性代数计算以及轻量化GEMM调用的算子库。 | 7 | ops-blas是[CANN](https://hiascend.com/software/cann) （Compute Architecture for Neural Networks）算子库中提供高性能线性代数计算以及轻量化GEMM调用的算子库。 |
| 8 | 8 | ||
| 9 | -## 📝环境部署 | ||
| 10 | -### 快速安装CANN软件 | ||
| 11 | -本节提供快速安装CANN软件的示例命令，更多安装步骤请参考[详细安装指南](#cann详细安装指南)。 | ||
| 12 | - | ||
| 13 | -#### 安装前准备 | ||
| 14 | -在线安装和离线安装时，需确保已具备Python环境及pip3，当前CANN支持Python3.7.x至3.11.4版本。 | ||
| 15 | -离线安装时，请单击[获取链接](https://www.hiascend.com/developer/download/community/result?module=cann)下载CANN软件包，并上传到安装环境任意路径。 | ||
| 16 | -#### 安装CANN | ||
| 17 | -```shell | ||
| 18 | -chmod +x Ascend-cann-toolkit_8.5.RC1_linux-$(arch).run | ||
| 19 | -./Ascend-cann-toolkit_8.5.RC1_linux-$(arch).run --install | ||
| 20 | -``` | ||
| 21 | -#### 安装后配置 | ||
| 22 | -配置环境变量脚本set_env.sh，当前安装路径以${HOME}/Ascend为例。 | ||
| 23 | -``` | ||
| 24 | -source ${HOME}/Ascend/ascend-toolkit/set_env.sh | ||
| 25 | -``` | ||
| 26 | - | ||
| 27 | -### CANN详细安装指南 | ||
| 28 | -开发者可访问[昇腾文档-昇腾社区](https://www.hiascend.com/document)->CANN社区版->软件安装，查看CANN软件安装引导，根据机器环境、操作系统和业务场景选择后阅读详细安装步骤。 | ||
| 29 | - | ||
| 30 | -### 基础工具版本要求与安装 | ||
| 31 | -安装CANN之后，您可安装一些工具方便后续开发，参见以下内容： | ||
| 32 | - | ||
| 33 | -* [CANN依赖列表](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/83RC1alpha002/softwareinst/instg/instg_0045.html?Mode=PmIns&InstallType=local&OS=Debian&Software=cannToolKit) | ||
| 34 | -* [CANN安装后操作](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/83RC1alpha002/softwareinst/instg/instg_0094.html?Mode=PmIns&InstallType=local&OS=Debian&Software=cannToolKit) | ||
| 35 | - | ||
| 36 | ## ⚡️快速入门 | 9 | ## ⚡️快速入门 |
| 37 | - - 加速库下载 | 10 | +若您希望**从零到一快速体验**项目能力，请访问下述简易教程。 |
| 38 | - ```sh | ||
| 39 | - git clone https://gitcode.com/cann/ops-blas.git | ||
| 40 | - ``` | ||
| 41 | - 您可自行选择需要的分支。 | ||
| 42 | - - 执行编译安装 | ||
| 43 | - 编译加速库，设置加速库环境变量： | ||
| 44 | - ```sh | ||
| 45 | - cd ops-blas | ||
| 46 | - bash build.sh --op=scopy --run # --op=<算子名> --run可选参数，执行测试样例 | ||
| 47 | - ``` | ||
| 48 | 11 | ||
| 49 | -## 📂目录结构 | 12 | +1. [环境部署](docs/zh/install/quick_install.md)：介绍基础环境搭建，包括软件包和三方依赖的获取和安装、源码下载等。 |
| 50 | -ops-blas库的目录结构如下： | ||
| 51 | 13 | ||
| 14 | + > **说明**：本步骤是QuickStart和各类教程的操作前提，请先完成基础环境搭建。 | ||
| 15 | +2. [QuickStart](docs/QUICKSTART.md)：提供快速上手本项目能力的指南，包括编译部署、算子调用/开发/调试等核心能力。 | ||
| 16 | + | ||
| 17 | +## 📖学习教程 | ||
| 18 | + | ||
| 19 | +若您已学习**环境部署和QuickStart**，对本项目有一定认知，并希望**深入了解和体验项目**，请访问下述详细教程。 | ||
| 20 | + | ||
| 21 | +1. [接口列表](docs/zh/api_list.md)：提供全量API信息，方便您查看aclblas和aclblasLt接口的分类和功能。 | ||
| 22 | + | ||
| 23 | + | ||
| 24 | +## 🔍目录结构 | ||
| 25 | +ops-blas仓关键目录结构如下。 | ||
| 52 | ``` | 26 | ``` |
| 53 | ops-blas | 27 | ops-blas |
| 54 | -├── build //可存放构建生成的文件 | 28 | +├── docs # 项目文档介绍 |
| 55 | -├── docs //文档文件 | 29 | +├── examples # 端到端算子开发和调用示例 |
| 56 | -├── example //算子调用示例代码，包含可直接运行的Demo | 30 | +├── include # 对外头文件 |
| 57 | -├── include //存放公共头文件 | 31 | +├── scripts # 脚本目录 |
| 58 | -├── scripts //脚本文件存放目录 | 32 | +├── blasLt # blasLt主体源代码目录 |
| 59 | -├── blaslt //blaslt主体源代码目录 | 33 | +├── blas # blas主体源代码目录 |
| 60 | -├── blas //blas主体源代码目录 | 34 | +│ ├── utils # 公共函数 |
| 61 | -│ ├── utils //公共函数 | 35 | +│ ├── dot # 向量点积算子实现 |
| 62 | -│ ├── dot //向量点积算子实现 | 36 | +│ ├── gemv # 一般矩阵向量乘法算子实现 |
| 63 | -│ ├── gemv //一般矩阵向量乘法算子实现 | 37 | +│ ├── ... # 其他算子实现 |
| 64 | -│ ├── ... //其他算子实现 | 38 | +│ └── CMakeLists.txt # 算子编译配置文件 |
| 65 | -│ └── CMakeLists.txt | 39 | +├── tests # 测试代码 |
| 66 | -├── tests //测试代码 | ||
| 67 | ``` | 40 | ``` |
| 68 | 41 | ||
| 69 | ## 💬相关信息 | 42 | ## 💬相关信息 |
| @@ -11,50 +11,223 @@ | |||
| 11 | 11 | ||
| 12 | 12 | ||
| 13 | 13 | ||
| 14 | + | ||
| 15 | + | ||
| 14 | 16 | ||
| 15 | 17 | ||
| 18 | + | ||
| 19 | + | ||
| 16 | 20 | ||
| 17 | - | 21 | +#include <unordered_map> |
| 22 | + | ||
| 18 | 23 | ||
| 19 | 24 | ||
| 20 | 25 | ||
| 21 | namespace { | 26 | namespace { |
| 22 | 27 | ||
| 23 | -constexpr int kAclblasLtVersionMajor = 1; | 28 | +constexpr int ACLBLASLT_VERSION_MAJOR = 1; |
| 24 | -constexpr int kAclblasLtVersionMinor = 0; | 29 | +constexpr int ACLBLASLT_VERSION_MINOR = 0; |
| 25 | -constexpr int kAclblasLtVersionPatch = 0; | 30 | +constexpr int ACLBLASLT_VERSION_PATCH = 0; |
| 26 | 31 | ||
| 27 | -struct LtHandle { | 32 | +constexpr uint32_t ACLBLASLT_HANDLE_MAGIC = 0xACBA1234; |
| 28 | - uint32_t magic = 0x4C54484C; // 'LTHL' | 33 | +constexpr uint32_t ACLBLASLT_LAYOUT_MAGIC = 0xACBB1234; |
| 29 | - int32_t deviceId = 0; | 34 | +constexpr uint32_t ACLBLASLT_DESC_MAGIC = 0xACBC1234; |
| 30 | - bool initialized = false; | 35 | +constexpr uint32_t ACLBLASLT_ALGO_MAGIC = 0xACBD1234; |
| 36 | + | ||
| 37 | +constexpr size_t DEFAULT_WORKSPACE_SIZE = 32 * 1024 * 1024; | ||
| 38 | +constexpr size_t L1_SIZE = 512 * 1024; | ||
| 39 | +constexpr size_t L0_SIZE = 256; | ||
| 40 | +constexpr uint32_t DEFAULT_AI_CORES = 8; | ||
| 41 | +constexpr double DEFAULT_PEAK_TFLOPS = 140.0; | ||
| 42 | +constexpr double DEFAULT_PEAK_GBPS = 900.0; | ||
| 43 | + | ||
| 44 | +struct AtlasA2 { | ||
| 45 | + static constexpr uint32_t BIAS_SIZE = 1024; | ||
| 46 | + static constexpr uint32_t FIXBUF_SIZE = 7 * 1024; | ||
| 47 | + static constexpr uint32_t UB_SIZE = 192 * 1024; | ||
| 48 | + static constexpr uint32_t L1_SIZE = 512 * 1024; | ||
| 49 | + static constexpr uint32_t L0A_SIZE = 64 * 1024; | ||
| 50 | + static constexpr uint32_t L0B_SIZE = 64 * 1024; | ||
| 51 | + static constexpr uint32_t L0C_SIZE = 128 * 1024; | ||
| 31 | }; | 52 | }; |
| 32 | 53 | ||
| 33 | -struct MatrixLayout { | 54 | +struct Ascend950 { |
| 55 | + static constexpr uint32_t BIAS_SIZE = 4 * 1024; | ||
| 56 | + static constexpr uint32_t FIXBUF_SIZE = 16 * 1024; | ||
| 57 | + static constexpr uint32_t UB_SIZE = 248 * 1024; | ||
| 58 | + static constexpr uint32_t L1_SIZE = 512 * 1024; | ||
| 59 | + static constexpr uint32_t L0A_SIZE = 64 * 1024; | ||
| 60 | + static constexpr uint32_t L0B_SIZE = 64 * 1024; | ||
| 61 | + static constexpr uint32_t L0C_SIZE = 256 * 1024; | ||
| 62 | +}; | ||
| 63 | + | ||
| 64 | +enum DispatchPolicyType : uint8_t { | ||
| 65 | + DISPATCH_POLICY_MMAD_SYNC = 0, | ||
| 66 | + DISPATCH_POLICY_MMAD_PINGPONG = 1, | ||
| 67 | + DISPATCH_POLICY_MMAD_MULTI_STAGE = 2, | ||
| 68 | +}; | ||
| 69 | + | ||
| 70 | +struct AlgoKey { | ||
| 71 | + uint64_t m = 0; | ||
| 72 | + uint64_t n = 0; | ||
| 73 | + uint64_t k = 0; | ||
| 74 | + aclDataType aType = ACL_FLOAT; | ||
| 75 | + aclDataType bType = ACL_DT_UNDEFINED; | ||
| 76 | + aclDataType cType = ACL_DT_UNDEFINED; | ||
| 77 | + aclDataType dType = ACL_DT_UNDEFINED; | ||
| 78 | + aclblasComputeType_t computeType = ACLBLAS_COMPUTE_32F; | ||
| 79 | + aclblasLtEpilogue_t epilogue = ACLBLASLT_EPILOGUE_DEFAULT; | ||
| 80 | + bool transA = false; | ||
| 81 | + bool transB = false; | ||
| 82 | + | ||
| 83 | + bool operator==(const AlgoKey& other) const | ||
| 84 | + { | ||
| 85 | + return m == other.m && n == other.n && k == other.k && aType == other.aType && bType == other.bType && | ||
| 86 | + cType == other.cType && dType == other.dType && computeType == other.computeType && epilogue == other.epilogue && | ||
| 87 | + transA == other.transA && transB == other.transB; | ||
| 88 | + } | ||
| 89 | +}; | ||
| 90 | + | ||
| 91 | +struct AlgoKeyHasher { | ||
| 92 | + size_t operator()(const AlgoKey& x) const | ||
| 93 | + { | ||
| 94 | + size_t h = 1469598103934665603ull; | ||
| 95 | + auto mix = [&](uint64_t v) { | ||
| 96 | + h ^= static_cast<size_t>(v + 0x9e3779b97f4a7c15ull + (h << 6) + (h >> 2)); | ||
| 97 | + }; | ||
| 98 | + mix(x.m); | ||
| 99 | + mix(x.n); | ||
| 100 | + mix(x.k); | ||
| 101 | + mix(static_cast<uint64_t>(x.aType)); | ||
| 102 | + mix(static_cast<uint64_t>(x.bType)); | ||
| 103 | + mix(static_cast<uint64_t>(x.cType)); | ||
| 104 | + mix(static_cast<uint64_t>(x.dType)); | ||
| 105 | + mix(static_cast<uint64_t>(x.computeType)); | ||
| 106 | + mix(static_cast<uint64_t>(x.epilogue)); | ||
| 107 | + mix(static_cast<uint64_t>(x.transA)); | ||
| 108 | + mix(static_cast<uint64_t>(x.transB)); | ||
| 109 | + return h; | ||
| 110 | + } | ||
| 111 | +}; | ||
| 112 | + | ||
| 113 | +struct CacheEntry { | ||
| 114 | + aclblasLtMatmulAlgo_t algo; | ||
| 115 | + std::list<AlgoKey>::iterator lruIter; | ||
| 116 | +}; | ||
| 117 | + | ||
| 118 | +struct aclblasLtHandle { | ||
| 119 | + uint32_t magic = ACLBLASLT_HANDLE_MAGIC; | ||
| 120 | + bool initialized = false; | ||
| 121 | + // version info | ||
| 122 | + int versionMajor = ACLBLASLT_VERSION_MAJOR; | ||
| 123 | + int versionMinor = ACLBLASLT_VERSION_MINOR; | ||
| 124 | + // AscendCL runtime | ||
| 125 | + aclrtContext context = nullptr; | ||
| 126 | + aclrtStream defaultStream = nullptr; | ||
| 127 | + int32_t deviceId = 0; | ||
| 128 | + // workspace | ||
| 129 | + void* internalWorkspace = nullptr; | ||
| 130 | + size_t workspaceSize = 0; | ||
| 131 | + // thread safety | ||
| 132 | + std::mutex* mutex = nullptr; | ||
| 133 | + // soc spec | ||
| 134 | + int npuArch = 0; | ||
| 135 | + size_t maxSharedMemory = 0; | ||
| 136 | + // algo cache | ||
| 137 | + std::unordered_map<AlgoKey, CacheEntry, AlgoKeyHasher>* algoCache = nullptr; | ||
| 138 | + size_t algoCacheMaxSize = 128; | ||
| 139 | + std::list<AlgoKey>* lruList = nullptr; | ||
| 140 | +}; | ||
| 141 | + | ||
| 142 | +struct aclblasLtMatrixLayoutImpl { | ||
| 143 | + uint32_t magic; | ||
| 34 | aclDataType type; | 144 | aclDataType type; |
| 35 | uint64_t rows; | 145 | uint64_t rows; |
| 36 | uint64_t cols; | 146 | uint64_t cols; |
| 37 | int64_t ld; | 147 | int64_t ld; |
| 38 | - aclblasLtOrder_t order; | 148 | + aclblasLtOrder_t order = ACLBLASLT_ORDER_COL; |
| 39 | int32_t batchCount = 1; | 149 | int32_t batchCount = 1; |
| 40 | int64_t stridedBatchOffset = 0; | 150 | int64_t stridedBatchOffset = 0; |
| 41 | }; | 151 | }; |
| 152 | +static_assert(sizeof(aclblasLtMatrixLayoutImpl) <= sizeof(aclblasLtMatrixLayoutOpaque_t), | ||
| 153 | + "Impl of aclblasLtMatrixLayout must fit in capsule!"); | ||
| 42 | 154 | ||
| 43 | -struct MatmulDesc { | 155 | +struct aclblasLtMatmulDescImpl { |
| 156 | + uint32_t magic; | ||
| 44 | aclblasComputeType_t computeType; | 157 | aclblasComputeType_t computeType; |
| 45 | aclDataType scaleType; | 158 | aclDataType scaleType; |
| 46 | - aclblasLtEpilogue_t epilogue; | ||
| 47 | - const void* bias; | ||
| 48 | aclblasOperation_t transA = ACLBLAS_OP_N; | 159 | aclblasOperation_t transA = ACLBLAS_OP_N; |
| 49 | aclblasOperation_t transB = ACLBLAS_OP_N; | 160 | aclblasOperation_t transB = ACLBLAS_OP_N; |
| 50 | - aclDataType biasDataType = ACL_FLOAT; | 161 | + aclblasLtEpilogue_t epilogue = ACLBLASLT_EPILOGUE_DEFAULT; |
| 162 | + const void* bias = nullptr; | ||
| 163 | + aclDataType biasDataType = ACL_DT_UNDEFINED; | ||
| 164 | +}; | ||
| 165 | +static_assert(sizeof(aclblasLtMatmulDescImpl) <= sizeof(aclblasLtMatmulDescOpaque_t), | ||
| 166 | + "Impl of aclblasLtMatmulDesc must fit in capsule!"); | ||
| 167 | + | ||
| 168 | +struct aclblasLtMatmulPreferenceImpl { | ||
| 169 | + uint32_t magic; | ||
| 170 | + uint32_t searchMode = 0; | ||
| 171 | + size_t maxWorkspaceBytes = DEFAULT_WORKSPACE_SIZE; | ||
| 172 | + int32_t maxResults = 3; | ||
| 173 | + bool allowMixedPrecision = true; | ||
| 174 | + bool allowSplitK = true; | ||
| 175 | + // tiling | ||
| 176 | + uint32_t preferredL0M = 0; | ||
| 177 | + uint32_t preferredL0N = 0; | ||
| 178 | + uint32_t preferredL0K = 0; | ||
| 179 | + // Scheduling | ||
| 180 | + bool preferPingpong = false; | ||
| 181 | + bool preferDoubleBuffer = false; | ||
| 182 | + float minEfficiency = 0.5f; | ||
| 183 | +}; | ||
| 184 | +static_assert(sizeof(aclblasLtMatmulPreferenceImpl) <= sizeof(aclblasLtMatmulPreferenceOpaque_t), | ||
| 185 | + "Impl of aclblasLtMatmulPreference must fit in capsule!"); | ||
| 186 | + | ||
| 187 | +struct AscendHardwareCaps { | ||
| 188 | + uint32_t numAICores = DEFAULT_AI_CORES; | ||
| 189 | + uint32_t l0CubeSize = L0_SIZE; | ||
| 190 | + size_t l1BufferSize = L1_SIZE; | ||
| 191 | + double memoryBandwidthGBps = DEFAULT_PEAK_GBPS; | ||
| 192 | + double peakTFlops = DEFAULT_PEAK_TFLOPS; | ||
| 193 | + double bandwidthBoundThreshold = 32.0; | ||
| 51 | }; | 194 | }; |
| 52 | 195 | ||
| 53 | -struct Preference { | 196 | +struct AlgoCandidate { |
| 54 | - size_t maxWorkspaceBytes = 0; | 197 | + uint32_t algoId = 0; |
| 55 | - uint32_t searchMode = 0; | 198 | + uint32_t l1TileM = 128; |
| 199 | + uint32_t l1TileN = 128; | ||
| 200 | + uint32_t l1TileK = 128; | ||
| 201 | + uint32_t l0TileM = 64; | ||
| 202 | + uint32_t l0TileN = 64; | ||
| 203 | + uint32_t l0TileK = 64; | ||
| 204 | + DispatchPolicyType policy = DISPATCH_POLICY_MMAD_SYNC; | ||
| 205 | + uint32_t numBuffers = 1; | ||
| 206 | + uint32_t splitKFactor = 1; | ||
| 207 | + size_t workspaceSize = 0; | ||
| 208 | + double peakPerformance = DEFAULT_PEAK_TFLOPS; | ||
| 56 | }; | 209 | }; |
| 57 | 210 | ||
| 211 | + | ||
| 212 | +struct ScoredResult { | ||
| 213 | + AlgoCandidate cand; | ||
| 214 | + double estimatedTimeMs = 0.0; | ||
| 215 | + double totalScore = 0.0; | ||
| 216 | + bool isEfficient = true; | ||
| 217 | +}; | ||
| 218 | + | ||
| 219 | +struct PackedAlgo { | ||
| 220 | + uint32_t magic; | ||
| 221 | + uint32_t algoId; | ||
| 222 | + uint16_t l1mDiv16; | ||
| 223 | + uint16_t l1nDiv16; | ||
| 224 | + uint8_t policy; | ||
| 225 | + uint8_t numBuffers; | ||
| 226 | + uint8_t splitK; | ||
| 227 | + uint8_t flags; | ||
| 228 | +}; | ||
| 229 | +static_assert(sizeof(PackedAlgo) == 16, "PackedAlgo must fit algo.data"); | ||
| 230 | + | ||
| 58 | template <typename T> | 231 | template <typename T> |
| 59 | static aclblasStatus_t AllocHandle(T** out) | 232 | static aclblasStatus_t AllocHandle(T** out) |
| 60 | { | 233 | { |
| @@ -80,6 +253,242 @@ static aclblasStatus_t FreeHandle(T* p) | |||
| 80 | return ACLBLAS_STATUS_SUCCESS; | 253 | return ACLBLAS_STATUS_SUCCESS; |
| 81 | } | 254 | } |
| 82 | 255 | ||
| 256 | +static size_t GetTypeSize(aclDataType dt) | ||
| 257 | +{ | ||
| 258 | + switch (dt) { | ||
| 259 | + case ACL_FLOAT16: | ||
| 260 | + return 2; | ||
| 261 | + case ACL_FLOAT: | ||
| 262 | + case ACL_INT32: | ||
| 263 | + return 4; | ||
| 264 | + case ACL_INT8: | ||
| 265 | + return 1; | ||
| 266 | + default: | ||
| 267 | + return 0; | ||
| 268 | + } | ||
| 269 | +} | ||
| 270 | + | ||
| 271 | +static bool IsDataTypeSupported(aclDataType dt) | ||
| 272 | +{ | ||
| 273 | + return GetTypeSize(dt) != 0; | ||
| 274 | +} | ||
| 275 | + | ||
| 276 | +static bool CheckComputeTypeCompatibility(aclblasComputeType_t ct, aclDataType typeA, aclDataType typeB) | ||
| 277 | +{ | ||
| 278 | + if (typeA != typeB) { | ||
| 279 | + return false; | ||
| 280 | + } | ||
| 281 | + if (typeA == ACL_FLOAT16) { | ||
| 282 | + return ct == ACLBLAS_COMPUTE_16F || ct == ACLBLAS_COMPUTE_16F_PEDANTIC || ct == ACLBLAS_COMPUTE_32F || | ||
| 283 | + ct == ACLBLAS_COMPUTE_32F_PEDANTIC || ct == ACLBLAS_COMPUTE_32F_FAST_16F; | ||
| 284 | + } | ||
| 285 | + if (typeA == ACL_FLOAT) { | ||
| 286 | + return ct == ACLBLAS_COMPUTE_32F || ct == ACLBLAS_COMPUTE_32F_PEDANTIC || ct == ACLBLAS_COMPUTE_32F_FAST_TF32; | ||
| 287 | + } | ||
| 288 | + if (typeA == ACL_INT8) { | ||
| 289 | + return ct == ACLBLAS_COMPUTE_32I || ct == ACLBLAS_COMPUTE_32I_PEDANTIC; | ||
| 290 | + } | ||
| 291 | + return false; | ||
| 292 | +} | ||
| 293 | + | ||
| 294 | +static uint32_t CeilDivU64(uint64_t a, uint64_t b) | ||
| 295 | +{ | ||
| 296 | + return static_cast<uint32_t>((a + b - 1) / b); | ||
| 297 | +} | ||
| 298 | + | ||
| 299 | +static uint32_t GenerateAlgoId(DispatchPolicyType policy, | ||
| 300 | + uint32_t l1m, | ||
| 301 | + uint32_t l1n, | ||
| 302 | + uint32_t l1k, | ||
| 303 | + uint32_t splitKFactor) | ||
| 304 | +{ | ||
| 305 | + return (static_cast<uint32_t>(policy) << 28) ^ (l1m << 16) ^ (l1n << 8) ^ (l1k << 2) ^ splitKFactor; | ||
| 306 | +} | ||
| 307 | + | ||
| 308 | +static aclblasLtMatmulAlgo_t BuildAlgoFromCandidate(const AlgoCandidate& cand) | ||
| 309 | +{ | ||
| 310 | + aclblasLtMatmulAlgo_t out{}; | ||
| 311 | + PackedAlgo packed{}; | ||
| 312 | + packed.magic = ACLBLASLT_ALGO_MAGIC; | ||
| 313 | + packed.algoId = cand.algoId; | ||
| 314 | + packed.l1mDiv16 = static_cast<uint16_t>(cand.l1TileM / 16); | ||
| 315 | + packed.l1nDiv16 = static_cast<uint16_t>(cand.l1TileN / 16); | ||
| 316 | + packed.policy = static_cast<uint8_t>(cand.policy); | ||
| 317 | + packed.numBuffers = static_cast<uint8_t>(cand.numBuffers); | ||
| 318 | + packed.splitK = static_cast<uint8_t>(cand.splitKFactor); | ||
| 319 | + std::memcpy(out.data, &packed, sizeof(packed)); | ||
| 320 | + out.max_workspace_bytes = cand.workspaceSize; | ||
| 321 | + return out; | ||
| 322 | +} | ||
| 323 | + | ||
| 324 | +static bool DecodeAlgo(const aclblasLtMatmulAlgo_t& algo, PackedAlgo* packed) | ||
| 325 | +{ | ||
| 326 | + if (packed == nullptr) { | ||
| 327 | + return false; | ||
| 328 | + } | ||
| 329 | + std::memcpy(packed, algo.data, sizeof(PackedAlgo)); | ||
| 330 | + return packed->magic == ACLBLASLT_ALGO_MAGIC; | ||
| 331 | +} | ||
| 332 | + | ||
| 333 | +static void GetAscendHardwareCaps(int32_t, AscendHardwareCaps* caps) | ||
| 334 | +{ | ||
| 335 | + if (caps == nullptr) { | ||
| 336 | + return; | ||
| 337 | + } | ||
| 338 | + // 当前仓库保持轻量默认能力值，后续可对接真实设备查询。 | ||
| 339 | + caps->numAICores = DEFAULT_AI_CORES; | ||
| 340 | + caps->l0CubeSize = L0_SIZE; | ||
| 341 | + caps->l1BufferSize = L1_SIZE; | ||
| 342 | + caps->memoryBandwidthGBps = DEFAULT_PEAK_GBPS; | ||
| 343 | + caps->peakTFlops = DEFAULT_PEAK_TFLOPS; | ||
| 344 | + caps->bandwidthBoundThreshold = 32.0; | ||
| 345 | +} | ||
| 346 | + | ||
| 347 | +static void SelectL1TileShape(uint64_t m, | ||
| 348 | + uint64_t n, | ||
| 349 | + uint64_t, | ||
| 350 | + uint32_t numAICores, | ||
| 351 | + uint32_t prefL0M, | ||
| 352 | + uint32_t prefL0N, | ||
| 353 | + uint32_t prefL0K, | ||
| 354 | + uint32_t* l1M, | ||
| 355 | + uint32_t* l1N, | ||
| 356 | + uint32_t* l1K) | ||
| 357 | +{ | ||
| 358 | + const uint32_t candidates[][3] = {{128, 256, 256}, {256, 128, 256}, {256, 256, 128}, {128, 128, 128}, {256, 256, 64}}; | ||
| 359 | + | ||
| 360 | + float bestScore = -1.0f; | ||
| 361 | + const uint32_t* best = candidates[0]; | ||
| 362 | + | ||
| 363 | + for (const auto& cand : candidates) { | ||
| 364 | + uint32_t cm = cand[0]; | ||
| 365 | + uint32_t cn = cand[1]; | ||
| 366 | + uint32_t ck = cand[2]; | ||
| 367 | + | ||
| 368 | + uint32_t tilesM = CeilDivU64(m, cm); | ||
| 369 | + uint32_t tilesN = CeilDivU64(n, cn); | ||
| 370 | + uint32_t totalTiles = tilesM * tilesN; | ||
| 371 | + | ||
| 372 | + float balanceScore = 1.0f - static_cast<float>(totalTiles % std::max(1u, numAICores)) / std::max(1u, numAICores); | ||
| 373 | + size_t l1Usage = static_cast<size_t>(cm) * ck * sizeof(float) + static_cast<size_t>(cn) * ck * sizeof(float); | ||
| 374 | + float l1Util = static_cast<float>(l1Usage) / static_cast<float>(L1_SIZE); | ||
| 375 | + | ||
| 376 | + float l0Match = 0.0f; | ||
| 377 | + if (prefL0M > 0 && cm % prefL0M == 0) { | ||
| 378 | + l0Match += 0.3f; | ||
| 379 | + } | ||
| 380 | + if (prefL0N > 0 && cn % prefL0N == 0) { | ||
| 381 | + l0Match += 0.3f; | ||
| 382 | + } | ||
| 383 | + if (prefL0K > 0 && ck % prefL0K == 0) { | ||
| 384 | + l0Match += 0.4f; | ||
| 385 | + } | ||
| 386 | + | ||
| 387 | + float score = balanceScore * 0.4f + std::min(l1Util, 1.0f) * 0.3f + l0Match * 0.3f; | ||
| 388 | + if (score > bestScore) { | ||
| 389 | + bestScore = score; | ||
| 390 | + best = cand; | ||
| 391 | + } | ||
| 392 | + } | ||
| 393 | + | ||
| 394 | + *l1M = best[0]; | ||
| 395 | + *l1N = best[1]; | ||
| 396 | + *l1K = best[2]; | ||
| 397 | +} | ||
| 398 | + | ||
| 399 | +static void SelectL0TileShape(uint32_t l1M, | ||
| 400 | + uint32_t l1N, | ||
| 401 | + uint32_t l1K, | ||
| 402 | + size_t, | ||
| 403 | + size_t, | ||
| 404 | + aclDataType, | ||
| 405 | + aclDataType, | ||
| 406 | + uint32_t* l0M, | ||
| 407 | + uint32_t* l0N, | ||
| 408 | + uint32_t* l0K) | ||
| 409 | +{ | ||
| 410 | + *l0K = std::min(64u, l1K); | ||
| 411 | + *l0M = std::min(128u, l1M); | ||
| 412 | + *l0N = std::min(256u, l1N); | ||
| 413 | + | ||
| 414 | + while (*l0M > 16 && (l1M % *l0M != 0)) { | ||
| 415 | + --(*l0M); | ||
| 416 | + } | ||
| 417 | + while (*l0N > 16 && (l1N % *l0N != 0)) { | ||
| 418 | + --(*l0N); | ||
| 419 | + } | ||
| 420 | +} | ||
| 421 | + | ||
| 422 | +static uint32_t SelectSplitKForAscend(uint32_t l1LoopsK, uint32_t numAICores) | ||
| 423 | +{ | ||
| 424 | + if (numAICores == 0) { | ||
| 425 | + return 1; | ||
| 426 | + } | ||
| 427 | + uint32_t candidate = std::min(l1LoopsK, numAICores); | ||
| 428 | + return std::max(1u, candidate); | ||
| 429 | +} | ||
| 430 | + | ||
| 431 | +static size_t CalculateWorkspaceForAscend(uint64_t m, | ||
| 432 | + uint64_t n, | ||
| 433 | + uint32_t splitKFactor, | ||
| 434 | + aclblasLtEpilogue_t epilogue) | ||
| 435 | +{ | ||
| 436 | + size_t workspace = 0; | ||
| 437 | + if (splitKFactor > 1) { | ||
| 438 | + workspace += static_cast<size_t>(splitKFactor) * static_cast<size_t>(m) * static_cast<size_t>(n) * sizeof(float); | ||
| 439 | + } | ||
| 440 | + | ||
| 441 | + switch (epilogue) { | ||
| 442 | + case ACLBLASLT_EPILOGUE_BIAS: | ||
| 443 | + case ACLBLASLT_EPILOGUE_RELU_BIAS: | ||
| 444 | + case ACLBLASLT_EPILOGUE_GELU_BIAS: | ||
| 445 | + workspace += static_cast<size_t>(m) * sizeof(float); | ||
| 446 | + break; | ||
| 447 | + case ACLBLASLT_EPILOGUE_GELU: | ||
| 448 | + case ACLBLASLT_EPILOGUE_RELU: | ||
| 449 | + workspace += 64 * 1024; | ||
| 450 | + break; | ||
| 451 | + default: | ||
| 452 | + break; | ||
| 453 | + } | ||
| 454 | + return workspace; | ||
| 455 | +} | ||
| 456 | + | ||
| 457 | +static bool CheckHandleValid(const aclblasLtHandle* h) | ||
| 458 | +{ | ||
| 459 | + return h != nullptr && h->magic == ACLBLASLT_HANDLE_MAGIC && h->initialized && h->algoCache != nullptr && h->lruList != nullptr; | ||
| 460 | +} | ||
| 461 | + | ||
| 462 | +static bool BuildGemmShape(const aclblasLtMatmulDescImpl* desc, | ||
| 463 | + const aclblasLtMatrixLayoutImpl* A, | ||
| 464 | + const aclblasLtMatrixLayoutImpl* B, | ||
| 465 | + const aclblasLtMatrixLayoutImpl* D, | ||
| 466 | + uint64_t* m, | ||
| 467 | + uint64_t* n, | ||
| 468 | + uint64_t* k) | ||
| 469 | +{ | ||
| 470 | + if (desc == nullptr || A == nullptr || B == nullptr || D == nullptr || m == nullptr || n == nullptr || k == nullptr) { | ||
| 471 | + return false; | ||
| 472 | + } | ||
| 473 | + const bool transA = (desc->transA != ACLBLAS_OP_N); | ||
| 474 | + const bool transB = (desc->transB != ACLBLAS_OP_N); | ||
| 475 | + | ||
| 476 | + const uint64_t mA = transA ? A->cols : A->rows; | ||
| 477 | + const uint64_t kA = transA ? A->rows : A->cols; | ||
| 478 | + const uint64_t kB = transB ? B->cols : B->rows; | ||
| 479 | + const uint64_t nB = transB ? B->rows : B->cols; | ||
| 480 | + | ||
| 481 | + if (mA != D->rows || kA != kB || nB != D->cols) { | ||
| 482 | + return false; | ||
| 483 | + } | ||
| 484 | + | ||
| 485 | + *m = mA; | ||
| 486 | + *n = nB; | ||
| 487 | + *k = kA; | ||
| 488 | + return true; | ||
| 489 | +} | ||
| 490 | + | ||
| 491 | + | ||
| 83 | } // namespace | 492 | } // namespace |
| 84 | 493 | ||
| 85 | extern void matmul_kernel_do(GM_ADDR a, | 494 | extern void matmul_kernel_do(GM_ADDR a, |
| @@ -100,9 +509,9 @@ aclblasStatus_t aclblasLtGetVersion(size_t* version) | |||
| 100 | return ACLBLAS_STATUS_INVALID_VALUE; | 509 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 101 | } | 510 | } |
| 102 | 511 | ||
| 103 | - *version = (static_cast<size_t>(kAclblasLtVersionMajor) << 24) | | 512 | + *version = (static_cast<size_t>(ACLBLASLT_VERSION_MAJOR) << 24) | |
| 104 | - (static_cast<size_t>(kAclblasLtVersionMinor) << 16) | | 513 | + (static_cast<size_t>(ACLBLASLT_VERSION_MINOR) << 16) | |
| 105 | - static_cast<size_t>(kAclblasLtVersionPatch); | 514 | + static_cast<size_t>(ACLBLASLT_VERSION_PATCH); |
| 106 | return ACLBLAS_STATUS_SUCCESS; | 515 | return ACLBLAS_STATUS_SUCCESS; |
| 107 | } | 516 | } |
| 108 | 517 | ||
| @@ -114,32 +523,31 @@ aclblasStatus_t aclblasLtGetProperty(aclblasLtPropertyType_t type, int* value) | |||
| 114 | 523 | ||
| 115 | switch (type) { | 524 | switch (type) { |
| 116 | case ACLBLASLT_PROPERTY_MAJOR_VERSION: | 525 | case ACLBLASLT_PROPERTY_MAJOR_VERSION: |
| 117 | - *value = kAclblasLtVersionMajor; | 526 | + *value = ACLBLASLT_VERSION_MAJOR; |
| 118 | return ACLBLAS_STATUS_SUCCESS; | 527 | return ACLBLAS_STATUS_SUCCESS; |
| 119 | case ACLBLASLT_PROPERTY_MINOR_VERSION: | 528 | case ACLBLASLT_PROPERTY_MINOR_VERSION: |
| 120 | - *value = kAclblasLtVersionMinor; | 529 | + *value = ACLBLASLT_VERSION_MINOR; |
| 121 | return ACLBLAS_STATUS_SUCCESS; | 530 | return ACLBLAS_STATUS_SUCCESS; |
| 122 | case ACLBLASLT_PROPERTY_PATCH_LEVEL: | 531 | case ACLBLASLT_PROPERTY_PATCH_LEVEL: |
| 123 | - *value = kAclblasLtVersionPatch; | 532 | + *value = ACLBLASLT_VERSION_PATCH; |
| 124 | return ACLBLAS_STATUS_SUCCESS; | 533 | return ACLBLAS_STATUS_SUCCESS; |
| 125 | default: | 534 | default: |
| 126 | return ACLBLAS_STATUS_INVALID_VALUE; | 535 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 127 | } | 536 | } |
| 128 | } | 537 | } |
| 129 | 538 | ||
| 130 | -aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* handle) | 539 | +aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* lightHandle) |
| 131 | { | 540 | { |
| 132 | - if (handle == nullptr) { | 541 | + if (lightHandle == nullptr) { |
| 133 | return ACLBLAS_STATUS_INVALID_VALUE; | 542 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 134 | } | 543 | } |
| 135 | 544 | ||
| 136 | - LtHandle* h = nullptr; | 545 | + aclblasLtHandle* h = nullptr; |
| 137 | auto st = AllocHandle(&h); | 546 | auto st = AllocHandle(&h); |
| 138 | if (st != ACLBLAS_STATUS_SUCCESS) { | 547 | if (st != ACLBLAS_STATUS_SUCCESS) { |
| 139 | return st; | 548 | return st; |
| 140 | } | 549 | } |
| 141 | 550 | ||
| 142 | - // Get current device ID | ||
| 143 | int32_t deviceId = 0; | 551 | int32_t deviceId = 0; |
| 144 | aclError aclRet = aclrtGetDevice(&deviceId); | 552 | aclError aclRet = aclrtGetDevice(&deviceId); |
| 145 | if (aclRet != ACL_SUCCESS) { | 553 | if (aclRet != ACL_SUCCESS) { |
| @@ -147,24 +555,63 @@ aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* handle) | |||
| 147 | return ACLBLAS_STATUS_NOT_INITIALIZED; | 555 | return ACLBLAS_STATUS_NOT_INITIALIZED; |
| 148 | } | 556 | } |
| 149 | 557 | ||
| 558 | + aclrtContext currentCtx = nullptr; | ||
| 559 | + aclRet = aclrtGetCurrentContext(¤tCtx); | ||
| 560 | + if (aclRet != ACL_SUCCESS || currentCtx == nullptr) { | ||
| 561 | + delete h; | ||
| 562 | + return ACLBLAS_STATUS_NOT_INITIALIZED; | ||
| 563 | + } | ||
| 564 | + | ||
| 150 | h->deviceId = deviceId; | 565 | h->deviceId = deviceId; |
| 566 | + h->context = currentCtx; | ||
| 567 | + h->defaultStream = nullptr; | ||
| 568 | + h->workspaceSize = DEFAULT_WORKSPACE_SIZE; | ||
| 569 | + h->internalWorkspace = std::malloc(h->workspaceSize); | ||
| 570 | + if (h->internalWorkspace == nullptr) { | ||
| 571 | + delete h; | ||
| 572 | + return ACLBLAS_STATUS_ALLOC_FAILED; | ||
| 573 | + } | ||
| 574 | + | ||
| 575 | + h->mutex = new (std::nothrow) std::mutex(); | ||
| 576 | + h->algoCache = new (std::nothrow) std::unordered_map<AlgoKey, CacheEntry, AlgoKeyHasher>(); | ||
| 577 | + h->lruList = new (std::nothrow) std::list<AlgoKey>(); | ||
| 578 | + if (h->mutex == nullptr || h->algoCache == nullptr || h->lruList == nullptr) { | ||
| 579 | + delete h->mutex; | ||
| 580 | + delete h->algoCache; | ||
| 581 | + delete h->lruList; | ||
| 582 | + std::free(h->internalWorkspace); | ||
| 583 | + delete h; | ||
| 584 | + return ACLBLAS_STATUS_ALLOC_FAILED; | ||
| 585 | + } | ||
| 586 | + | ||
| 587 | + h->npuArch = 2; | ||
| 588 | + h->maxSharedMemory = L1_SIZE; | ||
| 151 | h->initialized = true; | 589 | h->initialized = true; |
| 152 | - *handle = reinterpret_cast<aclblasLtHandle_t>(h); | 590 | + *lightHandle = reinterpret_cast<aclblasLtHandle_t>(h); |
| 153 | return ACLBLAS_STATUS_SUCCESS; | 591 | return ACLBLAS_STATUS_SUCCESS; |
| 154 | } | 592 | } |
| 155 | 593 | ||
| 156 | -aclblasStatus_t aclblasLtDestroy(const aclblasLtHandle_t handle) | 594 | +aclblasStatus_t aclblasLtDestroy(const aclblasLtHandle_t lightHandle) |
| 157 | { | 595 | { |
| 158 | - if (handle == nullptr) { | 596 | + if (lightHandle == nullptr) { |
| 159 | return ACLBLAS_STATUS_INVALID_VALUE; | 597 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 160 | } | 598 | } |
| 161 | 599 | ||
| 162 | - auto* h = reinterpret_cast<LtHandle*>(handle); | 600 | + auto* h = reinterpret_cast<aclblasLtHandle*>(lightHandle); |
| 163 | - if (h->magic != 0x4C54484C) { | 601 | + if (h->magic != ACLBLASLT_HANDLE_MAGIC) { |
| 164 | return ACLBLAS_STATUS_INVALID_VALUE; | 602 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 165 | } | 603 | } |
| 166 | 604 | ||
| 167 | h->initialized = false; | 605 | h->initialized = false; |
| 606 | + delete h->mutex; | ||
| 607 | + delete h->algoCache; | ||
| 608 | + delete h->lruList; | ||
| 609 | + std::free(h->internalWorkspace); | ||
| 610 | + h->mutex = nullptr; | ||
| 611 | + h->algoCache = nullptr; | ||
| 612 | + h->lruList = nullptr; | ||
| 613 | + h->internalWorkspace = nullptr; | ||
| 614 | + h->workspaceSize = 0; | ||
| 168 | return FreeHandle(h); | 615 | return FreeHandle(h); |
| 169 | } | 616 | } |
| 170 | 617 | ||
| @@ -174,35 +621,33 @@ aclblasStatus_t aclblasLtMatrixLayoutCreate(aclblasLtMatrixLayout_t* layout, | |||
| 174 | uint64_t cols, | 621 | uint64_t cols, |
| 175 | int64_t ld) | 622 | int64_t ld) |
| 176 | { | 623 | { |
| 177 | - if (layout == nullptr) { | 624 | + // 1. 参数校验 |
| 625 | + if (layout == nullptr || rows == 0 || cols == 0 || ld < 0) { | ||
| 178 | return ACLBLAS_STATUS_INVALID_VALUE; | 626 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 179 | } | 627 | } |
| 180 | - | 628 | + *layout = nullptr; |
| 181 | - if (rows == 0 || cols == 0) { | 629 | + // 2. 堆上分配胶囊 |
| 182 | - return ACLBLAS_STATUS_INVALID_VALUE; | 630 | + auto* capsule = new (std::nothrow) aclblasLtMatrixLayoutOpaque_t(); |
| 631 | + if (capsule == nullptr) { | ||
| 632 | + return ACLBLAS_STATUS_ALLOC_FAILED; | ||
| 183 | } | 633 | } |
| 184 | - | 634 | + // 3. 栈上创建Impl，初始化后拷贝进胶囊 |
| 185 | - // ld must be >= rows for column major, >= cols for row major | 635 | + aclblasLtMatrixLayoutImpl impl; |
| 186 | - // For now, allow ld == 0 to use default | 636 | + impl.magic = ACLBLASLT_LAYOUT_MAGIC; |
| 187 | - if (ld < 0) { | 637 | + impl.type = type; |
| 188 | - return ACLBLAS_STATUS_INVALID_VALUE; | 638 | + impl.rows = rows; |
| 639 | + impl.cols = cols; | ||
| 640 | + impl.ld = (ld == 0) ? static_cast<int64_t>(rows) : ld; | ||
| 641 | + // 4. Impl → 胶囊 | ||
| 642 | + static_assert(sizeof(impl) <= sizeof(*capsule), "aclblasLtMatrixLayoutImpl too large, not fit in capsule!"); | ||
| 643 | + memcpy(capsule, &impl, sizeof(impl)); | ||
| 644 | + if (sizeof(*capsule) > sizeof(impl)) { | ||
| 645 | + memset(reinterpret_cast<char*>(capsule) + sizeof(impl), | ||
| 646 | + 0, | ||
| 647 | + sizeof(*capsule) - sizeof(impl)); | ||
| 189 | } | 648 | } |
| 190 | - | 649 | + // 5. 返回胶囊指针 |
| 191 | - MatrixLayout* l = nullptr; | 650 | + *layout = capsule; |
| 192 | - auto st = AllocHandle(&l); | ||
| 193 | - if (st != ACLBLAS_STATUS_SUCCESS) { | ||
| 194 | - return st; | ||
| 195 | - } | ||
| 196 | - | ||
| 197 | - l->type = type; | ||
| 198 | - l->rows = rows; | ||
| 199 | - l->cols = cols; | ||
| 200 | - l->ld = (ld == 0) ? static_cast<int64_t>(rows) : ld; // Default ld = rows for col-major | ||
| 201 | - l->order = ACLBLASLT_ORDER_COL; | ||
| 202 | - l->batchCount = 1; | ||
| 203 | - l->stridedBatchOffset = 0; | ||
| 204 | - | ||
| 205 | - *layout = reinterpret_cast<aclblasLtMatrixLayout_t>(l); | ||
| 206 | return ACLBLAS_STATUS_SUCCESS; | 651 | return ACLBLAS_STATUS_SUCCESS; |
| 207 | } | 652 | } |
| 208 | 653 | ||
| @@ -211,78 +656,89 @@ aclblasStatus_t aclblasLtMatrixLayoutDestroy(const aclblasLtMatrixLayout_t layou | |||
| 211 | if (layout == nullptr) { | 656 | if (layout == nullptr) { |
| 212 | return ACLBLAS_STATUS_INVALID_VALUE; | 657 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 213 | } | 658 | } |
| 214 | - auto* l = reinterpret_cast<MatrixLayout*>(layout); | 659 | + |
| 215 | - return FreeHandle(l); | 660 | + auto* capsule = reinterpret_cast<aclblasLtMatrixLayoutOpaque_t*>(layout); |
| 661 | + delete capsule; | ||
| 662 | + | ||
| 663 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 216 | } | 664 | } |
| 217 | 665 | ||
| 218 | aclblasStatus_t aclblasLtMatrixLayoutSetAttribute(aclblasLtMatrixLayout_t layout, | 666 | aclblasStatus_t aclblasLtMatrixLayoutSetAttribute(aclblasLtMatrixLayout_t layout, |
| 219 | - aclblasLtMatrixLayoutAttribute_t attr, | 667 | + aclblasLtMatrixLayoutAttribute_t attr, |
| 220 | - const void* buf, | 668 | + const void* buf, |
| 221 | - size_t sizeInBytes) | 669 | + size_t sizeInBytes) |
| 222 | { | 670 | { |
| 223 | if (layout == nullptr || buf == nullptr) { | 671 | if (layout == nullptr || buf == nullptr) { |
| 224 | return ACLBLAS_STATUS_INVALID_VALUE; | 672 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 225 | } | 673 | } |
| 226 | 674 | ||
| 227 | - auto* l = reinterpret_cast<MatrixLayout*>(layout); | 675 | + // 解包到栈上 |
| 676 | + aclblasLtMatrixLayoutImpl impl; | ||
| 677 | + memcpy(&impl, layout, sizeof(impl)); | ||
| 228 | 678 | ||
| 229 | switch (attr) { | 679 | switch (attr) { |
| 230 | - case ACLBLASLT_MATRIX_LAYOUT_ORDER: { | 680 | + case ACLBLASLT_MATRIX_LAYOUT_TYPE: |
| 231 | - if (sizeInBytes != sizeof(aclblasLtOrder_t) && sizeInBytes != sizeof(int32_t)) { | 681 | + if (sizeInBytes != sizeof(impl.type)) { |
| 232 | return ACLBLAS_STATUS_INVALID_VALUE; | 682 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 233 | } | 683 | } |
| 234 | - int32_t v = 0; | 684 | + impl.type = *reinterpret_cast<const aclDataType*>(buf); |
| 235 | - std::memcpy(&v, buf, sizeof(int32_t)); | 685 | + break; |
| 236 | - l->order = static_cast<aclblasLtOrder_t>(v); | 686 | + |
| 237 | - return ACLBLAS_STATUS_SUCCESS; | 687 | + case ACLBLASLT_MATRIX_LAYOUT_ROWS: |
| 238 | - } | 688 | + if (sizeInBytes != sizeof(impl.rows)) { |
| 239 | - case ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT: { | ||
| 240 | - if (sizeInBytes != sizeof(int32_t)) { | ||
| 241 | return ACLBLAS_STATUS_INVALID_VALUE; | 689 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 242 | } | 690 | } |
| 243 | - std::memcpy(&l->batchCount, buf, sizeof(int32_t)); | 691 | + impl.rows = *reinterpret_cast<const uint64_t*>(buf); |
| 244 | - return ACLBLAS_STATUS_SUCCESS; | 692 | + break; |
| 245 | - } | 693 | + |
| 246 | - case ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET: { | 694 | + case ACLBLASLT_MATRIX_LAYOUT_COLS: |
| 247 | - if (sizeInBytes != sizeof(int64_t)) { | 695 | + if (sizeInBytes != sizeof(impl.cols)) { |
| 248 | return ACLBLAS_STATUS_INVALID_VALUE; | 696 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 249 | } | 697 | } |
| 250 | - std::memcpy(&l->stridedBatchOffset, buf, sizeof(int64_t)); | 698 | + impl.cols = *reinterpret_cast<const uint64_t*>(buf); |
| 251 | - return ACLBLAS_STATUS_SUCCESS; | 699 | + break; |
| 252 | - } | 700 | + |
| 253 | - case ACLBLASLT_MATRIX_LAYOUT_ROWS: { | 701 | + case ACLBLASLT_MATRIX_LAYOUT_LD: |
| 254 | - if (sizeInBytes != sizeof(uint64_t)) { | 702 | + if (sizeInBytes != sizeof(impl.ld)) { |
| 255 | return ACLBLAS_STATUS_INVALID_VALUE; | 703 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 256 | } | 704 | } |
| 257 | - std::memcpy(&l->rows, buf, sizeof(uint64_t)); | 705 | + impl.ld = *reinterpret_cast<const int64_t*>(buf); |
| 258 | - return ACLBLAS_STATUS_SUCCESS; | 706 | + break; |
| 259 | - } | 707 | + |
| 260 | - case ACLBLASLT_MATRIX_LAYOUT_COLS: { | 708 | + case ACLBLASLT_MATRIX_LAYOUT_ORDER: |
| 261 | - if (sizeInBytes != sizeof(uint64_t)) { | 709 | + if (sizeInBytes != sizeof(impl.order)) { |
| 262 | return ACLBLAS_STATUS_INVALID_VALUE; | 710 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 263 | } | 711 | } |
| 264 | - std::memcpy(&l->cols, buf, sizeof(uint64_t)); | 712 | + impl.order = *reinterpret_cast<const aclblasLtOrder_t*>(buf); |
| 265 | - return ACLBLAS_STATUS_SUCCESS; | 713 | + break; |
| 266 | - } | 714 | + |
| 267 | - case ACLBLASLT_MATRIX_LAYOUT_LD: { | 715 | + case ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT: |
| 268 | - if (sizeInBytes != sizeof(int64_t)) { | 716 | + if (sizeInBytes != sizeof(impl.batchCount)) { |
| 269 | return ACLBLAS_STATUS_INVALID_VALUE; | 717 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 270 | } | 718 | } |
| 271 | - std::memcpy(&l->ld, buf, sizeof(int64_t)); | 719 | + impl.batchCount = *reinterpret_cast<const int32_t*>(buf); |
| 272 | - return ACLBLAS_STATUS_SUCCESS; | 720 | + break; |
| 273 | - } | 721 | + |
| 274 | - case ACLBLASLT_MATRIX_LAYOUT_TYPE: { | 722 | + case ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET: |
| 275 | - if (sizeInBytes != sizeof(int32_t)) { | 723 | + if (sizeInBytes != sizeof(impl.stridedBatchOffset)) { |
| 276 | return ACLBLAS_STATUS_INVALID_VALUE; | 724 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 277 | } | 725 | } |
| 278 | - int32_t typeVal = 0; | 726 | + impl.stridedBatchOffset = *reinterpret_cast<const int64_t*>(buf); |
| 279 | - std::memcpy(&typeVal, buf, sizeof(int32_t)); | 727 | + break; |
| 280 | - l->type = static_cast<aclDataType>(typeVal); | 728 | + |
| 281 | - return ACLBLAS_STATUS_SUCCESS; | ||
| 282 | - } | ||
| 283 | default: | 729 | default: |
| 284 | - return ACLBLAS_STATUS_NOT_SUPPORTED; | 730 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 285 | } | 731 | } |
| 732 | + | ||
| 733 | + // 压缩回堆上 | ||
| 734 | + memcpy(layout, &impl, sizeof(impl)); | ||
| 735 | + if (sizeof(*layout) > sizeof(impl)) { | ||
| 736 | + memset(reinterpret_cast<char*>(layout) + sizeof(impl), | ||
| 737 | + 0, | ||
| 738 | + sizeof(*layout) - sizeof(impl)); | ||
| 739 | + } | ||
| 740 | + | ||
| 741 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 286 | } | 742 | } |
| 287 | 743 | ||
| 288 | aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t layout, | 744 | aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t layout, |
| @@ -295,82 +751,78 @@ aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t | |||
| 295 | return ACLBLAS_STATUS_INVALID_VALUE; | 751 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 296 | } | 752 | } |
| 297 | 753 | ||
| 298 | - auto* l = reinterpret_cast<const MatrixLayout*>(layout); | 754 | + aclblasLtMatrixLayoutImpl impl; |
| 755 | + static_assert(sizeof(impl) <= sizeof(*layout), "aclblasLtMatrixLayoutImpl too large for capsule"); | ||
| 756 | + memcpy(&impl, layout, sizeof(impl)); | ||
| 757 | + | ||
| 758 | + size_t actualSize = 0; | ||
| 299 | 759 | ||
| 300 | switch (attr) { | 760 | switch (attr) { |
| 301 | - case ACLBLASLT_MATRIX_LAYOUT_ORDER: { | 761 | + case ACLBLASLT_MATRIX_LAYOUT_TYPE: |
| 302 | - if (sizeInBytes < sizeof(aclblasLtOrder_t)) { | 762 | + actualSize = sizeof(impl.type); |
| 303 | - return ACLBLAS_STATUS_INVALID_VALUE; | 763 | + if (sizeInBytes < actualSize) { |
| 304 | - } | 764 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 305 | - std::memcpy(buf, &l->order, sizeof(aclblasLtOrder_t)); | 765 | + } |
| 306 | - if (sizeWritten != nullptr) { | 766 | + *reinterpret_cast<aclDataType*>(buf) = impl.type; |
| 307 | - *sizeWritten = sizeof(aclblasLtOrder_t); | 767 | + break; |
| 308 | - } | 768 | + |
| 309 | - return ACLBLAS_STATUS_SUCCESS; | 769 | + case ACLBLASLT_MATRIX_LAYOUT_ROWS: |
| 310 | - } | 770 | + actualSize = sizeof(impl.rows); |
| 311 | - case ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT: { | 771 | + if (sizeInBytes < actualSize) { |
| 312 | - if (sizeInBytes < sizeof(int32_t)) { | 772 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 313 | - return ACLBLAS_STATUS_INVALID_VALUE; | 773 | + } |
| 314 | - } | 774 | + *reinterpret_cast<uint64_t*>(buf) = impl.rows; |
| 315 | - std::memcpy(buf, &l->batchCount, sizeof(int32_t)); | 775 | + break; |
| 316 | - if (sizeWritten != nullptr) { | 776 | + |
| 317 | - *sizeWritten = sizeof(int32_t); | 777 | + case ACLBLASLT_MATRIX_LAYOUT_COLS: |
| 318 | - } | 778 | + actualSize = sizeof(impl.cols); |
| 319 | - return ACLBLAS_STATUS_SUCCESS; | 779 | + if (sizeInBytes < actualSize) { |
| 320 | - } | 780 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 321 | - case ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET: { | 781 | + } |
| 322 | - if (sizeInBytes < sizeof(int64_t)) { | 782 | + *reinterpret_cast<uint64_t*>(buf) = impl.cols; |
| 323 | - return ACLBLAS_STATUS_INVALID_VALUE; | 783 | + break; |
| 324 | - } | 784 | + |
| 325 | - std::memcpy(buf, &l->stridedBatchOffset, sizeof(int64_t)); | 785 | + case ACLBLASLT_MATRIX_LAYOUT_LD: |
| 326 | - if (sizeWritten != nullptr) { | 786 | + actualSize = sizeof(impl.ld); |
| 327 | - *sizeWritten = sizeof(int64_t); | 787 | + if (sizeInBytes < actualSize) { |
| 328 | - } | 788 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 329 | - return ACLBLAS_STATUS_SUCCESS; | 789 | + } |
| 330 | - } | 790 | + *reinterpret_cast<int64_t*>(buf) = impl.ld; |
| 331 | - case ACLBLASLT_MATRIX_LAYOUT_ROWS: { | 791 | + break; |
| 332 | - if (sizeInBytes < sizeof(uint64_t)) { | 792 | + |
| 333 | - return ACLBLAS_STATUS_INVALID_VALUE; | 793 | + case ACLBLASLT_MATRIX_LAYOUT_ORDER: |
| 334 | - } | 794 | + actualSize = sizeof(impl.order); |
| 335 | - std::memcpy(buf, &l->rows, sizeof(uint64_t)); | 795 | + if (sizeInBytes < actualSize) { |
| 336 | - if (sizeWritten != nullptr) { | 796 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 337 | - *sizeWritten = sizeof(uint64_t); | 797 | + } |
| 338 | - } | 798 | + *reinterpret_cast<aclblasLtOrder_t*>(buf) = impl.order; |
| 339 | - return ACLBLAS_STATUS_SUCCESS; | 799 | + break; |
| 340 | - } | 800 | + |
| 341 | - case ACLBLASLT_MATRIX_LAYOUT_COLS: { | 801 | + case ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT: |
| 342 | - if (sizeInBytes < sizeof(uint64_t)) { | 802 | + actualSize = sizeof(impl.batchCount); |
| 343 | - return ACLBLAS_STATUS_INVALID_VALUE; | 803 | + if (sizeInBytes < actualSize) { |
| 344 | - } | 804 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 345 | - std::memcpy(buf, &l->cols, sizeof(uint64_t)); | 805 | + } |
| 346 | - if (sizeWritten != nullptr) { | 806 | + *reinterpret_cast<int32_t*>(buf) = impl.batchCount; |
| 347 | - *sizeWritten = sizeof(uint64_t); | 807 | + break; |
| 348 | - } | 808 | + |
| 349 | - return ACLBLAS_STATUS_SUCCESS; | 809 | + case ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET: |
| 350 | - } | 810 | + actualSize = sizeof(impl.stridedBatchOffset); |
| 351 | - case ACLBLASLT_MATRIX_LAYOUT_LD: { | 811 | + if (sizeInBytes < actualSize) { |
| 352 | - if (sizeInBytes < sizeof(int64_t)) { | 812 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 353 | - return ACLBLAS_STATUS_INVALID_VALUE; | 813 | + } |
| 354 | - } | 814 | + *reinterpret_cast<int64_t*>(buf) = impl.stridedBatchOffset; |
| 355 | - std::memcpy(buf, &l->ld, sizeof(int64_t)); | 815 | + break; |
| 356 | - if (sizeWritten != nullptr) { | 816 | + |
| 357 | - *sizeWritten = sizeof(int64_t); | ||
| 358 | - } | ||
| 359 | - return ACLBLAS_STATUS_SUCCESS; | ||
| 360 | - } | ||
| 361 | - case ACLBLASLT_MATRIX_LAYOUT_TYPE: { | ||
| 362 | - if (sizeInBytes < sizeof(aclDataType)) { | ||
| 363 | - return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 364 | - } | ||
| 365 | - std::memcpy(buf, &l->type, sizeof(aclDataType)); | ||
| 366 | - if (sizeWritten != nullptr) { | ||
| 367 | - *sizeWritten = sizeof(aclDataType); | ||
| 368 | - } | ||
| 369 | - return ACLBLAS_STATUS_SUCCESS; | ||
| 370 | - } | ||
| 371 | default: | 817 | default: |
| 372 | - return ACLBLAS_STATUS_NOT_SUPPORTED; | 818 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 373 | } | 819 | } |
| 820 | + | ||
| 821 | + if (sizeWritten != nullptr) { | ||
| 822 | + *sizeWritten = actualSize; | ||
| 823 | + } | ||
| 824 | + | ||
| 825 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 374 | } | 826 | } |
| 375 | 827 | ||
| 376 | aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* desc, | 828 | aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* desc, |
| @@ -380,22 +832,25 @@ aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* desc, | |||
| 380 | if (desc == nullptr) { | 832 | if (desc == nullptr) { |
| 381 | return ACLBLAS_STATUS_INVALID_VALUE; | 833 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 382 | } | 834 | } |
| 835 | + *desc = nullptr; | ||
| 383 | 836 | ||
| 384 | - MatmulDesc* d = nullptr; | 837 | + auto* capsule = new (std::nothrow) aclblasLtMatmulDescOpaque_t(); |
| 385 | - auto st = AllocHandle(&d); | 838 | + if (capsule == nullptr) { |
| 386 | - if (st != ACLBLAS_STATUS_SUCCESS) { | 839 | + return ACLBLAS_STATUS_ALLOC_FAILED; |
| 387 | - return st; | ||
| 388 | } | 840 | } |
| 389 | 841 | ||
| 390 | - d->computeType = computeType; | 842 | + aclblasLtMatmulDescImpl impl; |
| 391 | - d->scaleType = scaleType; | 843 | + impl.magic = ACLBLASLT_DESC_MAGIC; |
| 392 | - d->epilogue = ACLBLASLT_EPILOGUE_DEFAULT; | 844 | + impl.computeType = computeType; |
| 393 | - d->bias = nullptr; | 845 | + impl.scaleType = scaleType; |
| 394 | - d->transA = ACLBLAS_OP_N; | ||
| 395 | - d->transB = ACLBLAS_OP_N; | ||
| 396 | - d->biasDataType = ACL_FLOAT; | ||
| 397 | 846 | ||
| 398 | - *desc = reinterpret_cast<aclblasLtMatmulDesc_t>(d); | 847 | + static_assert(sizeof(impl) <= sizeof(*capsule), "aclblasLtMatmulDescImpl too large, not fit in capsule!"); |
| 848 | + std::memcpy(capsule, &impl, sizeof(impl)); | ||
| 849 | + if (sizeof(*capsule) > sizeof(impl)) { | ||
| 850 | + std::memset(reinterpret_cast<char*>(capsule) + sizeof(impl), 0, sizeof(*capsule) - sizeof(impl)); | ||
| 851 | + } | ||
| 852 | + | ||
| 853 | + *desc = capsule; | ||
| 399 | return ACLBLAS_STATUS_SUCCESS; | 854 | return ACLBLAS_STATUS_SUCCESS; |
| 400 | } | 855 | } |
| 401 | 856 | ||
| @@ -404,20 +859,23 @@ aclblasStatus_t aclblasLtMatmulDescDestroy(const aclblasLtMatmulDesc_t desc) | |||
| 404 | if (desc == nullptr) { | 859 | if (desc == nullptr) { |
| 405 | return ACLBLAS_STATUS_INVALID_VALUE; | 860 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 406 | } | 861 | } |
| 407 | - auto* d = reinterpret_cast<MatmulDesc*>(desc); | 862 | + |
| 408 | - return FreeHandle(d); | 863 | + auto* capsule = reinterpret_cast<aclblasLtMatmulDescOpaque_t*>(desc); |
| 864 | + delete capsule; | ||
| 865 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 409 | } | 866 | } |
| 410 | 867 | ||
| 411 | aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t desc, | 868 | aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t desc, |
| 412 | - aclblasLtMatmulDescAttribute_t attr, | 869 | + aclblasLtMatmulDescAttribute_t attr, |
| 413 | - const void* buf, | 870 | + const void* buf, |
| 414 | - size_t sizeInBytes) | 871 | + size_t sizeInBytes) |
| 415 | { | 872 | { |
| 416 | if (desc == nullptr || buf == nullptr) { | 873 | if (desc == nullptr || buf == nullptr) { |
| 417 | return ACLBLAS_STATUS_INVALID_VALUE; | 874 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 418 | } | 875 | } |
| 419 | 876 | ||
| 420 | - auto* d = reinterpret_cast<MatmulDesc*>(desc); | 877 | + aclblasLtMatmulDescImpl impl; |
| 878 | + std::memcpy(&impl, desc, sizeof(impl)); | ||
| 421 | 879 | ||
| 422 | switch (attr) { | 880 | switch (attr) { |
| 423 | case ACLBLASLT_MATMUL_DESC_EPILOGUE: { | 881 | case ACLBLASLT_MATMUL_DESC_EPILOGUE: { |
| @@ -426,24 +884,23 @@ aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t desc, | |||
| 426 | } | 884 | } |
| 427 | uint32_t v = 0; | 885 | uint32_t v = 0; |
| 428 | std::memcpy(&v, buf, sizeof(uint32_t)); | 886 | std::memcpy(&v, buf, sizeof(uint32_t)); |
| 429 | - d->epilogue = static_cast<aclblasLtEpilogue_t>(v); | 887 | + impl.epilogue = static_cast<aclblasLtEpilogue_t>(v); |
| 430 | - return ACLBLAS_STATUS_SUCCESS; | 888 | + break; |
| 431 | } | 889 | } |
| 432 | - case ACLBLASLT_MATMUL_DESC_BIAS_POINTER: { | 890 | + case ACLBLASLT_MATMUL_DESC_BIAS_POINTER: |
| 433 | if (sizeInBytes != sizeof(void*)) { | 891 | if (sizeInBytes != sizeof(void*)) { |
| 434 | return ACLBLAS_STATUS_INVALID_VALUE; | 892 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 435 | } | 893 | } |
| 436 | - std::memcpy(&d->bias, buf, sizeof(void*)); | 894 | + std::memcpy(&impl.bias, buf, sizeof(void*)); |
| 437 | - return ACLBLAS_STATUS_SUCCESS; | 895 | + break; |
| 438 | - } | ||
| 439 | case ACLBLASLT_MATMUL_DESC_TRANSA: { | 896 | case ACLBLASLT_MATMUL_DESC_TRANSA: { |
| 440 | if (sizeInBytes != sizeof(int32_t)) { | 897 | if (sizeInBytes != sizeof(int32_t)) { |
| 441 | return ACLBLAS_STATUS_INVALID_VALUE; | 898 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 442 | } | 899 | } |
| 443 | int32_t v = 0; | 900 | int32_t v = 0; |
| 444 | std::memcpy(&v, buf, sizeof(int32_t)); | 901 | std::memcpy(&v, buf, sizeof(int32_t)); |
| 445 | - d->transA = static_cast<aclblasOperation_t>(v); | 902 | + impl.transA = static_cast<aclblasOperation_t>(v); |
| 446 | - return ACLBLAS_STATUS_SUCCESS; | 903 | + break; |
| 447 | } | 904 | } |
| 448 | case ACLBLASLT_MATMUL_DESC_TRANSB: { | 905 | case ACLBLASLT_MATMUL_DESC_TRANSB: { |
| 449 | if (sizeInBytes != sizeof(int32_t)) { | 906 | if (sizeInBytes != sizeof(int32_t)) { |
| @@ -451,8 +908,8 @@ aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t desc, | |||
| 451 | } | 908 | } |
| 452 | int32_t v = 0; | 909 | int32_t v = 0; |
| 453 | std::memcpy(&v, buf, sizeof(int32_t)); | 910 | std::memcpy(&v, buf, sizeof(int32_t)); |
| 454 | - d->transB = static_cast<aclblasOperation_t>(v); | 911 | + impl.transB = static_cast<aclblasOperation_t>(v); |
| 455 | - return ACLBLAS_STATUS_SUCCESS; | 912 | + break; |
| 456 | } | 913 | } |
| 457 | case ACLBLASLT_MATMUL_DESC_BIAS_DATA_TYPE: { | 914 | case ACLBLASLT_MATMUL_DESC_BIAS_DATA_TYPE: { |
| 458 | if (sizeInBytes != sizeof(int32_t)) { | 915 | if (sizeInBytes != sizeof(int32_t)) { |
| @@ -460,12 +917,86 @@ aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t desc, | |||
| 460 | } | 917 | } |
| 461 | int32_t v = 0; | 918 | int32_t v = 0; |
| 462 | std::memcpy(&v, buf, sizeof(int32_t)); | 919 | std::memcpy(&v, buf, sizeof(int32_t)); |
| 463 | - d->biasDataType = static_cast<aclDataType>(v); | 920 | + impl.biasDataType = static_cast<aclDataType>(v); |
| 464 | - return ACLBLAS_STATUS_SUCCESS; | 921 | + break; |
| 465 | } | 922 | } |
| 466 | default: | 923 | default: |
| 467 | return ACLBLAS_STATUS_NOT_SUPPORTED; | 924 | return ACLBLAS_STATUS_NOT_SUPPORTED; |
| 468 | } | 925 | } |
| 926 | + | ||
| 927 | + std::memcpy(desc, &impl, sizeof(impl)); | ||
| 928 | + if (sizeof(*desc) > sizeof(impl)) { | ||
| 929 | + std::memset(reinterpret_cast<char*>(desc) + sizeof(impl), 0, sizeof(*desc) - sizeof(impl)); | ||
| 930 | + } | ||
| 931 | + | ||
| 932 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 933 | +} | ||
| 934 | + | ||
| 935 | +aclblasStatus_t aclblasLtMatmulDescGetAttribute(aclblasLtMatmulDesc_t desc, | ||
| 936 | + aclblasLtMatmulDescAttribute_t attr, | ||
| 937 | + void* buf, | ||
| 938 | + size_t sizeInBytes, | ||
| 939 | + size_t* sizeWritten) | ||
| 940 | +{ | ||
| 941 | + if (desc == nullptr || buf == nullptr) { | ||
| 942 | + return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 943 | + } | ||
| 944 | + | ||
| 945 | + aclblasLtMatmulDescImpl impl; | ||
| 946 | + std::memcpy(&impl, desc, sizeof(impl)); | ||
| 947 | + | ||
| 948 | + // if (!impl.valid()) { | ||
| 949 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 950 | + // } | ||
| 951 | + | ||
| 952 | + size_t requiredSize = 0; | ||
| 953 | + const void* srcPtr = nullptr; | ||
| 954 | + | ||
| 955 | + switch (attr) { | ||
| 956 | + case ACLBLASLT_MATMUL_DESC_EPILOGUE: | ||
| 957 | + requiredSize = sizeof(impl.epilogue); | ||
| 958 | + srcPtr = &impl.epilogue; | ||
| 959 | + break; | ||
| 960 | + | ||
| 961 | + case ACLBLASLT_MATMUL_DESC_BIAS_POINTER: | ||
| 962 | + requiredSize = sizeof(impl.bias); | ||
| 963 | + srcPtr = &impl.bias; | ||
| 964 | + break; | ||
| 965 | + | ||
| 966 | + case ACLBLASLT_MATMUL_DESC_TRANSA: | ||
| 967 | + requiredSize = sizeof(impl.transA); | ||
| 968 | + srcPtr = &impl.transA; | ||
| 969 | + break; | ||
| 970 | + | ||
| 971 | + case ACLBLASLT_MATMUL_DESC_TRANSB: | ||
| 972 | + requiredSize = sizeof(impl.transB); | ||
| 973 | + srcPtr = &impl.transB; | ||
| 974 | + break; | ||
| 975 | + | ||
| 976 | + case ACLBLASLT_MATMUL_DESC_BIAS_DATA_TYPE: | ||
| 977 | + requiredSize = sizeof(impl.biasDataType); | ||
| 978 | + srcPtr = &impl.biasDataType; | ||
| 979 | + break; | ||
| 980 | + | ||
| 981 | + default: | ||
| 982 | + return ACLBLAS_STATUS_NOT_SUPPORTED; | ||
| 983 | + } | ||
| 984 | + | ||
| 985 | + // 检查用户缓冲区大小 | ||
| 986 | + if (sizeInBytes < requiredSize) { | ||
| 987 | + if (sizeWritten != nullptr) { | ||
| 988 | + *sizeWritten = requiredSize; | ||
| 989 | + } | ||
| 990 | + return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 991 | + } | ||
| 992 | + | ||
| 993 | + std::memcpy(buf, srcPtr, requiredSize); | ||
| 994 | + | ||
| 995 | + if (sizeWritten != nullptr) { | ||
| 996 | + *sizeWritten = requiredSize; | ||
| 997 | + } | ||
| 998 | + | ||
| 999 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 469 | } | 1000 | } |
| 470 | 1001 | ||
| 471 | aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pref) | 1002 | aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pref) |
| @@ -474,16 +1005,20 @@ aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pre | |||
| 474 | return ACLBLAS_STATUS_INVALID_VALUE; | 1005 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 475 | } | 1006 | } |
| 476 | 1007 | ||
| 477 | - Preference* p = nullptr; | 1008 | + *pref = nullptr; |
| 478 | - auto st = AllocHandle(&p); | 1009 | + auto* capsule = new (std::nothrow) aclblasLtMatmulPreferenceOpaque_t(); |
| 479 | - if (st != ACLBLAS_STATUS_SUCCESS) { | 1010 | + if (capsule == nullptr) { |
| 480 | - return st; | 1011 | + return ACLBLAS_STATUS_ALLOC_FAILED; |
| 1012 | + } | ||
| 1013 | + std::memset(capsule, 0, sizeof(*capsule)); | ||
| 1014 | + | ||
| 1015 | + aclblasLtMatmulPreferenceImpl impl; | ||
| 1016 | + std::memcpy(capsule, &impl, sizeof(impl)); | ||
| 1017 | + if (sizeof(*capsule) > sizeof(impl)) { | ||
| 1018 | + std::memset(reinterpret_cast<char*>(capsule) + sizeof(impl), 0, sizeof(*capsule) - sizeof(impl)); | ||
| 481 | } | 1019 | } |
| 482 | 1020 | ||
| 483 | - p->maxWorkspaceBytes = 0; | 1021 | + *pref = capsule; |
| 484 | - p->searchMode = 0; | ||
| 485 | - | ||
| 486 | - *pref = reinterpret_cast<aclblasLtMatmulPreference_t>(p); | ||
| 487 | return ACLBLAS_STATUS_SUCCESS; | 1022 | return ACLBLAS_STATUS_SUCCESS; |
| 488 | } | 1023 | } |
| 489 | 1024 | ||
| @@ -492,8 +1027,10 @@ aclblasStatus_t aclblasLtMatmulPreferenceDestroy(const aclblasLtMatmulPreference | |||
| 492 | if (pref == nullptr) { | 1027 | if (pref == nullptr) { |
| 493 | return ACLBLAS_STATUS_INVALID_VALUE; | 1028 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 494 | } | 1029 | } |
| 495 | - auto* p = reinterpret_cast<Preference*>(pref); | 1030 | + |
| 496 | - return FreeHandle(p); | 1031 | + auto* capsule = reinterpret_cast<aclblasLtMatmulPreferenceOpaque_t*>(pref); |
| 1032 | + delete capsule; | ||
| 1033 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 497 | } | 1034 | } |
| 498 | 1035 | ||
| 499 | aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_t pref, | 1036 | aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_t pref, |
| @@ -505,40 +1042,234 @@ aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_ | |||
| 505 | return ACLBLAS_STATUS_INVALID_VALUE; | 1042 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 506 | } | 1043 | } |
| 507 | 1044 | ||
| 508 | - auto* p = reinterpret_cast<Preference*>(pref); | 1045 | + aclblasLtMatmulPreferenceImpl impl; |
| 1046 | + std::memcpy(&impl, pref, sizeof(impl)); | ||
| 509 | 1047 | ||
| 510 | switch (attr) { | 1048 | switch (attr) { |
| 511 | - case ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES: { | ||
| 512 | - if (sizeInBytes != sizeof(uint64_t) && sizeInBytes != sizeof(size_t)) { | ||
| 513 | - return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 514 | - } | ||
| 515 | - uint64_t v = 0; | ||
| 516 | - std::memcpy(&v, buf, sizeInBytes); | ||
| 517 | - p->maxWorkspaceBytes = static_cast<size_t>(v); | ||
| 518 | - return ACLBLAS_STATUS_SUCCESS; | ||
| 519 | - } | ||
| 520 | case ACLBLASLT_MATMUL_PREF_SEARCH_MODE: { | 1049 | case ACLBLASLT_MATMUL_PREF_SEARCH_MODE: { |
| 521 | if (sizeInBytes != sizeof(uint32_t)) { | 1050 | if (sizeInBytes != sizeof(uint32_t)) { |
| 522 | return ACLBLAS_STATUS_INVALID_VALUE; | 1051 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 523 | } | 1052 | } |
| 524 | - std::memcpy(&p->searchMode, buf, sizeof(uint32_t)); | 1053 | + uint32_t v = 0; |
| 525 | - return ACLBLAS_STATUS_SUCCESS; | 1054 | + std::memcpy(&v, buf, sizeof(v)); |
| 1055 | + // 0=heuristic, 1=exhaustive, 2=fast | ||
| 1056 | + if (v > 2) { | ||
| 1057 | + return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1058 | + } | ||
| 1059 | + impl.searchMode = v; | ||
| 1060 | + break; | ||
| 526 | } | 1061 | } |
| 1062 | + | ||
| 1063 | + case ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES: { | ||
| 1064 | + if (sizeInBytes != sizeof(size_t) && sizeInBytes != sizeof(uint64_t)) { | ||
| 1065 | + return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1066 | + } | ||
| 1067 | + size_t v = 0; | ||
| 1068 | + std::memcpy(&v, buf, std::min(sizeInBytes, sizeof(v))); | ||
| 1069 | + if (v > INT64_MAX) { | ||
| 1070 | + return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1071 | + } | ||
| 1072 | + impl.maxWorkspaceBytes = v; | ||
| 1073 | + break; | ||
| 1074 | + } | ||
| 1075 | + | ||
| 1076 | + // case ACLBLASLT_MATMUL_PREF_MAX_RESULTS: { | ||
| 1077 | + // if (sizeInBytes != sizeof(int32_t)) { | ||
| 1078 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1079 | + // } | ||
| 1080 | + // int32_t v = 0; | ||
| 1081 | + // std::memcpy(&v, buf, sizeof(v)); | ||
| 1082 | + // if (v <= 0 || v > 10) { | ||
| 1083 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1084 | + // } | ||
| 1085 | + // impl.maxResults = v; | ||
| 1086 | + // break; | ||
| 1087 | + // } | ||
| 1088 | + | ||
| 1089 | + // case ACLBLASLT_MATMUL_PREF_ALLOW_MIXED_PRECISION: { | ||
| 1090 | + // if (sizeInBytes != sizeof(bool) && sizeInBytes != sizeof(int32_t)) { | ||
| 1091 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1092 | + // } | ||
| 1093 | + // bool v = false; | ||
| 1094 | + // std::memcpy(&v, buf, sizeof(bool)); | ||
| 1095 | + // impl.allowMixedPrecision = v; | ||
| 1096 | + // break; | ||
| 1097 | + // } | ||
| 1098 | + | ||
| 1099 | + // case ACLBLASLT_MATMUL_PREF_ALLOW_SPLIT_K: { | ||
| 1100 | + // if (sizeInBytes != sizeof(bool) && sizeInBytes != sizeof(int32_t)) { | ||
| 1101 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1102 | + // } | ||
| 1103 | + // bool v = false; | ||
| 1104 | + // std::memcpy(&v, buf, sizeof(bool)); | ||
| 1105 | + // impl.allowSplitK = v; | ||
| 1106 | + // break; | ||
| 1107 | + // } | ||
| 1108 | + | ||
| 1109 | + // case ACLBLASLT_MATMUL_PREF_L0_TILE_M: { | ||
| 1110 | + // if (sizeInBytes != sizeof(uint32_t)) { | ||
| 1111 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1112 | + // } | ||
| 1113 | + // std::memcpy(&impl.preferredL0M, buf, sizeof(uint32_t)); | ||
| 1114 | + // break; | ||
| 1115 | + // } | ||
| 1116 | + | ||
| 1117 | + // case ACLBLASLT_MATMUL_PREF_L0_TILE_N: { | ||
| 1118 | + // if (sizeInBytes != sizeof(uint32_t)) { | ||
| 1119 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1120 | + // } | ||
| 1121 | + // std::memcpy(&impl.preferredL0N, buf, sizeof(uint32_t)); | ||
| 1122 | + // break; | ||
| 1123 | + // } | ||
| 1124 | + | ||
| 1125 | + // case ACLBLASLT_MATMUL_PREF_L0_TILE_K: { | ||
| 1126 | + // if (sizeInBytes != sizeof(uint32_t)) { | ||
| 1127 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1128 | + // } | ||
| 1129 | + // std::memcpy(&impl.preferredL0K, buf, sizeof(uint32_t)); | ||
| 1130 | + // break; | ||
| 1131 | + // } | ||
| 1132 | + | ||
| 1133 | + // case ACLBLASLT_MATMUL_PREF_PREFER_PINGPONG: { | ||
| 1134 | + // if (sizeInBytes != sizeof(bool)) { | ||
| 1135 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1136 | + // } | ||
| 1137 | + // std::memcpy(&impl.preferPingpong, buf, sizeof(bool)); | ||
| 1138 | + // break; | ||
| 1139 | + // } | ||
| 1140 | + | ||
| 1141 | + // case ACLBLASLT_MATMUL_PREF_PREFER_DOUBLE_BUFFER: { | ||
| 1142 | + // if (sizeInBytes != sizeof(bool)) { | ||
| 1143 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1144 | + // } | ||
| 1145 | + // std::memcpy(&impl.preferDoubleBuffer, buf, sizeof(bool)); | ||
| 1146 | + // break; | ||
| 1147 | + // } | ||
| 1148 | + | ||
| 1149 | + // case ACLBLASLT_MATMUL_PREF_MIN_EFFICIENCY: { | ||
| 1150 | + // if (sizeInBytes != sizeof(float)) { | ||
| 1151 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1152 | + // } | ||
| 1153 | + // float v = 0.0f; | ||
| 1154 | + // std::memcpy(&v, buf, sizeof(v)); | ||
| 1155 | + // if (v < 0.0f || v > 1.0f) { | ||
| 1156 | + // return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1157 | + // } | ||
| 1158 | + // impl.minEfficiency = v; | ||
| 1159 | + // break; | ||
| 1160 | + // } | ||
| 1161 | + | ||
| 527 | default: | 1162 | default: |
| 528 | return ACLBLAS_STATUS_NOT_SUPPORTED; | 1163 | return ACLBLAS_STATUS_NOT_SUPPORTED; |
| 529 | } | 1164 | } |
| 1165 | + | ||
| 1166 | + std::memcpy(pref, &impl, sizeof(impl)); | ||
| 1167 | + | ||
| 1168 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 530 | } | 1169 | } |
| 531 | 1170 | ||
| 532 | -aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | 1171 | +aclblasStatus_t aclblasLtMatmulPreferenceGetAttribute(aclblasLtMatmulPreference_t pref, |
| 533 | - aclblasLtMatmulDesc_t matmulDesc, | 1172 | + aclblasLtMatmulPreferenceAttribute_t attr, |
| 534 | - aclblasLtMatrixLayout_t Adesc, | 1173 | + void* buf, |
| 535 | - aclblasLtMatrixLayout_t Bdesc, | 1174 | + size_t sizeInBytes, |
| 536 | - aclblasLtMatrixLayout_t Cdesc, | 1175 | + size_t* sizeWritten) |
| 537 | - aclblasLtMatrixLayout_t Ddesc, | 1176 | +{ |
| 538 | - aclblasLtMatmulPreference_t pref, | 1177 | + if (pref == nullptr || buf == nullptr) { |
| 539 | - int requestedAlgoCount, | 1178 | + return ACLBLAS_STATUS_INVALID_VALUE; |
| 540 | - aclblasLtMatmulHeuristicResult_t heuristicResultsArray[], | 1179 | + } |
| 541 | - int* returnAlgoCount) | 1180 | + |
| 1181 | + aclblasLtMatmulPreferenceImpl impl; | ||
| 1182 | + std::memcpy(&impl, pref, sizeof(impl)); | ||
| 1183 | + | ||
| 1184 | + size_t requiredSize = 0; | ||
| 1185 | + const void* srcPtr = nullptr; | ||
| 1186 | + | ||
| 1187 | + switch (attr) { | ||
| 1188 | + case ACLBLASLT_MATMUL_PREF_SEARCH_MODE: | ||
| 1189 | + requiredSize = sizeof(impl.searchMode); | ||
| 1190 | + srcPtr = &impl.searchMode; | ||
| 1191 | + break; | ||
| 1192 | + | ||
| 1193 | + case ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES: | ||
| 1194 | + requiredSize = sizeof(impl.maxWorkspaceBytes); | ||
| 1195 | + srcPtr = &impl.maxWorkspaceBytes; | ||
| 1196 | + break; | ||
| 1197 | + | ||
| 1198 | + // case ACLBLASLT_MATMUL_PREF_MAX_RESULTS: | ||
| 1199 | + // requiredSize = sizeof(impl.maxResults); | ||
| 1200 | + // srcPtr = &impl.maxResults; | ||
| 1201 | + // break; | ||
| 1202 | + | ||
| 1203 | + // case ACLBLASLT_MATMUL_PREF_ALLOW_MIXED_PRECISION: | ||
| 1204 | + // requiredSize = sizeof(impl.allowMixedPrecision); | ||
| 1205 | + // srcPtr = &impl.allowMixedPrecision; | ||
| 1206 | + // break; | ||
| 1207 | + | ||
| 1208 | + // case ACLBLASLT_MATMUL_PREF_ALLOW_SPLIT_K: | ||
| 1209 | + // requiredSize = sizeof(impl.allowSplitK); | ||
| 1210 | + // srcPtr = &impl.allowSplitK; | ||
| 1211 | + // break; | ||
| 1212 | + | ||
| 1213 | + // case ACLBLASLT_MATMUL_PREF_L0_TILE_M: | ||
| 1214 | + // requiredSize = sizeof(impl.preferredL0M); | ||
| 1215 | + // srcPtr = &impl.preferredL0M; | ||
| 1216 | + // break; | ||
| 1217 | + | ||
| 1218 | + // case ACLBLASLT_MATMUL_PREF_L0_TILE_N: | ||
| 1219 | + // requiredSize = sizeof(impl.preferredL0N); | ||
| 1220 | + // srcPtr = &impl.preferredL0N; | ||
| 1221 | + // break; | ||
| 1222 | + | ||
| 1223 | + // case ACLBLASLT_MATMUL_PREF_L0_TILE_K: | ||
| 1224 | + // requiredSize = sizeof(impl.preferredL0K); | ||
| 1225 | + // srcPtr = &impl.preferredL0K; | ||
| 1226 | + // break; | ||
| 1227 | + | ||
| 1228 | + // case ACLBLASLT_MATMUL_PREF_PREFER_PINGPONG: | ||
| 1229 | + // requiredSize = sizeof(impl.preferPingpong); | ||
| 1230 | + // srcPtr = &impl.preferPingpong; | ||
| 1231 | + // break; | ||
| 1232 | + | ||
| 1233 | + // case ACLBLASLT_MATMUL_PREF_PREFER_DOUBLE_BUFFER: | ||
| 1234 | + // requiredSize = sizeof(impl.preferDoubleBuffer); | ||
| 1235 | + // srcPtr = &impl.preferDoubleBuffer; | ||
| 1236 | + // break; | ||
| 1237 | + | ||
| 1238 | + // case ACLBLASLT_MATMUL_PREF_MIN_EFFICIENCY: | ||
| 1239 | + // requiredSize = sizeof(impl.minEfficiency); | ||
| 1240 | + // srcPtr = &impl.minEfficiency; | ||
| 1241 | + // break; | ||
| 1242 | + | ||
| 1243 | + default: | ||
| 1244 | + return ACLBLAS_STATUS_NOT_SUPPORTED; | ||
| 1245 | + } | ||
| 1246 | + | ||
| 1247 | + if (sizeInBytes < requiredSize) { | ||
| 1248 | + if (sizeWritten != nullptr) { | ||
| 1249 | + *sizeWritten = requiredSize; | ||
| 1250 | + } | ||
| 1251 | + return ACLBLAS_STATUS_INVALID_VALUE; | ||
| 1252 | + } | ||
| 1253 | + | ||
| 1254 | + std::memcpy(buf, srcPtr, requiredSize); | ||
| 1255 | + | ||
| 1256 | + if (sizeWritten != nullptr) { | ||
| 1257 | + *sizeWritten = requiredSize; | ||
| 1258 | + } | ||
| 1259 | + | ||
| 1260 | + return ACLBLAS_STATUS_SUCCESS; | ||
| 1261 | +} | ||
| 1262 | + | ||
| 1263 | +aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t lightHandle, | ||
| 1264 | + aclblasLtMatmulDesc_t computeDesc, | ||
| 1265 | + aclblasLtMatrixLayout_t Adesc, | ||
| 1266 | + aclblasLtMatrixLayout_t Bdesc, | ||
| 1267 | + aclblasLtMatrixLayout_t Cdesc, | ||
| 1268 | + aclblasLtMatrixLayout_t Ddesc, | ||
| 1269 | + aclblasLtMatmulPreference_t preference, | ||
| 1270 | + int requestedAlgoCount, | ||
| 1271 | + aclblasLtMatmulHeuristicResult_t heuristicResultsArray[], | ||
| 1272 | + int* returnAlgoCount) | ||
| 542 | { | 1273 | { |
| 543 | // Validate input parameters | 1274 | // Validate input parameters |
| 544 | if (returnAlgoCount == nullptr) { | 1275 | if (returnAlgoCount == nullptr) { |
| @@ -550,7 +1281,7 @@ aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | |||
| 550 | return ACLBLAS_STATUS_INVALID_VALUE; | 1281 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 551 | } | 1282 | } |
| 552 | 1283 | ||
| 553 | - if (handle == nullptr || matmulDesc == nullptr) { | 1284 | + if (lightHandle == nullptr || computeDesc == nullptr) { |
| 554 | return ACLBLAS_STATUS_INVALID_VALUE; | 1285 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 555 | } | 1286 | } |
| 556 | 1287 | ||
| @@ -560,16 +1291,16 @@ aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | |||
| 560 | 1291 | ||
| 561 | // Get workspace size from preference | 1292 | // Get workspace size from preference |
| 562 | size_t maxWorkspace = 0; | 1293 | size_t maxWorkspace = 0; |
| 563 | - if (pref != nullptr) { | 1294 | + if (preference != nullptr) { |
| 564 | - auto* p = reinterpret_cast<Preference*>(pref); | 1295 | + auto* p = reinterpret_cast<aclblasLtMatmulPreferenceImpl*>(preference); |
| 565 | maxWorkspace = p->maxWorkspaceBytes; | 1296 | maxWorkspace = p->maxWorkspaceBytes; |
| 566 | } | 1297 | } |
| 567 | 1298 | ||
| 568 | // Get matrix dimensions | 1299 | // Get matrix dimensions |
| 569 | - auto* A = reinterpret_cast<MatrixLayout*>(Adesc); | 1300 | + auto* A = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Adesc); |
| 570 | - auto* B = reinterpret_cast<MatrixLayout*>(Bdesc); | 1301 | + auto* B = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Bdesc); |
| 571 | - auto* D = reinterpret_cast<MatrixLayout*>(Ddesc); | 1302 | + auto* D = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Ddesc); |
| 572 | - auto* desc = reinterpret_cast<MatmulDesc*>(matmulDesc); | 1303 | + auto* desc = reinterpret_cast<aclblasLtMatmulDescImpl*>(computeDesc); |
| 573 | 1304 | ||
| 574 | // Validate dimensions for GEMM: D = A * B + C | 1305 | // Validate dimensions for GEMM: D = A * B + C |
| 575 | // A: m x k, B: k x n, C/D: m x n | 1306 | // A: m x k, B: k x n, C/D: m x n |
| @@ -584,7 +1315,6 @@ aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | |||
| 584 | } | 1315 | } |
| 585 | 1316 | ||
| 586 | // Fill heuristic result | 1317 | // Fill heuristic result |
| 587 | - // heuristicResultsArray[0].algo.algoId = 0; | ||
| 588 | heuristicResultsArray[0].algo.max_workspace_bytes = maxWorkspace; | 1318 | heuristicResultsArray[0].algo.max_workspace_bytes = maxWorkspace; |
| 589 | heuristicResultsArray[0].workspaceSize = maxWorkspace; | 1319 | heuristicResultsArray[0].workspaceSize = maxWorkspace; |
| 590 | heuristicResultsArray[0].state = ACLBLAS_STATUS_SUCCESS; | 1320 | heuristicResultsArray[0].state = ACLBLAS_STATUS_SUCCESS; |
| @@ -595,8 +1325,8 @@ aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | |||
| 595 | return ACLBLAS_STATUS_SUCCESS; | 1325 | return ACLBLAS_STATUS_SUCCESS; |
| 596 | } | 1326 | } |
| 597 | 1327 | ||
| 598 | -aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | 1328 | +aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t lightHandle, |
| 599 | - aclblasLtMatmulDesc_t matmulDesc, | 1329 | + aclblasLtMatmulDesc_t computeDesc, |
| 600 | const void* alpha, | 1330 | const void* alpha, |
| 601 | const void* A, | 1331 | const void* A, |
| 602 | aclblasLtMatrixLayout_t Adesc, | 1332 | aclblasLtMatrixLayout_t Adesc, |
| @@ -612,18 +1342,13 @@ aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | |||
| 612 | size_t workspaceSizeInBytes, | 1342 | size_t workspaceSizeInBytes, |
| 613 | aclrtStream stream) | 1343 | aclrtStream stream) |
| 614 | { | 1344 | { |
| 615 | - // Validate handle | 1345 | + // Validate lightHandle |
| 616 | - if (handle == nullptr) { | 1346 | + if (lightHandle == nullptr) { |
| 617 | - return ACLBLAS_STATUS_NOT_INITIALIZED; | ||
| 618 | - } | ||
| 619 | - | ||
| 620 | - auto* h = reinterpret_cast<LtHandle*>(handle); | ||
| 621 | - if (!h->initialized || h->magic != 0x4C54484C) { | ||
| 622 | return ACLBLAS_STATUS_NOT_INITIALIZED; | 1347 | return ACLBLAS_STATUS_NOT_INITIALIZED; |
| 623 | } | 1348 | } |
| 624 | 1349 | ||
| 625 | // Validate descriptors | 1350 | // Validate descriptors |
| 626 | - if (matmulDesc == nullptr || Adesc == nullptr || Bdesc == nullptr || | 1351 | + if (computeDesc == nullptr || Adesc == nullptr || Bdesc == nullptr || |
| 627 | Cdesc == nullptr || Ddesc == nullptr) { | 1352 | Cdesc == nullptr || Ddesc == nullptr) { |
| 628 | return ACLBLAS_STATUS_INVALID_VALUE; | 1353 | return ACLBLAS_STATUS_INVALID_VALUE; |
| 629 | } | 1354 | } |
| @@ -638,11 +1363,11 @@ aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | |||
| 638 | } | 1363 | } |
| 639 | 1364 | ||
| 640 | // Get layout info | 1365 | // Get layout info |
| 641 | - auto* ALayout = reinterpret_cast<MatrixLayout*>(Adesc); | 1366 | + auto* ALayout = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Adesc); |
| 642 | - auto* BLayout = reinterpret_cast<MatrixLayout*>(Bdesc); | 1367 | + auto* BLayout = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Bdesc); |
| 643 | - auto* CLayout = reinterpret_cast<MatrixLayout*>(Cdesc); | 1368 | + auto* CLayout = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Cdesc); |
| 644 | - auto* DLayout = reinterpret_cast<MatrixLayout*>(Ddesc); | 1369 | + auto* DLayout = reinterpret_cast<aclblasLtMatrixLayoutImpl*>(Ddesc); |
| 645 | - auto* desc = reinterpret_cast<MatmulDesc*>(matmulDesc); | 1370 | + auto* desc = reinterpret_cast<aclblasLtMatmulDescImpl*>(computeDesc); |
| 646 | 1371 | ||
| 647 | // Get dimensions | 1372 | // Get dimensions |
| 648 | uint64_t m = DLayout->rows; | 1373 | uint64_t m = DLayout->rows; |
| @@ -681,4 +1406,4 @@ aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | |||
| 681 | return ACLBLAS_STATUS_SUCCESS; | 1406 | return ACLBLAS_STATUS_SUCCESS; |
| 682 | } | 1407 | } |
| 683 | 1408 | ||
| 684 | -} // extern "C" | 1409 | +} // extern "C" |
| @@ -0,0 +1,163 @@ | |||
| 1 | +# 快速入门：基于ops-blas仓 | ||
| 2 | + | ||
| 3 | +## 使用须知 | ||
| 4 | + | ||
| 5 | +本指南旨在帮助您快速上手CANN和`ops-blas`算子仓的使用。为方便快速了解算子开发全流程，将以**Copy**算子为实践对象，其源文件位于`ops-blas/blas/copy`，具体操作流程如下： | ||
| 6 | + | ||
| 7 | +1. **[环境部署](zh/install/quick_install.md)**：完成软件包安装和源码下载，此处不再赘述。快速入门场景下，**推荐WebIDE或Docker环境**，安装操作简单。 | ||
| 8 | + | ||
| 9 | + > **说明**：当前WebIDE或Docker环境默认最新商发版CANN包；如需体验master分支最新能力，可手动安装CANN包，注意软件与源码版本配套。 | ||
| 10 | + | ||
| 11 | +2. **[编译运行](#一编译运行)**：编译自定义算子包并安装，实现快速调用算子。 | ||
| 12 | + | ||
| 13 | +3. **[算子开发](#二算子开发)**：通过修改现有算子Kernel，体验开发、编译、验证的完整闭环。 | ||
| 14 | + | ||
| 15 | +4. **[算子调试](#三算子调试)**：掌握算子打印和性能采集方法。 | ||
| 16 | + | ||
| 17 | +## 一、编译运行 | ||
| 18 | + | ||
| 19 | +本阶段目的是**快速体验项目标准流程**，验证环境能否成功进行算子源码编译、打包、安装和运行。 | ||
| 20 | + | ||
| 21 | +### 1. 编译Copy算子 | ||
| 22 | + | ||
| 23 | +环境准备好后（注意软件与源码版本配套），进入环境并访问项目源码根目录，编译指定算子。 | ||
| 24 | + | ||
| 25 | +通用编译命令格式：`bash build.sh --pkg --soc=<芯片版本> --op=<算子名>`。以Copy算子为例，编译命令如下： | ||
| 26 | + | ||
| 27 | +```bash | ||
| 28 | +bash build.sh --pkg --soc=ascend950 --op=scopy | ||
| 29 | +``` | ||
| 30 | + | ||
| 31 | +若提示如下信息，说明编译成功。 | ||
| 32 | +```bash | ||
| 33 | +Self-extractable archive "cann-ops-blas_${cann_version}_linux-${arch}.run" successfully created. | ||
| 34 | +``` | ||
| 35 | +编译成功后，run包存放于项目根目录的build_out目录下。 | ||
| 36 | + | ||
| 37 | +### 2. 安装Copy算子包 | ||
| 38 | +```bash | ||
| 39 | +./build_out/cann-ops-blas-*linux*.run | ||
| 40 | +``` | ||
| 41 | + | ||
| 42 | +### 3. 快速验证：运行算子样例 | ||
| 43 | + | ||
| 44 | +通用的运行命令格式：`bash build.sh --soc=<芯片版本> --op=<算子名> --run`。 | ||
| 45 | + | ||
| 46 | +以Copy为例，其提供了简单算子样例`test/scopy/scopy_test.cpp`，运行该样例验证算子功能是否正常。 | ||
| 47 | + | ||
| 48 | +```bash | ||
| 49 | +bash build.sh --pkg --soc=ascend950 --op=scopy --run | ||
| 50 | +``` | ||
| 51 | +预期输出：打印算子`Copy`的计算结果，表明算子已成功部署并正确执行。 | ||
| 52 | + | ||
| 53 | +``` | ||
| 54 | +Running scopy_test... | ||
| 55 | +Output: 1.2 1.2 1.2 1.2 1.2 1.2 1.2 1.2 ... | ||
| 56 | +Golden: 1.2 1.2 1.2 1.2 1.2 1.2 1.2 1.2 ... | ||
| 57 | +[Success] Case accuracy is verification passed. | ||
| 58 | +``` | ||
| 59 | + | ||
| 60 | +## 二、算子开发 | ||
| 61 | + | ||
| 62 | +本阶段目的是对已成功运行的Copy算子尝试**修改核函数代码**。 | ||
| 63 | + | ||
| 64 | +### 1. 修改Kernel实现 | ||
| 65 | +找到Copy算子的核心kernel实现文件`blas/copy/scopy_kernel.cpp`，尝试修改算子中的DataCopy操作： | ||
| 66 | + | ||
| 67 | +```cpp | ||
| 68 | +template <typename T> | ||
| 69 | +__aicore__ inline void CopyAIV<T>::SingleIteration(uint32_t curOffset, uint32_t dataCount) | ||
| 70 | +{ | ||
| 71 | + LocalTensor<T> inLocal = inQueue.AllocTensor<T>(); | ||
| 72 | + DataCopy(inLocal, inGM[curOffset], dataCount); | ||
| 73 | + inQueue.EnQue<T>(inLocal); | ||
| 74 | + int32_t eventIDMTE2ToMTE3 = static_cast<int32_t>(GetTPipePtr()->FetchEventID(AscendC::HardEvent::MTE2_MTE3)); | ||
| 75 | + AscendC::SetFlag<AscendC::HardEvent::MTE2_MTE3>(eventIDMTE2ToMTE3); | ||
| 76 | + AscendC::WaitFlag<AscendC::HardEvent::MTE2_MTE3>(eventIDMTE2ToMTE3); | ||
| 77 | + LocalTensor<T> outLocal = inQueue.DeQue<T>(); | ||
| 78 | + // DataCopy(outGM[curOffset], outLocal, dataCount); | ||
| 79 | + // 补充相应的AIV计算操作 | ||
| 80 | + inQueue.FreeTensor(outLocal); | ||
| 81 | + int32_t eventIDMTE3ToMTE2 = static_cast<int32_t>(GetTPipePtr()->FetchEventID(AscendC::HardEvent::MTE3_MTE2)); | ||
| 82 | + AscendC::SetFlag<AscendC::HardEvent::MTE3_MTE2>(eventIDMTE3ToMTE2); | ||
| 83 | + AscendC::WaitFlag<AscendC::HardEvent::MTE3_MTE2>(eventIDMTE3ToMTE2); | ||
| 84 | +} | ||
| 85 | +``` | ||
| 86 | + | ||
| 87 | +### 2. 编译与验证 | ||
| 88 | + | ||
| 89 | +重复[编译运行](#一编译运行)章节中的步骤： | ||
| 90 | + | ||
| 91 | +1. **重新编译**： | ||
| 92 | + 先回到项目根目录，编译命令如下： | ||
| 93 | + | ||
| 94 | + ```bash | ||
| 95 | + bash build.sh --pkg --soc=ascend950 --op=scopy | ||
| 96 | + ``` | ||
| 97 | + | ||
| 98 | +2. **重新安装**： | ||
| 99 | + ```bash | ||
| 100 | + ./build_out/cann-ops-blas-*linux*.run | ||
| 101 | + ``` | ||
| 102 | + | ||
| 103 | +3. **重新验证**： | ||
| 104 | + ```bash | ||
| 105 | + bash build.sh --soc=ascend950 --op=scopy --run | ||
| 106 | + ``` | ||
| 107 | + | ||
| 108 | +4. **成功标志**：输出结果精度比对成功。 | ||
| 109 | + ``` | ||
| 110 | + Running scopy_test... | ||
| 111 | + Output: 1.2 1.2 1.2 1.2 1.2 1.2 1.2 1.2 ... | ||
| 112 | + Golden: 1.2 1.2 1.2 1.2 1.2 1.2 1.2 1.2 ... | ||
| 113 | + [Success] Case accuracy is verification passed. | ||
| 114 | + ``` | ||
| 115 | + | ||
| 116 | +## 三、算子调试 | ||
| 117 | +本阶段以Copy为例，在算子中添加打印并采集算子性能数据，以便后续问题分析定位。 | ||
| 118 | + | ||
| 119 | +### 1. 打印 | ||
| 120 | +算子如果出现执行失败、精度异常等问题，添加打印进行问题分析和定位。 | ||
| 121 | + | ||
| 122 | +请在`blas/copy/scopy_kernel.cpp`中进行代码修改。 | ||
| 123 | + | ||
| 124 | +* **printf** | ||
| 125 | + | ||
| 126 | + 该接口支持打印Scalar类型数据，如整数、字符型、布尔型等，详细介绍请参见[《Ascend C API》](https://hiascend.com/document/redirect/CannCommunityAscendCApi)中“算子调测API > printf”。 | ||
| 127 | + | ||
| 128 | + ```cpp | ||
| 129 | + blockLength_ = (tilingData->totalLength + AscendC::GetBlockNum() - 1) / AscendC::GetBlockNum(); | ||
| 130 | + tileNum_ = tilingData->tileNum; | ||
| 131 | + tileLength_ = ((blockLength_ + tileNum_ - 1) / tileNum_ / BUFFER_NUM) ? | ||
| 132 | + ((blockLength_ + tileNum_ - 1) / tileNum_ / BUFFER_NUM) : 1; | ||
| 133 | + // 打印当前核计算Block长度 | ||
| 134 | + AscendC::PRINTF("Tiling blockLength is %llu\n", blockLength_); | ||
| 135 | + ``` | ||
| 136 | +* **DumpTensor** | ||
| 137 | + | ||
| 138 | + 该接口支持Dump指定Tensor的内容，同时支持打印自定义附加信息，比如当前行号等，详细介绍请参见[《Ascend C API》](https://hiascend.com/document/redirect/CannCommunityAscendCApi)中“算子调测API > DumpTensor”。 | ||
| 139 | + | ||
| 140 | + ```cpp | ||
| 141 | + AscendC::LocalTensor<T> zLocal = outputQueueZ.DeQue<T>(); | ||
| 142 | + // 打印zLocal Tensor信息 | ||
| 143 | + DumpTensor(zLocal, 0, 128); | ||
| 144 | + ``` | ||
| 145 | +### 2. 性能采集 | ||
| 146 | + | ||
| 147 | +当算子功能验证正确后，可通过`msprof`工具采集算子性能数据。 | ||
| 148 | + | ||
| 149 | +- **生成可执行文件** | ||
| 150 | + | ||
| 151 | + 调用Copy算子的test样例，生成可执行文件（scopy_test），该文件位于项目`ops-blas/build/test/scopy`目录。 | ||
| 152 | + ```bash | ||
| 153 | + bash build.sh --soc=ascend950 --op=scopy | ||
| 154 | + ``` | ||
| 155 | + | ||
| 156 | +- **采集性能数据** | ||
| 157 | + | ||
| 158 | + 进入Copy算子可执行文件目录`ops-blas/build/test/scopy`，执行如下命令： | ||
| 159 | + | ||
| 160 | + ```bash | ||
| 161 | + msprof --application="./scopy_test" | ||
| 162 | + ``` | ||
| 163 | +采集结果在项目`ops-blas/build/test/scopy`目录，msprof命令执行完后会自动解析并导出性能数据结果文件，详细内容请参见[msprof](https://www.hiascend.com/document/detail/zh/mindstudio/82RC1/T&ITools/Profiling/atlasprofiling_16_0110.html#ZH-CN_TOPIC_0000002504160251)。 | ||
Ddocs/aclblasLt.md+0-398
| @@ -1,398 +0,0 @@ | |||
| 1 | -# aclblasLt 接口文档 | ||
| 2 | - | ||
| 3 | -## 1. 模块简介 | ||
| 4 | - | ||
| 5 | -`aclblasLt` 是面向矩阵乘（GEMM）场景的轻量级高级接口，提供以下能力： | ||
| 6 | - | ||
| 7 | -- 句柄生命周期管理（Create/Destroy） | ||
| 8 | -- 矩阵布局描述（MatrixLayout）及属性配置/查询 | ||
| 9 | -- Matmul 操作描述（MatmulDesc）及属性配置 | ||
| 10 | -- 启发式算法查询（Heuristic） | ||
| 11 | -- 执行矩阵乘计算（Matmul） | ||
| 12 | -- 版本与属性查询（GetVersion/GetProperty） | ||
| 13 | - | ||
| 14 | -典型计算形式： | ||
| 15 | - | ||
| 16 | -\[ | ||
| 17 | -D = \alpha \cdot (A \times B) + \beta \cdot C | ||
| 18 | -\] | ||
| 19 | - | ||
| 20 | ---- | ||
| 21 | - | ||
| 22 | -## 2. 句柄与描述符类型 | ||
| 23 | - | ||
| 24 | -- `aclblasLtHandle_t`：库上下文句柄。 | ||
| 25 | -- `aclblasLtMatrixLayout_t`：矩阵布局描述符。 | ||
| 26 | -- `aclblasLtMatmulDesc_t`：矩阵乘操作描述符。 | ||
| 27 | -- `aclblasLtMatmulPreference_t`：算法搜索偏好描述符。 | ||
| 28 | -- `aclblasLtMatmulAlgo_t`：算法对象。 | ||
| 29 | -- `aclblasLtMatmulHeuristicResult_t`：启发式结果（含算法、工作区大小、状态等）。 | ||
| 30 | - | ||
| 31 | ---- | ||
| 32 | - | ||
| 33 | -## 3. 主要枚举 | ||
| 34 | - | ||
| 35 | -### 3.1 矩阵存储顺序 `aclblasLtOrder_t` | ||
| 36 | - | ||
| 37 | -- `ACLBLASLT_ORDER_COL`：列主序 | ||
| 38 | -- `ACLBLASLT_ORDER_ROW`：行主序 | ||
| 39 | - | ||
| 40 | -### 3.2 Epilogue 类型 `aclblasLtEpilogue_t` | ||
| 41 | - | ||
| 42 | -包含默认、ReLU、GELU、Bias 及其组合等后处理选项（详见头文件枚举定义）。 | ||
| 43 | - | ||
| 44 | -### 3.3 MatrixLayout 属性 `aclblasLtMatrixLayoutAttribute_t` | ||
| 45 | - | ||
| 46 | -常用属性包括： | ||
| 47 | - | ||
| 48 | -- `ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT` | ||
| 49 | -- `ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET` | ||
| 50 | -- `ACLBLASLT_MATRIX_LAYOUT_TYPE` | ||
| 51 | -- `ACLBLASLT_MATRIX_LAYOUT_ORDER` | ||
| 52 | -- `ACLBLASLT_MATRIX_LAYOUT_ROWS` | ||
| 53 | -- `ACLBLASLT_MATRIX_LAYOUT_COLS` | ||
| 54 | -- `ACLBLASLT_MATRIX_LAYOUT_LD` | ||
| 55 | - | ||
| 56 | -### 3.4 MatmulDesc 属性 `aclblasLtMatmulDescAttribute_t` | ||
| 57 | - | ||
| 58 | -包括转置、epilogue、bias 指针、数据类型、scale 指针、aux 指针等属性（详见头文件枚举定义）。 | ||
| 59 | - | ||
| 60 | -### 3.5 Preference 属性 `aclblasLtMatmulPreferenceAttribute_t` | ||
| 61 | - | ||
| 62 | -- `ACLBLASLT_MATMUL_PREF_SEARCH_MODE` | ||
| 63 | -- `ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES` | ||
| 64 | - | ||
| 65 | ---- | ||
| 66 | - | ||
| 67 | -## 4. API 参考 | ||
| 68 | - | ||
| 69 | -> 说明：以下所有接口返回类型均为 `aclblasStatus_t`。 | ||
| 70 | - | ||
| 71 | -## 4.1 版本与属性查询 | ||
| 72 | - | ||
| 73 | -### 4.1.1 `aclblasLtGetVersion` | ||
| 74 | - | ||
| 75 | -```c | ||
| 76 | -aclblasStatus_t aclblasLtGetVersion(size_t* version); | ||
| 77 | -``` | ||
| 78 | - | ||
| 79 | -**功能** | ||
| 80 | -- 查询 aclblasLt 打包版本号。 | ||
| 81 | - | ||
| 82 | -**参数** | ||
| 83 | -- `version`（输出）：版本值地址，不能为空。 | ||
| 84 | - | ||
| 85 | -**返回** | ||
| 86 | -- `ACLBLAS_STATUS_SUCCESS`：成功。 | ||
| 87 | -- `ACLBLAS_STATUS_INVALID_VALUE`：参数非法（如 `version == NULL`）。 | ||
| 88 | - | ||
| 89 | ---- | ||
| 90 | - | ||
| 91 | -### 4.1.2 `aclblasLtGetProperty` | ||
| 92 | - | ||
| 93 | -```c | ||
| 94 | -aclblasStatus_t aclblasLtGetProperty(aclblasLtPropertyType_t type, int* value); | ||
| 95 | -``` | ||
| 96 | - | ||
| 97 | -**功能** | ||
| 98 | -- 查询库属性（如主版本、次版本、补丁号等）。 | ||
| 99 | - | ||
| 100 | -**参数** | ||
| 101 | -- `type`（输入）：属性类型。 | ||
| 102 | -- `value`（输出）：属性值输出地址。 | ||
| 103 | - | ||
| 104 | -**返回** | ||
| 105 | -- `ACLBLAS_STATUS_SUCCESS`：成功。 | ||
| 106 | -- `ACLBLAS_STATUS_INVALID_VALUE`：参数非法或属性类型不支持。 | ||
| 107 | - | ||
| 108 | ---- | ||
| 109 | - | ||
| 110 | -## 4.2 库句柄管理 | ||
| 111 | - | ||
| 112 | -### 4.2.1 `aclblasLtCreate` | ||
| 113 | - | ||
| 114 | -```c | ||
| 115 | -aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* handle); | ||
| 116 | -``` | ||
| 117 | - | ||
| 118 | -**功能** | ||
| 119 | -- 创建 aclblasLt 上下文句柄。 | ||
| 120 | - | ||
| 121 | -**参数** | ||
| 122 | -- `handle`（输出）：返回创建的句柄。 | ||
| 123 | - | ||
| 124 | -**返回** | ||
| 125 | -- `ACLBLAS_STATUS_SUCCESS` | ||
| 126 | -- `ACLBLAS_STATUS_INVALID_VALUE` | ||
| 127 | -- `ACLBLAS_STATUS_NOT_INITIALIZED` | ||
| 128 | -- `ACLBLAS_STATUS_ALLOC_FAILED` | ||
| 129 | - | ||
| 130 | ---- | ||
| 131 | - | ||
| 132 | -### 4.2.2 `aclblasLtDestroy` | ||
| 133 | - | ||
| 134 | -```c | ||
| 135 | -aclblasStatus_t aclblasLtDestroy(const aclblasLtHandle_t handle); | ||
| 136 | -``` | ||
| 137 | - | ||
| 138 | -**功能** | ||
| 139 | -- 销毁句柄并释放资源。 | ||
| 140 | - | ||
| 141 | -**参数** | ||
| 142 | -- `handle`（输入）：待销毁句柄。 | ||
| 143 | - | ||
| 144 | ---- | ||
| 145 | - | ||
| 146 | -## 4.3 MatrixLayout 描述符 | ||
| 147 | - | ||
| 148 | -### 4.3.1 `aclblasLtMatrixLayoutCreate` | ||
| 149 | - | ||
| 150 | -```c | ||
| 151 | -aclblasStatus_t aclblasLtMatrixLayoutCreate(aclblasLtMatrixLayout_t* matLayout, | ||
| 152 | - aclDataType type, | ||
| 153 | - uint64_t rows, | ||
| 154 | - uint64_t cols, | ||
| 155 | - int64_t ld); | ||
| 156 | -``` | ||
| 157 | - | ||
| 158 | -**功能** | ||
| 159 | -- 创建矩阵布局描述符。 | ||
| 160 | - | ||
| 161 | -**参数说明** | ||
| 162 | -- `type`：数据类型。 | ||
| 163 | -- `rows/cols`：矩阵行列。 | ||
| 164 | -- `ld`：leading dimension。 | ||
| 165 | - | ||
| 166 | ---- | ||
| 167 | - | ||
| 168 | -### 4.3.2 `aclblasLtMatrixLayoutDestroy` | ||
| 169 | - | ||
| 170 | -```c | ||
| 171 | -aclblasStatus_t aclblasLtMatrixLayoutDestroy(const aclblasLtMatrixLayout_t matLayout); | ||
| 172 | -``` | ||
| 173 | - | ||
| 174 | ---- | ||
| 175 | - | ||
| 176 | -### 4.3.3 `aclblasLtMatrixLayoutSetAttribute` | ||
| 177 | - | ||
| 178 | -```c | ||
| 179 | -aclblasStatus_t aclblasLtMatrixLayoutSetAttribute(aclblasLtMatrixLayout_t matLayout, | ||
| 180 | - aclblasLtMatrixLayoutAttribute_t attr, | ||
| 181 | - const void* buf, | ||
| 182 | - size_t sizeInBytes); | ||
| 183 | -``` | ||
| 184 | - | ||
| 185 | -**功能** | ||
| 186 | -- 设置 MatrixLayout 属性。 | ||
| 187 | - | ||
| 188 | ---- | ||
| 189 | - | ||
| 190 | -### 4.3.4 `aclblasLtMatrixLayoutGetAttribute` | ||
| 191 | - | ||
| 192 | -```c | ||
| 193 | -aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t matLayout, | ||
| 194 | - aclblasLtMatrixLayoutAttribute_t attr, | ||
| 195 | - void* buf, | ||
| 196 | - size_t sizeInBytes, | ||
| 197 | - size_t* sizeWritten); | ||
| 198 | -``` | ||
| 199 | - | ||
| 200 | -**功能** | ||
| 201 | -- 查询 MatrixLayout 属性值。 | ||
| 202 | - | ||
| 203 | -**参数** | ||
| 204 | -- `matLayout`（输入）：矩阵布局描述符。 | ||
| 205 | -- `attr`（输入）：要查询的属性。 | ||
| 206 | -- `buf`（输出）：接收属性值的缓冲区。 | ||
| 207 | -- `sizeInBytes`（输入）：`buf` 大小。 | ||
| 208 | -- `sizeWritten`（输出，可选）：实际写入字节数。 | ||
| 209 | - | ||
| 210 | -**返回** | ||
| 211 | -- `ACLBLAS_STATUS_SUCCESS` | ||
| 212 | -- `ACLBLAS_STATUS_INVALID_VALUE` | ||
| 213 | -- `ACLBLAS_STATUS_NOT_SUPPORTED` | ||
| 214 | - | ||
| 215 | ---- | ||
| 216 | - | ||
| 217 | -## 4.4 MatmulDesc 描述符 | ||
| 218 | - | ||
| 219 | -### 4.4.1 `aclblasLtMatmulDescCreate` | ||
| 220 | - | ||
| 221 | -```c | ||
| 222 | -aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* matmulDesc, | ||
| 223 | - aclblasComputeType_t computeType, | ||
| 224 | - aclDataType scaleType); | ||
| 225 | -``` | ||
| 226 | - | ||
| 227 | -### 4.4.2 `aclblasLtMatmulDescDestroy` | ||
| 228 | - | ||
| 229 | -```c | ||
| 230 | -aclblasStatus_t aclblasLtMatmulDescDestroy(const aclblasLtMatmulDesc_t matmulDesc); | ||
| 231 | -``` | ||
| 232 | - | ||
| 233 | -### 4.4.3 `aclblasLtMatmulDescSetAttribute` | ||
| 234 | - | ||
| 235 | -```c | ||
| 236 | -aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t matmulDesc, | ||
| 237 | - aclblasLtMatmulDescAttribute_t attr, | ||
| 238 | - const void* buf, | ||
| 239 | - size_t sizeInBytes); | ||
| 240 | -``` | ||
| 241 | - | ||
| 242 | ---- | ||
| 243 | - | ||
| 244 | -## 4.5 MatmulPreference 描述符 | ||
| 245 | - | ||
| 246 | -### 4.5.1 `aclblasLtMatmulPreferenceCreate` | ||
| 247 | - | ||
| 248 | -```c | ||
| 249 | -aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pref); | ||
| 250 | -``` | ||
| 251 | - | ||
| 252 | -### 4.5.2 `aclblasLtMatmulPreferenceDestroy` | ||
| 253 | - | ||
| 254 | -```c | ||
| 255 | -aclblasStatus_t aclblasLtMatmulPreferenceDestroy(const aclblasLtMatmulPreference_t pref); | ||
| 256 | -``` | ||
| 257 | - | ||
| 258 | -### 4.5.3 `aclblasLtMatmulPreferenceSetAttribute` | ||
| 259 | - | ||
| 260 | -```c | ||
| 261 | -aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_t pref, | ||
| 262 | - aclblasLtMatmulPreferenceAttribute_t attr, | ||
| 263 | - const void* buf, | ||
| 264 | - size_t sizeInBytes); | ||
| 265 | -``` | ||
| 266 | - | ||
| 267 | ---- | ||
| 268 | - | ||
| 269 | -## 4.6 启发式算法查询 | ||
| 270 | - | ||
| 271 | -### 4.6.1 `aclblasLtMatmulAlgoGetHeuristic` | ||
| 272 | - | ||
| 273 | -```c | ||
| 274 | -aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | ||
| 275 | - aclblasLtMatmulDesc_t matmulDesc, | ||
| 276 | - aclblasLtMatrixLayout_t Adesc, | ||
| 277 | - aclblasLtMatrixLayout_t Bdesc, | ||
| 278 | - aclblasLtMatrixLayout_t Cdesc, | ||
| 279 | - aclblasLtMatrixLayout_t Ddesc, | ||
| 280 | - aclblasLtMatmulPreference_t pref, | ||
| 281 | - int requestedAlgoCount, | ||
| 282 | - aclblasLtMatmulHeuristicResult_t heuristicResultsArray[], | ||
| 283 | - int* returnAlgoCount); | ||
| 284 | -``` | ||
| 285 | - | ||
| 286 | -**功能** | ||
| 287 | -- 根据输入描述符和偏好，返回可用算法候选。 | ||
| 288 | - | ||
| 289 | ---- | ||
| 290 | - | ||
| 291 | -## 4.7 矩阵乘执行 | ||
| 292 | - | ||
| 293 | -### 4.7.1 `aclblasLtMatmul` | ||
| 294 | - | ||
| 295 | -```c | ||
| 296 | -aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | ||
| 297 | - aclblasLtMatmulDesc_t matmulDesc, | ||
| 298 | - const void* alpha, | ||
| 299 | - const void* A, | ||
| 300 | - aclblasLtMatrixLayout_t Adesc, | ||
| 301 | - const void* B, | ||
| 302 | - aclblasLtMatrixLayout_t Bdesc, | ||
| 303 | - const void* beta, | ||
| 304 | - const void* C, | ||
| 305 | - aclblasLtMatrixLayout_t Cdesc, | ||
| 306 | - void* D, | ||
| 307 | - aclblasLtMatrixLayout_t Ddesc, | ||
| 308 | - const aclblasLtMatmulAlgo_t* algo, | ||
| 309 | - void* workspace, | ||
| 310 | - size_t workspaceSizeInBytes, | ||
| 311 | - aclrtStream stream); | ||
| 312 | -``` | ||
| 313 | - | ||
| 314 | -**功能** | ||
| 315 | -- 执行矩阵乘及线性组合。 | ||
| 316 | - | ||
| 317 | -**说明** | ||
| 318 | -- 支持 `C == D` 的原位计算。 | ||
| 319 | -- `workspace` 建议满足 16B 对齐。 | ||
| 320 | -- 若 `algo == NULL`，实现可采用默认策略。 | ||
| 321 | - | ||
| 322 | ---- | ||
| 323 | - | ||
| 324 | -## 5. 常见调用流程（推荐） | ||
| 325 | - | ||
| 326 | -1. `aclblasLtCreate` | ||
| 327 | -2. 创建 `MatrixLayout`（A/B/C/D）并配置属性 | ||
| 328 | -3. 创建 `MatmulDesc` 并设置转置、epilogue 等属性 | ||
| 329 | -4. 创建 `MatmulPreference` 并设置工作区上限 | ||
| 330 | -5. `aclblasLtMatmulAlgoGetHeuristic` 查询算法 | ||
| 331 | -6. 调用 `aclblasLtMatmul` | ||
| 332 | -7. 销毁 Preference/Desc/Layout/Handle | ||
| 333 | - | ||
| 334 | ---- | ||
| 335 | - | ||
| 336 | -## 6. 最小示例（伪代码） | ||
| 337 | - | ||
| 338 | -```c | ||
| 339 | -aclblasLtHandle_t handle; | ||
| 340 | -aclblasLtCreate(&handle); | ||
| 341 | - | ||
| 342 | -aclblasLtMatrixLayout_t Adesc, Bdesc, Cdesc, Ddesc; | ||
| 343 | -aclblasLtMatrixLayoutCreate(&Adesc, ACL_FLOAT16, m, k, lda); | ||
| 344 | -aclblasLtMatrixLayoutCreate(&Bdesc, ACL_FLOAT16, k, n, ldb); | ||
| 345 | -aclblasLtMatrixLayoutCreate(&Cdesc, ACL_FLOAT16, m, n, ldc); | ||
| 346 | -aclblasLtMatrixLayoutCreate(&Ddesc, ACL_FLOAT16, m, n, ldd); | ||
| 347 | - | ||
| 348 | -aclblasLtMatmulDesc_t opDesc; | ||
| 349 | -aclblasLtMatmulDescCreate(&opDesc, ACLBLAS_COMPUTE_32F, ACL_FLOAT); | ||
| 350 | - | ||
| 351 | -aclblasLtMatmulPreference_t pref; | ||
| 352 | -aclblasLtMatmulPreferenceCreate(&pref); | ||
| 353 | -size_t workspaceCap = 32 * 1024 * 1024; | ||
| 354 | -aclblasLtMatmulPreferenceSetAttribute(pref, | ||
| 355 | - ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES, | ||
| 356 | - &workspaceCap, | ||
| 357 | - sizeof(workspaceCap)); | ||
| 358 | - | ||
| 359 | -aclblasLtMatmulHeuristicResult_t heur[8]; | ||
| 360 | -int algoCount = 0; | ||
| 361 | -aclblasLtMatmulAlgoGetHeuristic(handle, opDesc, Adesc, Bdesc, Cdesc, Ddesc, | ||
| 362 | - pref, 8, heur, &algoCount); | ||
| 363 | - | ||
| 364 | -aclblasLtMatmul(handle, opDesc, | ||
| 365 | - &alpha, A, Adesc, | ||
| 366 | - B, Bdesc, | ||
| 367 | - &beta, C, Cdesc, | ||
| 368 | - D, Ddesc, | ||
| 369 | - &heur[0].algo, | ||
| 370 | - workspace, workspaceBytes, | ||
| 371 | - stream); | ||
| 372 | - | ||
| 373 | -aclblasLtMatmulPreferenceDestroy(pref); | ||
| 374 | -aclblasLtMatmulDescDestroy(opDesc); | ||
| 375 | -aclblasLtMatrixLayoutDestroy(Adesc); | ||
| 376 | -aclblasLtMatrixLayoutDestroy(Bdesc); | ||
| 377 | -aclblasLtMatrixLayoutDestroy(Cdesc); | ||
| 378 | -aclblasLtMatrixLayoutDestroy(Ddesc); | ||
| 379 | -aclblasLtDestroy(handle); | ||
| 380 | -``` | ||
| 381 | - | ||
| 382 | ---- | ||
| 383 | - | ||
| 384 | -## 7. 返回码说明（通用） | ||
| 385 | - | ||
| 386 | -- `ACLBLAS_STATUS_SUCCESS`：成功 | ||
| 387 | -- `ACLBLAS_STATUS_INVALID_VALUE`：参数非法 | ||
| 388 | -- `ACLBLAS_STATUS_NOT_INITIALIZED`：上下文未初始化 | ||
| 389 | -- `ACLBLAS_STATUS_NOT_SUPPORTED`：当前配置或属性不支持 | ||
| 390 | -- `ACLBLAS_STATUS_ALLOC_FAILED`：内存分配失败 | ||
| 391 | -- `ACLBLAS_STATUS_EXECUTION_FAILED`：设备执行失败（执行类接口） | ||
| 392 | - | ||
| 393 | ---- | ||
| 394 | - | ||
| 395 | -## 8. 备注 | ||
| 396 | - | ||
| 397 | -- 接口能力与属性支持范围以当前实现版本为准。 | ||
| 398 | -- 文档与头文件不一致时，请以头文件声明和实际实现行为为准。 | ||
| @@ -0,0 +1,535 @@ | |||
| 1 | +# aclblasLt 接口文档 | ||
| 2 | + | ||
| 3 | +## 0. 头文件与目录结构 | ||
| 4 | + | ||
| 5 | +- **aclblasLt API 头文件** | ||
| 6 | + - 接口声明、句柄与描述符结构、Lt 专用枚举： | ||
| 7 | + - `ops-blas/include/cann_ops_blasLt.h` | ||
| 8 | +- **通用 BLAS 状态码与基础枚举** | ||
| 9 | + - 返回码 `aclblasStatus_t` / `aclblasLtStatus`、计算类型 `aclblasComputeType_t`、矩阵转置枚举 `aclblasOperation_t`： | ||
| 10 | + - `ops-blas/include/cann_ops_blas_common.h` | ||
| 11 | +- **ACL 基础类型** | ||
| 12 | + - `aclDataType`、`aclrtStream` 等类型来自： | ||
| 13 | + - `acl/acl.h`（通过 `cann_ops_blasLt.h` 间接包含） | ||
| 14 | + | ||
| 15 | +> **查找接口/枚举定义**： | ||
| 16 | +> 如需查看某个接口或枚举的完整注释与声明，可在上述目录下打开对应头文件（`cann_ops_blasLt.h` / `cann_ops_blas_common.h`）检索名称。 | ||
| 17 | + | ||
| 18 | +--- | ||
| 19 | + | ||
| 20 | +## 1. 模块简介 | ||
| 21 | + | ||
| 22 | +`aclblasLt` 是面向矩阵乘（GEMM）场景的轻量级高级接口，提供以下能力： | ||
| 23 | + | ||
| 24 | +- 句柄生命周期管理（Create/Destroy） | ||
| 25 | +- 矩阵布局描述（MatrixLayout）及属性配置/查询 | ||
| 26 | +- Matmul 操作描述（MatmulDesc）及属性配置 | ||
| 27 | +- 算法搜索偏好设置（Preference） | ||
| 28 | +- 启发式算法查询（Heuristic） | ||
| 29 | +- 执行矩阵乘计算（Matmul） | ||
| 30 | +- 版本与属性查询（GetVersion/GetProperty） | ||
| 31 | + | ||
| 32 | + | ||
| 33 | +--- | ||
| 34 | + | ||
| 35 | +## 2. 框架层基础结构与类型关系 | ||
| 36 | + | ||
| 37 | +### 2.1 基础句柄与描述符类型（定义于 `cann_ops_blasLt.h`） | ||
| 38 | + | ||
| 39 | +- **库上下文句柄** | ||
| 40 | + - `aclblasLtHandle_t`：`typedef void* aclblasLtHandle_t;` | ||
| 41 | +- **矩阵布局描述符** | ||
| 42 | + - `aclblasLtMatrixLayoutOpaque_t`：内部保存布局信息的 opaque 结构 | ||
| 43 | + - `aclblasLtMatrixLayout_t`：`typedef aclblasLtMatrixLayoutOpaque_t* aclblasLtMatrixLayout_t;` | ||
| 44 | +- **矩阵乘操作描述符** | ||
| 45 | + - `aclblasLtMatmulDescOpaque_t` | ||
| 46 | + - `aclblasLtMatmulDesc_t`：`typedef aclblasLtMatmulDescOpaque_t* aclblasLtMatmulDesc_t;` | ||
| 47 | +- **算法偏好描述符** | ||
| 48 | + - `aclblasLtMatmulPreferenceOpaque_t` | ||
| 49 | + - `aclblasLtMatmulPreference_t`：`typedef aclblasLtMatmulPreferenceOpaque_t* aclblasLtMatmulPreference_t;` | ||
| 50 | +- **算法与启发式结构** | ||
| 51 | + - `aclblasLtMatmulAlgo_t`：算法描述（包含内部编码和 `max_workspace_bytes`） | ||
| 52 | + - `aclblasLtMatmulHeuristicResult_t`：启发式结果（包含 `algo`、`workspaceSize`、`state`、`wavesCount` 等） | ||
| 53 | + | ||
| 54 | +### 2.2 通用状态码与基础枚举（定义于 `cann_ops_blas_common.h`） | ||
| 55 | + | ||
| 56 | +- **返回状态枚举** | ||
| 57 | + - `aclblasStatus_t`：主返回码类型 | ||
| 58 | + - `aclblasLtStatus`：`typedef aclblasStatus_t aclblasLtStatus;`（Lt 接口复用相同状态码集合） | ||
| 59 | +- **矩阵操作枚举** | ||
| 60 | + - `aclblasOperation_t`：`ACLBLAS_OP_N` / `ACLBLAS_OP_T` / `ACLBLAS_OP_C` | ||
| 61 | +- **计算类型枚举** | ||
| 62 | + - `aclblasComputeType_t`：`ACLBLAS_COMPUTE_16F` / `ACLBLAS_COMPUTE_32F` 等 | ||
| 63 | + | ||
| 64 | +### 2.3 Lt 专用枚举与属性（定义于 `cann_ops_blasLt.h`） | ||
| 65 | + | ||
| 66 | +- **矩阵存储顺序** | ||
| 67 | + - `aclblasLtOrder_t` | ||
| 68 | +- **库属性类型** | ||
| 69 | + - `aclblasLtPropertyType_t` | ||
| 70 | +- **Epilogue 类型** | ||
| 71 | + - `aclblasLtEpilogue_t` | ||
| 72 | +- **矩阵布局属性** | ||
| 73 | + - `aclblasLtMatrixLayoutAttribute_t` | ||
| 74 | +- **Matmul 描述符属性** | ||
| 75 | + - `aclblasLtMatmulDescAttribute_t` | ||
| 76 | +- **Preference 属性** | ||
| 77 | + - `aclblasLtMatmulPreferenceAttribute_t` | ||
| 78 | + | ||
| 79 | +### 2.4 类型关系简图 | ||
| 80 | + | ||
| 81 | +> 下图仅表达“包含/别名关系”，非 C++ 继承。 | ||
| 82 | + | ||
| 83 | +```text | ||
| 84 | +aclblasStatus_t <------------------------------+ | ||
| 85 | + ^ | | ||
| 86 | + | typedef | | ||
| 87 | +aclblasLtStatus --------------------------------+ | ||
| 88 | + | ||
| 89 | +aclblasLtMatmulHeuristicResult_t | ||
| 90 | + ├── aclblasLtMatmulAlgo_t algo | ||
| 91 | + ├── size_t workspaceSize | ||
| 92 | + └── aclblasStatus_t state | ||
| 93 | + | ||
| 94 | +aclblasLtMatrixLayout_t | ||
| 95 | + └── typedef aclblasLtMatrixLayoutOpaque_t* | ||
| 96 | + | ||
| 97 | +aclblasLtMatmulDesc_t | ||
| 98 | + └── typedef aclblasLtMatmulDescOpaque_t* | ||
| 99 | + | ||
| 100 | +aclblasLtMatmulPreference_t | ||
| 101 | + └── typedef aclblasLtMatmulPreferenceOpaque_t* | ||
| 102 | +``` | ||
| 103 | + | ||
| 104 | +--- | ||
| 105 | + | ||
| 106 | +## 3. 主要枚举与属性 | ||
| 107 | + | ||
| 108 | +> **头文件位置**：以下枚举均定义在 `ops-blas/include/cann_ops_blasLt.h` 或 `ops-blas/include/cann_ops_blas_common.h` 中，对应关系见每小节说明。 | ||
| 109 | + | ||
| 110 | +### 3.1 矩阵存储顺序 `aclblasLtOrder_t`（`cann_ops_blasLt.h`） | ||
| 111 | + | ||
| 112 | +- `ACLBLASLT_ORDER_COL`：列主序 | ||
| 113 | +- `ACLBLASLT_ORDER_ROW`：行主序 | ||
| 114 | + | ||
| 115 | +### 3.2 Epilogue 类型 `aclblasLtEpilogue_t`（`cann_ops_blasLt.h`） | ||
| 116 | + | ||
| 117 | +- 包含默认、ReLU、GELU、Bias、Sigmoid、Swish、Clamp 及其 Aux/Bias 组合等多种后处理选项： | ||
| 118 | + - 例如：`ACLBLASLT_EPILOGUE_DEFAULT`、`ACLBLASLT_EPILOGUE_RELU`、`ACLBLASLT_EPILOGUE_BIAS`、 | ||
| 119 | + `ACLBLASLT_EPILOGUE_RELU_BIAS`、`ACLBLASLT_EPILOGUE_GELU` 等。 | ||
| 120 | +- 具体枚举值与行为说明可查阅 `cann_ops_blasLt.h` 中 `aclblasLtEpilogue` 的注释。 | ||
| 121 | + | ||
| 122 | +### 3.3 MatrixLayout 属性 `aclblasLtMatrixLayoutAttribute_t`（`cann_ops_blasLt.h`） | ||
| 123 | + | ||
| 124 | +常用属性包括（括号中为类型/含义）： | ||
| 125 | + | ||
| 126 | +- `ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT`（`int32_t`，批次数，默认 1） | ||
| 127 | +- `ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET`（`int64_t`，跨 batch 偏移元素数） | ||
| 128 | +- `ACLBLASLT_MATRIX_LAYOUT_TYPE`（`uint32_t`，矩阵数据类型，对应 `aclDataType`） | ||
| 129 | +- `ACLBLASLT_MATRIX_LAYOUT_ORDER`（`int32_t`，存储顺序，对应 `aclblasLtOrder_t`） | ||
| 130 | +- `ACLBLASLT_MATRIX_LAYOUT_ROWS`（`uint64_t`，行数） | ||
| 131 | +- `ACLBLASLT_MATRIX_LAYOUT_COLS`（`uint64_t`，列数） | ||
| 132 | +- `ACLBLASLT_MATRIX_LAYOUT_LD`（`int64_t`，leading dimension） | ||
| 133 | + | ||
| 134 | +### 3.4 MatmulDesc 属性 `aclblasLtMatmulDescAttribute_t`（`cann_ops_blasLt.h`） | ||
| 135 | + | ||
| 136 | +包括但不限于： | ||
| 137 | + | ||
| 138 | +- 转置控制： | ||
| 139 | + - `ACLBLASLT_MATMUL_DESC_TRANSA` / `ACLBLASLT_MATMUL_DESC_TRANSB`（`int32_t`，对应 `aclblasOperation_t`） | ||
| 140 | +- 后处理与 Bias： | ||
| 141 | + - `ACLBLASLT_MATMUL_DESC_EPILOGUE`（`uint32_t`，对应 `aclblasLtEpilogue_t`） | ||
| 142 | + - `ACLBLASLT_MATMUL_DESC_BIAS_POINTER`、`ACLBLASLT_MATMUL_DESC_BIAS_DATA_TYPE` | ||
| 143 | +- Scale 相关： | ||
| 144 | + - `ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER` / `B/C/D_SCALE_POINTER` | ||
| 145 | +- Aux 相关： | ||
| 146 | + - `ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER` / | ||
| 147 | + `ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD` / | ||
| 148 | + `ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE` / | ||
| 149 | + `ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_DATA_TYPE` | ||
| 150 | +- Pointer 模式与扩展： | ||
| 151 | + - `ACLBLASLT_MATMUL_DESC_POINTER_MODE` | ||
| 152 | + - `ACLBLASLT_MATMUL_DESC_A_SCALE_MODE` / `B_SCALE_MODE` | ||
| 153 | + - `ACLBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_A_EXT` / `B_EXT` | ||
| 154 | + - 激活扩展参数：`ACLBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG0_EXT` / `ACT_ARG1_EXT` | ||
| 155 | + | ||
| 156 | +### 3.5 Preference 属性 `aclblasLtMatmulPreferenceAttribute_t`（`cann_ops_blasLt.h`） | ||
| 157 | + | ||
| 158 | +- `ACLBLASLT_MATMUL_PREF_SEARCH_MODE`（`uint32_t`，搜索模式） | ||
| 159 | +- `ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES`（`uint64_t`，最大可用 workspace 大小） | ||
| 160 | + | ||
| 161 | +--- | ||
| 162 | + | ||
| 163 | +## 4. API 参考 | ||
| 164 | + | ||
| 165 | +> **说明**：以下所有接口的返回类型为 `aclblasStatus_t` / `aclblasLtStatus`，返回码含义详见第 7 章。 | ||
| 166 | +> **头文件路径统一为**：`ops-blas/include/cann_ops_blasLt.h`。 | ||
| 167 | + | ||
| 168 | +### 4.1 版本与属性查询 | ||
| 169 | + | ||
| 170 | +#### 4.1.1 `aclblasLtGetVersion`（`cann_ops_blasLt.h`） | ||
| 171 | + | ||
| 172 | +```c | ||
| 173 | +aclblasStatus_t aclblasLtGetVersion(size_t* version); | ||
| 174 | +``` | ||
| 175 | + | ||
| 176 | +- **功能** | ||
| 177 | + - 查询 aclblasLt 打包版本号。 | ||
| 178 | +- **参数** | ||
| 179 | + - `version`（输出）：版本值地址，不能为空。 | ||
| 180 | +- **返回** | ||
| 181 | + - `ACLBLAS_STATUS_SUCCESS`：成功。 | ||
| 182 | + - `ACLBLAS_STATUS_INVALID_VALUE`：`version == NULL` 等非法参数。 | ||
| 183 | + | ||
| 184 | +#### 4.1.2 `aclblasLtGetProperty`（`cann_ops_blasLt.h`） | ||
| 185 | + | ||
| 186 | +```c | ||
| 187 | +aclblasStatus_t aclblasLtGetProperty(aclblasLtPropertyType_t type, int* value); | ||
| 188 | +``` | ||
| 189 | + | ||
| 190 | +- **功能** | ||
| 191 | + - 查询库属性（主版本、次版本、补丁号等）。 | ||
| 192 | +- **参数** | ||
| 193 | + - `type`（输入）：属性类型。 | ||
| 194 | + - `value`（输出）：属性值输出地址。 | ||
| 195 | +- **返回** | ||
| 196 | + - `ACLBLAS_STATUS_SUCCESS`：成功。 | ||
| 197 | + - `ACLBLAS_STATUS_INVALID_VALUE`：参数非法或属性类型不支持。 | ||
| 198 | + | ||
| 199 | +--- | ||
| 200 | + | ||
| 201 | +### 4.2 库句柄管理 | ||
| 202 | + | ||
| 203 | +#### 4.2.1 `aclblasLtCreate`（`cann_ops_blasLt.h`） | ||
| 204 | + | ||
| 205 | +```c | ||
| 206 | +aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* handle); | ||
| 207 | +``` | ||
| 208 | + | ||
| 209 | +- **功能** | ||
| 210 | + - 创建 aclblasLt 上下文句柄，初始化库并绑定当前设备。 | ||
| 211 | +- **参数** | ||
| 212 | + - `handle`（输出）：返回创建的句柄。 | ||
| 213 | +- **返回** | ||
| 214 | + - `ACLBLAS_STATUS_SUCCESS` | ||
| 215 | + - `ACLBLAS_STATUS_INVALID_VALUE`：`handle == NULL` | ||
| 216 | + - `ACLBLAS_STATUS_ALLOC_FAILED`：内部资源分配失败 | ||
| 217 | + | ||
| 218 | +#### 4.2.2 `aclblasLtDestroy`（`cann_ops_blasLt.h`） | ||
| 219 | + | ||
| 220 | +```c | ||
| 221 | +aclblasStatus_t aclblasLtDestroy(const aclblasLtHandle_t handle); | ||
| 222 | +``` | ||
| 223 | + | ||
| 224 | +- **功能** | ||
| 225 | + - 销毁句柄并释放资源，可触发设备同步。 | ||
| 226 | +- **参数** | ||
| 227 | + - `handle`（输入）：待销毁句柄。 | ||
| 228 | +- **返回** | ||
| 229 | + - `ACLBLAS_STATUS_SUCCESS` | ||
| 230 | + - `ACLBLAS_STATUS_NOT_INITIALIZED`：库未初始化 | ||
| 231 | + - `ACLBLAS_STATUS_INVALID_VALUE`：`handle == NULL` | ||
| 232 | + | ||
| 233 | +--- | ||
| 234 | + | ||
| 235 | +### 4.3 MatrixLayout 描述符（`cann_ops_blasLt.h`） | ||
| 236 | + | ||
| 237 | +#### 4.3.1 `aclblasLtMatrixLayoutCreate` | ||
| 238 | + | ||
| 239 | +```c | ||
| 240 | +aclblasStatus_t aclblasLtMatrixLayoutCreate(aclblasLtMatrixLayout_t* matLayout, | ||
| 241 | + aclDataType type, | ||
| 242 | + uint64_t rows, | ||
| 243 | + uint64_t cols, | ||
| 244 | + int64_t ld); | ||
| 245 | +``` | ||
| 246 | + | ||
| 247 | +- **功能** | ||
| 248 | + - 创建矩阵布局描述符，配置基础形状与数据类型。 | ||
| 249 | +- **参数说明** | ||
| 250 | + - `type`：数据类型（`aclDataType`）。 | ||
| 251 | + - `rows/cols`：矩阵行列。 | ||
| 252 | + - `ld`：leading dimension。 | ||
| 253 | +- **返回** | ||
| 254 | + - `ACLBLAS_STATUS_SUCCESS` | ||
| 255 | + - `ACLBLAS_STATUS_ALLOC_FAILED` | ||
| 256 | + | ||
| 257 | +#### 4.3.2 `aclblasLtMatrixLayoutDestroy` | ||
| 258 | + | ||
| 259 | +```c | ||
| 260 | +aclblasStatus_t aclblasLtMatrixLayoutDestroy(const aclblasLtMatrixLayout_t matLayout); | ||
| 261 | +``` | ||
| 262 | + | ||
| 263 | +- **功能** | ||
| 264 | + - 销毁矩阵布局描述符。 | ||
| 265 | +- **返回** | ||
| 266 | + - `ACLBLAS_STATUS_SUCCESS` | ||
| 267 | + | ||
| 268 | +#### 4.3.3 `aclblasLtMatrixLayoutSetAttribute` | ||
| 269 | + | ||
| 270 | +```c | ||
| 271 | +aclblasStatus_t aclblasLtMatrixLayoutSetAttribute(aclblasLtMatrixLayout_t matLayout, | ||
| 272 | + aclblasLtMatrixLayoutAttribute_t attr, | ||
| 273 | + const void* buf, | ||
| 274 | + size_t sizeInBytes); | ||
| 275 | +``` | ||
| 276 | + | ||
| 277 | +- **功能** | ||
| 278 | + - 设置 MatrixLayout 属性值。 | ||
| 279 | +- **返回** | ||
| 280 | + - `ACLBLAS_STATUS_SUCCESS` | ||
| 281 | + - `ACLBLAS_STATUS_INVALID_VALUE`：`buf == NULL` 或 `sizeInBytes` 不匹配 | ||
| 282 | + | ||
| 283 | +#### 4.3.4 `aclblasLtMatrixLayoutGetAttribute` | ||
| 284 | + | ||
| 285 | +```c | ||
| 286 | +aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t matLayout, | ||
| 287 | + aclblasLtMatrixLayoutAttribute_t attr, | ||
| 288 | + void* buf, | ||
| 289 | + size_t sizeInBytes, | ||
| 290 | + size_t* sizeWritten); | ||
| 291 | +``` | ||
| 292 | + | ||
| 293 | +- **功能** | ||
| 294 | + - 查询 MatrixLayout 属性值。 | ||
| 295 | +- **典型失败返回** | ||
| 296 | + - `ACLBLAS_STATUS_INVALID_VALUE` | ||
| 297 | + - `ACLBLAS_STATUS_NOT_SUPPORTED` | ||
| 298 | + | ||
| 299 | +--- | ||
| 300 | + | ||
| 301 | +### 4.4 MatmulDesc 描述符（`cann_ops_blasLt.h`） | ||
| 302 | + | ||
| 303 | +#### 4.4.1 `aclblasLtMatmulDescCreate` | ||
| 304 | + | ||
| 305 | +```c | ||
| 306 | +aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* matmulDesc, | ||
| 307 | + aclblasComputeType_t computeType, | ||
| 308 | + aclDataType scaleType); | ||
| 309 | +``` | ||
| 310 | + | ||
| 311 | +- **功能** | ||
| 312 | + - 创建矩阵乘操作描述符，配置计算类型与 scale 类型。 | ||
| 313 | +- **返回** | ||
| 314 | + - `ACLBLAS_STATUS_SUCCESS` | ||
| 315 | + - `ACLBLAS_STATUS_ALLOC_FAILED` | ||
| 316 | + | ||
| 317 | +#### 4.4.2 `aclblasLtMatmulDescDestroy` | ||
| 318 | + | ||
| 319 | +```c | ||
| 320 | +aclblasStatus_t aclblasLtMatmulDescDestroy(const aclblasLtMatmulDesc_t matmulDesc); | ||
| 321 | +``` | ||
| 322 | + | ||
| 323 | +#### 4.4.3 `aclblasLtMatmulDescSetAttribute` | ||
| 324 | + | ||
| 325 | +```c | ||
| 326 | +aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t matmulDesc, | ||
| 327 | + aclblasLtMatmulDescAttribute_t attr, | ||
| 328 | + const void* buf, | ||
| 329 | + size_t sizeInBytes); | ||
| 330 | +``` | ||
| 331 | + | ||
| 332 | +- **典型失败返回** | ||
| 333 | + - `ACLBLAS_STATUS_INVALID_VALUE`：`buf == NULL` 或大小不匹配 | ||
| 334 | + | ||
| 335 | +#### 4.4.4 `aclblasLtMatmulDescGetAttribute`（`cann_ops_blasLt.h`） | ||
| 336 | + | ||
| 337 | +```c | ||
| 338 | +aclblasStatus_t aclblasLtMatmulDescGetAttribute(aclblasLtMatmulDesc_t desc, | ||
| 339 | + aclblasLtMatmulDescAttribute_t attr, | ||
| 340 | + void* buf, | ||
| 341 | + size_t sizeInBytes, | ||
| 342 | + size_t* sizeWritten); | ||
| 343 | +``` | ||
| 344 | + | ||
| 345 | +--- | ||
| 346 | + | ||
| 347 | +### 4.5 MatmulPreference 描述符（`cann_ops_blasLt.h`） | ||
| 348 | + | ||
| 349 | +#### 4.5.1 `aclblasLtMatmulPreferenceCreate` | ||
| 350 | + | ||
| 351 | +```c | ||
| 352 | +aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pref); | ||
| 353 | +``` | ||
| 354 | + | ||
| 355 | +#### 4.5.2 `aclblasLtMatmulPreferenceDestroy` | ||
| 356 | + | ||
| 357 | +```c | ||
| 358 | +aclblasStatus_t aclblasLtMatmulPreferenceDestroy(const aclblasLtMatmulPreference_t pref); | ||
| 359 | +``` | ||
| 360 | + | ||
| 361 | +#### 4.5.3 `aclblasLtMatmulPreferenceSetAttribute` | ||
| 362 | + | ||
| 363 | +```c | ||
| 364 | +aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_t pref, | ||
| 365 | + aclblasLtMatmulPreferenceAttribute_t attr, | ||
| 366 | + const void* buf, | ||
| 367 | + size_t sizeInBytes); | ||
| 368 | +``` | ||
| 369 | + | ||
| 370 | +- **典型失败返回** | ||
| 371 | + - `ACLBLAS_STATUS_INVALID_VALUE` | ||
| 372 | + | ||
| 373 | +#### 4.5.4 `aclblasLtMatmulPreferenceGetAttribute` | ||
| 374 | + | ||
| 375 | +```c | ||
| 376 | +aclblasStatus_t aclblasLtMatmulPreferenceGetAttribute(aclblasLtMatmulPreference_t pref, | ||
| 377 | + aclblasLtMatmulPreferenceAttribute_t attr, | ||
| 378 | + void* buf, | ||
| 379 | + size_t sizeInBytes, | ||
| 380 | + size_t* sizeWritten); | ||
| 381 | +``` | ||
| 382 | + | ||
| 383 | +--- | ||
| 384 | + | ||
| 385 | +### 4.6 启发式算法查询（`cann_ops_blasLt.h`） | ||
| 386 | + | ||
| 387 | +#### 4.6.1 `aclblasLtMatmulAlgoGetHeuristic` | ||
| 388 | + | ||
| 389 | +```c | ||
| 390 | +aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | ||
| 391 | + aclblasLtMatmulDesc_t matmulDesc, | ||
| 392 | + aclblasLtMatrixLayout_t Adesc, | ||
| 393 | + aclblasLtMatrixLayout_t Bdesc, | ||
| 394 | + aclblasLtMatrixLayout_t Cdesc, | ||
| 395 | + aclblasLtMatrixLayout_t Ddesc, | ||
| 396 | + aclblasLtMatmulPreference_t pref, | ||
| 397 | + int requestedAlgoCount, | ||
| 398 | + aclblasLtMatmulHeuristicResult_t heuristicResultsArray[], | ||
| 399 | + int* returnAlgoCount); | ||
| 400 | +``` | ||
| 401 | + | ||
| 402 | +- **功能** | ||
| 403 | + - 基于输入布局与偏好，返回若干候选算法及其估计性能。 | ||
| 404 | +- **典型失败返回** | ||
| 405 | + - `ACLBLAS_STATUS_NOT_SUPPORTED`：当前配置无可用启发式 | ||
| 406 | + - `ACLBLAS_STATUS_INVALID_VALUE`：`requestedAlgoCount <= 0` 等非法参数 | ||
| 407 | + | ||
| 408 | +--- | ||
| 409 | + | ||
| 410 | +### 4.7 矩阵乘执行（`cann_ops_blasLt.h`） | ||
| 411 | + | ||
| 412 | +#### 4.7.1 `aclblasLtMatmul` | ||
| 413 | + | ||
| 414 | +```c | ||
| 415 | +aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | ||
| 416 | + aclblasLtMatmulDesc_t matmulDesc, | ||
| 417 | + const void* alpha, | ||
| 418 | + const void* A, | ||
| 419 | + aclblasLtMatrixLayout_t Adesc, | ||
| 420 | + const void* B, | ||
| 421 | + aclblasLtMatrixLayout_t Bdesc, | ||
| 422 | + const void* beta, | ||
| 423 | + const void* C, | ||
| 424 | + aclblasLtMatrixLayout_t Cdesc, | ||
| 425 | + void* D, | ||
| 426 | + aclblasLtMatrixLayout_t Ddesc, | ||
| 427 | + const aclblasLtMatmulAlgo_t* algo, | ||
| 428 | + void* workspace, | ||
| 429 | + size_t workspaceSizeInBytes, | ||
| 430 | + aclrtStream stream); | ||
| 431 | +``` | ||
| 432 | + | ||
| 433 | +- **功能** | ||
| 434 | + - 执行矩阵乘及线性组合，支持 `C == D` 原位更新。 | ||
| 435 | +- **说明** | ||
| 436 | + - `workspace` 建议满足 16B 对齐。 | ||
| 437 | + - 若 `algo == NULL`，内部可进行一次隐式启发式查询选择算法。 | ||
| 438 | +- **典型失败返回** | ||
| 439 | + - `ACLBLAS_STATUS_EXECUTION_FAILED`：设备执行失败 | ||
| 440 | + - `ACLBLAS_STATUS_ARCH_MISMATCH`：配置与设备架构不匹配 | ||
| 441 | + - `ACLBLAS_STATUS_NOT_SUPPORTED`：当前设备不支持该配置 | ||
| 442 | + - `ACLBLAS_STATUS_INVALID_VALUE`：参数为 NULL 或配置冲突 | ||
| 443 | + - `ACLBLAS_STATUS_NOT_INITIALIZED`：库未初始化 | ||
| 444 | + | ||
| 445 | +--- | ||
| 446 | + | ||
| 447 | +## 5. 推荐调用流程 | ||
| 448 | + | ||
| 449 | +1. 调用 `aclblasLtCreate` 创建句柄。 | ||
| 450 | +2. 使用 `aclblasLtMatrixLayoutCreate` 创建 A/B/C/D 的 `MatrixLayout`，并通过 `SetAttribute` 配置属性。 | ||
| 451 | +3. 使用 `aclblasLtMatmulDescCreate` 创建 `MatmulDesc`，设置转置、epilogue 等属性。 | ||
| 452 | +4. 使用 `aclblasLtMatmulPreferenceCreate` 创建 `MatmulPreference`，设置最大 workspace 等偏好。 | ||
| 453 | +5. 调用 `aclblasLtMatmulAlgoGetHeuristic` 获取可用算法。 | ||
| 454 | +6. 调用 `aclblasLtMatmul` 执行计算。 | ||
| 455 | +7. 依次销毁 Preference/Desc/Layout/Handle。 | ||
| 456 | + | ||
| 457 | +--- | ||
| 458 | + | ||
| 459 | +## 6. 最小示例（伪代码） | ||
| 460 | + | ||
| 461 | +```c | ||
| 462 | +aclblasLtHandle_t handle; | ||
| 463 | +aclblasLtCreate(&handle); | ||
| 464 | + | ||
| 465 | +aclblasLtMatrixLayout_t Adesc, Bdesc, Cdesc, Ddesc; | ||
| 466 | +aclblasLtMatrixLayoutCreate(&Adesc, ACL_FLOAT16, m, k, lda); | ||
| 467 | +aclblasLtMatrixLayoutCreate(&Bdesc, ACL_FLOAT16, k, n, ldb); | ||
| 468 | +aclblasLtMatrixLayoutCreate(&Cdesc, ACL_FLOAT16, m, n, ldc); | ||
| 469 | +aclblasLtMatrixLayoutCreate(&Ddesc, ACL_FLOAT16, m, n, ldd); | ||
| 470 | + | ||
| 471 | +aclblasLtMatmulDesc_t opDesc; | ||
| 472 | +aclblasLtMatmulDescCreate(&opDesc, ACLBLAS_COMPUTE_32F, ACL_FLOAT); | ||
| 473 | + | ||
| 474 | +aclblasLtMatmulPreference_t pref; | ||
| 475 | +aclblasLtMatmulPreferenceCreate(&pref); | ||
| 476 | +size_t workspaceCap = 32 * 1024 * 1024; | ||
| 477 | +aclblasLtMatmulPreferenceSetAttribute(pref, | ||
| 478 | + ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES, | ||
| 479 | + &workspaceCap, | ||
| 480 | + sizeof(workspaceCap)); | ||
| 481 | + | ||
| 482 | +aclblasLtMatmulHeuristicResult_t heur[8]; | ||
| 483 | +int algoCount = 0; | ||
| 484 | +aclblasLtMatmulAlgoGetHeuristic(handle, opDesc, Adesc, Bdesc, Cdesc, Ddesc, | ||
| 485 | + pref, 8, heur, &algoCount); | ||
| 486 | + | ||
| 487 | +aclblasLtMatmul(handle, opDesc, | ||
| 488 | + &alpha, A, Adesc, | ||
| 489 | + B, Bdesc, | ||
| 490 | + &beta, C, Cdesc, | ||
| 491 | + D, Ddesc, | ||
| 492 | + &heur[0].algo, | ||
| 493 | + workspace, workspaceBytes, | ||
| 494 | + stream); | ||
| 495 | + | ||
| 496 | +aclblasLtMatmulPreferenceDestroy(pref); | ||
| 497 | +aclblasLtMatmulDescDestroy(opDesc); | ||
| 498 | +aclblasLtMatrixLayoutDestroy(Adesc); | ||
| 499 | +aclblasLtMatrixLayoutDestroy(Bdesc); | ||
| 500 | +aclblasLtMatrixLayoutDestroy(Cdesc); | ||
| 501 | +aclblasLtMatrixLayoutDestroy(Ddesc); | ||
| 502 | +aclblasLtDestroy(handle); | ||
| 503 | +``` | ||
| 504 | + | ||
| 505 | +--- | ||
| 506 | + | ||
| 507 | +## 7. 返回码对照表（含定义位置） | ||
| 508 | + | ||
| 509 | +> **定义头文件**：`ops-blas/include/cann_ops_blas_common.h`（`aclblasStatus_t`，`typedef aclblasStatus_t aclblasLtStatus;`） | ||
| 510 | + | ||
| 511 | +| 枚举名 | 数值 | 含义 | 典型触发场景示例 | 头文件路径 | | ||
| 512 | +|--------------------------------------|------|----------------------------------------|----------------------------------------------------------------------------------|-------------------------------------------------| | ||
| 513 | +| `ACLBLAS_STATUS_SUCCESS` | 0 | 调用成功 | 接口执行成功，无错误。 | `ops-blas/include/cann_ops_blas_common.h` | | ||
| 514 | +| `ACLBLAS_STATUS_NOT_INITIALIZED` | 1 | 库未初始化 | 使用未创建或已销毁的 `aclblasLtHandle_t` 调用接口；环境未正确初始化。 | 同上 | | ||
| 515 | +| `ACLBLAS_STATUS_ALLOC_FAILED` | 2 | 资源/内存分配失败 | 创建描述符、句柄或内部 buffer 时内存不足；设备侧资源分配失败。 | 同上 | | ||
| 516 | +| `ACLBLAS_STATUS_INVALID_VALUE` | 3 | 非法参数 | 传入 NULL 指针、size 不匹配、不支持的属性类型/组合、无效 `requestedAlgoCount`。 | 同上 | | ||
| 517 | +| `ACLBLAS_STATUS_MAPPING_ERROR` | 4 | 内存映射/访问失败 | 访问非法设备内存地址；设备侧 DMA/映射失败。 | 同上 | | ||
| 518 | +| `ACLBLAS_STATUS_EXECUTION_FAILED` | 5 | 程序执行失败 | `aclblasLtMatmul` 在设备上执行过程中出现运行时错误。 | 同上 | | ||
| 519 | +| `ACLBLAS_STATUS_INTERNAL_ERROR` | 6 | 内部错误 | 内部逻辑异常或未预期状态，通常为实现内部错误。 | 同上 | | ||
| 520 | +| `ACLBLAS_STATUS_NOT_SUPPORTED` | 7 | 功能/配置不支持 | 当前设备/实现不支持的矩阵尺寸、数据类型、Epilogue 组合或启发式不可用。 | 同上 | | ||
| 521 | +| `ACLBLAS_STATUS_ARCH_MISMATCH` | 8 | 架构不匹配 | 算法或配置与当前 NPU 架构不兼容。 | 同上 | | ||
| 522 | +| `ACLBLAS_STATUS_HANDLE_IS_NULLPTR` | 9 | 句柄为 nullptr | 调用接口时传入的 `aclblasLtHandle_t` 或 BLAS 句柄为 NULL。 | 同上 | | ||
| 523 | +| `ACLBLAS_STATUS_INVALID_ENUM` | 10 | 不支持的枚举值 | 传入未定义的 `aclblasOperation_t`/`aclblasComputeType_t` 等枚举。 | 同上 | | ||
| 524 | +| `ACLBLAS_STATUS_UNKNOWN` | 11 | 后端返回未知状态码 | 后端库返回了当前版本未识别的错误码。 | 同上 | | ||
| 525 | + | ||
| 526 | +> **说明**： | ||
| 527 | +> Lt 接口（如 `aclblasLtMatmul`、`aclblasLtMatmulAlgoGetHeuristic` 等）返回的 `aclblasStatus_t` 与上述枚举一一对应，错误语义以此枚举为准。 | ||
| 528 | + | ||
| 529 | +--- | ||
| 530 | + | ||
| 531 | +## 8. 备注 | ||
| 532 | + | ||
| 533 | +- 接口能力与属性支持范围以当前实现版本为准，详细限制请参考头文件内注释。 | ||
| 534 | +- 若文档描述与头文件声明不一致，请**以头文件声明与实际实现行为为准**。 | ||
| 535 | +- 开发调试时，建议结合返回码对照表和具体接口注释快速定位问题。 | ||
| @@ -0,0 +1,189 @@ | |||
| 1 | +# 环境部署 | ||
| 2 | +请您在学习QuickStart或各类教程操作之前，请先参考下面步骤完成基础环境搭建。 | ||
| 3 | + | ||
| 4 | +注意本文提到的编译态和运行态场景含义如下，请按需安装： | ||
| 5 | + | ||
| 6 | +- 编译态：针对仅编译不运行本项目的场景，只需安装前置依赖和CANN toolkit包。 | ||
| 7 | +- 运行态：针对运行本项目的场景（编译运行或纯运行），除了安装前置依赖和CANN toolkit包，还需安装驱动与固件、CANN ops包。 | ||
| 8 | + | ||
| 9 | +## 前提条件 | ||
| 10 | + | ||
| 11 | +编译本项目前，请确保编译环境的基础库依赖、NPU驱动和固件已安装。 | ||
| 12 | + | ||
| 13 | +1. **安装依赖** | ||
| 14 | + | ||
| 15 | + - python >= 3.7.0（建议版本 <= 3.10） | ||
| 16 | + - gcc >= 7.3.0 | ||
| 17 | + - cmake >= 3.16.0 | ||
| 18 | + - pigz（可选，安装后可提升打包速度，建议版本 >= 2.4） | ||
| 19 | + - dos2unix | ||
| 20 | + - gawk | ||
| 21 | + - patch | ||
| 22 | + - make | ||
| 23 | + | ||
| 24 | + 上述依赖包请注意版本号，也可通过项目根目录install\_deps.sh一键安装，命令如下，若遇到不支持系统，请参考该文件自行适配。 | ||
| 25 | + ```bash | ||
| 26 | + bash install_deps.sh | ||
| 27 | + ``` | ||
| 28 | + | ||
| 29 | +2. **安装驱动与固件**（运行态依赖） | ||
| 30 | + | ||
| 31 | + 运行算子时必须安装驱动与固件，若仅编译算子，可跳过本操作。 | ||
| 32 | + | ||
| 33 | + 单击[下载链接](https://www.hiascend.com/hardware/firmware-drivers/community)，根据实际产品型号和环境架构，获取对应的`Ascend-hdk-<chip_type>-npu-driver_<version>_linux-<arch>.run`、`Ascend-hdk-<chip_type>-npu-firmware_<version>.run`包。 | ||
| 34 | + | ||
| 35 | + 安装指导详见《[CANN 软件安装指南](https://www.hiascend.com/document/redirect/CannCommunityInstSoftware)》。 | ||
| 36 | + | ||
| 37 | +## 环境准备 | ||
| 38 | + | ||
| 39 | +本项目提供了多种CANN包（`Ascend-cann-toolkit`和`Ascend-cann-ops`）安装方式，请按需选择。 | ||
| 40 | + | ||
| 41 | +| CANN安装方式 | 说明 |使用场景| | ||
| 42 | +| :--- | :--- | :--- | | ||
| 43 | +| WebIDE | 一站式开发平台，提供了在线直接运行的昇腾环境。当前可提供单机算力，**默认安装最新商发版CANN软件包**（目前是CANN 8.5.0）。 | 适用于没有昇腾设备的开发者。| | ||
| 44 | +| Docker | Docker镜像是一种高效部署方式，目前仅适用于Atlas A2系列产品，OS仅支持Ubuntu操作系统。**默认安装最新商发版CANN软件包**（目前是CANN 8.5.0） |适用有昇腾设备，需要快速搭建环境的开发者。| | ||
| 45 | +| 手动安装 | - |适用有昇腾设备，想体验手动安装CANN包或体验最新master分支能力的开发者。| | ||
| 46 | + | ||
| 47 | +### 方式1：WebIDE环境 | ||
| 48 | + | ||
| 49 | +对于无昇腾设备的开发者，可直接使用WebIDE开发平台，即“**算子一站式开发平台**”，该平台为您提供在线可直接运行的昇腾环境，环境中已安装必备的软件包，无需手动安装。更多关于开发平台的介绍请参考[LINK](https://gitcode.com/org/cann/discussions/54)。 | ||
| 50 | + | ||
| 51 | +1. 进入开源项目，单击“`云开发`”按钮，使用已认证过的华为云账号登录。若未注册或认证，请根据页面提示进行注册和认证。 | ||
| 52 | + | ||
| 53 | + <img src="../figures/cloudIDE.png" alt="云平台" width="750px" height="90px"> | ||
| 54 | + | ||
| 55 | +2. 根据页面提示创建并启动云开发环境，单击“`连接 > WebIDE `”进入算子一站式开发平台，开源项目的资源默认在`/mnt/workspace`目录下。 | ||
| 56 | + | ||
| 57 | + <img src="../figures/webIDE.png" alt="云平台" width="1000px" height="150px"> | ||
| 58 | + | ||
| 59 | + | ||
| 60 | +### 方式2：Docker部署 | ||
| 61 | + | ||
| 62 | +对于有昇腾设备的开发者，若您想快速搭建昇腾环境，可使用Docker镜像部署。 | ||
| 63 | + | ||
| 64 | +> **说明**：镜像文件比较大，下载需要一定时间，请您耐心等待。 | ||
| 65 | + | ||
| 66 | +1.**下载镜像** | ||
| 67 | + | ||
| 68 | +- 步骤1：以root用户登录宿主机。确保宿主机已安装Docker引擎（版本1.11.2及以上）。 | ||
| 69 | +- 步骤2：从[昇腾镜像仓库](https://www.hiascend.com/developer/ascendhub/detail/17da20d1c2b6493cb38765adeba85884)拉取已预集成CANN软件包及`ops-blas`所需依赖的镜像。命令如下，根据实际架构选择： | ||
| 70 | + | ||
| 71 | + ```bash | ||
| 72 | + # 示例：拉取ARM架构的CANN开发镜像 | ||
| 73 | + docker pull --platform=arm64 swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops | ||
| 74 | + # 示例：拉取X86架构的CANN开发镜像 | ||
| 75 | + docker pull --platform=amd64 swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops | ||
| 76 | + ``` | ||
| 77 | + | ||
| 78 | +2.**运行Docker** | ||
| 79 | +拉取镜像后，需要以特定参数启动容器，以便容器内能访问宿主的昇腾设备。 | ||
| 80 | + | ||
| 81 | +```bash | ||
| 82 | +docker run --name cann_container --device /dev/davinci0 --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -v /etc/ascend_install.info:/etc/ascend_install.info -it swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops bash | ||
| 83 | +``` | ||
| 84 | +| 参数 | 说明 | 注意事项 | | ||
| 85 | +| :--- | :--- | :--- | | ||
| 86 | +| `--name cann_container` | 为容器指定名称，便于管理。 | 可自定义。 | | ||
| 87 | +| `--device /dev/davinci0` | 核心：将宿主机的NPU设备卡映射到容器内，可指定映射多张NPU设备卡。 | 必须根据实际情况调整：`davinci0`对应系统中的第0张NPU卡。请先在宿主机执行 `npu-smi info`命令，根据输出显示的设备号（如`NPU 0`, `NPU 1`）来修改此编号。| | ||
| 88 | +| `--device /dev/davinci_manager` | 映射NPU设备管理接口。 | - | | ||
| 89 | +| `--device /dev/devmm_svm` | 映射设备内存管理接口。 | - | | ||
| 90 | +| `--device /dev/hisi_hdc` | 映射主机与设备间的通信接口。 | - | | ||
| 91 | +| `-v /usr/local/dcmi:/usr/local/dcmi` | 挂载设备容器管理接口（DCMI）相关工具和库。 | - | | ||
| 92 | +| `-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi` | 挂载`npu-smi`工具。 | 使容器内可以直接运行此命令来查询NPU状态和性能信息。| | ||
| 93 | +| `-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/` | 关键挂载：将宿主机的NPU驱动库映射到容器内。 | - | | ||
| 94 | +| `-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info` | 挂载驱动版本信息文件。 | - | | ||
| 95 | +| `-v /etc/ascend_install.info:/etc/ascend_install.info` | 挂载CANN软件安装信息文件。 | - | | ||
| 96 | +| `-it` | `-i`（交互式）和 `-t`（分配伪终端）的组合参数。 | - | | ||
| 97 | +| `swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops` | 指定要运行的Docker镜像。 |请确保此镜像名和标签（tag）与你通过`docker pull`拉取的镜像完全一致。 | | ||
| 98 | +| `bash` | 容器启动后立即执行的命令。 | - | | ||
| 99 | + | ||
| 100 | +### 方式3：手动安装CANN | ||
| 101 | + | ||
| 102 | +对于有昇腾设备的开发者，若您想手动安装CANN包，请根据下述描述，选择对应的安装指导。 | ||
| 103 | + | ||
| 104 | +**场景1：已发布版本** | ||
| 105 | + | ||
| 106 | +若您想体验**官网正式发布的CANN包**能力，请访问[CANN官网下载中心](https://www.hiascend.com/cann/download)，根据产品和环境架构选择对应版本的软件包（仅支持CANN 8.5.0及后续版本）进行安装。 | ||
| 107 | + | ||
| 108 | + | ||
| 109 | +**场景2：master版本** | ||
| 110 | + | ||
| 111 | +若您想体验**master分支最新能力**，请单击[下载链接](https://ascend.devcloud.huaweicloud.com/artifactory/cann-run-mirror/software/master/)，根据产品和环境架构选择对应版本的软件包，关键安装命令如下，更多安装指导参考《[CANN软件安装指南](https://www.hiascend.com/document/redirect/CannCommunityInstWizard)》。 | ||
| 112 | + | ||
| 113 | +1. 安装CANN toolkit包。 | ||
| 114 | + | ||
| 115 | + ```bash | ||
| 116 | + # 确保安装包具有可执行权限 | ||
| 117 | + chmod +x Ascend-cann-toolkit_${cann_version}_linux-${arch}.run | ||
| 118 | + # 安装命令 | ||
| 119 | + ./Ascend-cann-toolkit_${cann_version}_linux-${arch}.run --install --install-path=${install_path} | ||
| 120 | + ``` | ||
| 121 | + - \$\{cann\_version\}：表示CANN包版本号。 | ||
| 122 | + - \$\{arch\}：表示CPU架构，如aarch64、x86_64。 | ||
| 123 | + - \$\{install\_path\}：表示指定安装路径，默认安装在`/usr/local/Ascend`目录。 | ||
| 124 | + | ||
| 125 | +2. 安装CANN ops包（运行态依赖）。 | ||
| 126 | + | ||
| 127 | + ops包是运行态依赖，若仅编译算子，可以不安装此包。 | ||
| 128 | + | ||
| 129 | + ```bash | ||
| 130 | + # 确保安装包具有可执行权限 | ||
| 131 | + chmod +x Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run | ||
| 132 | + # 安装命令 | ||
| 133 | + ./Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run --install --install-path=${install_path} | ||
| 134 | + ``` | ||
| 135 | + | ||
| 136 | + - \$\{cann\_version\}：表示CANN包版本号。 | ||
| 137 | + - \$\{arch\}：表示CPU架构，如aarch64、x86_64。 | ||
| 138 | + - \$\{soc\_name\}：表示NPU型号名称。 | ||
| 139 | + - \$\{install\_path\}：表示指定安装路径，ops包需与toolkit包安装在相同路径，root用户默认安装在`/usr/local/Ascend`目录。 | ||
| 140 | + | ||
| 141 | +## 环境验证 | ||
| 142 | + | ||
| 143 | +安装完CANN包后，需验证环境和驱动是否正常。 | ||
| 144 | + | ||
| 145 | +- **检查NPU设备** | ||
| 146 | + | ||
| 147 | + ```bash | ||
| 148 | + # 运行npu-smi，若能正常显示设备信息，则驱动正常 | ||
| 149 | + npu-smi info | ||
| 150 | + ``` | ||
| 151 | +- **检查CANN安装** | ||
| 152 | + | ||
| 153 | + ```bash | ||
| 154 | + # 查看CANN Toolkit版本信息（默认路径安装） | ||
| 155 | + cat /usr/local/Ascend/ascend-toolkit/latest/opp/version.info | ||
| 156 | + ``` | ||
| 157 | + | ||
| 158 | +## 环境变量配置 | ||
| 159 | + | ||
| 160 | +按需选择合适的命令使环境变量生效。 | ||
| 161 | +```bash | ||
| 162 | +# 默认路径安装，以root用户为例（非root用户，将/usr/local替换为${HOME}） | ||
| 163 | +source /usr/local/Ascend/cann/set_env.sh | ||
| 164 | +# 指定路径安装 | ||
| 165 | +# source ${install_path}/cann/set_env.sh | ||
| 166 | +``` | ||
| 167 | + | ||
| 168 | +## 源码下载 | ||
| 169 | + | ||
| 170 | +通过如下命令下载项目源码，\$\{tag\_version\}请替换为版本分支标签名，源码版本与CANN版本配套关系参见[release仓库](https://gitcode.com/cann/release-management)。 | ||
| 171 | + | ||
| 172 | +```bash | ||
| 173 | +# 下载项目对应分支源码 | ||
| 174 | +git clone -b ${tag_version} https://gitcode.com/cann/ops-blas.git | ||
| 175 | +``` | ||
| 176 | + | ||
| 177 | +对于WebIDE或Docker环境，已默认提供最新商发版本的项目源码，如需获取其他版本的源码，也需通过上述命令下载源码。 | ||
| 178 | + | ||
| 179 | +对于手动安装CANN场景，安装CANN包和下载源码后，还需额外安装python基础库依赖。 | ||
| 180 | + | ||
| 181 | +```bash | ||
| 182 | +# 安装根目录requirements.txt依赖 | ||
| 183 | +cd ops-blas | ||
| 184 | +pip3 install -r requirements.txt | ||
| 185 | +``` | ||
| 186 | +> [!NOTE] 注意 | ||
| 187 | +> | ||
| 188 | +> - gitcode平台在使用HTTPS协议的时候要配置并使用个人访问令牌代替登录密码进行克隆，推送等操作。 | ||
| 189 | +> - 若您的编译环境无法访问网络，无法通过git指令下载代码，请先在联网环境中下载源码，再手动上传至目标环境。 | ||
| @@ -1,601 +1,634 @@ | |||
| 1 | -/** | 1 | +/** |
| 2 | - * Copyright (c) 2026 Huawei Technologies Co., Ltd. | 2 | + * Copyright (c) 2026 Huawei Technologies Co., Ltd. |
| 3 | - * This program is free software, you can redistribute it and/or modify it under the terms and conditions of | 3 | + * This program is free software, you can redistribute it and/or modify it under the terms and conditions of |
| 4 | - * CANN Open Software License Agreement Version 2.0 (the "License"). | 4 | + * CANN Open Software License Agreement Version 2.0 (the "License"). |
| 5 | - * Please refer to the License for details. You may not use this file except in compliance with the License. | 5 | + * Please refer to the License for details. You may not use this file except in compliance with the License. |
| 6 | - * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | 6 | + * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, |
| 7 | - * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | 7 | + * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. |
| 8 | - * See LICENSE in the root of the software repository for the full text of the License. | 8 | + * See LICENSE in the root of the software repository for the full text of the License. |
| 9 | - */ | 9 | + */ |
| 10 | - | 10 | + |
| 11 | -#pragma once | 11 | +#pragma once |
| 12 | - | 12 | + |
| 13 | -#include "cann_ops_blas_common.h" | 13 | +#include "cann_ops_blas_common.h" |
| 14 | -#include <acl/acl.h> | 14 | +#include <acl/acl.h> |
| 15 | - | 15 | + |
| 16 | -#ifdef __cplusplus | 16 | +#ifdef __cplusplus |
| 17 | -#include <cstddef> | 17 | +#include <cstddef> |
| 18 | -#include <cstdint> | 18 | +#include <cstdint> |
| 19 | -#else | 19 | +#else |
| 20 | -#include <stddef.h> | 20 | +#include <stddef.h> |
| 21 | -#include <stdint.h> | 21 | +#include <stdint.h> |
| 22 | -#endif | 22 | +#endif |
| 23 | - | 23 | + |
| 24 | -#ifdef __cplusplus | 24 | +#ifdef __cplusplus |
| 25 | -extern "C" { | 25 | +extern "C" { |
| 26 | -#endif | 26 | +#endif |
| 27 | - | 27 | + |
| 28 | -/*! \ingroup types_module | 28 | +/*! \ingroup types_module |
| 29 | - * \brief Descriptor of the matrix layout. | 29 | + * \brief Descriptor of the library context. |
| 30 | - */ | 30 | + */ |
| 31 | -typedef struct | 31 | +typedef void* aclblasLtHandle_t; |
| 32 | -{ | 32 | + |
| 33 | - uint64_t data[4]; | 33 | +/*! \ingroup types_module |
| 34 | -} aclblasLtMatrixLayoutOpaque_t; | 34 | + * \brief Descriptor of the matrix layout. |
| 35 | - | 35 | + */ |
| 36 | -typedef aclblasLtMatrixLayoutOpaque_t* aclblasLtMatrixLayout_t; | 36 | +typedef struct { |
| 37 | - | 37 | + uint64_t data[8]; |
| 38 | -/*! \ingroup types_module | 38 | +} aclblasLtMatrixLayoutOpaque_t; |
| 39 | - * \brief Descriptor of the library context. | 39 | + |
| 40 | - */ | 40 | +typedef aclblasLtMatrixLayoutOpaque_t* aclblasLtMatrixLayout_t; |
| 41 | -typedef void* aclblasLtHandle_t; | 41 | + |
| 42 | - | 42 | +/*! \ingroup types_module |
| 43 | -/*! \ingroup types_module | 43 | + * \brief Descriptor of the matmul operation. |
| 44 | - * \brief Descriptor of the matmul operation. | 44 | + */ |
| 45 | - */ | 45 | +typedef struct { |
| 46 | -typedef struct | 46 | + uint64_t data[24]; |
| 47 | -{ | 47 | +} aclblasLtMatmulDescOpaque_t; |
| 48 | - uint64_t data[4]; | 48 | + |
| 49 | -} aclblasLtMatmulDescOpaque_t; | 49 | +typedef aclblasLtMatmulDescOpaque_t* aclblasLtMatmulDesc_t; |
| 50 | - | 50 | + |
| 51 | -typedef aclblasLtMatmulDescOpaque_t* aclblasLtMatmulDesc_t; | 51 | +/*! \ingroup types_module |
| 52 | - | 52 | + * \brief Descriptor of the matmul preference. |
| 53 | -/*! \ingroup types_module | 53 | + */ |
| 54 | - * \brief Descriptor of the matmul preference. | 54 | +typedef struct { |
| 55 | - */ | 55 | + uint64_t data[8]; |
| 56 | -typedef struct | 56 | +} aclblasLtMatmulPreferenceOpaque_t; |
| 57 | -{ | 57 | + |
| 58 | - uint64_t data[5]; | 58 | +typedef aclblasLtMatmulPreferenceOpaque_t* aclblasLtMatmulPreference_t; |
| 59 | -} aclblasLtMatmulPreferenceOpaque_t; | 59 | + |
| 60 | - | 60 | +/*! \ingroup types_module |
| 61 | -typedef aclblasLtMatmulPreferenceOpaque_t* aclblasLtMatmulPreference_t; | 61 | + * \struct aclblasLtMatmulAlgo_t |
| 62 | - | 62 | + * \brief Description of the matrix multiplication algorithm. |
| 63 | -/*! \ingroup types_module | 63 | + */ |
| 64 | - * \struct aclblasLtMatmulAlgo_t | 64 | +typedef struct _aclblasLtMatmulAlgo_t { |
| 65 | - * \brief Description of the matrix multiplication algorithm. | 65 | +#ifdef __cplusplus |
| 66 | - */ | 66 | + uint8_t data[16] = {0}; |
| 67 | -typedef struct _aclblasLtMatmulAlgo_t { | 67 | + size_t max_workspace_bytes = 0; |
| 68 | -#ifdef __cplusplus | 68 | +#else |
| 69 | - uint8_t data[16] = {0}; | 69 | + uint8_t data[16]; |
| 70 | - size_t max_workspace_bytes = 0; | 70 | + size_t max_workspace_bytes; |
| 71 | -#else | 71 | +#endif |
| 72 | - uint8_t data[16]; | 72 | +} aclblasLtMatmulAlgo_t; |
| 73 | - size_t max_workspace_bytes; | 73 | + |
| 74 | -#endif | 74 | +/*! \ingroup types_module |
| 75 | -} aclblasLtMatmulAlgo_t; | 75 | + * \brief Matmul Heuristic Result. |
| 76 | - | 76 | + */ |
| 77 | -/*! \ingroup types_module | 77 | +typedef struct _aclblasLtMatmulHeuristicResult_t { |
| 78 | - * \brief Matmul Heuristic Result. | 78 | + aclblasLtMatmulAlgo_t algo; /**< Algo struct */ |
| 79 | - */ | 79 | + size_t workspaceSize = 0; /**< Actual size of workspace memory required. */ |
| 80 | -typedef struct _aclblasLtMatmulHeuristicResult_t { | 80 | + aclblasStatus_t state = ACLBLAS_STATUS_SUCCESS; /**< Result status. */ |
| 81 | - aclblasLtMatmulAlgo_t algo; /**< Algo struct */ | 81 | + float wavesCount = 1.0; /**< Waves count is a device utilization metric. */ |
| 82 | - size_t workspaceSize; /**< Actual size of workspace memory required. */ | 82 | + int reserved[4]; /**< Reserved. */ |
| 83 | - aclblasStatus_t state; /**< Result status. */ | 83 | +} aclblasLtMatmulHeuristicResult_t; |
| 84 | - float wavesCount; /**< Waves count is a device utilization metric. */ | 84 | + |
| 85 | - int reserved[4]; /**< Reserved. */ | 85 | +/*! \ingroup types_module |
| 86 | -} aclblasLtMatmulHeuristicResult_t; | 86 | + * \brief Matrix order. |
| 87 | - | 87 | + */ |
| 88 | -/*! \ingroup types_module | 88 | +typedef enum aclblasLtOrder { |
| 89 | - * \brief Matrix order. | 89 | + ACLBLASLT_ORDER_COL = 0, /**< Column major */ |
| 90 | - */ | 90 | + ACLBLASLT_ORDER_ROW = 1, /**< Row major */ |
| 91 | -typedef enum aclblasLtOrder { | 91 | +} aclblasLtOrder_t; |
| 92 | - ACLBLASLT_ORDER_COL = 0, /**< Column major */ | 92 | + |
| 93 | - ACLBLASLT_ORDER_ROW = 1, /**< Row major */ | 93 | +/*! \ingroup types_module |
| 94 | -} aclblasLtOrder_t; | 94 | + * \brief Property type. |
| 95 | - | 95 | + */ |
| 96 | -/*! \ingroup types_module | 96 | +typedef enum aclblasLtPropertyType { |
| 97 | - * \brief Property type. | 97 | + ACLBLASLT_PROPERTY_MAJOR_VERSION = 0, /**<Major version number. */ |
| 98 | - */ | 98 | + ACLBLASLT_PROPERTY_MINOR_VERSION = 1, /**<Minor version number. */ |
| 99 | -typedef enum aclblasLtPropertyType { | 99 | + ACLBLASLT_PROPERTY_PATCH_LEVEL = 2, /**<Patch version number. */ |
| 100 | - ACLBLASLT_PROPERTY_MAJOR_VERSION = 0, /**<Major version number. */ | 100 | +} aclblasLtPropertyType_t; |
| 101 | - ACLBLASLT_PROPERTY_MINOR_VERSION = 1, /**<Minor version number. */ | 101 | + |
| 102 | - ACLBLASLT_PROPERTY_PATCH_LEVEL = 2, /**<Patch version number. */ | 102 | +/*! \ingroup types_module |
| 103 | -} aclblasLtPropertyType_t; | 103 | + * \brief Specifies the enumeration type to set the postprocessing options for the epilogue. |
| 104 | - | 104 | + */ |
| 105 | -/*! \ingroup types_module | 105 | +typedef enum aclblasLtEpilogue { |
| 106 | - * \brief Specifies the enumeration type to set the postprocessing options for the epilogue. | 106 | + ACLBLASLT_EPILOGUE_DEFAULT = 1, /**<No special postprocessing. Scale and quantize the results if necessary.*/ |
| 107 | - */ | 107 | + ACLBLASLT_EPILOGUE_RELU = 2, /**<Apply ReLU pointwise transform to the results (``x:=max(x, 0)``)*/ |
| 108 | -typedef enum aclblasLtEpilogue { | 108 | + ACLBLASLT_EPILOGUE_BIAS = 4, /**<Apply (broadcast) bias from the bias vector. The bias vector length must match the number of rows in matrix D, and it must be packed (so the stride between vector elements is one). The bias vector is broadcast to all columns and added before applying the final postprocessing.*/ |
| 109 | - ACLBLASLT_EPILOGUE_DEFAULT = 1, /**<No special postprocessing. Scale and quantize the results if necessary.*/ | 109 | + ACLBLASLT_EPILOGUE_RELU_BIAS = 6, /**<Apply bias and then ReLU transform.*/ |
| 110 | - ACLBLASLT_EPILOGUE_RELU = 2, /**<Apply ReLU pointwise transform to the results (``x:=max(x, 0)``)*/ | 110 | + ACLBLASLT_EPILOGUE_GELU = 32, /**<Apply GELU pointwise transform to the results (``x:=GELU(x)``).*/ |
| 111 | - ACLBLASLT_EPILOGUE_BIAS = 4, /**<Apply (broadcast) bias from the bias vector. The bias vector length must match the number of rows in matrix D, and it must be packed (so the stride between vector elements is one). The bias vector is broadcast to all columns and added before applying the final postprocessing.*/ | 111 | + ACLBLASLT_EPILOGUE_GELU_BIAS = 36, /**<Apply Bias and then GELU transform.*/ |
| 112 | - ACLBLASLT_EPILOGUE_RELU_BIAS = 6, /**<Apply bias and then ReLU transform.*/ | 112 | + ACLBLASLT_EPILOGUE_RELU_AUX = 130, /**<Output GEMM results before applying RELU transform.*/ |
| 113 | - ACLBLASLT_EPILOGUE_GELU = 32, /**<Apply GELU pointwise transform to the results (``x:=GELU(x)``).*/ | 113 | + ACLBLASLT_EPILOGUE_RELU_AUX_BIAS = 134, /**<Output GEMM results after applying bias but before applying RELU transform.*/ |
| 114 | - ACLBLASLT_EPILOGUE_GELU_BIAS = 36, /**<Apply Bias and then GELU transform.*/ | 114 | + ACLBLASLT_EPILOGUE_DRELU = 136, |
| 115 | - ACLBLASLT_EPILOGUE_RELU_AUX = 130, /**<Output GEMM results before applying RELU transform.*/ | 115 | + ACLBLASLT_EPILOGUE_DRELU_BGRAD = 152, /**<Apply gradient RELU transform and bias gradient to the results. Requires additional auxiliary input. */ /**<Apply gradient RELU transform. Requires additional auxiliary input. */ |
| 116 | - ACLBLASLT_EPILOGUE_RELU_AUX_BIAS = 134, /**<Output GEMM results after applying bias but before applying RELU transform.*/ | 116 | + ACLBLASLT_EPILOGUE_GELU_AUX = 160, /**<Output GEMM results before applying GELU transform.*/ |
| 117 | - ACLBLASLT_EPILOGUE_DRELU = 136, | 117 | + ACLBLASLT_EPILOGUE_GELU_AUX_BIAS = 164, /**<Output GEMM results after applying bias but before applying GELU transform.*/ |
| 118 | - ACLBLASLT_EPILOGUE_DRELU_BGRAD = 152, /**<Apply gradient RELU transform and bias gradient to the results. Requires additional auxiliary input. */ /**<Apply gradient RELU transform. Requires additional auxiliary input. */ | 118 | + ACLBLASLT_EPILOGUE_DGELU = 192, /**<Apply gradient GELU transform. Requires additional auxiliary input. */ |
| 119 | - ACLBLASLT_EPILOGUE_GELU_AUX = 160, /**<Output GEMM results before applying GELU transform.*/ | 119 | + ACLBLASLT_EPILOGUE_DGELU_BGRAD = 208, /**<Apply gradient GELU transform and bias gradient to the results. Requires additional auxiliary input. */ |
| 120 | - ACLBLASLT_EPILOGUE_GELU_AUX_BIAS = 164, /**<Output GEMM results after applying bias but before applying GELU transform.*/ | 120 | + ACLBLASLT_EPILOGUE_BGRADA = 256, /**<Apply bias gradient to A and output GEMM result. */ |
| 121 | - ACLBLASLT_EPILOGUE_DGELU = 192, /**<Apply gradient GELU transform. Requires additional auxiliary input. */ | 121 | + ACLBLASLT_EPILOGUE_BGRADB = 512, /**<Apply bias gradient to B and output GEMM result. */ |
| 122 | - ACLBLASLT_EPILOGUE_DGELU_BGRAD = 208, /**<Apply gradient GELU transform and bias gradient to the results. Requires additional auxiliary input. */ | 122 | + ACLBLASLT_EPILOGUE_SIGMOID = 1024, /**<Apply sigmoid activation function pointwise. */ |
| 123 | - ACLBLASLT_EPILOGUE_BGRADA = 256, /**<Apply bias gradient to A and output GEMM result. */ | 123 | + ACLBLASLT_EPILOGUE_SWISH_EXT = 65536, /**<Apply Swish pointwise transform to the results (``x:=Swish(x, 1)``).*/ |
| 124 | - ACLBLASLT_EPILOGUE_BGRADB = 512, /**<Apply bias gradient to B and output GEMM result. */ | 124 | + ACLBLASLT_EPILOGUE_SWISH_BIAS_EXT = 65540, /**<Apply Bias and then Swish transform.*/ |
| 125 | - ACLBLASLT_EPILOGUE_SIGMOID = 1024, /**<Apply sigmoid activation function pointwise. */ | 125 | + ACLBLASLT_EPILOGUE_CLAMP_EXT = 131072, /**<Apply pointwise clamp to the results (``x:=max(alpha, min(x, beta))``).*/ |
| 126 | - ACLBLASLT_EPILOGUE_SWISH_EXT = 65536, /**<Apply Swish pointwise transform to the results (``x:=Swish(x, 1)``).*/ | 126 | + ACLBLASLT_EPILOGUE_CLAMP_BIAS_EXT = 131076, /**<Apply Bias and then clamp.*/ |
| 127 | - ACLBLASLT_EPILOGUE_SWISH_BIAS_EXT = 65540, /**<Apply Bias and then Swish transform.*/ | 127 | + ACLBLASLT_EPILOGUE_CLAMP_AUX_EXT = 131200, /**<Output GEMM results before applying clamp transform.*/ |
| 128 | - ACLBLASLT_EPILOGUE_CLAMP_EXT = 131072, /**<Apply pointwise clamp to the results (``x:=max(alpha, min(x, beta))``).*/ | 128 | + ACLBLASLT_EPILOGUE_CLAMP_AUX_BIAS_EXT = 131204, /**<Output GEMM results after applying bias but before applying clamp transform.*/ |
| 129 | - ACLBLASLT_EPILOGUE_CLAMP_BIAS_EXT = 131076, /**<Apply Bias and then clamp.*/ | 129 | +} aclblasLtEpilogue_t; |
| 130 | - ACLBLASLT_EPILOGUE_CLAMP_AUX_EXT = 131200, /**<Output GEMM results before applying clamp transform.*/ | 130 | + |
| 131 | - ACLBLASLT_EPILOGUE_CLAMP_AUX_BIAS_EXT = 131204, /**<Output GEMM results after applying bias but before applying clamp transform.*/ | 131 | +/*! \ingroup types_module |
| 132 | -} aclblasLtEpilogue_t; | 132 | + * \brief Matrix layout attributes. |
| 133 | - | 133 | + */ |
| 134 | -/*! \ingroup types_module | 134 | +typedef enum aclblasLtMatrixLayoutAttribute { |
| 135 | - * \brief Matrix layout attributes. | 135 | + ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT = 0, /**< Batch count. Default: 1. Type: ``int32_t``. */ |
| 136 | - */ | 136 | + ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET = 1, /**< Strided-batch offset (elements). Default: 0. Type: ``int64_t``. */ |
| 137 | -typedef enum aclblasLtMatrixLayoutAttribute { | 137 | + ACLBLASLT_MATRIX_LAYOUT_TYPE = 2, /**< Matrix data type. See ``aclDataType``. Type: ``uint32_t``. */ |
| 138 | - ACLBLASLT_MATRIX_LAYOUT_BATCH_COUNT = 0, /**<Number of batches of this matrix. Default value is 1. Data type: ``int32_t``. */ | 138 | + ACLBLASLT_MATRIX_LAYOUT_ORDER = 3, /**< Memory order. See ``aclblasLtOrder_t``. Default: ``ACLBLASLT_ORDER_COL``. Type: ``int32_t``. */ |
| 139 | - ACLBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET = 1, /**<Stride (in elements) to the next matrix for the strided batch operation. Default value is 0. Data type: ``int64_t``. */ | 139 | + ACLBLASLT_MATRIX_LAYOUT_ROWS = 4, /**< Row count. Type: ``uint64_t``. */ |
| 140 | - /** Data type. See ``aclDataType``. | 140 | + ACLBLASLT_MATRIX_LAYOUT_COLS = 5, /**< Column count. Type: ``uint64_t``. */ |
| 141 | - * | 141 | + ACLBLASLT_MATRIX_LAYOUT_LD = 6, /**< Leading dimension (elements). Type: ``int64_t``. */ |
| 142 | - * ``uint32_t`` | 142 | +} aclblasLtMatrixLayoutAttribute_t; |
| 143 | - */ | 143 | + |
| 144 | - ACLBLASLT_MATRIX_LAYOUT_TYPE = 2, | 144 | +/*! \ingroup types_module |
| 145 | - | 145 | + * \brief Matmul operation descriptor attributes. |
| 146 | - /** Memory order of the data. See ``aclblasLtOrder_t``. | 146 | + */ |
| 147 | - * | 147 | +typedef enum aclblasLtMatmulDescAttribute { |
| 148 | - * ``int32_t``, default: ``ACLBLASLT_ORDER_COL``. | 148 | + ACLBLASLT_MATMUL_DESC_TRANSA = 0, /**<Specifies the type of transformation operation that should be performed on matrix A. Default value is ``ACLBLAS_OP_N`` (for example, non-transpose operation). See ``aclblasOperation_t``. Data type: ``int32_t``. */ |
| 149 | - */ | 149 | + ACLBLASLT_MATMUL_DESC_TRANSB = 1, /**<Specifies the type of transformation operation that should be performed on matrix B. Default value is ``ACLBLAS_OP_N`` (for example, non-transpose operation). See ``aclblasOperation_t``. Data type: ``int32_t``. */ |
| 150 | - ACLBLASLT_MATRIX_LAYOUT_ORDER = 3, | 150 | + ACLBLASLT_MATMUL_DESC_EPILOGUE = 2, /**<Epilogue function. See ``aclblasLtEpilogue_t``. Default value is ``ACLBLASLT_EPILOGUE_DEFAULT``. Data type: ``uint32_t``. */ |
| 151 | - | 151 | + ACLBLASLT_MATMUL_DESC_BIAS_POINTER = 3, /**<Bias or bias gradient vector pointer in the device memory. Data type: ``void*`` / ``const void*``. */ |
| 152 | - /** Number of rows. | 152 | + ACLBLASLT_MATMUL_DESC_BIAS_DATA_TYPE = 4, /**<Type of the bias vector in the device memory. Can be set the same as the D matrix type or Scale type. Bias case: see ``ACLBLASLT_EPILOGUE_BIAS``. Data type: ``int32_t`` based on ``aclDataType``. */ |
| 153 | - * | 153 | + ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER = 5, /**<Device pointer to the scale factor value that converts data in matrix A to the compute data type range. The scaling factor must have the same type as the compute type. If not specified, or set to NULL, the scaling factor is assumed to be ``1``. If set for an unsupported matrix data, scale, and compute type combination, calling aclblasLtMatmul() will return ``ACLBLAS_INVALID_VALUE``. Default value: NULL. Data type: ``void*`` ``/const void*``. */ |
| 154 | - * Typically only values that can be expressed as ``int32_t`` are supported. | 154 | + ACLBLASLT_MATMUL_DESC_B_SCALE_POINTER = 6, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix B. Default value: NULL. Data type: ``void*`` / ``const void*``. */ |
| 155 | - * | 155 | + ACLBLASLT_MATMUL_DESC_C_SCALE_POINTER = 7, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix C. Default value: NULL. Data type: ``void*`` / ``const void*``. */ |
| 156 | - * ``uint64_t`` | 156 | + ACLBLASLT_MATMUL_DESC_D_SCALE_POINTER = 8, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix D. Default value: NULL. Data type: ``void*`` / ``const void*``. */ |
| 157 | - */ | 157 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_SCALE_POINTER = 9, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix AUX. Default value: NULL. Data type: ``void*`` / ``const void*``. */ |
| 158 | - ACLBLASLT_MATRIX_LAYOUT_ROWS = 4, | 158 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER = 10, /**<Epilogue auxiliary buffer pointer in the device memory. Data type: ``void*`` / ``const void*``. */ |
| 159 | - | 159 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD = 11, /**<The leading dimension of the epilogue auxiliary buffer pointer in the device memory. Data type: ``int64_t``. */ |
| 160 | - /** Number of columns. | 160 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE = 12, /**<The batch stride of the epilogue auxiliary buffer pointer in the device memory. Data type: ``int64_t``. */ |
| 161 | - * | 161 | + ACLBLASLT_MATMUL_DESC_POINTER_MODE = 13, /**<Specifies that alpha and beta are passed by reference, whether they are scalars on the host or on the device, or device vectors. Default value is: ``ACLBLASLT_POINTER_MODE_HOST`` (on the host). Data type: ``int32_t`` based on ``aclblasLtPointerMode_t``. */ |
| 162 | - * Typically only values that can be expressed as ``int32_t`` are supported. | 162 | + ACLBLASLT_MATMUL_DESC_AMAX_D_POINTER = 14, /**<Device pointer to the memory location that on completion will be set to the maximum of the absolute values in the output matrix. Data type: ``void*`` / ``const void*``. */ |
| 163 | - * | 163 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_DATA_TYPE = 22, /**<Type of the auxiliary vector in the device memory. Default value is: ``ACLBLASLT_DATATYPE_INVALID`` (using D matrix type). Data type: ``int32_t`` based on ``aclDataType``. */ |
| 164 | - * ``uint64_t`` | 164 | + ACLBLASLT_MATMUL_DESC_A_SCALE_MODE = 31, /**<Scaling mode that defines how the matrix scaling factor for matrix A is interpreted. See ``aclblasLtMatmulMatrixScale_t``. */ |
| 165 | - */ | 165 | + ACLBLASLT_MATMUL_DESC_B_SCALE_MODE = 32, /**<Scaling mode that defines how the matrix scaling factor for matrix B is interpreted. See ``aclblasLtMatmulMatrixScale_t``. */ |
| 166 | - ACLBLASLT_MATRIX_LAYOUT_COLS = 5, | 166 | + ACLBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_A_EXT = 100, /**<Compute input A types. Defines the data type used for the input A of a matrix multiply. */ |
| 167 | - | 167 | + ACLBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_B_EXT = 101, /**<Compute input B types. Defines the data type used for the input B of a matrix multiply. */ |
| 168 | - /** Matrix leading dimension. | 168 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG0_EXT = 102, /**<First extra argument for the activation function. Data type: ``float``. */ |
| 169 | - * | 169 | + ACLBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG1_EXT = 103, /**<Second extra argument for the activation function. Data type: ``float``. */ |
| 170 | - * For ``ACLBLASLT_ORDER_COL``, this is the stride (in elements) of the matrix column. For more details and documentation for | 170 | + ACLBLASLT_MATMUL_DESC_MAX, |
| 171 | - * other memory orders, see the documentation for ``a c lblasLtOrder_t`` values. | 171 | +} aclblasLtMatmulDescAttribute_t; |
| 172 | - * | 172 | + |
| 173 | - * Currently only non-negative values are supported. The value must be large enough so that matrix memory locations are not | 173 | +/*! \ingroup types_module |
| 174 | - * overlapping (that is, greater or equal to ``ACLBLASLT_MATRIX_LAYOUT_ROWS`` in the case of ``ACLBLASLT_ORDER_COL``). | 174 | + * \brief Matmul preference attributes. |
| 175 | - * | 175 | + */ |
| 176 | - * ``int64_t`` | 176 | +typedef enum aclblasLtMatmulPreferenceAttribute { |
| 177 | - */ | 177 | + ACLBLASLT_MATMUL_PREF_SEARCH_MODE = 0, /**<Search mode. Data type: ``uint32_t``. */ |
| 178 | - ACLBLASLT_MATRIX_LAYOUT_LD = 6, | 178 | + ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES = 1, /**<Maximum allowed workspace memory. Default is 0 (no workspace memory allowed). Data type: ``uint64_t``. */ |
| 179 | -} aclblasLtMatrixLayoutAttribute_t; | 179 | + ACLBLASLT_MATMUL_PREF_MAX = 2 |
| 180 | - | 180 | +} aclblasLtMatmulPreferenceAttribute_t; |
| 181 | -/*! \ingroup types_module | 181 | + |
| 182 | - * \brief Matmul operation descriptor attributes. | 182 | +/*! \ingroup library_module |
| 183 | - */ | 183 | + * \brief Query aclBLASLt packed version number. |
| 184 | -typedef enum aclblasLtMatmulDescAttribute { | 184 | + */ |
| 185 | - ACLBLASLT_MATMUL_DESC_TRANSA = 0, /**<Specifies the type of transformation operation that should be performed on matrix A. Default value is ``ACLBLAS_OP_N`` (for example, non-transpose operation). See ``aclblasOperation_t``. Data type: ``int32_t``. */ | 185 | + aclblasStatus_t aclblasLtGetVersion(size_t* version); |
| 186 | - ACLBLASLT_MATMUL_DESC_TRANSB = 1, /**<Specifies the type of transformation operation that should be performed on matrix B. Default value is ``ACLBLAS_OP_N`` (for example, non-transpose operation). See ``aclblasOperation_t``. Data type: ``int32_t``. */ | 186 | + |
| 187 | - ACLBLASLT_MATMUL_DESC_EPILOGUE = 2, /**<Epilogue function. See ``aclblasLtEpilogue_t``. Default value is ``ACLBLASLT_EPILOGUE_DEFAULT``. Data type: ``uint32_t``. */ | 187 | + /*! \ingroup library_module |
| 188 | - ACLBLASLT_MATMUL_DESC_BIAS_POINTER = 3, /**<Bias or bias gradient vector pointer in the device memory. Data type: ``void*`` / ``const void*``. */ | 188 | + * \brief Query aclBLASLt property value. |
| 189 | - ACLBLASLT_MATMUL_DESC_BIAS_DATA_TYPE = 4, /**<Type of the bias vector in the device memory. Can be set the same as the D matrix type or Scale type. Bias case: see ``ACLBLASLT_EPILOGUE_BIAS``. Data type: ``int32_t`` based on ``aclDataType``. */ | 189 | + */ |
| 190 | - ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER = 5, /**<Device pointer to the scale factor value that converts data in matrix A to the compute data type range. The scaling factor must have the same type as the compute type. If not specified, or set to NULL, the scaling factor is assumed to be ``1``. If set for an unsupported matrix data, scale, and compute type combination, calling aclblasLtMatmul() will return ``ACLBLAS_INVALID_VALUE``. Default value: NULL. Data type: ``void*`` ``/const void*``. */ | 190 | + aclblasStatus_t aclblasLtGetProperty(aclblasLtPropertyType_t type, int* value); |
| 191 | - ACLBLASLT_MATMUL_DESC_B_SCALE_POINTER = 6, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix B. Default value: NULL. Data type: ``void*`` / ``const void*``. */ | 191 | + |
| 192 | - ACLBLASLT_MATMUL_DESC_C_SCALE_POINTER = 7, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix C. Default value: NULL. Data type: ``void*`` / ``const void*``. */ | 192 | +// Library management |
| 193 | - ACLBLASLT_MATMUL_DESC_D_SCALE_POINTER = 8, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix D. Default value: NULL. Data type: ``void*`` / ``const void*``. */ | 193 | +/*! \ingroup library_module |
| 194 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_SCALE_POINTER = 9, /**<Equivalent to ``ACLBLASLT_MATMUL_DESC_A_SCALE_POINTER`` for matrix AUX. Default value: NULL. Data type: ``void*`` / ``const void*``. */ | 194 | + * \brief Create a aclBLASLt handle. |
| 195 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER = 10, /**<Epilogue auxiliary buffer pointer in the device memory. Data type: ``void*`` / ``const void*``. */ | 195 | + * |
| 196 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD = 11, /**<The leading dimension of the epilogue auxiliary buffer pointer in the device memory. Data type: ``int64_t``. */ | 196 | + * \details |
| 197 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE = 12, /**<The batch stride of the epilogue auxiliary buffer pointer in the device memory. Data type: ``int64_t``. */ | 197 | + * This function initializes the aclBLASLt library and creates a handle to an |
| 198 | - ACLBLASLT_MATMUL_DESC_POINTER_MODE = 13, /**<Specifies that alpha and beta are passed by reference, whether they are scalars on the host or on the device, or device vectors. Default value is: ``ACLBLASLT_POINTER_MODE_HOST`` (on the host). Data type: ``int32_t`` based on ``aclblasLtPointerMode_t``. */ | 198 | + * opaque structure holding the aclBLASLt library context. It allocates light |
| 199 | - ACLBLASLT_MATMUL_DESC_AMAX_D_POINTER = 14, /**<Device pointer to the memory location that on completion will be set to the maximum of the absolute values in the output matrix. Data type: ``void*`` / ``const void*``. */ | 199 | + * hardware resources on the host and device and must be called prior to making |
| 200 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_AUX_DATA_TYPE = 22, /**<Type of the auxiliary vector in the device memory. Default value is: ``ACLBLASLT_DATATYPE_INVALID`` (using D matrix type). Data type: ``int32_t`` based on ``aclDataType``. */ | 200 | + * any other aclBLASLt library calls. The aclBLASLt library context is tied to |
| 201 | - ACLBLASLT_MATMUL_DESC_A_SCALE_MODE = 31, /**<Scaling mode that defines how the matrix scaling factor for matrix A is interpreted. See ``aclblasLtMatmulMatrixScale_t``. */ | 201 | + * the current CANN device. To use the library on multiple devices, one |
| 202 | - ACLBLASLT_MATMUL_DESC_B_SCALE_MODE = 32, /**<Scaling mode that defines how the matrix scaling factor for matrix B is interpreted. See ``aclblasLtMatmulMatrixScale_t``. */ | 202 | + * aclBLASLt handle should be created for each device. |
| 203 | - ACLBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_A_EXT = 100, /**<Compute input A types. Defines the data type used for the input A of a matrix multiply. */ | 203 | + * |
| 204 | - ACLBLASLT_MATMUL_DESC_COMPUTE_INPUT_TYPE_B_EXT, /**<Compute input B types. Defines the data type used for the input B of a matrix multiply. */ | 204 | + * @param[out] |
| 205 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG0_EXT, /**<First extra argument for the activation function. Data type: ``float``. */ | 205 | + * lightHandle Pointer to the allocated aclBLASLt handle for the created aclBLASLt |
| 206 | - ACLBLASLT_MATMUL_DESC_EPILOGUE_ACT_ARG1_EXT, /**<Second extra argument for the activation function. Data type: ``float``. */ | 206 | + * context. |
| 207 | - ACLBLASLT_MATMUL_DESC_MAX, | 207 | + * |
| 208 | -} aclblasLtMatmulDescAttribute_t; | 208 | + * \retval ACLBLAS_STATUS_SUCCESS The allocation completed successfully. |
| 209 | - | 209 | + * \retval ACLBLAS_STATUS_INVALID_VALUE \p lightHandle == NULL. |
| 210 | -/*! \ingroup types_module | 210 | + */ |
| 211 | - * \brief Matmul preference attributes. | 211 | +aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* lightHandle); |
| 212 | - */ | 212 | + |
| 213 | -typedef enum aclblasLtMatmulPreferenceAttribute { | 213 | +/*! \ingroup library_module |
| 214 | - ACLBLASLT_MATMUL_PREF_SEARCH_MODE = 0, /**<Search mode. Data type: ``uint32_t``. */ | 214 | + * \brief Destroy a aclBLASLt handle. |
| 215 | - ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES = 1, /**<Maximum allowed workspace memory. Default is 0 (no workspace memory allowed). Data type: ``uint64_t``. */ | 215 | + * |
| 216 | - ACLBLASLT_MATMUL_PREF_MAX = 2 | 216 | + * \details |
| 217 | -} aclblasLtMatmulPreferenceAttribute_t; | 217 | + * This function releases hardware resources used by the aclBLASLt library. |
| 218 | - | 218 | + * It is usually the last call with a particular handle to the |
| 219 | -/*! \ingroup library_module | 219 | + * aclBLASLt library. Because aclblasLtCreate() allocates some internal |
| 220 | - * \brief Query aclBLASLt packed version number. | 220 | + * resources and the release of those resources by calling aclblasLtDestroy() |
| 221 | - */ | 221 | + * implicitly calls device synchronization, it is recommended to minimize |
| 222 | - aclblasStatus_t aclblasLtGetVersion(size_t* version); | 222 | + * the number of aclblasLtCreate() / aclblasLtDestroy() occurrences. |
| 223 | - | 223 | + * |
| 224 | - /*! \ingroup library_module | 224 | + * @param[in] |
| 225 | - * \brief Query aclBLASLt property value. | 225 | + * lightHandle Pointer to the aclBLASLt handle to be destroyed. |
| 226 | - */ | 226 | + * |
| 227 | - aclblasStatus_t aclblasLtGetProperty(aclblasLtPropertyType_t type, int* value); | 227 | + * \retval ACLBLAS_STATUS_SUCCESS The aclBLASLt context was successfully |
| 228 | - | 228 | + * destroyed. |
| 229 | -// Library management | 229 | + * \retval ACLBLAS_STATUS_NOT_INITIALIZED The aclBLASLt library was |
| 230 | -/*! \ingroup library_module | 230 | + * not initialized. |
| 231 | - * \brief Create a aclBLASLt handle. | 231 | + * \retval ACLBLAS_STATUS_INVALID_VALUE \p lightHandle == NULL. |
| 232 | - * | 232 | + */ |
| 233 | - * \details | 233 | +aclblasStatus_t aclblasLtDestroy(const aclblasLtHandle_t lightHandle); |
| 234 | - * This function initializes the aclBLASLt library and creates a handle to an | 234 | + |
| 235 | - * opaque structure holding the aclBLASLt library context. It allocates light | 235 | +// Matrix layout descriptor |
| 236 | - * hardware resources on the host and device and must be called prior to making | 236 | +/*! \ingroup library_module |
| 237 | - * any other aclBLASLt library calls. The aclBLASLt library context is tied to | 237 | + * \brief Create a matrix layout descriptor. |
| 238 | - * the current CANN device. To use the library on multiple devices, one | 238 | + * |
| 239 | - * aclBLASLt handle should be created for each device. | 239 | + * \details |
| 240 | - * | 240 | + * This function creates a matrix layout descriptor by allocating the memory |
| 241 | - * @param[out] | 241 | + * needed to hold its opaque structure. |
| 242 | - * handle Pointer to the allocated aclBLASLt handle for the created aclBLASLt | 242 | + * |
| 243 | - * context. | 243 | + * @param[out] |
| 244 | - * | 244 | + * matLayout Pointer to the structure holding the matrix layout descriptor |
| 245 | - * \retval ACLBLAS_STATUS_SUCCESS The allocation completed successfully. | 245 | + * created by this function. See \ref aclblasLtMatrixLayout_t. |
| 246 | - * \retval ACLBLAS_STATUS_INVALID_VALUE \p handle == NULL. | 246 | + * @param[in] |
| 247 | - */ | 247 | + * type Enumerant that specifies the data precision for the matrix layout |
| 248 | -aclblasStatus_t aclblasLtCreate(aclblasLtHandle_t* handle); | 248 | + * descriptor created by this function. See aclDataType. |
| 249 | - | 249 | + * @param[in] |
| 250 | -/*! \ingroup library_module | 250 | + * rows Number of rows of the matrix. |
| 251 | - * \brief Destroy a aclBLASLt handle. | 251 | + * @param[in] |
| 252 | - * | 252 | + * cols Number of columns of the matrix. |
| 253 | - * \details | 253 | + * @param[in] |
| 254 | - * This function releases hardware resources used by the aclBLASLt library. | 254 | + * ld The leading dimension of the matrix. In column major layout, this is the |
| 255 | - * It is usually the last call with a particular handle to the | 255 | + * number of elements to jump to reach the next column. Therefore, ld >= m (number of |
| 256 | - * aclBLASLt library. Because aclblasLtCreate() allocates some internal | 256 | + * rows). |
| 257 | - * resources and the release of those resources by calling aclblasLtDestroy() | 257 | + * |
| 258 | - * implicitly calls device synchronization, it is recommended to minimize | 258 | + * \retval ACLBLAS_STATUS_SUCCESS If the descriptor was created successfully. |
| 259 | - * the number of aclblasLtCreate() / aclblasLtDestroy() occurrences. | 259 | + * \retval ACLBLAS_STATUS_ALLOC_FAILED If the memory could not be allocated. |
| 260 | - * | 260 | + */ |
| 261 | - * @param[in] | 261 | +aclblasStatus_t aclblasLtMatrixLayoutCreate(aclblasLtMatrixLayout_t* matLayout, |
| 262 | - * handle Pointer to the aclBLASLt handle to be destroyed. | 262 | + aclDataType type, |
| 263 | - * | 263 | + uint64_t rows, |
| 264 | - * \retval ACLBLAS_STATUS_SUCCESS The aclBLASLt context was successfully | 264 | + uint64_t cols, |
| 265 | - * destroyed. \retval ACLBLAS_STATUS_NOT_INITIALIZED The aclBLASLt library was | 265 | + int64_t ld); |
| 266 | - * not initialized. \retval ACLBLAS_STATUS_INVALID_VALUE \p handle == NULL. | 266 | + |
| 267 | - */ | 267 | +/*! \ingroup library_module |
| 268 | -aclblasStatus_t aclblasLtDestroy(const aclblasLtHandle_t handle); | 268 | + * \brief Destroy a matrix layout descriptor. |
| 269 | - | 269 | + * |
| 270 | -// Matrix layout descriptor | 270 | + * \details |
| 271 | -/*! \ingroup library_module | 271 | + * This function destroys a previously created matrix layout descriptor object. |
| 272 | - * \brief Create a matrix layout descriptor. | 272 | + * |
| 273 | - * | 273 | + * @param[in] |
| 274 | - * \details | 274 | + * matLayout Pointer to the structure holding the matrix layout descriptor to |
| 275 | - * This function creates a matrix layout descriptor by allocating the memory | 275 | + * be destroyed by this function. See \ref aclblasLtMatrixLayout_t. |
| 276 | - * needed to hold its opaque structure. | 276 | + * |
| 277 | - * | 277 | + * \retval ACLBLAS_STATUS_SUCCESS If the operation was successful. |
| 278 | - * @param[out] | 278 | + */ |
| 279 | - * matLayout Pointer to the structure holding the matrix layout descriptor | 279 | +aclblasStatus_t aclblasLtMatrixLayoutDestroy(const aclblasLtMatrixLayout_t matLayout); |
| 280 | - * created by this function. See \ref aclblasLtMatrixLayout_t. | 280 | + |
| 281 | - * @param[in] | 281 | +/*! \ingroup library_module |
| 282 | - * type Enumerant that specifies the data precision for the matrix layout | 282 | + * \brief Set an attribute for a matrix descriptor. |
| 283 | - * descriptor created by this function. See aclDataType. | 283 | + * |
| 284 | - * @param[in] | 284 | + * \details |
| 285 | - * rows Number of rows of the matrix. | 285 | + * This function sets the value of the specified attribute belonging to a |
| 286 | - * @param[in] | 286 | + * previously created matrix descriptor. |
| 287 | - * cols Number of columns of the matrix. | 287 | + * |
| 288 | - * @param[in] | 288 | + * @param[in] |
| 289 | - * ld The leading dimension of the matrix. In column major layout, this is the | 289 | + * matLayout Pointer to the previously created structure holding the matrix |
| 290 | - * number of elements to jump to reach the next column. Therefore, ld >= m (number of | 290 | + * descriptor queried by this function. See \ref aclblasLtMatrixLayout_t. |
| 291 | - * rows). | 291 | + * @param[in] |
| 292 | - * | 292 | + * attr The attribute that will be set by this function. See \ref |
| 293 | - * \retval ACLBLAS_STATUS_SUCCESS If the descriptor was created successfully. | 293 | + * aclblasLtMatrixLayoutAttribute_t. |
| 294 | - * \retval ACLBLAS_STATUS_ALLOC_FAILED If the memory could not be allocated. | 294 | + * @param[in] |
| 295 | - */ | 295 | + * buf The value to which the specified attribute should be set. |
| 296 | -aclblasStatus_t aclblasLtMatrixLayoutCreate(aclblasLtMatrixLayout_t* matLayout, | 296 | + * @param[in] |
| 297 | - aclDataType type, | 297 | + * sizeInBytes Size of the buf buffer (in bytes) for verification. |
| 298 | - uint64_t rows, | 298 | + * |
| 299 | - uint64_t cols, | 299 | + * \retval ACLBLAS_STATUS_SUCCESS If the attribute was set successfully. |
| 300 | - int64_t ld); | 300 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If \p buf is NULL or \p sizeInBytes |
| 301 | - | 301 | + * doesn't match the size of the internal storage for the selected attribute. |
| 302 | -/*! \ingroup library_module | 302 | + */ |
| 303 | - * \brief Destroy a matrix layout descriptor. | 303 | +aclblasStatus_t aclblasLtMatrixLayoutSetAttribute(aclblasLtMatrixLayout_t matLayout, |
| 304 | - * | 304 | + aclblasLtMatrixLayoutAttribute_t attr, |
| 305 | - * \details | 305 | + const void* buf, |
| 306 | - * This function destroys a previously created matrix layout descriptor object. | 306 | + size_t sizeInBytes); |
| 307 | - * | 307 | + |
| 308 | - * @param[in] | 308 | +/*! \ingroup library_module |
| 309 | - * matLayout Pointer to the structure holding the matrix layout descriptor to | 309 | + * \brief Get an attribute for a matrix descriptor. |
| 310 | - * be destroyed by this function. See \ref aclblasLtMatrixLayout_t. | 310 | + */ |
| 311 | - * | 311 | +aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t matLayout, |
| 312 | - * \retval ACLBLAS_STATUS_SUCCESS If the operation was successful. | 312 | + aclblasLtMatrixLayoutAttribute_t attr, |
| 313 | - */ | 313 | + void* buf, |
| 314 | -aclblasStatus_t aclblasLtMatrixLayoutDestroy(const aclblasLtMatrixLayout_t matLayout); | 314 | + size_t sizeInBytes, |
| 315 | - | 315 | + size_t* sizeWritten); |
| 316 | -/*! \ingroup library_module | 316 | + |
| 317 | - * \brief Set an attribute for a matrix descriptor. | 317 | +// Matmul operation descriptor |
| 318 | - * | 318 | +/*! \ingroup library_module |
| 319 | - * \details | 319 | + * \brief Create a matrix multiply descriptor. |
| 320 | - * This function sets the value of the specified attribute belonging to a | 320 | + * |
| 321 | - * previously created matrix descriptor. | 321 | + * \details |
| 322 | - * | 322 | + * This function creates a matrix multiply descriptor by allocating the memory |
| 323 | - * @param[in] | 323 | + * needed to hold its opaque structure. |
| 324 | - * matLayout Pointer to the previously created structure holding the matrix | 324 | + * |
| 325 | - * descriptor queried by this function. See \ref aclblasLtMatrixLayout_t. | 325 | + * @param[out] |
| 326 | - * @param[in] | 326 | + * matmulDesc Pointer to the structure holding the matrix multiply descriptor |
| 327 | - * attr The attribute that will be set by this function. See \ref | 327 | + * created by this function. See \ref aclblasLtMatmulDesc_t. |
| 328 | - * aclblasLtMatrixLayoutAttribute_t. | 328 | + * @param[in] |
| 329 | - * @param[in] | 329 | + * computeType Enumerant that specifies the data precision for the matrix |
| 330 | - * buf The value to which the specified attribute should be set. | 330 | + * multiply descriptor this function creates. See aclblasComputeType_t. |
| 331 | - * @param[in] | 331 | + * @param[in] |
| 332 | - * sizeInBytes Size of the buf buffer (in bytes) for verification. | 332 | + * scaleType Enumerant that specifies the data precision for the matrix |
| 333 | - * | 333 | + * transform descriptor this function creates. See aclDataType. |
| 334 | - * \retval ACLBLAS_STATUS_SUCCESS If the attribute was set successfully. | 334 | + * |
| 335 | - * \retval ACLBLAS_STATUS_INVALID_VALUE If \p buf is NULL or \p sizeInBytes | 335 | + * \retval ACLBLAS_STATUS_SUCCESS If the descriptor was created successfully. |
| 336 | - * doesn't match the size of the internal storage for the selected attribute. | 336 | + * \retval ACLBLAS_STATUS_ALLOC_FAILED If the memory could not be allocated. |
| 337 | - */ | 337 | + */ |
| 338 | -aclblasStatus_t aclblasLtMatrixLayoutSetAttribute(aclblasLtMatrixLayout_t matLayout, | 338 | +aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* matmulDesc, |
| 339 | - aclblasLtMatrixLayoutAttribute_t attr, | 339 | + aclblasComputeType_t computeType, |
| 340 | - const void* buf, | 340 | + aclDataType scaleType); |
| 341 | - size_t sizeInBytes); | 341 | + |
| 342 | - | 342 | +/*! \ingroup library_module |
| 343 | -/*! \ingroup library_module | 343 | + * \brief Destroy a matrix multiply descriptor. |
| 344 | - * \brief Get an attribute for a matrix descriptor. | 344 | + * |
| 345 | - */ | 345 | + * \details |
| 346 | -aclblasStatus_t aclblasLtMatrixLayoutGetAttribute(const aclblasLtMatrixLayout_t matLayout, | 346 | + * This function destroys a previously created matrix multiply descriptor |
| 347 | - aclblasLtMatrixLayoutAttribute_t attr, | 347 | + * object. |
| 348 | - void* buf, | 348 | + * |
| 349 | - size_t sizeInBytes, | 349 | + * @param[in] |
| 350 | - size_t* sizeWritten); | 350 | + * matmulDesc Pointer to the structure holding the matrix multiply descriptor |
| 351 | - | 351 | + * to be destroyed by this function. See \ref aclblasLtMatmulDesc_t. |
| 352 | -// Matmul operation descriptor | 352 | + * |
| 353 | -/*! \ingroup library_module | 353 | + * \retval ACLBLAS_STATUS_SUCCESS If operation was successful. |
| 354 | - * \brief Create a matrix multiply descriptor. | 354 | + */ |
| 355 | - * | 355 | +aclblasStatus_t aclblasLtMatmulDescDestroy(const aclblasLtMatmulDesc_t matmulDesc); |
| 356 | - * \details | 356 | + |
| 357 | - * This function creates a matrix multiply descriptor by allocating the memory | 357 | +/*! \ingroup library_module |
| 358 | - * needed to hold its opaque structure. | 358 | + * \brief Set attribute to a matrix multiply descriptor. |
| 359 | - * | 359 | + * |
| 360 | - * @param[out] | 360 | + * \details |
| 361 | - * matmulDesc Pointer to the structure holding the matrix multiply descriptor | 361 | + * This function sets the value of the specified attribute belonging to a |
| 362 | - * created by this function. See \ref aclblasLtMatmulDesc_t. | 362 | + * previously created matrix multiply descriptor. |
| 363 | - * @param[in] | 363 | + * |
| 364 | - * computeType Enumerant that specifies the data precision for the matrix | 364 | + * @param[in] |
| 365 | - * multiply descriptor this function creates. See aclblasComputeType_t. | 365 | + * matmulDesc Pointer to the previously created structure holding the matrix |
| 366 | - * @param[in] | 366 | + * multiply descriptor queried by this function. See \ref aclblasLtMatmulDesc_t. |
| 367 | - * scaleType Enumerant that specifies the data precision for the matrix | 367 | + * @param[in] |
| 368 | - * transform descriptor this function creates. See aclDataType. | 368 | + * attr The attribute that will be set by this function. See \ref |
| 369 | - * | 369 | + * aclblasLtMatmulDescAttribute_t. |
| 370 | - * \retval ACLBLAS_STATUS_SUCCESS If the descriptor was created successfully. | 370 | + * @param[in] |
| 371 | - * \retval ACLBLAS_STATUS_ALLOC_FAILED If the memory could not be allocated. | 371 | + * buf The value to which the specified attribute should be set. |
| 372 | - */ | 372 | + * @param[in] |
| 373 | -aclblasStatus_t aclblasLtMatmulDescCreate(aclblasLtMatmulDesc_t* matmulDesc, | 373 | + * sizeInBytes Size of the buf buffer (in bytes) for verification. |
| 374 | - aclblasComputeType_t computeType, | 374 | + * |
| 375 | - aclDataType scaleType); | 375 | + * \retval ACLBLAS_STATUS_SUCCESS If the attribute was set successfully. |
| 376 | - | 376 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If \p buf is NULL or \p sizeInBytes |
| 377 | -/*! \ingroup library_module | 377 | + * doesn't match the size of the internal storage for the selected attribute. |
| 378 | - * \brief Destroy a matrix multiply descriptor. | 378 | + */ |
| 379 | - * | 379 | +aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t matmulDesc, |
| 380 | - * \details | 380 | + aclblasLtMatmulDescAttribute_t attr, |
| 381 | - * This function destroys a previously created matrix multiply descriptor | 381 | + const void* buf, |
| 382 | - * object. | 382 | + size_t sizeInBytes); |
| 383 | - * | 383 | + |
| 384 | - * @param[in] | 384 | +/*! \ingroup library_module |
| 385 | - * matmulDesc Pointer to the structure holding the matrix multiply descriptor | 385 | + * \brief Get an attribute from a matrix multiply descriptor. |
| 386 | - * to be destroyed by this function. See \ref aclblasLtMatmulDesc_t. | 386 | + * |
| 387 | - * | 387 | + * \details |
| 388 | - * \retval ACLBLAS_STATUS_SUCCESS If operation was successful. | 388 | + * This function retrieves the value of the specified attribute from a |
| 389 | - */ | 389 | + * previously created matrix multiply descriptor. |
| 390 | -aclblasStatus_t aclblasLtMatmulDescDestroy(const aclblasLtMatmulDesc_t matmulDesc); | 390 | + * |
| 391 | - | 391 | + * @param[in] |
| 392 | -/*! \ingroup library_module | 392 | + * desc Pointer to the previously created matrix multiply descriptor. See |
| 393 | - * \brief Set attribute to a matrix multiply descriptor. | 393 | + * \ref aclblasLtMatmulDesc_t. |
| 394 | - * | 394 | + * @param[in] |
| 395 | - * \details | 395 | + * attr The attribute to query. See \ref aclblasLtMatmulDescAttribute_t. |
| 396 | - * This function sets the value of the specified attribute belonging to a | 396 | + * @param[out] |
| 397 | - * previously created matrix multiply descriptor. | 397 | + * buf Output buffer used to store the queried attribute value. |
| 398 | - * | 398 | + * @param[in] |
| 399 | - * @param[in] | 399 | + * sizeInBytes Size of \p buf in bytes. |
| 400 | - * matmulDesc Pointer to the previously created structure holding the matrix | 400 | + * @param[out] |
| 401 | - * multiply descriptor queried by this function. See \ref aclblasLtMatmulDesc_t. | 401 | + * sizeWritten Number of bytes actually written to \p buf. Can be NULL. |
| 402 | - * @param[in] | 402 | + * |
| 403 | - * attr The attribute that will be set by this function. See \ref | 403 | + * \retval ACLBLAS_STATUS_SUCCESS If the attribute was retrieved successfully. |
| 404 | - * aclblasLtMatmulDescAttribute_t. | 404 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If \p desc or \p buf is NULL, or |
| 405 | - * @param[in] | 405 | + * \p sizeInBytes is smaller than the required size for the selected |
| 406 | - * buf The value to which the specified attribute should be set. | 406 | + * attribute. |
| 407 | - * @param[in] | 407 | + */ |
| 408 | - * sizeInBytes Size of the buf buffer (in bytes) for verification. | 408 | +aclblasStatus_t aclblasLtMatmulDescGetAttribute(aclblasLtMatmulDesc_t desc, |
| 409 | - * | 409 | + aclblasLtMatmulDescAttribute_t attr, |
| 410 | - * \retval ACLBLAS_STATUS_SUCCESS If the attribute was set successfully. | 410 | + void* buf, |
| 411 | - * \retval ACLBLAS_STATUS_INVALID_VALUE If \p buf is NULL or \p sizeInBytes | 411 | + size_t sizeInBytes, |
| 412 | - * doesn't match the size of the internal storage for the selected attribute. | 412 | + size_t* sizeWritten); |
| 413 | - */ | 413 | + |
| 414 | -aclblasStatus_t aclblasLtMatmulDescSetAttribute(aclblasLtMatmulDesc_t matmulDesc, | 414 | +// Preference |
| 415 | - aclblasLtMatmulDescAttribute_t attr, | 415 | +/*! \ingroup library_module |
| 416 | - const void* buf, | 416 | + * \brief Create a preference descriptor. |
| 417 | - size_t sizeInBytes); | 417 | + * |
| 418 | - | 418 | + * \details |
| 419 | -// Preference | 419 | + * This function creates a matrix multiply heuristic search preferences |
| 420 | -/*! \ingroup library_module | 420 | + * descriptor by allocating the memory needed to hold its opaque structure. |
| 421 | - * \brief Create a preference descriptor. | 421 | + * |
| 422 | - * | 422 | + * @param[out] |
| 423 | - * \details | 423 | + * pref Pointer to the structure holding the matrix multiply preferences |
| 424 | - * This function creates a matrix multiply heuristic search preferences | 424 | + * descriptor created by this function. see \ref aclblasLtMatmulPreference_t. |
| 425 | - * descriptor by allocating the memory needed to hold its opaque structure. | 425 | + * |
| 426 | - * | 426 | + * \retval ACLBLAS_STATUS_SUCCESS If the descriptor was created |
| 427 | - * @param[out] | 427 | + * successfully. |
| 428 | - * pref Pointer to the structure holding the matrix multiply preferences | 428 | + * \retval ACLBLAS_STATUS_ALLOC_FAILED If memory could not be |
| 429 | - * descriptor created by this function. see \ref aclblasLtMatmulPreference_t. | 429 | + * allocated. |
| 430 | - * | 430 | + */ |
| 431 | - * \retval ACLBLAS_STATUS_SUCCESS If the descriptor was created | 431 | +aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pref); |
| 432 | - * successfully. \retval ACLBLAS_STATUS_ALLOC_FAILED If memory could not be | 432 | + |
| 433 | - * allocated. | 433 | +/*! \ingroup library_module |
| 434 | - */ | 434 | + * \brief Destroy a preference descriptor. |
| 435 | -aclblasStatus_t aclblasLtMatmulPreferenceCreate(aclblasLtMatmulPreference_t* pref); | 435 | + * |
| 436 | - | 436 | + * \details |
| 437 | -/*! \ingroup library_module | 437 | + * This function destroys a previously created matrix multiply preferences |
| 438 | - * \brief Destroy a preference descriptor. | 438 | + * descriptor object. |
| 439 | - * | 439 | + * |
| 440 | - * \details | 440 | + * @param[in] |
| 441 | - * This function destroys a previously created matrix multiply preferences | 441 | + * pref Pointer to the structure holding the matrix multiply preferences |
| 442 | - * descriptor object. | 442 | + * descriptor to be destroyed by this function. See \ref |
| 443 | - * | 443 | + * aclblasLtMatmulPreference_t. |
| 444 | - * @param[in] | 444 | + * |
| 445 | - * pref Pointer to the structure holding the matrix multiply preferences | 445 | + * \retval ACLBLAS_STATUS_SUCCESS If operation was successful. |
| 446 | - * descriptor to be destroyed by this function. See \ref | 446 | + */ |
| 447 | - * aclblasLtMatmulPreference_t. | 447 | +aclblasStatus_t aclblasLtMatmulPreferenceDestroy(const aclblasLtMatmulPreference_t pref); |
| 448 | - * | 448 | + |
| 449 | - * \retval ACLBLAS_STATUS_SUCCESS If operation was successful. | 449 | +/*! \ingroup library_module |
| 450 | - */ | 450 | + * \brief Set attribute in a preference descriptor. |
| 451 | -aclblasStatus_t aclblasLtMatmulPreferenceDestroy(const aclblasLtMatmulPreference_t pref); | 451 | + * |
| 452 | - | 452 | + * \details |
| 453 | -/*! \ingroup library_module | 453 | + * This function sets the value of the specified attribute belonging to a |
| 454 | - * \brief Set attribute in a preference descriptor. | 454 | + * previously created matrix multiply preferences descriptor. |
| 455 | - * | 455 | + * |
| 456 | - * \details | 456 | + * @param[in] |
| 457 | - * This function sets the value of the specified attribute belonging to a | 457 | + * pref Pointer to the previously created structure holding the matrix |
| 458 | - * previously created matrix multiply preferences descriptor. | 458 | + * multiply preferences descriptor queried by this function. See \ref |
| 459 | - * | 459 | + * aclblasLtMatmulPreference_t. |
| 460 | - * @param[in] | 460 | + * @param[in] |
| 461 | - * pref Pointer to the previously created structure holding the matrix | 461 | + * attr The attribute that will be set by this function. See \ref |
| 462 | - * multiply preferences descriptor queried by this function. See \ref | 462 | + * aclblasLtMatmulPreferenceAttribute_t. |
| 463 | - * aclblasLtMatmulPreference_t. | 463 | + * @param[in] |
| 464 | - * @param[in] | 464 | + * buf The value to which the specified attribute should be set. |
| 465 | - * attr The attribute that will be set by this function. See \ref | 465 | + * @param[in] |
| 466 | - * aclblasLtMatmulPreferenceAttribute_t. | 466 | + * sizeInBytes Size of the \p buf buffer (in bytes) for verification. |
| 467 | - * @param[in] | 467 | + * |
| 468 | - * buf The value to which the specified attribute should be set. | 468 | + * \retval ACLBLAS_STATUS_SUCCESS If the attribute was set successfully. |
| 469 | - * @param[in] | 469 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If \p buf is NULL or \p sizeInBytes |
| 470 | - * sizeInBytes Size of the \p buf buffer (in bytes) for verification. | 470 | + * doesn't match the size of the internal storage for the selected attribute. |
| 471 | - * | 471 | + */ |
| 472 | - * \retval ACLBLAS_STATUS_SUCCESS If the attribute was set successfully. | 472 | +aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_t pref, |
| 473 | - * \retval ACLBLAS_STATUS_INVALID_VALUE If \p buf is NULL or \p sizeInBytes | 473 | + aclblasLtMatmulPreferenceAttribute_t attr, |
| 474 | - * doesn't match the size of the internal storage for the selected attribute. | 474 | + const void* buf, |
| 475 | - */ | 475 | + size_t sizeInBytes); |
| 476 | -aclblasStatus_t aclblasLtMatmulPreferenceSetAttribute(aclblasLtMatmulPreference_t pref, | 476 | + |
| 477 | - aclblasLtMatmulPreferenceAttribute_t attr, | 477 | + |
| 478 | - const void* buf, | 478 | +/*! \ingroup library_module |
| 479 | - size_t sizeInBytes); | 479 | + * \brief Get an attribute from a preference descriptor. |
| 480 | - | 480 | + * |
| 481 | -// Heuristic + matmul | 481 | + * \details |
| 482 | -/*! \ingroup library_module | 482 | + * This function retrieves the value of the specified attribute from a |
| 483 | - * \brief Retrieve the possible algorithms. | 483 | + * previously created matrix multiply preference descriptor. |
| 484 | - * | 484 | + * |
| 485 | - * \details | 485 | + * @param[in] |
| 486 | - * This function retrieves the possible algorithms for the matrix multiply | 486 | + * pref Pointer to the previously created preference descriptor. See |
| 487 | - * operation aclblasLtMatmul() with the given input matrices A, B, and | 487 | + * \ref aclblasLtMatmulPreference_t. |
| 488 | - * C, and the output matrix D. The output is placed in ``heuristicResultsArray`` | 488 | + * @param[in] |
| 489 | - * in order of increasing estimated compute time. | 489 | + * attr The attribute to query. See |
| 490 | - * | 490 | + * \ref aclblasLtMatmulPreferenceAttribute_t. |
| 491 | - * @param[in] | 491 | + * @param[out] |
| 492 | - * handle Pointer to the allocated aclBLASLt handle for the | 492 | + * buf Output buffer used to store the queried attribute value. |
| 493 | - * aclBLASLt context. See \ref aclblasLtHandle_t. | 493 | + * @param[in] |
| 494 | - * @param[in] | 494 | + * sizeInBytes Size of \p buf in bytes. |
| 495 | - * matmulDesc Handle to a previously created matrix multiplication | 495 | + * @param[out] |
| 496 | - * descriptor of type \ref aclblasLtMatmulDesc_t. | 496 | + * sizeWritten Number of bytes actually written to \p buf. Can be NULL. |
| 497 | - * @param[in] | 497 | + * |
| 498 | - * Adesc,Bdesc,Cdesc,Ddesc Handles to the previously created matrix layout | 498 | + * \retval ACLBLAS_STATUS_SUCCESS If the attribute was retrieved successfully. |
| 499 | - * descriptors of the type \ref aclblasLtMatrixLayout_t. | 499 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If \p pref or \p buf is NULL, or |
| 500 | - * @param[in] | 500 | + * \p sizeInBytes is smaller than the required size for the selected |
| 501 | - * pref Pointer to the structure holding the heuristic | 501 | + * attribute. |
| 502 | - * search preferences descriptor. See \ref aclblasLtMatmulPreference_t. | 502 | + */ |
| 503 | - * @param[in] | 503 | +aclblasStatus_t aclblasLtMatmulPreferenceGetAttribute(aclblasLtMatmulPreference_t pref, |
| 504 | - * requestedAlgoCount Size of the \p heuristicResultsArray (in elements). | 504 | + aclblasLtMatmulPreferenceAttribute_t attr, |
| 505 | - * This is the requested maximum number of algorithms to return. | 505 | + void* buf, |
| 506 | - * @param[out] | 506 | + size_t sizeInBytes, |
| 507 | - * heuristicResultsArray[] Array containing the algorithm heuristics and | 507 | + size_t* sizeWritten); |
| 508 | - * associated runtime characteristics returned by this function, in order | 508 | + |
| 509 | - * of increasing estimated compute time. | 509 | +// Heuristic |
| 510 | - * @param[out] | 510 | +/*! \ingroup library_module |
| 511 | - * returnAlgoCount Number of algorithms returned by this function. This | 511 | + * \brief Retrieve the possible algorithms. |
| 512 | - * is the number of \p heuristicResultsArray elements written. | 512 | + * |
| 513 | - * | 513 | + * \details |
| 514 | - * \retval ACLBLAS_STATUS_SUCCESS If query was successful. Inspect | 514 | + * This function retrieves the possible algorithms for the matrix multiply |
| 515 | - * ``heuristicResultsArray[0 to (returnAlgoCount -1)].state`` for the status of the | 515 | + * operation aclblasLtMatmul() with the given input matrices A, B, and |
| 516 | - * results. \retval ACLBLAS_STATUS_NOT_SUPPORTED If no heuristic function is | 516 | + * C, and the output matrix D. The output is placed in ``heuristicResultsArray`` |
| 517 | - * available for current configuration. \retval ACLBLAS_STATUS_INVALID_VALUE If | 517 | + * in order of increasing estimated compute time. |
| 518 | - * \p requestedAlgoCount is less than or equal to zero. | 518 | + * |
| 519 | - */ | 519 | + * @param[in] |
| 520 | -aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t handle, | 520 | + * lightHandle Pointer to the allocated aclBLASLt handle for the |
| 521 | - aclblasLtMatmulDesc_t matmulDesc, | 521 | + * aclBLASLt context. See \ref aclblasLtHandle_t. |
| 522 | - aclblasLtMatrixLayout_t Adesc, | 522 | + * @param[in] |
| 523 | - aclblasLtMatrixLayout_t Bdesc, | 523 | + * matmulDesc Handle to a previously created matrix multiplication |
| 524 | - aclblasLtMatrixLayout_t Cdesc, | 524 | + * descriptor of type \ref aclblasLtMatmulDesc_t. |
| 525 | - aclblasLtMatrixLayout_t Ddesc, | 525 | + * @param[in] |
| 526 | - aclblasLtMatmulPreference_t pref, | 526 | + * Adesc,Bdesc,Cdesc,Ddesc Handles to the previously created matrix layout |
| 527 | - int requestedAlgoCount, | 527 | + * descriptors of the type \ref aclblasLtMatrixLayout_t. |
| 528 | - aclblasLtMatmulHeuristicResult_t heuristicResultsArray[], | 528 | + * @param[in] |
| 529 | - int* returnAlgoCount); | 529 | + * pref Pointer to the structure holding the heuristic |
| 530 | - | 530 | + * search preferences descriptor. See \ref aclblasLtMatmulPreference_t. |
| 531 | -/*! \ingroup library_module | 531 | + * @param[in] |
| 532 | - * \brief Matrix-matrix multiplication. | 532 | + * requestedAlgoCount Size of the \p heuristicResultsArray (in elements). |
| 533 | - * | 533 | + * This is the requested maximum number of algorithms to return. |
| 534 | - * \details | 534 | + * @param[out] |
| 535 | - * This function computes the matrix multiplication of matrices A and B to | 535 | + * heuristicResultsArray[] Array containing the algorithm heuristics and |
| 536 | - * produce the output matrix D, according to the following operation: \p D = \p | 536 | + * associated runtime characteristics returned by this function, in order |
| 537 | - * alpha*( \p A *\p B) + \p beta*( \p C ), where \p A, \p B, and \p C are input | 537 | + * of increasing estimated compute time. |
| 538 | - * matrices, and \p alpha and \p beta are input scalars. Note: This function | 538 | + * @param[out] |
| 539 | - * supports both in-place matrix multiplication (``C == D`` and ``Cdesc == Ddesc``) and | 539 | + * returnAlgoCount Number of algorithms returned by this function. This |
| 540 | - * out-of-place matrix multiplication (``C != D``). | 540 | + * is the number of \p heuristicResultsArray elements written. |
| 541 | - * | 541 | + * |
| 542 | - * @param[in] | 542 | + * \retval ACLBLAS_STATUS_SUCCESS If query was successful. Inspect |
| 543 | - * handle Pointer to the allocated aclBLASLt handle for the | 543 | + * ``heuristicResultsArray[0 to (returnAlgoCount -1)].state`` for the status of the |
| 544 | - * aclBLASLt context. See \ref aclblasLtHandle_t. | 544 | + * results. |
| 545 | - * @param[in] | 545 | + * \retval ACLBLAS_STATUS_NOT_SUPPORTED If no heuristic function is |
| 546 | - * matmulDesc Handle to a previously created matrix multiplication | 546 | + * available for current configuration. |
| 547 | - * descriptor of type \ref aclblasLtMatmulDesc_t. | 547 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If |
| 548 | - * @param[in] | 548 | + * \p requestedAlgoCount is less than or equal to zero. |
| 549 | - * alpha,beta Pointers to the scalars used in the multiplication. | 549 | + */ |
| 550 | - * @param[in] | 550 | +aclblasStatus_t aclblasLtMatmulAlgoGetHeuristic(aclblasLtHandle_t lightHandle, |
| 551 | - * Adesc,Bdesc,Cdesc,Ddesc Handles to the previously created matrix layout | 551 | + aclblasLtMatmulDesc_t matmulDesc, |
| 552 | - * descriptors of the type \ref aclblasLtMatrixLayout_t. | 552 | + aclblasLtMatrixLayout_t Adesc, |
| 553 | - * @param[in] | 553 | + aclblasLtMatrixLayout_t Bdesc, |
| 554 | - * A,B,C Pointers to the memory associated with the | 554 | + aclblasLtMatrixLayout_t Cdesc, |
| 555 | - * corresponding descriptors \p Adesc, \p Bdesc, and \p Cdesc. | 555 | + aclblasLtMatrixLayout_t Ddesc, |
| 556 | - * @param[out] | 556 | + aclblasLtMatmulPreference_t pref, |
| 557 | - * D Pointer to the memory associated with the | 557 | + int requestedAlgoCount, |
| 558 | - * descriptor \p Ddesc. | 558 | + aclblasLtMatmulHeuristicResult_t heuristicResultsArray[], |
| 559 | - * @param[in] | 559 | + int* returnAlgoCount); |
| 560 | - * algo Handle for matrix multiplication algorithm to be | 560 | + |
| 561 | - * used. See \ref aclblasLtMatmulAlgo_t. When NULL, an implicit heuristics query | 561 | +// Matmul |
| 562 | - * with default search preferences will be performed to determine the actual | 562 | +/*! \ingroup library_module |
| 563 | - * algorithm to use. | 563 | + * \brief Matrix-matrix multiplication. |
| 564 | - * @param[in] | 564 | + * |
| 565 | - * workspace Pointer to the workspace buffer allocated in the GPU | 565 | + * \details |
| 566 | - * memory. Pointer must be 16B aligned. | 566 | + * This function computes the matrix multiplication of matrices A and B to |
| 567 | - * @param[in] | 567 | + * produce the output matrix D, according to the following operation: \p D = \p |
| 568 | - * workspaceSizeInBytes Size of the workspace. | 568 | + * alpha*( \p A *\p B) + \p beta*( \p C ), where \p A, \p B, and \p C are input |
| 569 | - * @param[in] | 569 | + * matrices, and \p alpha and \p beta are input scalars. Note: This function |
| 570 | - * stream The stream where all device work is submitted. | 570 | + * supports both in-place matrix multiplication (``C == D`` and ``Cdesc == Ddesc``) and |
| 571 | - * | 571 | + * out-of-place matrix multiplication (``C != D``). |
| 572 | - * \retval ACLBLAS_STATUS_SUCCESS If the operation completed | 572 | + * |
| 573 | - * successfully. \retval ACLBLAS_STATUS_EXECUTION_FAILED If device reported an | 573 | + * @param[in] |
| 574 | - * execution error. \retval ACLBLAS_STATUS_ARCH_MISMATCH If | 574 | + * lightHandle Pointer to the allocated aclBLASLt handle for the |
| 575 | - * the configured operation cannot be run using the selected device. \retval | 575 | + * aclBLASLt context. See \ref aclblasLtHandle_t. |
| 576 | - * ACLBLAS_STATUS_NOT_SUPPORTED If the current implementation on the | 576 | + * @param[in] |
| 577 | - * selected device doesn't support the configured operation. \retval | 577 | + * computeDesc Handle to a previously created matrix multiplication |
| 578 | - * ACLBLAS_STATUS_INVALID_VALUE If the parameters are unexpectedly NULL, in | 578 | + * descriptor of type \ref aclblasLtMatmulDesc_t. |
| 579 | - * conflict, or in an impossible configuration. \retval ACLBLAS_STATUS_NOT_INITIALIZED | 579 | + * @param[in] |
| 580 | - * If the aclBLASLt handle has not been initialized. | 580 | + * alpha,beta Pointers to the scalars used in the multiplication. |
| 581 | - */ | 581 | + * @param[in] |
| 582 | -aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t handle, | 582 | + * Adesc,Bdesc,Cdesc,Ddesc Handles to the previously created matrix layout |
| 583 | - aclblasLtMatmulDesc_t matmulDesc, | 583 | + * descriptors of the type \ref aclblasLtMatrixLayout_t. |
| 584 | - const void* alpha, | 584 | + * @param[in] |
| 585 | - const void* A, | 585 | + * A,B,C Pointers to the memory associated with the |
| 586 | - aclblasLtMatrixLayout_t Adesc, | 586 | + * corresponding descriptors \p Adesc, \p Bdesc, and \p Cdesc. |
| 587 | - const void* B, | 587 | + * @param[out] |
| 588 | - aclblasLtMatrixLayout_t Bdesc, | 588 | + * D Pointer to the memory associated with the |
| 589 | - const void* beta, | 589 | + * descriptor \p Ddesc. |
| 590 | - const void* C, | 590 | + * @param[in] |
| 591 | - aclblasLtMatrixLayout_t Cdesc, | 591 | + * algo Handle for matrix multiplication algorithm to be |
| 592 | - void* D, | 592 | + * used. See \ref aclblasLtMatmulAlgo_t. When NULL, an implicit heuristics query |
| 593 | - aclblasLtMatrixLayout_t Ddesc, | 593 | + * with default search preferences will be performed to determine the actual |
| 594 | - const aclblasLtMatmulAlgo_t* algo, | 594 | + * algorithm to use. |
| 595 | - void* workspace, | 595 | + * @param[in] |
| 596 | - size_t workspaceSizeInBytes, | 596 | + * workspace Pointer to the workspace buffer allocated in the GPU |
| 597 | - aclrtStream stream); | 597 | + * memory. Pointer must be 16B aligned. |
| 598 | - | 598 | + * @param[in] |
| 599 | -#ifdef __cplusplus | 599 | + * workspaceSizeInBytes Size of the workspace. |
| 600 | -} | 600 | + * @param[in] |
| 601 | + * stream The stream where all device work is submitted. | ||
| 602 | + * | ||
| 603 | + * \retval ACLBLAS_STATUS_SUCCESS If the operation completed | ||
| 604 | + * successfully. | ||
| 605 | + * \retval ACLBLAS_STATUS_EXECUTION_FAILED If device reported an | ||
| 606 | + * execution error. | ||
| 607 | + * \retval ACLBLAS_STATUS_ARCH_MISMATCH If | ||
| 608 | + * the configured operation cannot be run using the selected device. | ||
| 609 | + * \retval ACLBLAS_STATUS_NOT_SUPPORTED If the current implementation on the | ||
| 610 | + * selected device doesn't support the configured operation. | ||
| 611 | + * \retval ACLBLAS_STATUS_INVALID_VALUE If the parameters are unexpectedly NULL, in | ||
| 612 | + * conflict, or in an impossible configuration. | ||
| 613 | + * \retval ACLBLAS_STATUS_NOT_INITIALIZED If the aclBLASLt handle has not been initialized. | ||
| 614 | + */ | ||
| 615 | +aclblasStatus_t aclblasLtMatmul(aclblasLtHandle_t lightHandle, | ||
| 616 | + aclblasLtMatmulDesc_t computeDesc, | ||
| 617 | + const void* alpha, | ||
| 618 | + const void* A, | ||
| 619 | + aclblasLtMatrixLayout_t Adesc, | ||
| 620 | + const void* B, | ||
| 621 | + aclblasLtMatrixLayout_t Bdesc, | ||
| 622 | + const void* beta, | ||
| 623 | + const void* C, | ||
| 624 | + aclblasLtMatrixLayout_t Cdesc, | ||
| 625 | + void* D, | ||
| 626 | + aclblasLtMatrixLayout_t Ddesc, | ||
| 627 | + const aclblasLtMatmulAlgo_t* algo, | ||
| 628 | + void* workspace, | ||
| 629 | + size_t workspaceSizeInBytes, | ||
| 630 | + aclrtStream stream); | ||
| 631 | + | ||
| 632 | + | ||
| 633 | +} | ||
| 601 | 634 | ||
| @@ -0,0 +1,489 @@ | |||
| 1 | +#!/bin/bash | ||
| 2 | +# ============================================================================ | ||
| 3 | +# Copyright (c) 2025-2026 Huawei Technologies Co., Ltd. | ||
| 4 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 5 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 6 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 7 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 8 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 9 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 10 | +# ============================================================================ | ||
| 11 | + | ||
| 12 | +set -euo pipefail | ||
| 13 | + | ||
| 14 | +run_command() { | ||
| 15 | + local cmd="$*" | ||
| 16 | + echo "Executing command: $cmd" | ||
| 17 | + | ||
| 18 | + if ! output=$("$@" 2>&1); then | ||
| 19 | + local exit_code=$? | ||
| 20 | + echo -e "\nCommand execution failed!" | ||
| 21 | + echo -e "\nFailed command: $cmd" | ||
| 22 | + echo -e "\nError output: $output" | ||
| 23 | + echo -e "\nExit code: $exit_code" | ||
| 24 | + exit $exit_code | ||
| 25 | + fi | ||
| 26 | +} | ||
| 27 | + | ||
| 28 | +version_ge() { | ||
| 29 | + # Version comparison, format: xx.xx.xx | ||
| 30 | + IFS='.' read -r -a curr_arr <<< "$1" | ||
| 31 | + IFS='.' read -r -a req_arr <<< "$2" | ||
| 32 | + | ||
| 33 | + for ((i=0; i<${#req_arr[@]}; i++)); do | ||
| 34 | + curr=${curr_arr[i]:-0} | ||
| 35 | + req=${req_arr[i]} | ||
| 36 | + if (( curr > req )); then | ||
| 37 | + return 0 | ||
| 38 | + elif (( curr < req )); then | ||
| 39 | + return 1 | ||
| 40 | + fi | ||
| 41 | + done | ||
| 42 | + return 0 | ||
| 43 | +} | ||
| 44 | + | ||
| 45 | +detect_os() { | ||
| 46 | + # OS detection, supports debian (uses apt), rhel (uses dnf or yum), macos | ||
| 47 | + if [[ "$(uname -s)" == "Linux" ]]; then | ||
| 48 | + if [[ -f /etc/debian_version ]]; then | ||
| 49 | + OS="debian" | ||
| 50 | + PKG_MANAGER="apt" | ||
| 51 | + elif [[ -f /etc/redhat-release ]]; then | ||
| 52 | + OS="rhel" | ||
| 53 | + if command -v dnf &> /dev/null; then | ||
| 54 | + PKG_MANAGER="dnf" | ||
| 55 | + else | ||
| 56 | + PKG_MANAGER="yum" | ||
| 57 | + fi | ||
| 58 | + elif grep -qE '^NAME="openEuler"$|^NAME="EulerOS"$' /etc/os-release 2>/dev/null; then | ||
| 59 | + OS="euler" | ||
| 60 | + PKG_MANAGER="dnf" | ||
| 61 | + else | ||
| 62 | + echo "Unsupported Linux distribution, please install manually" | ||
| 63 | + exit 1 | ||
| 64 | + fi | ||
| 65 | + elif [[ "$(uname -s)" == "Darwin" ]]; then | ||
| 66 | + OS="macos" | ||
| 67 | + if ! command -v brew &> /dev/null; then | ||
| 68 | + echo "Please install Homebrew first" | ||
| 69 | + exit 1 | ||
| 70 | + fi | ||
| 71 | + PKG_MANAGER="brew" | ||
| 72 | + else | ||
| 73 | + echo "Unsupported OS type, please install manually" | ||
| 74 | + exit 1 | ||
| 75 | + fi | ||
| 76 | +} | ||
| 77 | + | ||
| 78 | +install_gawk() { | ||
| 79 | + echo -e "\n==== Checking gawk ====" | ||
| 80 | + | ||
| 81 | + if command -v gawk &> /dev/null; then | ||
| 82 | + echo "gawk has been installed" | ||
| 83 | + return | ||
| 84 | + fi | ||
| 85 | + | ||
| 86 | + echo "Installing gawk..." | ||
| 87 | + case "$OS" in | ||
| 88 | + debian) | ||
| 89 | + run_command sudo $PKG_MANAGER update | ||
| 90 | + run_command sudo $PKG_MANAGER install -y gawk | ||
| 91 | + ;; | ||
| 92 | + rhel|euler) | ||
| 93 | + run_command sudo $PKG_MANAGER install -y gawk | ||
| 94 | + ;; | ||
| 95 | + macos) | ||
| 96 | + run_command brew install gawk | ||
| 97 | + ;; | ||
| 98 | + esac | ||
| 99 | + | ||
| 100 | + if command -v gawk &> /dev/null; then | ||
| 101 | + echo "gawk installed successfully" | ||
| 102 | + else | ||
| 103 | + echo "gawk installation failed" | ||
| 104 | + exit 1 | ||
| 105 | + fi | ||
| 106 | +} | ||
| 107 | + | ||
| 108 | +install_python() { | ||
| 109 | + # Python version >= 3.7.0 | ||
| 110 | + echo -e "\n==== Checking Python ====" | ||
| 111 | + local req_ver="3.7.0" | ||
| 112 | + local curr_ver="" | ||
| 113 | + | ||
| 114 | + if command -v python3 &> /dev/null; then | ||
| 115 | + curr_ver=$(python3 --version 2>&1 | awk '{print $2}') | ||
| 116 | + echo "Current Python version: $curr_ver" | ||
| 117 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 118 | + echo "Python version meets requirements" | ||
| 119 | + return | ||
| 120 | + fi | ||
| 121 | + fi | ||
| 122 | + echo "Installing Python..." | ||
| 123 | + case "$OS" in | ||
| 124 | + debian) | ||
| 125 | + run_command sudo $PKG_MANAGER update | ||
| 126 | + run_command sudo $PKG_MANAGER install -y python3 python3-pip python3-dev | ||
| 127 | + ;; | ||
| 128 | + rhel) | ||
| 129 | + if grep -q "release 7" /etc/redhat-release; then | ||
| 130 | + run_command sudo $PKG_MANAGER install -y centos-release-scl | ||
| 131 | + run_command sudo $PKG_MANAGER install -y rh-python38 rh-python38-python-devel | ||
| 132 | + run_command source /opt/rh/rh-python38/enable | ||
| 133 | + echo "Need to execute 'source /opt/rh/rh-python38/enable' to activate python3.8" | ||
| 134 | + else | ||
| 135 | + run_command sudo $PKG_MANAGER install -y python3 python3-pip python3-devel | ||
| 136 | + fi | ||
| 137 | + ;; | ||
| 138 | + macos) | ||
| 139 | + run_command brew install python@3.10 | ||
| 140 | + echo 'export PATH="/usr/local/opt/python@3.10/bin:$PATH"' >> ~/.zshrc | ||
| 141 | + run_command source ~/.zshrc | ||
| 142 | + ;; | ||
| 143 | + euler) | ||
| 144 | + run_command sudo $PKG_MANAGER install -y python3 python3-pip python3-devel | ||
| 145 | + ;; | ||
| 146 | + esac | ||
| 147 | + | ||
| 148 | + if command -v python3 &> /dev/null; then | ||
| 149 | + curr_ver=$(python3 --version 2>&1 | awk '{print $2}') | ||
| 150 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 151 | + echo "Python installed successfully ($curr_ver)" | ||
| 152 | + else | ||
| 153 | + echo "Python version still doesn't meet requirements, please install manually" | ||
| 154 | + exit 1 | ||
| 155 | + fi | ||
| 156 | + else | ||
| 157 | + echo "Python installation failed" | ||
| 158 | + exit 1 | ||
| 159 | + fi | ||
| 160 | +} | ||
| 161 | + | ||
| 162 | +install_gcc() { | ||
| 163 | + # GCC version >= 7.3.0 | ||
| 164 | + echo -e "\n==== Checking GCC ====" | ||
| 165 | + local req_ver="7.3.0" | ||
| 166 | + local curr_ver="" | ||
| 167 | + | ||
| 168 | + if command -v gcc &> /dev/null; then | ||
| 169 | + curr_ver=$(gcc --version | awk '/^gcc/ {print $NF}') | ||
| 170 | + elif command -v g++ &> /dev/null; then | ||
| 171 | + curr_ver=$(g++ --version | awk '/^g\+\+/ {print $NF}') | ||
| 172 | + else | ||
| 173 | + curr_ver="0.0.0" | ||
| 174 | + fi | ||
| 175 | + echo "Current GCC version: $curr_ver" | ||
| 176 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 177 | + echo "GCC version meets requirements ($curr_ver)" | ||
| 178 | + return | ||
| 179 | + fi | ||
| 180 | + | ||
| 181 | + echo "Installing GCC..." | ||
| 182 | + case "$OS" in | ||
| 183 | + debian) | ||
| 184 | + run_command sudo $PKG_MANAGER update | ||
| 185 | + run_command sudo $PKG_MANAGER install -y gcc-9 g++-9 | ||
| 186 | + run_command sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-9 90 \ | ||
| 187 | + --slave /usr/bin/g++ g++ /usr/bin/g++-9 | ||
| 188 | + ;; | ||
| 189 | + rhel) | ||
| 190 | + if grep -q "release 7" /etc/redhat-release; then | ||
| 191 | + run_command sudo $PKG_MANAGER install -y centos-release-scl | ||
| 192 | + run_command sudo $PKG_MANAGER install -y devtoolset-9-gcc devtoolset-9-gcc-c++ | ||
| 193 | + run_command source /opt/rh/devtoolset-9/enable | ||
| 194 | + echo "Need to execute 'source /opt/rh/devtoolset-9/enable' to activate GCC9" | ||
| 195 | + else | ||
| 196 | + run_command sudo $PKG_MANAGER install -y gcc gcc-c++ | ||
| 197 | + fi | ||
| 198 | + ;; | ||
| 199 | + macos) | ||
| 200 | + if ! xcode-select -p &> /dev/null; then | ||
| 201 | + xcode-select --install | ||
| 202 | + fi | ||
| 203 | + run_command brew install gcc@11 | ||
| 204 | + echo 'export CC=/usr/local/bin/gcc-11' >> ~/.zshrc | ||
| 205 | + echo 'export CXX=/usr/local/bin/g++-11' >> ~/.zshrc | ||
| 206 | + run_command source ~/.zshrc | ||
| 207 | + ;; | ||
| 208 | + euler) | ||
| 209 | + run_command sudo $PKG_MANAGER install -y gcc gcc-c++ | ||
| 210 | + ;; | ||
| 211 | + esac | ||
| 212 | + | ||
| 213 | + if command -v gcc &> /dev/null; then | ||
| 214 | + curr_ver=$(gcc --version | awk '/^gcc/ {print $NF}') | ||
| 215 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 216 | + echo "GCC installed successfully ($curr_ver)" | ||
| 217 | + else | ||
| 218 | + echo "GCC version still doesn't meet requirements, please install manually." | ||
| 219 | + exit 1 | ||
| 220 | + fi | ||
| 221 | + else | ||
| 222 | + echo "GCC installation failed" | ||
| 223 | + exit 1 | ||
| 224 | + fi | ||
| 225 | +} | ||
| 226 | + | ||
| 227 | +install_cmake() { | ||
| 228 | + # CMake version >= 3.16.0 | ||
| 229 | + echo -e "\n==== Checking CMake ====" | ||
| 230 | + local req_ver="3.16.0" | ||
| 231 | + local curr_ver="" | ||
| 232 | + | ||
| 233 | + if command -v cmake &> /dev/null; then | ||
| 234 | + curr_ver=$(cmake --version | awk '/^cmake/ {print $3}') | ||
| 235 | + echo "Current CMake version: $curr_ver" | ||
| 236 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 237 | + echo "CMake meets requirements" | ||
| 238 | + return | ||
| 239 | + fi | ||
| 240 | + fi | ||
| 241 | + | ||
| 242 | + echo "Installing CMake..." | ||
| 243 | + case "$OS" in | ||
| 244 | + debian) | ||
| 245 | + if grep -q "Ubuntu 18.04" /etc/os-release; then | ||
| 246 | + run_command wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | gpg --dearmor - | sudo tee /usr/share/keyrings/kitware-archive-keyring.gpg >/dev/null | ||
| 247 | + run_command echo 'deb [signed-by=/usr/share/keyrings/kitware-archive-keyring.gpg] https://apt.kitware.com/ubuntu/ bionic main' | sudo tee /etc/apt/sources.list.d/kitware.list >/dev/null | ||
| 248 | + run_command sudo apt update | ||
| 249 | + run_command sudo apt install -y cmake make | ||
| 250 | + else | ||
| 251 | + run_command sudo $PKG_MANAGER update | ||
| 252 | + run_command sudo $PKG_MANAGER install -y cmake make | ||
| 253 | + fi | ||
| 254 | + ;; | ||
| 255 | + rhel) | ||
| 256 | + if grep -q "release 7" /etc/redhat-release; then | ||
| 257 | + run_command sudo $PKG_MANAGER install -y epel-release | ||
| 258 | + run_command sudo $PKG_MANAGER install -y cmake3 make | ||
| 259 | + run_command sudo ln -s /usr/bin/cmake3 /usr/bin/cmake | ||
| 260 | + else | ||
| 261 | + run_command sudo $PKG_MANAGER install -y cmake make | ||
| 262 | + fi | ||
| 263 | + ;; | ||
| 264 | + macos) | ||
| 265 | + run_command brew install cmake | ||
| 266 | + ;; | ||
| 267 | + euler) | ||
| 268 | + run_command sudo $PKG_MANAGER install -y cmake make | ||
| 269 | + ;; | ||
| 270 | + esac | ||
| 271 | + | ||
| 272 | + if command -v cmake &> /dev/null; then | ||
| 273 | + curr_ver=$(cmake --version | awk '/^cmake/ {print $3}') | ||
| 274 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 275 | + echo "CMake installed successfully ($curr_ver)" | ||
| 276 | + else | ||
| 277 | + echo "CMake version still doesn't meet requirements, please install manually" | ||
| 278 | + exit 1 | ||
| 279 | + fi | ||
| 280 | + else | ||
| 281 | + echo "CMake installation failed" | ||
| 282 | + exit 1 | ||
| 283 | + fi | ||
| 284 | +} | ||
| 285 | + | ||
| 286 | +install_pigz() { | ||
| 287 | + # pigz version >= 2.4 | ||
| 288 | + echo -e "\n==== Checking pigz ====" | ||
| 289 | + local req_ver="2.4" | ||
| 290 | + local curr_ver="" | ||
| 291 | + | ||
| 292 | + if command -v pigz &> /dev/null; then | ||
| 293 | + curr_ver=$(pigz --version 2>&1 | awk '{print $2}') | ||
| 294 | + echo "Current pigz version: $curr_ver" | ||
| 295 | + if version_ge "$curr_ver" "$req_ver"; then | ||
| 296 | + echo "pigz meets requirements" | ||
| 297 | + return | ||
| 298 | + fi | ||
| 299 | + fi | ||
| 300 | + | ||
| 301 | + read -p "Install pigz? [Y/n] " -n 1 -r | ||
| 302 | + echo | ||
| 303 | + if [[ ! $REPLY =~ ^[Yy]$ ]]; then | ||
| 304 | + echo "Skipping pigz installation" | ||
| 305 | + return | ||
| 306 | + fi | ||
| 307 | + | ||
| 308 | + echo "Installing pigz..." | ||
| 309 | + case "$OS" in | ||
| 310 | + debian|rhel|euler) | ||
| 311 | + run_command sudo $PKG_MANAGER install -y pigz | ||
| 312 | + ;; | ||
| 313 | + macos) | ||
| 314 | + run_command brew install pigz | ||
| 315 | + ;; | ||
| 316 | + esac | ||
| 317 | + | ||
| 318 | + if command -v pigz &> /dev/null; then | ||
| 319 | + curr_ver=$(pigz --version 2>&1 | awk '{print $2}') | ||
| 320 | + echo "pigz installed successfully ($curr_ver)" | ||
| 321 | + else | ||
| 322 | + echo "pigz installation failed, can be ignored" | ||
| 323 | + fi | ||
| 324 | +} | ||
| 325 | + | ||
| 326 | +install_dos2unix() { | ||
| 327 | + echo -e "\n==== Checking dos2unix ====" | ||
| 328 | + | ||
| 329 | + if command -v dos2unix &> /dev/null; then | ||
| 330 | + echo "dos2unix has been installed" | ||
| 331 | + return | ||
| 332 | + fi | ||
| 333 | + | ||
| 334 | + echo "Installing dos2unix..." | ||
| 335 | + case "$OS" in | ||
| 336 | + debian|rhel|euler) | ||
| 337 | + run_command sudo $PKG_MANAGER install -y dos2unix | ||
| 338 | + ;; | ||
| 339 | + macos) | ||
| 340 | + run_command brew install dos2unix | ||
| 341 | + ;; | ||
| 342 | + esac | ||
| 343 | + | ||
| 344 | + if command -v dos2unix &> /dev/null; then | ||
| 345 | + echo "dos2unix installed successfully" | ||
| 346 | + else | ||
| 347 | + echo "dos2unix installation failed" | ||
| 348 | + exit 1 | ||
| 349 | + fi | ||
| 350 | +} | ||
| 351 | + | ||
| 352 | +install_patch() { | ||
| 353 | + echo -e "\n==== Checking patch ====" | ||
| 354 | + | ||
| 355 | + if command -v patch &> /dev/null; then | ||
| 356 | + echo "patch has been installed" | ||
| 357 | + return | ||
| 358 | + fi | ||
| 359 | + | ||
| 360 | + echo "Installing patch..." | ||
| 361 | + case "$OS" in | ||
| 362 | + debian|rhel) | ||
| 363 | + run_command sudo $PKG_MANAGER install -y patch | ||
| 364 | + ;; | ||
| 365 | + macos) | ||
| 366 | + run_command brew install patch | ||
| 367 | + ;; | ||
| 368 | + esac | ||
| 369 | + | ||
| 370 | + if command -v patch &> /dev/null; then | ||
| 371 | + echo "patch installed successfully" | ||
| 372 | + else | ||
| 373 | + echo "patch installation failed" | ||
| 374 | + exit 1 | ||
| 375 | + fi | ||
| 376 | +} | ||
| 377 | + | ||
| 378 | +check_dependencies_silent() { | ||
| 379 | + local args=("$@") | ||
| 380 | + local check_pkgz="false" | ||
| 381 | + local check_dos2unix="false" | ||
| 382 | + | ||
| 383 | + for arg in "${args[@]}"; do | ||
| 384 | + case "$arg" in | ||
| 385 | + --pkg) | ||
| 386 | + check_pkgz="true" | ||
| 387 | + check_dos2unix="true" | ||
| 388 | + ;; | ||
| 389 | + --opkernel) | ||
| 390 | + check_dos2unix="true" | ||
| 391 | + ;; | ||
| 392 | + esac | ||
| 393 | + done | ||
| 394 | + | ||
| 395 | + local missing_deps=() | ||
| 396 | + declare -A req_versions | ||
| 397 | + req_versions["gawk"]="" | ||
| 398 | + req_versions["Python"]="3.7.0" | ||
| 399 | + req_versions["GCC"]="7.3.0" | ||
| 400 | + req_versions["CMake"]="3.16.0" | ||
| 401 | + req_versions["pigz"]="2.4" | ||
| 402 | + req_versions["dos2unix"]="" | ||
| 403 | + | ||
| 404 | + check_deps() { | ||
| 405 | + local name="$1" | ||
| 406 | + local cmd="$2" | ||
| 407 | + local req_ver="$3" | ||
| 408 | + | ||
| 409 | + if ! command -v "$cmd" &> /dev/null; then | ||
| 410 | + missing_deps+=("$name") | ||
| 411 | + return | ||
| 412 | + fi | ||
| 413 | + | ||
| 414 | + if [[ -n "$req_ver" ]]; then | ||
| 415 | + local curr_ver="" | ||
| 416 | + case "$cmd" in | ||
| 417 | + python3) | ||
| 418 | + curr_ver=$(python3 --version 2>&1 | awk '{print $2}') | ||
| 419 | + ;; | ||
| 420 | + gcc|g++) | ||
| 421 | + curr_ver=$(gcc --version | awk '/^gcc/ {print $NF}') | ||
| 422 | + ;; | ||
| 423 | + cmake) | ||
| 424 | + curr_ver=$(cmake --version | awk '/^cmake/ {print $3}') | ||
| 425 | + ;; | ||
| 426 | + pigz) | ||
| 427 | + curr_ver=$(pigz --version 2>&1 | awk '{print $2}') | ||
| 428 | + ;; | ||
| 429 | + esac | ||
| 430 | + | ||
| 431 | + if [[ -z "$curr_ver" ]] || ! version_ge "$curr_ver" "$req_ver"; then | ||
| 432 | + missing_deps+=("$name") | ||
| 433 | + fi | ||
| 434 | + fi | ||
| 435 | + } | ||
| 436 | + | ||
| 437 | + check_deps "gawk" "gawk" "${req_versions["gawk"]}" | ||
| 438 | + check_deps "Python" "python3" "${req_versions["Python"]}" | ||
| 439 | + check_deps "GCC" "gcc" "${req_versions["GCC"]}" | ||
| 440 | + check_deps "CMake" "cmake" "${req_versions["CMake"]}" | ||
| 441 | + if [[ "$check_dos2unix" == "true" ]]; then | ||
| 442 | + check_deps "dos2unix" "dos2unix" "${req_versions["dos2unix"]}" | ||
| 443 | + fi | ||
| 444 | + if [[ "$check_pkgz" == "true" ]]; then | ||
| 445 | + check_deps "pigz" "pigz" "${req_versions["pigz"]}" | ||
| 446 | + fi | ||
| 447 | + | ||
| 448 | + if [[ ${#missing_deps[@]} -eq 0 ]]; then | ||
| 449 | + return 0 | ||
| 450 | + else | ||
| 451 | + echo -e "\n Missing dependencies:" | ||
| 452 | + for dep in "${missing_deps[@]}"; do | ||
| 453 | + local req_ver="${req_versions[$dep]}" | ||
| 454 | + if [[ -n "$req_ver" ]]; then | ||
| 455 | + echo " - $dep (required: >= $req_ver)" | ||
| 456 | + else | ||
| 457 | + echo " - $dep" | ||
| 458 | + fi | ||
| 459 | + done | ||
| 460 | + echo -e "\n Please run:" | ||
| 461 | + echo -e "\n bash install_deps.sh\n" | ||
| 462 | + echo -e " to install all missing dependencies." | ||
| 463 | + echo -e " After installation, re-run this script.\n" | ||
| 464 | + return 1 | ||
| 465 | + fi | ||
| 466 | +} | ||
| 467 | + | ||
| 468 | +main() { | ||
| 469 | + echo "====================================================" | ||
| 470 | + echo "Starting project dependency installation" | ||
| 471 | + echo "====================================================" | ||
| 472 | + | ||
| 473 | + detect_os | ||
| 474 | + install_gawk | ||
| 475 | + install_python | ||
| 476 | + install_gcc | ||
| 477 | + install_cmake | ||
| 478 | + install_pigz | ||
| 479 | + install_dos2unix | ||
| 480 | + install_patch | ||
| 481 | + | ||
| 482 | + echo -e "====================================================" | ||
| 483 | + echo "All dependencies installed successfully!" | ||
| 484 | + echo "====================================================" | ||
| 485 | +} | ||
| 486 | + | ||
| 487 | +if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then | ||
| 488 | + main "$@" | ||
| 489 | +fi | ||
| @@ -0,0 +1,10 @@ | |||
| 1 | +numpy | ||
| 2 | +decorator | ||
| 3 | +sympy | ||
| 4 | +scipy | ||
| 5 | +attrs | ||
| 6 | +psutil | ||
| 7 | +protobuf | ||
| 8 | +pyyaml | ||
| 9 | +setuptools | ||
| 10 | +wheel | ||
| @@ -28,7 +28,7 @@ | |||
| 28 | 28 | ||
| 29 | ## 算子描述 | 29 | ## 算子描述 |
| 30 | 30 | ||
| 31 | -- 算子功能： | 31 | +- 算子功能： |
| 32 | `blasLtMatmul` 用于完成矩阵乘法，并可通过 epilogue 融合后处理（如 bias、ReLU 等）。其基础数学表达式为： | 32 | `blasLtMatmul` 用于完成矩阵乘法，并可通过 epilogue 融合后处理（如 bias、ReLU 等）。其基础数学表达式为： |
| 33 | 33 | ||
| 34 | ``` | 34 | ``` |
| @@ -80,7 +80,7 @@ D = alpha * op(A) * op(B) + beta * C | |||
| 80 | 80 | ||
| 81 | 在本样例根目录下执行如下步骤，编译并执行算子。 | 81 | 在本样例根目录下执行如下步骤，编译并执行算子。 |
| 82 | 82 | ||
| 83 | -- 配置环境变量 | 83 | +- 配置环境变量 |
| 84 | 请根据当前环境上 CANN 开发套件包的安装方式，选择对应配置环境变量的命令。 | 84 | 请根据当前环境上 CANN 开发套件包的安装方式，选择对应配置环境变量的命令。 |
| 85 | 85 | ||
| 86 | - 默认路径，root 用户安装 CANN 软件包 | 86 | - 默认路径，root 用户安装 CANN 软件包 |
| @@ -104,7 +104,7 @@ source ${install_path}/cann/set_env.sh | |||
| 104 | - 样例执行 | 104 | - 样例执行 |
| 105 | 105 | ||
| 106 | ```bash | 106 | ```bash |
| 107 | -bash build.sh --op=blasLtMatmul --run # --op=<算子名> --run 可选参数，执行测试样例 | 107 | +bash build.sh --op=blasLtMatmul --soc=ascend950 --run |
| 108 | ``` | 108 | ``` |
| 109 | 109 | ||
| 110 | 执行结果如下，说明精度对比成功。 | 110 | 执行结果如下，说明精度对比成功。 |
| @@ -17,19 +17,17 @@ | |||
| 17 | 17 | ||
| 18 | 18 | ||
| 19 | 19 | ||
| 20 | -#include "../common/util.h" | 20 | +#include "../utils/error_check.h" |
| 21 | + | ||
| 21 | 22 | ||
| 22 | 23 | ||
| 23 | 24 | ||
| 24 | int Init(int32_t deviceId, aclrtStream* stream) | 25 | int Init(int32_t deviceId, aclrtStream* stream) |
| 25 | { | 26 | { |
| 26 | // 固定写法，资源初始化 | 27 | // 固定写法，资源初始化 |
| 27 | - auto ret = aclInit(nullptr); | 28 | + CHECK_ACLRT(aclInit(nullptr)); |
| 28 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret); | 29 | + CHECK_ACLRT(aclrtSetDevice(deviceId)); |
| 29 | - ret = aclrtSetDevice(deviceId); | 30 | + CHECK_ACLRT(aclrtCreateStream(stream)); |
| 30 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret); | ||
| 31 | - ret = aclrtCreateStream(stream); | ||
| 32 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret); | ||
| 33 | return 0; | 31 | return 0; |
| 34 | } | 32 | } |
| 35 | 33 | ||
| @@ -50,7 +48,7 @@ aclError aclblasLtMatmulTest(int32_t deviceId, aclrtStream& stream) | |||
| 50 | 48 | ||
| 51 | // 2. 创建 BLASLt 句柄 | 49 | // 2. 创建 BLASLt 句柄 |
| 52 | aclblasLtHandle_t ltHandle; | 50 | aclblasLtHandle_t ltHandle; |
| 53 | - CHECK_ACLBLASLT(aclblasLtCreate(<Handle)); | 51 | + CHECK_ACLBLAS(aclblasLtCreate(<Handle)); |
| 54 | 52 | ||
| 55 | // 3. 在Device上分配和初始化数据 (示例使用随机初始化) | 53 | // 3. 在Device上分配和初始化数据 (示例使用随机初始化) |
| 56 | std::vector<float> hostInput(m * k, 0); | 54 | std::vector<float> hostInput(m * k, 0); |
| @@ -76,37 +74,37 @@ aclError aclblasLtMatmulTest(int32_t deviceId, aclrtStream& stream) | |||
| 76 | aclblasLtMatrixLayout_t Adesc, Bdesc, Ddesc; | 74 | aclblasLtMatrixLayout_t Adesc, Bdesc, Ddesc; |
| 77 | aclblasLtOrder_t order = ACLBLASLT_ORDER_ROW; | 75 | aclblasLtOrder_t order = ACLBLASLT_ORDER_ROW; |
| 78 | // 矩阵 A: FP32, 行主序, 维度 m x k | 76 | // 矩阵 A: FP32, 行主序, 维度 m x k |
| 79 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutCreate(&Adesc, ACL_FLOAT, m, k, m)); | 77 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutCreate(&Adesc, ACL_FLOAT, m, k, m)); |
| 80 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutSetAttribute(Adesc, ACLBLASLT_MATRIX_LAYOUT_ORDER, &order, sizeof(int))); | 78 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutSetAttribute(Adesc, ACLBLASLT_MATRIX_LAYOUT_ORDER, &order, sizeof(int))); |
| 81 | // 矩阵 B: FP32, 行主序, 维度 k x n | 79 | // 矩阵 B: FP32, 行主序, 维度 k x n |
| 82 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutCreate(&Bdesc, ACL_FLOAT, k, n, k)); | 80 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutCreate(&Bdesc, ACL_FLOAT, k, n, k)); |
| 83 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutSetAttribute(Bdesc, ACLBLASLT_MATRIX_LAYOUT_ORDER, &order, sizeof(int))); | 81 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutSetAttribute(Bdesc, ACLBLASLT_MATRIX_LAYOUT_ORDER, &order, sizeof(int))); |
| 84 | // 矩阵 D: FP32, 行主序, 维度 m x n | 82 | // 矩阵 D: FP32, 行主序, 维度 m x n |
| 85 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutCreate(&Ddesc, ACL_FLOAT, m, n, m)); | 83 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutCreate(&Ddesc, ACL_FLOAT, m, n, m)); |
| 86 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutSetAttribute(Ddesc, ACLBLASLT_MATRIX_LAYOUT_ORDER, &order, sizeof(int))); | 84 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutSetAttribute(Ddesc, ACLBLASLT_MATRIX_LAYOUT_ORDER, &order, sizeof(int))); |
| 87 | 85 | ||
| 88 | // 5. 创建并设置计算描述符 | 86 | // 5. 创建并设置计算描述符 |
| 89 | aclblasLtMatmulDesc_t operationDesc; | 87 | aclblasLtMatmulDesc_t operationDesc; |
| 90 | - CHECK_ACLBLASLT(aclblasLtMatmulDescCreate(&operationDesc, ACLBLAS_COMPUTE_32F, ACL_FLOAT)); | 88 | + CHECK_ACLBLAS(aclblasLtMatmulDescCreate(&operationDesc, ACLBLAS_COMPUTE_32F, ACL_FLOAT)); |
| 91 | // 设置 epilogue | 89 | // 设置 epilogue |
| 92 | aclblasLtEpilogue_t epilogue = ACLBLASLT_EPILOGUE_DEFAULT; | 90 | aclblasLtEpilogue_t epilogue = ACLBLASLT_EPILOGUE_DEFAULT; |
| 93 | - CHECK_ACLBLASLT( | 91 | + CHECK_ACLBLAS( |
| 94 | aclblasLtMatmulDescSetAttribute(operationDesc, ACLBLASLT_MATMUL_DESC_EPILOGUE, &epilogue, sizeof(epilogue))); | 92 | aclblasLtMatmulDescSetAttribute(operationDesc, ACLBLASLT_MATMUL_DESC_EPILOGUE, &epilogue, sizeof(epilogue))); |
| 95 | 93 | ||
| 96 | // 6. 算法选择 | 94 | // 6. 算法选择 |
| 97 | aclblasLtMatmulPreference_t preference; | 95 | aclblasLtMatmulPreference_t preference; |
| 98 | - CHECK_ACLBLASLT(aclblasLtMatmulPreferenceCreate(&preference)); | 96 | + CHECK_ACLBLAS(aclblasLtMatmulPreferenceCreate(&preference)); |
| 99 | // 为算法最多预留 12MB 的临时工作空间 | 97 | // 为算法最多预留 12MB 的临时工作空间 |
| 100 | size_t max_workspaceSize = 12 * 1024 * 1024; | 98 | size_t max_workspaceSize = 12 * 1024 * 1024; |
| 101 | - CHECK_ACLBLASLT(aclblasLtMatmulPreferenceSetAttribute(preference, ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES, | 99 | + CHECK_ACLBLAS(aclblasLtMatmulPreferenceSetAttribute(preference, ACLBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES, |
| 102 | - &max_workspaceSize, sizeof(max_workspaceSize))); | 100 | + &max_workspaceSize, sizeof(max_workspaceSize))); |
| 103 | 101 | ||
| 104 | // 用于接收推荐算法 | 102 | // 用于接收推荐算法 |
| 105 | int returnedAlgoCount = 0; | 103 | int returnedAlgoCount = 0; |
| 106 | const int request_solutions = 1; | 104 | const int request_solutions = 1; |
| 107 | aclblasLtMatmulHeuristicResult_t heuristicResult[request_solutions]; | 105 | aclblasLtMatmulHeuristicResult_t heuristicResult[request_solutions]; |
| 108 | - CHECK_ACLBLASLT(aclblasLtMatmulAlgoGetHeuristic(ltHandle, operationDesc, Adesc, Bdesc, Ddesc, Ddesc, preference, | 106 | + CHECK_ACLBLAS(aclblasLtMatmulAlgoGetHeuristic(ltHandle, operationDesc, Adesc, Bdesc, Ddesc, Ddesc, preference, |
| 109 | - request_solutions, heuristicResult, &returnedAlgoCount)); | 107 | + request_solutions, heuristicResult, &returnedAlgoCount)); |
| 110 | 108 | ||
| 111 | if (returnedAlgoCount == 0) { | 109 | if (returnedAlgoCount == 0) { |
| 112 | std::cerr << "No valid algorithm found for the given problem!" << std::endl; | 110 | std::cerr << "No valid algorithm found for the given problem!" << std::endl; |
| @@ -122,32 +120,29 @@ aclError aclblasLtMatmulTest(int32_t deviceId, aclrtStream& stream) | |||
| 122 | } | 120 | } |
| 123 | void *d_workspace = nullptr; | 121 | void *d_workspace = nullptr; |
| 124 | CHECK_ACLRT(aclrtMalloc(&d_workspace, workspace_size, ACL_MEM_MALLOC_HUGE_FIRST)); | 122 | CHECK_ACLRT(aclrtMalloc(&d_workspace, workspace_size, ACL_MEM_MALLOC_HUGE_FIRST)); |
| 125 | - CHECK_ACLBLASLT(aclblasLtMatmul(ltHandle, | 123 | + CHECK_ACLBLAS(aclblasLtMatmul(ltHandle, |
| 126 | - operationDesc, | 124 | + operationDesc, |
| 127 | - &alpha, | 125 | + &alpha, |
| 128 | - d_A, | 126 | + d_A, |
| 129 | - Adesc, | 127 | + Adesc, |
| 130 | - d_B, | 128 | + d_B, |
| 131 | - Bdesc, | 129 | + Bdesc, |
| 132 | - &beta, | 130 | + &beta, |
| 133 | - d_D, | 131 | + d_D, |
| 134 | - Ddesc, | 132 | + Ddesc, |
| 135 | - d_D, | 133 | + d_D, |
| 136 | - Ddesc, | 134 | + Ddesc, |
| 137 | - &algo, | 135 | + &algo, |
| 138 | - d_workspace, | 136 | + d_workspace, |
| 139 | - workspace_size, | 137 | + workspace_size, |
| 140 | - stream)); | 138 | + stream)); |
| 141 | 139 | ||
| 142 | // 8. 同步并检查错误 | 140 | // 8. 同步并检查错误 |
| 143 | - auto ret = aclInit(nullptr); | 141 | + CHECK_ACLRT(aclrtSynchronizeStream(stream)); |
| 144 | - ret = aclrtSynchronizeStream(stream); | ||
| 145 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret); | ||
| 146 | std::cout << "aclblasLtMatmul executed successfully!" << std::endl; | 142 | std::cout << "aclblasLtMatmul executed successfully!" << std::endl; |
| 147 | std::cout << "Heuristic returned " << returnedAlgoCount << " algorithm(s). Using the first one." << std::endl; | 143 | std::cout << "Heuristic returned " << returnedAlgoCount << " algorithm(s). Using the first one." << std::endl; |
| 148 | // 输出数据Device To Host | 144 | // 输出数据Device To Host |
| 149 | - ret = aclrtMemcpy(hostOutput.data(), sizeOutput, d_D, sizeOutput, ACL_MEMCPY_DEVICE_TO_HOST); | 145 | + CHECK_ACLRT(aclrtMemcpy(hostOutput.data(), sizeOutput, d_D, sizeOutput, ACL_MEMCPY_DEVICE_TO_HOST)); |
| 150 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy deviceOutput failed. ERROR: %d\n", ret); return ret); | ||
| 151 | 146 | ||
| 152 | // 9. 计算golden，对比精度 | 147 | // 9. 计算golden，对比精度 |
| 153 | ComputeGolden<float>(m, k, n, hostInput, hostWeight, goldenOutput); | 148 | ComputeGolden<float>(m, k, n, hostInput, hostWeight, goldenOutput); |
| @@ -169,12 +164,12 @@ aclError aclblasLtMatmulTest(int32_t deviceId, aclrtStream& stream) | |||
| 169 | CHECK_ACLRT(aclrtFree(d_D)); | 164 | CHECK_ACLRT(aclrtFree(d_D)); |
| 170 | CHECK_ACLRT(aclrtFree(d_B)); | 165 | CHECK_ACLRT(aclrtFree(d_B)); |
| 171 | CHECK_ACLRT(aclrtFree(d_A)); | 166 | CHECK_ACLRT(aclrtFree(d_A)); |
| 172 | - CHECK_ACLBLASLT(aclblasLtMatmulPreferenceDestroy(preference)); | 167 | + CHECK_ACLBLAS(aclblasLtMatmulPreferenceDestroy(preference)); |
| 173 | - CHECK_ACLBLASLT(aclblasLtMatmulDescDestroy(operationDesc)); | 168 | + CHECK_ACLBLAS(aclblasLtMatmulDescDestroy(operationDesc)); |
| 174 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutDestroy(Ddesc)); | 169 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutDestroy(Ddesc)); |
| 175 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutDestroy(Bdesc)); | 170 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutDestroy(Bdesc)); |
| 176 | - CHECK_ACLBLASLT(aclblasLtMatrixLayoutDestroy(Adesc)); | 171 | + CHECK_ACLBLAS(aclblasLtMatrixLayoutDestroy(Adesc)); |
| 177 | - CHECK_ACLBLASLT(aclblasLtDestroy(ltHandle)); | 172 | + CHECK_ACLBLAS(aclblasLtDestroy(ltHandle)); |
| 178 | 173 | ||
| 179 | return ACL_SUCCESS; | 174 | return ACL_SUCCESS; |
| 180 | } | 175 | } |
| @@ -190,14 +185,13 @@ int main(int argc, char* argv[]) | |||
| 190 | // printUsage(argv[0]); | 185 | // printUsage(argv[0]); |
| 191 | // return 1; | 186 | // return 1; |
| 192 | // } | 187 | // } |
| 188 | + | ||
| 193 | // 固定写法，device/stream初始化，参考acl API手册 | 189 | // 固定写法，device/stream初始化，参考acl API手册 |
| 194 | // 根据自己的实际device填写deviceId | 190 | // 根据自己的实际device填写deviceId |
| 195 | int32_t deviceId = 0; | 191 | int32_t deviceId = 0; |
| 196 | aclrtStream stream; | 192 | aclrtStream stream; |
| 197 | - auto ret = Init(deviceId, &stream); | 193 | + CHECK_ACLRT(Init(deviceId, &stream)); |
| 198 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret); | 194 | + CHECK_ACLRT(aclblasLtMatmulTest(deviceId, stream)); |
| 199 | - ret = aclblasLtMatmulTest(deviceId, stream); | ||
| 200 | - CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclblasLtMatmulTest failed. ERROR: %d\n", ret); return ret); | ||
| 201 | // 固定写法，释放资源 | 195 | // 固定写法，释放资源 |
| 202 | Finalize(deviceId, stream); | 196 | Finalize(deviceId, stream); |
| 203 | return 0; | 197 | return 0; |
| @@ -0,0 +1,28 @@ | |||
| 1 | + | ||
| 2 | + | ||
| 3 | + | ||
| 4 | + | ||
| 5 | + | ||
| 6 | + | ||
| 7 | + | ||
| 8 | + | ||
| 9 | + | ||
| 10 | + | ||
| 11 | + { \ | ||
| 12 | + aclError status = (func); \ | ||
| 13 | + if (status != ACL_SUCCESS) { \ | ||
| 14 | + std::cerr << "ACL Runtime Error at " << __FILE__ << ":" << __LINE__ << " (error code: " << status << ")" << std::endl; \ | ||
| 15 | + exit(EXIT_FAILURE); \ | ||
| 16 | + } \ | ||
| 17 | + } | ||
| 18 | + | ||
| 19 | + | ||
| 20 | + { \ | ||
| 21 | + aclblasStatus_t status = (func); \ | ||
| 22 | + if (status != ACLBLAS_STATUS_SUCCESS) { \ | ||
| 23 | + std::cerr << "BLASLT Error at " << __FILE__ << ":" << __LINE__ << " (error code: " << status << ")" << std::endl; \ | ||
| 24 | + exit(EXIT_FAILURE); \ | ||
| 25 | + } \ | ||
| 26 | + } | ||
| 27 | + | ||
| 28 | + | ||
| @@ -1,16 +1,14 @@ | |||
| 1 | -#ifndef OPS_BLAS_TEST_COMMON_UTIL_H | 1 | +#ifndef OPS_BLAS_TEST_UTILS_GOLDEN_H |
| 2 | -#define OPS_BLAS_TEST_COMMON_UTIL_H | 2 | +#define OPS_BLAS_TEST_UTILS_GOLDEN_H |
| 3 | 3 | ||
| 4 | 4 | ||
| 5 | 5 | ||
| 6 | 6 | ||
| 7 | - | ||
| 8 | 7 | ||
| 9 | 8 | ||
| 10 | 9 | ||
| 11 | 10 | ||
| 12 | 11 | ||
| 13 | - | ||
| 14 | 12 | ||
| 15 | template <typename T> | 13 | template <typename T> |
| 16 | void FillRandomData(std::vector<T>& data, T min, T max) | 14 | void FillRandomData(std::vector<T>& data, T min, T max) |
| @@ -94,34 +92,4 @@ inline void parseArguments(int argc, char* argv[], int& m, int& k, int& n) | |||
| 94 | } | 92 | } |
| 95 | } | 93 | } |
| 96 | 94 | ||
| 97 | -#define CHECK_RET(cond, return_expr) \ | 95 | +#endif // OPS_BLAS_TEST_UTILS_GOLDEN_H |
| 98 | - do { \ | ||
| 99 | - if (!(cond)) { \ | ||
| 100 | - return_expr; \ | ||
| 101 | - } \ | ||
| 102 | - } while (0) | ||
| 103 | - | ||
| 104 | - | ||
| 105 | - do { \ | ||
| 106 | - printf(message, ##__VA_ARGS__); \ | ||
| 107 | - } while (0) | ||
| 108 | - | ||
| 109 | - | ||
| 110 | - { \ | ||
| 111 | - aclError status = (func); \ | ||
| 112 | - if (status != ACL_SUCCESS) { \ | ||
| 113 | - std::cerr << "ACL Runtime Error at " << __FILE__ << ":" << __LINE__ << " (error code: " << status << ")" << std::endl; \ | ||
| 114 | - exit(EXIT_FAILURE); \ | ||
| 115 | - } \ | ||
| 116 | - } | ||
| 117 | - | ||
| 118 | - | ||
| 119 | - { \ | ||
| 120 | - aclblasStatus_t status = (func); \ | ||
| 121 | - if (status != ACLBLAS_STATUS_SUCCESS) { \ | ||
| 122 | - std::cerr << "BLASLT Error at " << __FILE__ << ":" << __LINE__ << " (error code: " << status << ")" << std::endl; \ | ||
| 123 | - exit(EXIT_FAILURE); \ | ||
| 124 | - } \ | ||
| 125 | - } | ||
| 126 | - | ||
| 127 | - | ||