已合并
【自研】【PyTorch离线推理】【cv】Nasnetlarge模型代码与资料整改 #3228
AtomGit-Bot创建于 2022年12月28日
【自研】【PyTorch离线推理】【cv】Nasnetlarge模型代码与资料整改 #3228
已合并
从refs/pull/3228/head合入到master
共 11 个文件变更+223-637
| @@ -1,331 +1,276 @@ | |||
| 1 | -# Nasnetlarge Onnx模型端到端推理指导 | 1 | +# Nasnetlarge模型-推理指导 |
| 2 | - | ||
| 3 | -- [1 模型概述](#1-模型概述) | ||
| 4 | - - [1.1 论文地址](#11-论文地址) | ||
| 5 | - - [1.2 代码地址](#12-代码地址) | ||
| 6 | -- [2 环境说明](#2-环境说明) | ||
| 7 | - - [2.1 深度学习框架](#21-深度学习框架) | ||
| 8 | - - [2.2 python第三方库](#22-python第三方库) | ||
| 9 | -- [3 模型转换](#3-模型转换) | ||
| 10 | - - [3.1 pth转onnx模型](#31-pth转onnx模型) | ||
| 11 | - - [3.2 onnx转om模型](#32-onnx转om模型) | ||
| 12 | -- [4 数据集预处理](#4-数据集预处理) | ||
| 13 | - - [4.1 数据集获取](#41-数据集获取) | ||
| 14 | - - [4.2 数据集预处理](#42-数据集预处理) | ||
| 15 | - - [4.3 生成数据集信息文件](#43-生成数据集信息文件) | ||
| 16 | -- [5 离线推理](#5-离线推理) | ||
| 17 | - - [5.1 benchmark工具概述](#51-benchmark工具概述) | ||
| 18 | - - [5.2 离线推理](#52-离线推理) | ||
| 19 | -- [6 精度对比](#6-精度对比) | ||
| 20 | - - [6.1 离线推理精度统计](#61-离线推理精度统计) | ||
| 21 | - - [6.2 开源精度](#62-开源精度) | ||
| 22 | - - [6.3 精度对比](#63-精度对比) | ||
| 23 | -- [7 性能对比](#7-性能对比) | ||
| 24 | - - [7.1 npu性能数据](#71-npu性能数据) | ||
| 25 | - - [7.2 基准性能数据](#72-基准性能数据) | ||
| 26 | - - [7.3 性能对比](#73-性能对比) | ||
| 27 | 2 | ||
| 28 | 3 | ||
| 4 | +- [概述](#ZH-CN_TOPIC_0000001172161501) | ||
| 29 | 5 | ||
| 30 | -## 1 模型概述 | 6 | + - [输入输出数据](#section540883920406) |
| 31 | 7 | ||
| 32 | -- **[论文地址](#11-论文地址)** | ||
| 33 | 8 | ||
| 34 | -- **[代码地址](#12-代码地址)** | ||
| 35 | 9 | ||
| 36 | -### 1.1 论文地址 | 10 | +- [推理环境准备](#ZH-CN_TOPIC_0000001126281702) |
| 37 | -[论文地址](https://arxiv.org/abs/1707.07012) | ||
| 38 | 11 | ||
| 39 | -### 1.2 代码地址 | 12 | +- [快速上手](#ZH-CN_TOPIC_0000001126281700) |
| 40 | -[代码地址](https://github.com/Cadene/pretrained-models.pytorch#nasnet) | ||
| 41 | -branch:master | ||
| 42 | -commit id：b8134c79b34d8baf88fe0815ce6776f28f54dbfe | ||
| 43 | -## 2 环境说明 | ||
| 44 | 13 | ||
| 45 | -- **[深度学习框架](#21-深度学习框架)** | 14 | + - [获取源码](#section4622531142816) |
| 15 | + - [准备数据集](#section183221994411) | ||
| 16 | + - [模型推理](#section741711594517) | ||
| 46 | 17 | ||
| 47 | -- **[python第三方库](#22-python第三方库)** | 18 | +- [模型推理性能&精度](#ZH-CN_TOPIC_0000001172201573) |
| 48 | 19 | ||
| 49 | -### 2.1 深度学习框架 | 20 | + ****** |
| 50 | -``` | ||
| 51 | -python3.7.5 | ||
| 52 | -CANN 5.1.RC1 | ||
| 53 | 21 | ||
| 54 | -pytorch >= 1.8.0 | 22 | + |
| 55 | -torchvision >= 0.9.0 | ||
| 56 | -onnx >= 1.8.0 | ||
| 57 | -``` | ||
| 58 | -### 2.2 python第三方库 | ||
| 59 | 23 | ||
| 60 | -``` | 24 | +# 概述<a name="ZH-CN_TOPIC_0000001172161501"></a> |
| 61 | -numpy == 1.20.3 | ||
| 62 | -munch == 2.5.1.dev12 | ||
| 63 | -tqdm == 4.36.1 | ||
| 64 | -scipy == 1.3.1 | ||
| 65 | -onnx-simplifier == 0.3.5 | ||
| 66 | -skl2onnx | ||
| 67 | -tqdm | ||
| 68 | -``` | ||
| 69 | 25 | ||
| 70 | -**说明：** | 26 | +SPNASNet100是通过网络搜索技术得到的精度与效率权衡的卷积神经网络，用于图像分类任务。 |
| 71 | -> X86架构：pytorch，torchvision和onnx可以通过官方下载whl包安装，其它可以通过pip3.7 install 包名 安装 | ||
| 72 | -> | ||
| 73 | -> Arm架构：pytorch，torchvision和onnx可以通过源码编译安装，其它可以通过pip3.7 install 包名 安装 | ||
| 74 | 27 | ||
| 75 | -## 3 模型转换 | ||
| 76 | 28 | ||
| 77 | -- **[pth转onnx模型](#31-pth转onnx模型)** | 29 | +- 参考实现： |
| 78 | 30 | ||
| 79 | -- **[onnx转om模型](#32-onnx转om模型)** | 31 | + ``` |
| 32 | + url=https://github.com/Cadene/pretrained-models.pytorch.git | ||
| 33 | + commit_id=b8134c79b34d8baf88fe0815ce6776f28f54dbfe | ||
| 34 | + code_path=ACL_PyTorch/contrib/cv/detection/Nasnetlarge | ||
| 35 | + model_name=Nasnetlarge | ||
| 36 | + ``` | ||
| 37 | + | ||
| 80 | 38 | ||
| 81 | -### 3.1 pth转onnx模型 | ||
| 82 | 39 | ||
| 83 | -1.下载pth权重文件 | ||
| 84 | -[nasnetlarge预训练pth权重文件](http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth) | ||
| 85 | -md5sum：78a73e51ee50997294f1f35a34b4de66 | ||
| 86 | -``` | ||
| 87 | -wget http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth | ||
| 88 | -``` | ||
| 89 | -安装onnx_tools | ||
| 90 | 40 | ||
| 91 | -``` | 41 | +## 输入输出数据<a name="section540883920406"></a> |
| 92 | -git clone https://gitee.com/zheng-wengang1/onnx_tools.git test/onnx_tools | ||
| 93 | -``` | ||
| 94 | 42 | ||
| 95 | -2.编写pth2onnx脚本nasnetlarge_pth2onnx.py | 43 | +- 输入数据 |
| 96 | 44 | ||
| 97 | - **说明：** | 45 | + | 输入数据 | 数据类型 | 大小 | 数据排布格式 | |
| 98 | ->目前ATC支持的onnx算子版本为11 | 46 | + | -------- | -------- | ------------------------ | ------------ | |
| 47 | + | input | RGB_FP32 | batchsize x 3 x 331x 331 | NCHW | | ||
| 99 | 48 | ||
| 100 | -3.执行pth2onnx脚本，生成onnx模型文件 | ||
| 101 | 49 | ||
| 102 | -``` | 50 | +- 输出数据 |
| 103 | -python3.7 nasnetlarge_pth2onnx.py nasnetalarge-a1897284.pth nasnetlarge.onnx | ||
| 104 | -``` | ||
| 105 | 51 | ||
| 106 | -4.使用onnxsim，生成onnx_sim模型文件 | 52 | + | 输出数据 | 数据类型 | 大小 | 数据排布格式 | |
| 53 | + | -------- | -------- | ---------------- | ------------ | | ||
| 54 | + | output1 | FP32 | Batchsize x 1000 | ND | | ||
| 107 | 55 | ||
| 108 | -``` | ||
| 109 | -python3.7 -m onnxsim --input-shape="1,3,331,331" nasnetlarge.onnx nasnetlarge_sim.onnx | ||
| 110 | -``` | ||
| 111 | 56 | ||
| 112 | -5.算子融合优化 | 57 | +# 推理环境准备<a name="ZH-CN_TOPIC_0000001126281702"></a> |
| 113 | 58 | ||
| 114 | -``` | 59 | +- 该模型需要以下插件与驱动 |
| 115 | -python3.7 merge_sliced.py nasnetlarge_sim.onnx nasnetlarge_sim_merge.onnx | ||
| 116 | -``` | ||
| 117 | 60 | ||
| 61 | + **表 1** 版本配套表 | ||
| 118 | 62 | ||
| 119 | -### 3.2 onnx转om模型 | 63 | + | 配套 | 版本 | 环境准备指导 | |
| 64 | + | ------------------------------------------------------------ | ------- | ------------------------------------------------------------ | | ||
| 65 | + | 固件与驱动 | 1.0.17 | [Pytorch框架推理环境准备](https://www.hiascend.com/document/detail/zh/ModelZoo/pytorchframework/pies) | | ||
| 66 | + | CANN | 6.0.RC1 | - | | ||
| 67 | + | Python | 3.7.5 | - | | ||
| 68 | + | PyTorch | 1.8.0 | - | | ||
| 69 | + | 说明：Atlas 300I Duo 推理卡请以CANN版本选择实际固件与驱动版本。 | \ | \ | | ||
| 120 | 70 | ||
| 121 | -1.设置环境变量 | ||
| 122 | 71 | ||
| 123 | -``` | 72 | +# 快速上手<a name="ZH-CN_TOPIC_0000001126281700"></a> |
| 124 | -source /usr/local/Ascend/ascend-toolkit/set_env.sh | ||
| 125 | -``` | ||
| 126 | 73 | ||
| 127 | -2.使用atc将onnx模型转换为om模型文件，工具使用方法可以参考[CANN V100R020C10 开发辅助工具指南 (推理) 01](https://support.huawei.com/enterprise/zh/doc/EDOC1100164868?idPath=23710424%7C251366513%7C22892968%7C251168373) | 74 | +## 获取源码<a name="section4622531142816"></a> |
| 128 | 75 | ||
| 129 | -``` | 76 | +1. 获取源码。 |
| 130 | -atc --framework=5 --model=nasnetlarge_sim_merge.onnx --input_format=NCHW --input_shape="image:1,3,331,331" --output=nasnetlarge_sim_bs1 --log=debug --soc_version=Ascend${chip_name} | ||
| 131 | -``` | ||
| 132 | 77 | ||
| 133 | -注：${chip_name}由“npu-smi info”命令查看处理器获得。 | 78 | + ``` |
| 134 | - | 79 | + git clone https://github.com/Cadene/pretrained-models.pytorch |
| 80 | + cd pretrained-models.pytorch | ||
| 81 | + python3 setup.py install | ||
| 82 | + ``` | ||
| 83 | + | ||
| 84 | +2. 安装依赖。 | ||
| 135 | 85 | ||
| 136 | -## 4 数据集预处理 | 86 | + ``` |
| 87 | + pip3 install -r requirements.txt | ||
| 88 | + ``` | ||
| 89 | + | ||
| 90 | + >**注**：在获取源码前建议先执行2.安装依赖，获取源码中python3 setup.py install 需要用到一些依赖文件。已在requirements.txt中列出。 | ||
| 137 | 91 | ||
| 138 | -- **[数据集获取](#41-数据集获取)** | 92 | +## 准备数据集<a name="section183221994411"></a> |
| 139 | -- **[数据集预处理](#42-数据集预处理)** | ||
| 140 | -- **[生成数据集信息文件](#43-生成数据集信息文件)** | ||
| 141 | 93 | ||
| 142 | -### 4.1 数据集获取 | 94 | +1. 获取原始数据集。（解压命令参考tar –xvf \*.tar与 unzip \*.zip） |
| 143 | -对于图像分类任务，该模型使用[ImageNet官网](http://www.image-net.org)的5万张验证集进行测试，图片与标签分别存放在/opt/npu/imagenet/val与/opt/npu/imagenet/val_label.txt | ||
| 144 | 95 | ||
| 145 | -### 4.2 数据集预处理 | 96 | + 本模型支持[ImageNet 50000](https://gitee.com/link?target=http%3A%2F%2Fwww.image-net.org)张图片的验证集。以ILSVRC2012为例，请用户需自行获取ILSVRC2012数据集，上传数据集到服务器任意目录并解压（如：/opt/npu/）。本模型将使用到ILSVRC2012_img_val.tar验证集及ILSVRC2012_devkit_t12.gz中的val_label.txt数据标签。目录结构如下： |
| 146 | -1.预处理脚本preprocess_img.py | ||
| 147 | 97 | ||
| 148 | -2.执行预处理脚本，生成数据集预处理后的bin文件 | 98 | + ``` |
| 99 | + ├── ImageNet | ||
| 100 | + ├── val | ||
| 101 | + ├── val_label.txt | ||
| 102 | + ``` | ||
| 149 | 103 | ||
| 150 | -``` | 104 | +2. 数据预处理，将原始数据集转换为模型输入的数据。 |
| 151 | -python3.7 preprocess_img.py /opt/npu/imagenet/val ./prep_dataset | ||
| 152 | -``` | ||
| 153 | 105 | ||
| 154 | -第一个参数为验证集路径，第二个参数为预处理后生成的二进制文件的存储路径 | 106 | + 将原始数据（.jpeg）转化为二进制文件（.bin）。 |
| 155 | 107 | ||
| 156 | -### 4.3 生成数据集信息文件 | 108 | + 执行preprocess_img.py脚本，完成预处理。 |
| 157 | -1.生成数据集信息文件脚本gen_dataset_info.py | ||
| 158 | 109 | ||
| 159 | -2.执行生成数据集信息脚本，生成数据集信息文件 | 110 | + ``` |
| 111 | + python3 preprocess_img.py ${datasets} ${prep_dataset} | ||
| 112 | + ``` | ||
| 160 | 113 | ||
| 161 | -``` | 114 | + + 参数说明： |
| 162 | -python3.7 gen_dataset_info.py bin ./prep_dataset ./nasnetlarge_prep_bin.info 331 331 | 115 | + + ${datasets}：原始数据验证集（.jpeg）所在路径,例如：/opt/npu/ImageNet/val。 |
| 163 | -``` | 116 | + + ${prep_dataset}：输出的二进制文件（.bin）所在路径。 |
| 164 | 117 | ||
| 165 | -第一个参数为模型输入的类型，第二个参数为生成的bin文件路径，第三个为输出的info文件，后面为宽高信息 | 118 | + 每个图像对应生成一个二进制文件。 |
| 166 | 119 | ||
| 167 | -## 5 离线推理 | ||
| 168 | 120 | ||
| 169 | -- **[benchmark工具概述](#51-benchmark工具概述)** | 121 | +## 模型推理<a name="section741711594517"></a> |
| 170 | -- **[离线推理](#52-离线推理)** | ||
| 171 | 122 | ||
| 172 | -### 5.1 benchmark工具概述 | 123 | +1. 模型转换。 |
| 173 | -benchmark工具为华为自研的模型推理工具，支持多种模型的离线推理，能够迅速统计出模型在Ascend310上的性能，支持真实数据和纯推理两种模式，配合后处理脚本，可以实现诸多模型的端到端过程，获取工具及使用方法可以参考[CANN V100R020C10 推理benchmark工具用户指南 01](https://support.huawei.com/enterprise/zh/doc/EDOC1100164874?idPath=23710424%7C251366513%7C22892968%7C251168373) | ||
| 174 | 124 | ||
| 175 | -### 5.2 离线推理 | 125 | + 使用PyTorch将模型权重文件.pth转换为.onnx文件，再使用ATC工具将.onnx文件转为离线推理模型文件.om文件。 |
| 176 | -1.设置环境变量 | ||
| 177 | 126 | ||
| 178 | -``` | 127 | + 1. 获取权重文件。 |
| 179 | -source /usr/local/Ascend/ascend-toolkit/set_env.sh | ||
| 180 | -``` | ||
| 181 | 128 | ||
| 182 | -2.执行离线推理 | 129 | + 从源码包中获取权重文件[nasnetalarge-a1897284.pth](http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth)。 |
| 183 | 130 | ||
| 184 | -``` | 131 | + ``` |
| 185 | - ./benchmark.x86_64 -model_type=vision -device_id=0 -batch_size=1 -om_path=nasnetlarge_sim_bs1.om -input_text_path=./nasnetlarge_prep_bin.info -input_width=331 -input_height=331 -output_binary=False -useDvpp=False | 132 | + wget http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth |
| 186 | -``` | 133 | + ``` |
| 187 | 134 | ||
| 188 | -输出结果默认保存在当前目录result/dumpOutput_devicex，模型只有一个名为class的输出，shape为bs * 1000，数据类型为FP32，对应1000个分类的预测结果，每个输入对应的输出对应一个_x.bin文件。 | 135 | + 2. 导出onnx文件。 |
| 189 | 136 | ||
| 190 | -## 6 精度对比 | 137 | + 1. 使用nasnetlarge_pth2onnx.py导出onnx文件。 |
| 191 | 138 | ||
| 192 | -- **[离线推理TopN精度](#61-离线推理TopN精度)** | 139 | + 运行nasnetlarge_pth2onnx.py脚本。 |
| 193 | -- **[开源TopN精度](#62-开源TopN精度)** | ||
| 194 | -- **[精度对比](#63-精度对比)** | ||
| 195 | 140 | ||
| 196 | -### 6.1 离线推理TopN精度统计 | 141 | + ``` |
| 142 | + python3 nasnetlarge_pth2onnx.py nasnetalarge-a1897284.pth nasnetlarge.onnx | ||
| 143 | + ``` | ||
| 197 | 144 | ||
| 198 | -调用imagenet_acc_eval.py脚本推理结果与label比对，可以获得Accuracy Top5数据。 | 145 | + 获得nasnetlarge.onnx文件。 |
| 199 | 146 | ||
| 200 | -``` | 147 | + 2. 优化ONNX文件。 |
| 201 | -python3.7 imagenet_acc_eval.py result/dumpOutput_device0/ /opt/npu/imagenet/val_label.txt ./ result.json | ||
| 202 | -``` | ||
| 203 | 148 | ||
| 204 | -第一个参数为benchmark输出目录，第二个为数据集配套标签，第三个是生成文件的保存目录，第四个是生成的文件名，其中存有推理的Top5精度。 | 149 | + 使用onnxsim，生成不同batch size的onnx_sim模型文件 |
| 205 | -对batch1和batch16的模型分别调用benchmark进行推理，并统计其Top5的精度。查看其输出结果： | ||
| 206 | 150 | ||
| 207 | -``` | 151 | + ``` |
| 208 | -{"title": "Overall statistical evaluation", "value": [{"key": "Number of images", "value": "50000"}, {"key": "Number of classes", "value": "1001"}, {"key": "Top1 accuracy", "value": "82.53%"}, {"key": "Top2 accuracy", "value": "91.12%"}, {"key": "Top3 accuracy", "value": "93.91%"}, {"key": "Top4 accuracy", "value": "95.2%"}, {"key": "Top5 accuracy", "value": "95.99%"}]} | 152 | + python3 -m onnxsim --input-shape="1,3,331,331" nasnetlarge.onnx nasnetlarge_sim1.onnx |
| 209 | -``` | 153 | + ``` |
| 154 | + | ||
| 155 | + 获得nasnetlarge_sim1.onnx文件。 | ||
| 156 | + | ||
| 157 | + 3. 不同bs算子融合优化。 | ||
| 210 | 158 | ||
| 211 | -经过对bs1与bs16的om测试，本模型batch1与batch16的精度没有差别，精度数据均如上。 | 159 | + ``` |
| 212 | - | 160 | + git clone https://gitee.com/ascend/msadvisor.git |
| 213 | -### 6.2 开源TopN精度 | 161 | + cd /msadvisor/auto-optimizer |
 | |||
| 214 | -[github开源代码仓精度](https://github.com/Cadene/pretrained-models.pytorch/blob/master/README.md) | 162 | + pip3 install -r requirements.txt |
| 215 | - | 163 | + python3 setup.py install |
| 216 | -Model Acc@1 Acc@5 | 164 | + cd ../.. |
| 217 | -NASNet-Large 82.566 96.086 | 165 | + python3 -m auto_optimizer opt nasnetlarge_sim1.onnx nasnetlarge_sim1_merge.onnx -k KnowledgeMergeConsecutiveSlice |
| 218 | - | 166 | + ``` |
| 219 | -### 6.3 精度对比 | 167 | + |
| 220 | -将得到的om离线模型推理TopN精度与该模型github代码仓上公布的精度对比，Top1与Top5精度均达标。 | 168 | + 获得nasnetlarge_sim1_merge.onnx文件。 |
| 221 | - **精度调试：** | ||
| 222 | ->没有遇到精度不达标的问题，故不需要进行精度调试 | ||
| 223 | - | ||
| 224 | -## 7 性能对比 | ||
| 225 | - | ||
| 226 | -- **[npu性能数据](#71-npu性能数据)** | ||
| 227 | -- **[基准性能数据](#72-基准性能数据)** | ||
| 228 | -- **[性能对比](#73-性能对比)** | ||
| 229 | - | ||
| 230 | -### 7.1 npu性能数据 | ||
| 231 | -benchmark工具在整个数据集上推理时会统计性能数据，存储于result/perf_vision_batchsize_bs_device_0.txt中。但是推理整个数据集较慢，如此测性能时需要确保benchmark独占device，使用npu-smi info可以查看device是否空闲。 | ||
| 232 | -除此之外，也可以使用benchmark纯推理功能测得性能数据，但是由于随机数不能模拟数据分布，纯推理功能测的有些模型性能数据可能不太准，benchmark纯推理功能测性能仅为快速获取大概的性能数据以便调试优化使用，可初步确认benchmark工具在整个数据集上推理时由于device也被其它推理任务使用了导致的性能不准的问题。 | ||
| 233 | -模型的性能以使用benchmark工具推理得到bs1与bs16的性能数据为准；对于使用benchmark工具测试的batch4，8，32，64的性能数据仅在README.md中作如下记录。 | ||
| 234 | -1.benchmark工具推理获得性能数据 | ||
| 235 | -使用benchmark工具的推理功能测试模型的推理性能，命令如下： | ||
| 236 | - | ||
| 237 | -``` | ||
| 238 | -./benchmark.x86_64 -model_type=vision -device_id=0 -batch_size=1 -om_path=nasnetlarge_sim1_merge.om -input_text_path=./nasnetlarge_prep_bin.info -input_width=331 -input_height=331 -output_binary=False -useDvpp=False | ||
| 239 | -``` | ||
| 240 | - | ||
| 241 | -benchmark工具进行推理后测得的性能数据存储于result/perf_vision_batchsize_x_device_0.txt，其中x为模型的batch_size。 | ||
| 242 | - | ||
| 243 | -batch1性能： | ||
| 244 | - | ||
| 245 | -``` | ||
| 246 | -[e2e] throughputRate: 83.5628, latency: 598352 | ||
| 247 | -[data read] throughputRate: 86.2813, moduleLatency: 11.59 | ||
| 248 | -[preprocess] throughputRate: 86.0436, moduleLatency: 11.622 | ||
| 249 | -[inference] throughputRate: 83.7909, Interface throughputRate: 113.609, moduleLatency: 11.51 | ||
| 250 | -[postprocess] throughputRate: 83.7924, moduleLatency: 11.9343 | ||
| 251 | -``` | ||
| 252 | - | ||
| 253 | -batch1的310P吞吐率为113.609fps | ||
| 254 | - | ||
| 255 | -batch16性能： | ||
| 256 | - | ||
| 257 | -``` | ||
| 258 | -[e2e] throughputRate: 70.9337, latency: 704884 | ||
| 259 | -[data read] throughputRate: 72.9184, moduleLatency: 13.714 | ||
| 260 | -[preprocess] throughputRate: 72.5527, moduleLatency: 13.7831 | ||
| 261 | -[inference] throughputRate: 71.1929, Interface throughputRate: 143.496, moduleLatency: 11.3749 | ||
| 262 | -[postprocess] throughputRate: 4.4509, moduleLatency: 224.674 | ||
| 263 | -``` | ||
| 264 | - | ||
| 265 | -batch16 310P吞吐率 ：143.496fps | ||
| 266 | - | ||
| 267 | -batch4性能： | ||
| 268 | - | ||
| 269 | -``` | ||
| 270 | -[e2e] throughputRate: 74.6563, latency: 669736 | ||
| 271 | -[data read] throughputRate: 76.6128, moduleLatency: 13.0526 | ||
| 272 | -[preprocess] throughputRate: 76.2606, moduleLatency: 13.1129 | ||
| 273 | -[inference] throughputRate: 74.9629, Interface throughputRate: 163.917, moduleLatency: 11.0111 | ||
| 274 | -[postprocess] throughputRate: 18.7421, moduleLatency: 53.3559 | ||
| 275 | -``` | ||
| 276 | - | ||
| 277 | -batch4 310P吞吐率：163.917fps | ||
| 278 | - | ||
| 279 | -batch8性能： | ||
| 280 | - | ||
| 281 | -``` | ||
| 282 | -[e2e] throughputRate: 74.2414, latency: 673479 | ||
| 283 | -[data read] throughputRate: 75.6931, moduleLatency: 13.2112 | ||
| 284 | -[preprocess] throughputRate: 75.41, moduleLatency: 13.2608 | ||
| 285 | -[inference] throughputRate: 74.5419, Interface throughputRate: 159.078, moduleLatency: 10.8562 | ||
| 286 | -[postprocess] throughputRate: 9.31917, moduleLatency: 107.306 | ||
| 287 | -``` | ||
| 288 | - | ||
| 289 | -batch8 310P吞吐率：159.078fps | ||
| 290 | - | ||
| 291 | -batch32性能： | ||
| 292 | - | ||
| 293 | -``` | ||
| 294 | -[e2e] throughputRate: 87.7866, latency: 569563 | ||
| 295 | -[data read] throughputRate: 90.4474, moduleLatency: 11.0561 | ||
| 296 | -[preprocess] throughputRate: 89.7505, moduleLatency: 11.142 | ||
| 297 | -[inference] throughputRate: 88.4308, Interface throughputRate: 134.228, moduleLatency: 10.2621 | ||
| 298 | -[postprocess] throughputRate: 2.76599, moduleLatency: 361.534 | ||
| 299 | -``` | ||
| 300 | - | ||
| 301 | -batch32 310P吞吐率：134.228fps | ||
| 302 | - | ||
| 303 | -batch64性能： | ||
| 304 | - | ||
| 305 | -``` | ||
| 306 | -[e2e] throughputRate: 76.969, latency: 649613 | ||
| 307 | -[data read] throughputRate: 79.2339, moduleLatency: 12.6209 | ||
| 308 | -[preprocess] throughputRate: 78.8025, moduleLatency: 12.6899 | ||
| 309 | -[inference] throughputRate: 77.487, Interface throughputRate: 113.828, moduleLatency: 11.7346 | ||
| 310 | -[postprocess] throughputRate: 1.21343, moduleLatency: 824.11 | ||
| 311 | -``` | ||
| 312 | - | ||
| 313 | -batch64 310P吞吐率：113.828fps | ||
| 314 | - | ||
| 315 | -### 7.2 基准性能数据 | ||
| 316 | -使用onnxruntime在T4服务器测试模型基准性能数据。 | ||
| 317 | - | ||
| 318 | -### 7.3 性能对比 | ||
| 319 | -| Throughput | 310 | 310P | T4 | 310P/310 | 310P/T4 | | ||
| 320 | -| ---------- | ------- | ------- | ------ | ----------- | ----------- | | ||
| 321 | -| bs1 | 120.712 | 122.147 | 31.746 | 1.011887799 | 3.847634348 | | ||
| 322 | -| bs4 | 131.478 | 165.939 | 39.468 | 1.262104687 | 4.204393433 | | ||
| 323 | -| bs8 | 123.164 | 155.383 | 38.319 | 1.261594297 | 4.054985777 | | ||
| 324 | -| bs16 | 120.508 | 143.066 | 30.119 | 1.187190892 | 4.750024901 | | ||
| 325 | -| bs32 | 98.14 | 136.04 | 29.357 | 1.386183004 | 4.633988487 | | ||
| 326 | -| bs64 | 114.13 | 114.448 | 41.057 | 1.002786296 | 2.787539275 | | ||
| 327 | -| | | | | | | | ||
| 328 | -| 最优batch | 131.478 | 165.939 | 41.057 | 1.262104687 | 4.041673771 | | ||
| 329 | 169 | ||
| 170 | + 3. 使用ATC工具将ONNX模型转OM模型。 | ||
| 330 | 171 | ||
| 172 | + 1. 配置环境变量。 | ||
| 331 | 173 | ||
| 174 | + ``` | ||
| 175 | + source /usr/local/Ascend/ascend-toolkit/set_env.sh | ||
| 176 | + ``` | ||
| 177 | + | ||
| 178 | + 2. 执行命令查看芯片名称（$\{chip\_name\}）。 | ||
| 179 | + | ||
| 180 | + ``` | ||
| 181 | + npu-smi info | ||
| 182 | + #该设备芯片名为Ascend310P3 （自行替换） | ||
| 183 | + 回显如下： | ||
| 184 | + +-------------------+-----------------+------------------------------------------------------+ | ||
| 185 | + | NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) | | ||
| 186 | + | Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) | | ||
| 187 | + +===================+=================+======================================================+ | ||
| 188 | + | 0 310P3 | OK | 15.8 42 0 / 0 | | ||
| 189 | + | 0 0 | 0000:82:00.0 | 0 1074 / 21534 | | ||
| 190 | + +===================+=================+======================================================+ | ||
| 191 | + | 1 310P3 | OK | 15.4 43 0 / 0 | | ||
| 192 | + | 0 1 | 0000:89:00.0 | 0 1070 / 21534 | | ||
| 193 | + +===================+=================+======================================================+ | ||
| 194 | + ``` | ||
| 195 | + | ||
| 196 | + 3. 执行ATC命令。 | ||
| 197 | + | ||
| 198 | + ``` | ||
| 199 | + atc --framework=5 --model=nasnetlarge_sim1_merge.onnx --input_format=NCHW --input_shape="image:1,3,331,331" --output=nasnetlarge_sim1_merge.onnx --log=debug --soc_version=Ascend${chip_name} | ||
| 200 | + ``` | ||
| 201 | + | ||
| 202 | + - 参数说明： | ||
| 203 | + | ||
| 204 | + - --model：为ONNX模型文件。 | ||
| 205 | + - --framework：5代表ONNX模型。 | ||
| 206 | + - --output：输出的OM模型。 | ||
| 207 | + - --input\_format：输入数据的格式。 | ||
| 208 | + - --input\_shape：输入数据的shape。 | ||
| 209 | + - --log：日志级别。 | ||
| 210 | + - --soc\_version：处理器型号。 | ||
| 211 | + | ||
| 212 | + | ||
| 213 | + 运行成功后生成nasnetlarge_sim1_merge.onnx模型文件。 | ||
| 214 | + | ||
| 215 | +2. 开始推理验证。 | ||
| 216 | + | ||
| 217 | + 1. 使用ais-infer工具进行推理。 | ||
| 218 | + | ||
| 219 | + ais-infer工具获取及使用方式请点击查看[[ais_infer 推理工具使用文档](https://gitee.com/ascend/tools/tree/master/ais-bench_workload/tool/ais_infer)] | ||
| 220 | + | ||
| 221 | + 2. 执行推理。 | ||
| 222 | + | ||
| 223 | + ``` | ||
| 224 | + python3 -m ais_bench --model nasnetlarge_sim1_merge.onnx --input ./prep_dataset --output ./result/ --output_dirname bs1 --outfmt TXT --batchsize 1 | ||
| 225 | + ``` | ||
| 226 | + | ||
| 227 | + - 参数说明： | ||
| 228 | + | ||
| 229 | + - --model：om模型。 | ||
| 230 | + - --input：预处理数据集路径。 | ||
| 231 | + - --output：推理结果所在路径。 | ||
| 232 | + - ----output_dirname：推理结果输出子文件夹。可选参数。与参数output搭配使用，单独使用无效。设置该值时输出结果将保存到 output/output_dirname文件夹中 。 | ||
| 233 | + - --outfmt：推理结果文件格式。 | ||
建议添加   | |||
| 234 | + - --batchsize：不同的batchsize。 | ||
| 235 | + | ||
| 236 | + 推理后的输出默认在当前目录result下。 | ||
| 237 | + | ||
| 238 | + >**说明：** | ||
| 239 | + >执行ais-infer工具请选择与运行环境架构相同的命令。参数详情请参见。 | ||
| 240 | + | ||
| 241 | + 3. 精度验证。 | ||
| 242 | + | ||
| 243 | + 调用imagenet_acc_eval.py脚本推理结果与label比对，可以获得Accuracy Top5数据。，结果保存在result.json中。 | ||
| 244 | + | ||
| 245 | + ``` | ||
| 246 | + python3 imagenet_acc_eval.py result/bs1/ /opt/npu/imagenet/val_label.txt ./ result.json | ||
| 247 | + ``` | ||
| 248 | + | ||
| 249 | + - 参数说明： | ||
| 250 | + | ||
| 251 | + - result/bs1/：为生成推理结果所在路径 | ||
| 252 | + - /opt/npu/imagenet/val_label.txt：标签数据。 | ||
| 253 | + - ./ ： 生成结果文件路径 | ||
| 254 | + - result.json：生成结果文件名称。 | ||
| 255 | + | ||
| 256 | +# 模型推理性能&精度<a name="ZH-CN_TOPIC_0000001172201573"></a> | ||
| 257 | + | ||
| 258 | +调用ACL接口推理计算，性能参考下列数据。 | ||
| 259 | + | ||
| 260 | +精度对比： | ||
| 261 | + | ||
| 262 | +| Model | Nasnetlarge | | ||
| 263 | +| --------- | -------------------------- | | ||
| 264 | +| 标杆精度 | top1：82.56% top5：96.08% | | ||
| 265 | +| 310P3精度 | top1：82.5% top5：96.02% | | ||
| 266 | + | ||
| 267 | +性能对比： | ||
| 268 | + | ||
| 269 | +| 芯片型号 | Batch Size | 数据集 | 性能 | | ||
| 270 | +| --------- | ---------------- | ---------- | --------------- | | ||
| 271 | +| 310P3 | 1 | ILSVRC2012 | 153.316 | | ||
| 272 | +| 310P3 | 4 | ILSVRC2012 | 175.744 | | ||
| 273 | +| 310P3 | 8 | ILSVRC2012 | 162.623 | | ||
| 274 | +| 310P3 | 16 | ILSVRC2012 | 146.502 | | ||
| 275 | +| 310P3 | 32 | ILSVRC2012 | 135.831 | | ||
| 276 | +| 310P3 | 64 | ILSVRC2012 | 113.459 | | ||
| @@ -1,61 +0,0 @@ | |||
| 1 | -""" | ||
| 2 | -Copyright 2020 Huawei Technologies Co., Ltd | ||
| 3 | - | ||
| 4 | -Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 5 | -you may not use this file except in compliance with the License. | ||
| 6 | -You may obtain a copy of the License at | ||
| 7 | - | ||
| 8 | - http://www.apache.org/licenses/LICENSE-2.0 | ||
| 9 | - | ||
| 10 | -Unless required by applicable law or agreed to in writing, software | ||
| 11 | -distributed under the License is distributed on an "AS IS" BASIS, | ||
| 12 | -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 13 | -See the License for the specific language governing permissions and | ||
| 14 | -limitations under the License. | ||
| 15 | -""" | ||
| 16 | -import os | ||
| 17 | -import sys | ||
| 18 | -from glob import glob | ||
| 19 | -import cv2 | ||
| 20 | - | ||
| 21 | - | ||
| 22 | -def get_bin_info(file_path, info_name, width, height): | ||
| 23 | - """get input bin data info""" | ||
| 24 | - bin_images = glob(os.path.join(file_path, '*.bin')) | ||
| 25 | - with open(info_name, 'w') as out_file: | ||
| 26 | - for index, img in enumerate(bin_images): | ||
| 27 | - content = ' '.join([str(index), img, width, height]) | ||
| 28 | - out_file.write(content) | ||
| 29 | - out_file.write('\n') | ||
| 30 | - | ||
| 31 | - | ||
| 32 | -def get_jpg_info(file_path, info_name): | ||
| 33 | - """get input jpg data info""" | ||
| 34 | - extensions = ['jpg', 'jpeg', 'JPG', 'JPEG'] | ||
| 35 | - image_names = [] | ||
| 36 | - for extension in extensions: | ||
| 37 | - image_names.append(glob(os.path.join(file_path, '*.' + extension))) | ||
| 38 | - with open(info_name, 'w') as out_file: | ||
| 39 | - for image_name in image_names: | ||
| 40 | - if image_name: | ||
| 41 | - for index, img in enumerate(image_name): | ||
| 42 | - img_cv = cv2.imread(img) | ||
| 43 | - shape = img_cv.shape | ||
| 44 | - img_w, img_h = shape[1], shape[0] | ||
| 45 | - content = ' '.join([str(index), img, str(img_w), str(img_h)]) | ||
| 46 | - out_file.write(content) | ||
| 47 | - out_file.write('\n') | ||
| 48 | - | ||
| 49 | - | ||
| 50 | -if __name__ == '__main__': | ||
| 51 | - input_type = sys.argv[1] | ||
| 52 | - input_path = sys.argv[2] | ||
| 53 | - input_name = sys.argv[3] | ||
| 54 | - if input_type == 'bin': | ||
| 55 | - input_width = sys.argv[4] | ||
| 56 | - input_height = sys.argv[5] | ||
| 57 | - assert len(sys.argv) == 6, 'The number of input parameters must be equal to 5' | ||
| 58 | - get_bin_info(input_path, input_name, input_width, input_height) | ||
| 59 | - elif input_type == 'jpg': | ||
| 60 | - assert len(sys.argv) == 4, 'The number of input parameters must be equal to 3' | ||
| 61 | - get_jpg_info(input_path, input_name) | ||
| @@ -1,120 +0,0 @@ | |||
| 1 | -""" | ||
| 2 | -Copyright 2020 Huawei Technologies Co., Ltd | ||
| 3 | - | ||
| 4 | -Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 5 | -you may not use this file except in compliance with the License. | ||
| 6 | -You may obtain a copy of the License at | ||
| 7 | - | ||
| 8 | - http://www.apache.org/licenses/LICENSE-2.0 | ||
| 9 | - | ||
| 10 | -Unless required by applicable law or agreed to in writing, software | ||
| 11 | -distributed under the License is distributed on an "AS IS" BASIS, | ||
| 12 | -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 13 | -See the License for the specific language governing permissions and | ||
| 14 | -limitations under the License. | ||
| 15 | -""" | ||
| 16 | -import sys | ||
| 17 | -import numpy as np | ||
| 18 | -from tqdm import tqdm | ||
| 19 | -sys.path.append('./test/onnx_tools') | ||
| 20 | -from OXInterface.OXInterface import OXGraph | ||
| 21 | - | ||
| 22 | - | ||
| 23 | -def mergeIntializer(oxgraph, initializer1, initializer2): | ||
| 24 | - """ | ||
| 25 | - :param oxgraph: input onnx graph | ||
| 26 | - :param initializer1: initializer need to be merged | ||
| 27 | - :param initializer2: initializer need to be merged | ||
| 28 | - :return: merged initializer | ||
| 29 | - """ | ||
| 30 | - merged_name = '{}_{}'.format(initializer1.get_name(), initializer2.get_name()) | ||
| 31 | - merged_data = np.append( | ||
| 32 | - initializer1.get_data(), | ||
| 33 | - initializer2.get_data(), | ||
| 34 | - ) | ||
| 35 | - try: | ||
| 36 | - return oxgraph.get_oxinitializer_by_name(merged_name) | ||
| 37 | - except RuntimeError: | ||
| 38 | - print("Insert a new initializer.") | ||
| 39 | - merged_initializer = oxgraph.add_initializer(merged_name, merged_data) | ||
| 40 | - return merged_initializer | ||
| 41 | - | ||
| 42 | - | ||
| 43 | -def mergeSlicedOp(oxgraph, slice_node1, slice_node2): | ||
| 44 | - """ | ||
| 45 | - :param oxgraph: input onnx graph | ||
| 46 | - :param slice_node1: slice node1 need to be merged | ||
| 47 | - :param slice_node2: slice node2 need to be merged | ||
| 48 | - :return: merged graph | ||
| 49 | - """ | ||
| 50 | - # modify slice_node1 -> merge_node | ||
| 51 | - slice_node1.input[1] = mergeIntializer( | ||
| 52 | - oxgraph, | ||
| 53 | - oxgraph.get_oxinitializer_by_name(slice_node1.input[1]), | ||
| 54 | - oxgraph.get_oxinitializer_by_name(slice_node2.input[1])).get_name() | ||
| 55 | - slice_node1.input[2] = mergeIntializer( | ||
| 56 | - oxgraph, | ||
| 57 | - oxgraph.get_oxinitializer_by_name(slice_node1.input[2]), | ||
| 58 | - oxgraph.get_oxinitializer_by_name(slice_node2.input[2])).get_name() | ||
| 59 | - slice_node1.input[3] = mergeIntializer( | ||
| 60 | - oxgraph, | ||
| 61 | - oxgraph.get_oxinitializer_by_name(slice_node1.input[3]), | ||
| 62 | - oxgraph.get_oxinitializer_by_name(slice_node2.input[3])).get_name() | ||
| 63 | - slice_node1.input[4] = mergeIntializer( | ||
| 64 | - oxgraph, | ||
| 65 | - oxgraph.get_oxinitializer_by_name(slice_node1.input[4]), | ||
| 66 | - oxgraph.get_oxinitializer_by_name(slice_node2.input[4])).get_name() | ||
| 67 | - oxgraph.remove_node(slice_node2.get_name()) | ||
| 68 | - return oxgraph | ||
| 69 | - | ||
| 70 | - | ||
| 71 | -def getContinuousOp(oxgraph, op_type='Slice'): | ||
| 72 | - """ | ||
| 73 | - :param oxgraph: input onnx graph | ||
| 74 | - :param op_type: op_type to be searched | ||
| 75 | - :return: continuous op list | ||
| 76 | - """ | ||
| 77 | - all_slice_ops = oxgraph.get_oxnode_by_op_type(op_type) | ||
| 78 | - flags = [-1] * len(all_slice_ops) | ||
| 79 | - res = [] | ||
| 80 | - for idx, node in enumerate(all_slice_ops): | ||
| 81 | - next_node = oxgraph.get_next_oxnode(node.get_name())[0] | ||
| 82 | - if next_node in all_slice_ops: | ||
| 83 | - next_idx = all_slice_ops.index(next_node) | ||
| 84 | - if flags[idx] == -1 and flags[next_idx] == -1: | ||
| 85 | - res.append([node, next_node]) | ||
| 86 | - flags[idx] = flags[next_idx] = len(res) - 1 | ||
| 87 | - elif flags[idx] != -1 and flags[next_idx] == -1: | ||
| 88 | - res[flags[idx]].append(next_node) | ||
| 89 | - flags[next_idx] = flags[idx] | ||
| 90 | - elif flags[idx] == -1 and flags[next_idx] != -1: | ||
| 91 | - res_idx = res[flags[next_idx]].index(next_node) | ||
| 92 | - res[flags[next_idx]].insert(res_idx, node) | ||
| 93 | - flags[idx] = flags[next_idx] | ||
| 94 | - else: | ||
| 95 | - res[flags[idx]] = res[flags[idx]] + res[flags[next_idx]] | ||
| 96 | - flags[next_idx] = flags[idx] | ||
| 97 | - flags = list(filter(lambda x: x != -1, flags)) | ||
| 98 | - uniq_flags = [] | ||
| 99 | - for f in flags: | ||
| 100 | - if f not in uniq_flags: | ||
| 101 | - uniq_flags.append(f) | ||
| 102 | - return [res[idx] for idx in uniq_flags] | ||
| 103 | - | ||
| 104 | - | ||
| 105 | -def main(model_path, out_model): | ||
| 106 | - """main function""" | ||
| 107 | - oxgraph = OXGraph(model_path) | ||
| 108 | - continuous_slice_nodes = getContinuousOp(oxgraph) | ||
| 109 | - for nodes in tqdm(continuous_slice_nodes): | ||
| 110 | - if len(nodes) > 2: | ||
| 111 | - raise NotImplementedError() | ||
| 112 | - slice_node1, slice_node2 = nodes | ||
| 113 | - oxgraph = mergeSlicedOp(oxgraph, slice_node1, slice_node2) | ||
| 114 | - oxgraph.save_new_model(out_model) | ||
| 115 | - | ||
| 116 | - | ||
| 117 | -if __name__ == '__main__': | ||
| 118 | - input_path = sys.argv[1] | ||
| 119 | - out_path = sys.argv[2] | ||
| 120 | - main(input_path, out_path) | ||
| @@ -19,6 +19,7 @@ import numpy as np | |||
| 19 | import pretrainedmodels | 19 | import pretrainedmodels |
| 20 | from pretrainedmodels import utils | 20 | from pretrainedmodels import utils |
| 21 | from PIL import Image | 21 | from PIL import Image |
| 22 | +from tqdm import tqdm | ||
| 22 | 23 | ||
| 23 | 24 | ||
| 24 | def init_transform(): | 25 | def init_transform(): |
| @@ -38,7 +39,7 @@ def preprecess(src_path, save_path): | |||
| 38 | """ | 39 | """ |
| 39 | in_files = os.listdir(src_path) | 40 | in_files = os.listdir(src_path) |
| 40 | transform = init_transform() | 41 | transform = init_transform() |
| 41 | - for out_file in in_files: | 42 | + for out_file in tqdm(in_files): |
| 42 | input_image = Image.open(src_path + '/' + out_file).convert('RGB') | 43 | input_image = Image.open(src_path + '/' + out_file).convert('RGB') |
| 43 | input_tensor = transform(input_image) | 44 | input_tensor = transform(input_image) |
| 44 | img = np.array(input_tensor).astype(np.float32) | 45 | img = np.array(input_tensor).astype(np.float32) |
| @@ -1,6 +1,10 @@ | |||
| 1 | -torch | 1 | +torch == 1.8.0 |
| 2 | -torchvision | 2 | +torchvision == 0.9.0 |
| 3 | -munch | 3 | +munch == 2.5.1.dev12 |
| 4 | tqdm | 4 | tqdm |
| 5 | -scipy | 5 | +scipy == 1.3.1 |
| 6 | -onnx-simplefier | 6 | +onnx |
| 7 | +numpy == 1.20.3 | ||
| 8 | +onnx-simplifier == 0.3.6 | ||
| 9 | +skl2onnx == 1.8.0 | ||
| 10 | +Pillow == 7.2.0 | ||
| @@ -1,67 +0,0 @@ | |||
| 1 | -datasets_path="/opt/npu" | ||
| 2 | - | ||
| 3 | -for para in $* | ||
| 4 | -do | ||
| 5 | - if [[ $para == --datasets_path* ]]; then | ||
| 6 | - datasets_path=`echo ${para#*=}` | ||
| 7 | - fi | ||
| 8 | -done | ||
| 9 | - | ||
| 10 | -arch=`uname -m` | ||
| 11 | -rm -rf ./prep_dataset | ||
| 12 | -python3.7 preprocess_img.py ${datasets_path}/imagenet/val ./prep_dataset | ||
| 13 | -if [ $? != 0 ]; then | ||
| 14 | - echo "fail!" | ||
| 15 | - exit -1 | ||
| 16 | -fi | ||
| 17 | -python3.7 gen_dataset_info.py bin ./prep_dataset ./nasnetlarge_prep_bin.info 331 331 | ||
| 18 | -if [ $? != 0 ]; then | ||
| 19 | - echo "fail!" | ||
| 20 | - exit -1 | ||
| 21 | -fi | ||
| 22 | -source /usr/local/Ascend/ascend-toolkit/set_env.sh | ||
| 23 | -rm -rf result/dumpOutput_device0 | ||
| 24 | -./benchmark.${arch} -model_type=vision -device_id=0 -batch_size=1 -om_path=nasnetlarge_sim_bs1.om -input_text_path=./nasnetlarge_prep_bin.info -input_width=331 -input_height=331 -output_binary=False -useDvpp=False | ||
| 25 | -if [ $? != 0 ]; then | ||
| 26 | - echo "fail!" | ||
| 27 | - exit -1 | ||
| 28 | -fi | ||
| 29 | -rm -rf result/dumpOutput_device1 | ||
| 30 | -./benchmark.${arch} -model_type=vision -device_id=0 -batch_size=16 -om_path=nasnetlarge_sim_bs16.om -input_text_path=./nasnetlarge_prep_bin.info -input_width=331 -input_height=331 -output_binary=False -useDvpp=False | ||
| 31 | -if [ $? != 0 ]; then | ||
| 32 | - echo "fail!" | ||
| 33 | - exit -1 | ||
| 34 | -fi | ||
| 35 | -python3.7 imagenet_acc_eval.py result/dumpOutput_device0/ ${datasets_path}/imagenet/val_label.txt ./ result_bs1.json | ||
| 36 | -if [ $? != 0 ]; then | ||
| 37 | - echo "fail!" | ||
| 38 | - exit -1 | ||
| 39 | -fi | ||
| 40 | -python3.7 imagenet_acc_eval.py result/dumpOutput_device0/ ${datasets_path}/imagenet/val_label.txt ./ result_bs16.json | ||
| 41 | -if [ $? != 0 ]; then | ||
| 42 | - echo "fail!" | ||
| 43 | - exit -1 | ||
| 44 | -fi | ||
| 45 | -echo "====accuracy data====" | ||
| 46 | -python3.7 test/parse.py result_bs1.json | ||
| 47 | -if [ $? != 0 ]; then | ||
| 48 | - echo "fail!" | ||
| 49 | - exit -1 | ||
| 50 | -fi | ||
| 51 | -python3.7 test/parse.py result_bs16.json | ||
| 52 | -if [ $? != 0 ]; then | ||
| 53 | - echo "fail!" | ||
| 54 | - exit -1 | ||
| 55 | -fi | ||
| 56 | -echo "====310 performance data====" | ||
| 57 | -python3.7 test/parse.py result/perf_vision_batchsize_1_device_0.txt | ||
| 58 | -if [ $? != 0 ]; then | ||
| 59 | - echo "fail!" | ||
| 60 | - exit -1 | ||
| 61 | -fi | ||
| 62 | -python3.7 test/parse.py result/perf_vision_batchsize_16_device_1.txt | ||
| 63 | -if [ $? != 0 ]; then | ||
| 64 | - echo "fail!" | ||
| 65 | - exit -1 | ||
| 66 | -fi | ||
| 67 | -echo "success" | ||
| @@ -1,34 +0,0 @@ | |||
| 1 | -""" | ||
| 2 | -Copyright 2021 Huawei Technologies Co., Ltd | ||
| 3 | - | ||
| 4 | -Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 5 | -you may not use this file except in compliance with the License. | ||
| 6 | -You may obtain a copy of the License at | ||
| 7 | - | ||
| 8 | - http://www.apache.org/licenses/LICENSE-2.0 | ||
| 9 | - | ||
| 10 | -Unless required by applicable law or agreed to in writing, software | ||
| 11 | -distributed under the License is distributed on an "AS IS" BASIS, | ||
| 12 | -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 13 | -See the License for the specific language governing permissions and | ||
| 14 | -limitations under the License. | ||
| 15 | -""" | ||
| 16 | -import sys | ||
| 17 | -import json | ||
| 18 | -import re | ||
| 19 | - | ||
| 20 | - | ||
| 21 | -if __name__ == '__main__': | ||
| 22 | - if sys.argv[1].endswith('.json'): | ||
| 23 | - result_json = sys.argv[1] | ||
| 24 | - with open(result_json, 'r') as f: | ||
| 25 | - content = f.read() | ||
| 26 | - tops = [i.get('value') for i in json.loads(content).get('value') if 'Top' in i.get('key')] | ||
| 27 | - print('om {} top1:{} top5:{}'.format(result_json.split('_')[1].split('.')[0], tops[0], tops[4])) | ||
| 28 | - elif sys.argv[1].endswith('.txt'): | ||
| 29 | - result_txt = sys.argv[1] | ||
| 30 | - with open(result_txt, 'r') as f: | ||
| 31 | - content = f.read() | ||
| 32 | - txt_data_list = [i.strip() for i in re.findall(r':(.*?),', content.replace('\n', ',') + ',')] | ||
| 33 | - fps = float(txt_data_list[7].replace('samples/s', '')) * 4 | ||
| 34 | - print('310 bs{} fps:{}'.format(result_txt.split('_')[3], fps)) | ||
| @@ -1,23 +0,0 @@ | |||
| 1 | - | ||
| 2 | -trtexec --onnx=nasnetlarge_sim.onnx --fp16 --shapes=image:1x3x331x331 --threads > nasnetlarge_bs1.log | ||
| 3 | - | ||
| 4 | -perf_str=`grep "GPU.* mean.*ms$" nasnetlarge_bs1.log` | ||
| 5 | -if [ -n "$perf_str" ]; then | ||
| 6 | - perf_num=`echo $perf_str | awk -F' ' '{print $16}'` | ||
| 7 | -else | ||
| 8 | - perf_str=`grep "mean.*ms$" nasnetlarge_bs1.log` | ||
| 9 | - perf_num=`echo $perf_str | awk -F' ' '{print $4}'` | ||
| 10 | -fi | ||
| 11 | -awk 'BEGIN{printf "benchmark bs1 fps:%.3f\n", 1000*1/('$perf_num'/1)}' | ||
| 12 | - | ||
| 13 | -trtexec --onnx=nasnetlarge_sim.onnx --fp16 --shapes=image:16x3x331x331 --threads > nasnetlarge_bs16.log | ||
| 14 | - | ||
| 15 | -perf_str=`grep "GPU.* mean.*ms$" nasnetlarge_bs16.log` | ||
| 16 | -if [ -n "$perf_str" ]; then | ||
| 17 | - perf_num=`echo $perf_str | awk -F' ' '{print $16}'` | ||
| 18 | -else | ||
| 19 | - perf_str=`grep "mean.*ms$" nasnetlarge_bs16.log` | ||
| 20 | - perf_num=`echo $perf_str | awk -F' ' '{print $4}'` | ||
| 21 | -fi | ||
| 22 | -awk 'BEGIN{printf "benchmark bs16 fps:%.3f\n", 1000*1/('$perf_num'/16)}' | ||
| 23 | - | ||
| @@ -1,20 +0,0 @@ | |||
| 1 | -source /usr/local/Ascend/ascend-toolkit/set_env.sh | ||
| 2 | -rm -rf nasnetlarge.onnx | ||
| 3 | - | ||
| 4 | -python3.7 nasnetlarge_pth2onnx.py nasnetalarge-a1897284.pth nasnetlarge.onnx | ||
| 5 | - | ||
| 6 | -python3.7 -m onnxsim --input-shape="1,3,331,331" nasnetlarge.onnx nasnetlarge_sim_bs1.onnx | ||
| 7 | -python3.7 -m onnxsim --input-shape="16,3,331,331" nasnetlarge.onnx nasnetlarge_sim_bs16.onnx | ||
| 8 | - | ||
| 9 | -python merge_sliced.py nasnetlarge_sim_bs1.onnx nasnetlarge_sim_merge_bs1.onnx | ||
| 10 | -python merge_sliced.py nasnetlarge_sim_bs16.onnx nasnetlarge_sim_merge_bs16.onnx | ||
| 11 | - | ||
| 12 | -rm -rf nasnetlarge_bs1.om nasnetlarge_bs16.om | ||
| 13 | -atc --framework=5 --model=nasnetlarge_sim_merge_bs1.onnx --input_format=NCHW --input_shape="image:1,3,331,331" --output=nasnetlarge_sim_bs1 --log=debug --soc_version=Ascend310 --auto_tune_mode="RL,GA" | ||
| 14 | -atc --framework=5 --model=nasnetlarge_sim_merge_bs16.onnx --input_format=NCHW --input_shape="image:16,3,331,331" --output=nasnetlarge_sim_bs16 --log=debug --soc_version=Ascend310 --auto_tune_mode="RL,GA" | ||
| 15 | - | ||
| 16 | -if [ -f "nasnetlarge_sim_bs1.om" ] && [ -f "nasnetlarge_sim_bs16.om" ]; then | ||
| 17 | - echo "success" | ||
| 18 | -else | ||
| 19 | - echo "fail!" | ||
| 20 | -fi | ||
| @@ -1,39 +0,0 @@ | |||
| 1 | -环境准备： | ||
| 2 | - | ||
| 3 | -1.数据集路径 | ||
| 4 | -通用的数据集统一放在/root/datasets/或/opt/npu/ | ||
| 5 | -本模型数据集放在/opt/npu/ | ||
| 6 | - | ||
| 7 | -2.进入工作目录 | ||
| 8 | -cd nasnetlarge | ||
| 9 | - | ||
| 10 | -3.安装必要的依赖，测试环境可能已经安装其中的一些不同版本的库了，故手动测试时不推荐使用该命令安装 | ||
| 11 | -pip3.7 install -r requirements.txt | ||
| 12 | - | ||
| 13 | -4.获取模型代码 | ||
| 14 | -[代码地址](https://github.com/Cadene/pretrained-models.pytorch#nasnet) | ||
| 15 | -branch:master | ||
| 16 | -commit id：b8134c79b34d8baf88fe0815ce6776f28f54dbfe | ||
| 17 | - | ||
| 18 | -5.本模型代码需要安装 | ||
| 19 | - | ||
| 20 | -cd pretrained-models.pytorch | ||
| 21 | - | ||
| 22 | -python3.7 setup.py install | ||
| 23 | - | ||
| 24 | -6.获取权重文件 | ||
| 25 | -wget http://data.lip6.fr/cadene/pretrainedmodels/nasnetalarge-a1897284.pth | ||
| 26 | - | ||
| 27 | -7.获取benchmark工具 | ||
| 28 | -将benchmark.x86_64 benchmark.aarch64 放在当前目录 | ||
| 29 | -获取onnx优化工具： | ||
| 30 | -cd test | ||
| 31 | -git clone https://gitee.com/zheng-wengang1/onnx_tools.git ./onnx_tools | ||
| 32 | -cd .. | ||
| 33 | - | ||
| 34 | -8.310上执行，执行时确保device空闲 | ||
| 35 | -bash test/pth2om.sh | ||
| 36 | -bash test/eval_acc_perf.sh | ||
| 37 | - | ||
| 38 | -9.在gpu环境上将onnx文件与perform_benchmark.sh放在同一目录 | ||
| 39 | -然后执行bash perform_benchmark.sh，执行时确保gpu空闲 | ||
这里
/msadvisor路径是不对的