【深圳大学】【高校贡献】【PyTorch离线推理】【SSD300】-初次提交-ModelZoo-PyTorch-AtomGit

已合并

【深圳大学】【高校贡献】【PyTorch离线推理】【SSD300】-初次提交 #1925

AtomGit-Bot创建于 2022年9月23日从refs/pull/1925/head 合入到 master

【深圳大学】【高校贡献】【PyTorch离线推理】【SSD300】-初次提交 #1925

已合并

AtomGit-Bot创建于 2022年9月23日

从refs/pull/1925/head合入到master

共 10 个文件变更+1419-1058

MACL_PyTorch/contrib/cv/detection/SSD/README.md+311-865

	@@ -1,873 +1,319 @@
1	-# 基于开源mmdetection预训练的SSD Onnx模型端到端推理指导	1	+# SSD300模型-推理指导
2	-- [1 模型概述](#1-模型概述)
3	- - [1.1 论文地址](#11-论文地址)
4	- - [1.2 代码地址](#12-代码地址)
5	-- [2 环境说明](#2-环境说明)
6	- - [2.1 深度学习框架](#21-深度学习框架)
7	- - [2.2 python第三方库](#22-python第三方库)
8	-- [3 模型转换](#3-模型转换)
9	- - [3.1 pth转onnx模型](#31-pth转onnx模型)
10	- - [3.2 onnx转om模型](#32-onnx转om模型)
11	-- [4 数据集预处理](#4-数据集预处理)
12	- - [4.1 数据集获取](#41-数据集获取)
13	- - [4.2 数据集预处理](#42-数据集预处理)
14	- - [4.3 生成数据集信息文件](#43-生成数据集信息文件)
15	-- [5 离线推理](#5-离线推理)
16	- - [5.1 benchmark工具概述](#51-benchmark工具概述)
17	- - [5.2 离线推理](#52-离线推理)
18	-- [6 精度对比](#6-精度对比)
19	- - [6.1 离线推理精度统计](#61-离线推理精度统计)
20	- - [6.2 开源精度](#62-开源精度)
21	- - [6.3 精度对比](#63-精度对比)
22	-- [7 性能对比](#7-性能对比)
23	- - [7.1 npu性能数据](#71-npu性能数据)
24	- - [7.2 基准性能数据](#72-基准性能数据)
25	- - [7.3 性能对比](#73-性能对比)
26		2
27	-## 1 模型概述
28		3
29	-- [论文地址](#11-论文地址)	4	+- [概述](#ZH-CN_TOPIC_0000001172161501)
30		5
31	-- [代码地址](#12-代码地址)	6	+- [推理环境准备](#ZH-CN_TOPIC_0000001126281702)
		7	+
		8	+- [快速上手](#ZH-CN_TOPIC_0000001126281700)
		9	+
		10	+ - [获取源码](#section4622531142816)
		11	+ - [准备数据集](#section183221994411)
		12	+ - [模型推理](#section741711594517)
		13	+
		14	+- [模型推理性能](#ZH-CN_TOPIC_0000001172201573)
		15	+
		16	+- [配套环境](#ZH-CN_TOPIC_0000001126121892)
		17	+
		18	+
		19	+
		20	+
		21	+
		22	+# 概述<a name="ZH-CN_TOPIC_0000001172161501"></a>
32		23
33	-### 1.1 论文地址
34	-[SSD论文](https://arxiv.org/abs/1512.02325)
35	SSD将detection转化为regression的思路，可以一次完成目标定位与分类。该算法基于Faster RCNN中的Anchor，提出了相似的Prior box；该算法修改了传统的VGG16网络：将VGG16的FC6和FC7层转化为卷积层，去掉所有的Dropout层和FC8层。同时加入基于特征金字塔的检测方式，在不同感受野的feature map上预测目标。	24	SSD将detection转化为regression的思路，可以一次完成目标定位与分类。该算法基于Faster RCNN中的Anchor，提出了相似的Prior box；该算法修改了传统的VGG16网络：将VGG16的FC6和FC7层转化为卷积层，去掉所有的Dropout层和FC8层。同时加入基于特征金字塔的检测方式，在不同感受野的feature map上预测目标。
36		25
37	-### 1.2 代码地址
38	-[mmdetection框架SSD代码](https://github.com/open-mmlab/mmdetection/tree/master/configs/ssd)
39	-
40	-## 2 环境说明
41	-
42	-- [深度学习框架](#21-深度学习框架)
43	-
44	-- [python第三方库](#22-python第三方库)
45	-
46	-### 2.1 深度学习框架
47	-```
48	-onnx==1.7.0
49	-torch==1.8.1
50	-torchvision==0.9.1
51	-```
52	-
53	-### 2.2 python第三方库
54	-
55	-```
56	-numpy==1.18.5
57	-opencv-python==4.2.0.34
58	-mmdet==2.8.0
59	-mmcv-full==1.2.4
60	-mmpycocotools==12.0.3
61	-```
62	-
63	-说明：
64	-> X86架构：opencv,pytorch,torchvision和onnx可以通过官方下载whl包安装，其它可以通过pip3.7 install 包名安装
65	->
66	-> Arm架构：opencv,pytorch,torchvision和onnx可以通过源码编译安装，其它可以通过pip3.7 install 包名安装
67	-
68	-## 3 模型转换
69	-
70	-- [pth转onnx模型](#31-pth转onnx模型)
71	-
72	-- [onnx转om模型](#32-onnx转om模型)
73	-
74	-atc暂不支持动态shape小算子，可以使用大颗粒算子替换这些小算子规避，这些小算子可以在转onnx时的verbose打印中找到其对应的python代码，从而根据功能用大颗粒算子替换，onnx能推导出变量正确的shape与算子属性正确即可，变量实际的数值无关紧要，因此这些大算子函数的功能实现无关紧要，因包含自定义算子需要去掉对onnx模型的校验。
75	-
76	-### 3.1 pth转onnx模型
77	-
78	-1.下载pth权重文件
79	-[SSD300预训练pth权重文件](http://download.openmmlab.com/mmdetection/v2.0/ssd/ssd300_coco/ssd300_coco_20200307-a92d2092.pth)
80	-文件md5sum: 496e671b20bda2b4f53051f298947bba
81	-
82	-```
83	-wget http://download.openmmlab.com/mmdetection/v2.0/ssd/ssd300_coco/ssd300_coco_20200307-a92d2092.pth
84	-```
85	-
86	-2.mmdetection源码安装
87	-
88	-```shell
89	-git clone https://github.com/open-mmlab/mmdetection.git
90	-cd mmdetection
91	-git reset --hard a21eb25535f31634cef332b09fc27d28956fb24b
92	-pip install -r requirements/build.txt
93	-pip install -v -e .
94	-```
95	-
96	- 说明：
97	-> 安装所需的依赖说明请参考mmdetection/docs/get_started.md
98	->
99	-
100	-3.转原始onnx
101	-
102	-```shell
103	-python3.7 mmdetection/tools/pytorch2onnx.py mmdetection/configs/ssd/ssd300_coco.py ./ssd300_coco_20200307-a92d2092.pth --output-file=ssd_300_coco.onnx --shape=300 --verify --show --mean 123.675 116.28 103.53 --std 1 1 1
104	-```
105	-4.修改mmdetection代码，参见ssd_mmdetection.diff
106	-
107	-~~~python
108	-diff --git a/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py b/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py
109	-index e9eb3579..e8b53dce 100644
110	---- a/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py
111	-+++ b/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py
112	-@@ -168,8 +168,13 @@ def delta2bbox(rois,
113	- [0.0000, 0.3161, 4.1945, 0.6839],
114	- [5.0000, 5.0000, 5.0000, 5.0000]])
115	- """
116	-- means = deltas.new_tensor(means).view(1, -1).repeat(1, deltas.size(1) // 4)
117	-- stds = deltas.new_tensor(stds).view(1, -1).repeat(1, deltas.size(1) // 4)
118	-+ # fix shape for means and stds for onnx
119	-+ if torch.onnx.is_in_onnx_export():
120	-+ means = deltas.new_tensor(means).view(1, -1).repeat(1, deltas.size(1).numpy() // 4)
121	-+ stds = deltas.new_tensor(stds).view(1, -1).repeat(1, deltas.size(1).numpy() // 4)
122	-+ else:
123	-+ means = deltas.new_tensor(means).view(1, -1).repeat(1, deltas.size(1) // 4)
124	-+ stds = deltas.new_tensor(stds).view(1, -1).repeat(1, deltas.size(1) // 4)
125	- denorm_deltas = deltas * stds + means
126	- dx = denorm_deltas[:, 0::4]
127	- dy = denorm_deltas[:, 1::4]
128	-@@ -178,12 +183,22 @@ def delta2bbox(rois,
129	- max_ratio = np.abs(np.log(wh_ratio_clip))
130	- dw = dw.clamp(min=-max_ratio, max=max_ratio)
131	- dh = dh.clamp(min=-max_ratio, max=max_ratio)
132	-- # Compute center of each roi
133	-- px = ((rois[:, 0] + rois[:, 2]) * 0.5).unsqueeze(1).expand_as(dx)
134	-- py = ((rois[:, 1] + rois[:, 3]) * 0.5).unsqueeze(1).expand_as(dy)
135	-- # Compute width/height of each roi
136	-- pw = (rois[:, 2] - rois[:, 0]).unsqueeze(1).expand_as(dw)
137	-- ph = (rois[:, 3] - rois[:, 1]).unsqueeze(1).expand_as(dh)
138	-+ # improve gather performance on NPU
139	-+ if torch.onnx.is_in_onnx_export():
140	-+ rois_perf = rois.permute(1, 0)
141	-+ # Compute center of each roi
142	-+ px = ((rois_perf[0, :] + rois_perf[2, :]) * 0.5).unsqueeze(1).expand_as(dx)
143	-+ py = ((rois_perf[1, :] + rois_perf[3, :]) * 0.5).unsqueeze(1).expand_as(dy)
144	-+ # Compute width/height of each roi
145	-+ pw = (rois_perf[2, :] - rois_perf[0, :]).unsqueeze(1).expand_as(dw)
146	-+ ph = (rois_perf[3, :] - rois_perf[1, :]).unsqueeze(1).expand_as(dh)
147	-+ else:
148	-+ # Compute center of each roi
149	-+ px = ((rois[:, 0] + rois[:, 2]) * 0.5).unsqueeze(1).expand_as(dx)
150	-+ py = ((rois[:, 1] + rois[:, 3]) * 0.5).unsqueeze(1).expand_as(dy)
151	-+ # Compute width/height of each roi
152	-+ pw = (rois[:, 2] - rois[:, 0]).unsqueeze(1).expand_as(dw)
153	-+ ph = (rois[:, 3] - rois[:, 1]).unsqueeze(1).expand_as(dh)
154	- # Use exp(network energy) to enlarge/shrink each roi
155	- gw = pw * dw.exp()
156	- gh = ph * dh.exp()
157	-diff --git a/mmdet/core/post_processing/bbox_nms.py b/mmdet/core/post_processing/bbox_nms.py
158	-index 463fe2e4..1f8ad5a8 100644
159	---- a/mmdet/core/post_processing/bbox_nms.py
160	-+++ b/mmdet/core/post_processing/bbox_nms.py
161	-@@ -4,6 +4,57 @@ from mmcv.ops.nms import batched_nms
162	- from mmdet.core.bbox.iou_calculators import bbox_overlaps
163	-
164	-
165	-+class BatchNMSOp(torch.autograd.Function):
166	-+ @staticmethod
167	-+ def forward(ctx, bboxes, scores, score_threshold, iou_threshold, max_size_per_class, max_total_size):
168	-+ """
169	-+ boxes (torch.Tensor): boxes in shape (batch, N, C, 4).
170	-+ scores (torch.Tensor): scores in shape (batch, N, C).
171	-+ return:
172	-+ nmsed_boxes: (1, N, 4)
173	-+ nmsed_scores: (1, N)
174	-+ nmsed_classes: (1, N)
175	-+ nmsed_num: (1,)
176	-+ """
177	-+
178	-+ # Phony implementation for onnx export
179	-+ nmsed_boxes = bboxes[:, :max_total_size, 0, :]
180	-+ nmsed_scores = scores[:, :max_total_size, 0]
181	-+ nmsed_classes = torch.arange(max_total_size, dtype=torch.long)
182	-+ nmsed_num = torch.Tensor([max_total_size])
183	-+
184	-+ return nmsed_boxes, nmsed_scores, nmsed_classes, nmsed_num
185	-+
186	-+ @staticmethod
187	-+ def symbolic(g, bboxes, scores, score_thr, iou_thr, max_size_p_class, max_t_size):
188	-+ nmsed_boxes, nmsed_scores, nmsed_classes, nmsed_num = g.op('BatchMultiClassNMS',
189	-+ bboxes, scores, score_threshold_f=score_thr, iou_threshold_f=iou_thr,
190	-+ max_size_per_class_i=max_size_p_class, max_total_size_i=max_t_size, outputs=4)
191	-+ return nmsed_boxes, nmsed_scores, nmsed_classes, nmsed_num
192	-+
193	-+def batch_nms_op(bboxes, scores, score_threshold, iou_threshold, max_size_per_class, max_total_size):
194	-+ """
195	-+ boxes (torch.Tensor): boxes in shape (N, 4).
196	-+ scores (torch.Tensor): scores in shape (N, ).
197	-+ """
198	-+
199	-+ if bboxes.dtype == torch.float32:
200	-+ bboxes = bboxes.reshape(1, bboxes.shape[0].numpy(), -1, 4).half()
201	-+ scores = scores.reshape(1, scores.shape[0].numpy(), -1).half()
202	-+ else:
203	-+ bboxes = bboxes.reshape(1, bboxes.shape[0].numpy(), -1, 4)
204	-+ scores = scores.reshape(1, scores.shape[0].numpy(), -1)
205	-+
206	-+ nmsed_boxes, nmsed_scores, nmsed_classes, nmsed_num = BatchNMSOp.apply(bboxes, scores,
207	-+ score_threshold, iou_threshold, max_size_per_class, max_total_size)
208	-+ nmsed_boxes = nmsed_boxes.float()
209	-+ nmsed_scores = nmsed_scores.float()
210	-+ nmsed_classes = nmsed_classes.long()
211	-+ dets = torch.cat((nmsed_boxes.reshape((max_total_size, 4)), nmsed_scores.reshape((max_total_size, 1))), -1)
212	-+ labels = nmsed_classes.reshape((max_total_size, ))
213	-+ return dets, labels
214	-+
215	-+
216	- def multiclass_nms(multi_bboxes,
217	- multi_scores,
218	- score_thr,
219	-@@ -36,13 +87,25 @@ def multiclass_nms(multi_bboxes,
220	- if multi_bboxes.shape[1] > 4:
221	- bboxes = multi_bboxes.view(multi_scores.size(0), -1, 4)
222	- else:
223	-- bboxes = multi_bboxes[:, None].expand(
224	-- multi_scores.size(0), num_classes, 4)
225	-+ # export expand operator to onnx more nicely
226	-+ if torch.onnx.is_in_onnx_export:
227	-+ bbox_shape_tensor = torch.ones(multi_scores.size(0), num_classes, 4)
228	-+ bboxes = multi_bboxes[:, None].expand_as(bbox_shape_tensor)
229	-+ else:
230	-+ bboxes = multi_bboxes[:, None].expand(
231	-+ multi_scores.size(0), num_classes, 4)
232	-+
233	-
234	- scores = multi_scores[:, :-1]
235	- if score_factors is not None:
236	- scores = scores * score_factors[:, None]
237	-
238	-+ # npu
239	-+ if torch.onnx.is_in_onnx_export():
240	-+ dets, labels = batch_nms_op(bboxes, scores, score_thr, nms_cfg.get("iou_threshold"), max_num, max_num)
241	-+ return dets, labels
242	-+
243	-+ # cpu and gpu
244	- labels = torch.arange(num_classes, dtype=torch.long)
245	- labels = labels.view(1, -1).expand_as(scores)
246	-
247	-@@ -53,6 +116,8 @@ def multiclass_nms(multi_bboxes,
248	- # remove low scoring boxes
249	- valid_mask = scores > score_thr
250	- inds = valid_mask.nonzero(as_tuple=False).squeeze(1)
251	-+ # vals, inds = torch.topk(scores, 1000)
252	-+
253	- bboxes, scores, labels = bboxes[inds], scores[inds], labels[inds]
254	- if inds.numel() == 0:
255	- if torch.onnx.is_in_onnx_export():
256	-@@ -76,6 +141,7 @@ def multiclass_nms(multi_bboxes,
257	- return dets, labels[keep]
258	-
259	-
260	-+
261	- def fast_nms(multi_bboxes,
262	- multi_scores,
263	- multi_coeffs,
264	-
265	-~~~
266	-
267	-修改依据：
268	-
269	-> 1. 在bbox_nms.py文件中用NPU算子BatchMultiNMS代替原mmdetection中的NMS层算子，替换后精度无损失。同时等价换一个expand算子，使导出的onnx中不含动态shape。
270	->
271	-> 4. delta_xywh_bbox_coder.py 中修改坐标的轴顺序，使切片操作在NPU上效率更高，整网性能提升约7%；修改means和std计算方法使其表现为固定shape。
272	-
273	-
274	-通过打补丁的方式修改mmdetection：
275	-```shell
276	-patch -p1 < ../ssd_mmdetection.diff
277	-```
278	-5.修改pytorch代码去除导出onnx时进行检查
279	-将/usr/local/python3.7.5/lib/python3.7/site-packages/torch/onnx/utils.py文件的_check_onnx_proto(proto)改为pass
280	-
281	-6.运行如下命令，生成含有npu自定义算子的onnx：
282	-
283	-```shell
284	-python3.7 mmdetection/tools/pytorch2onnx.py mmdetection/configs/ssd/ssd300_coco.py ./ssd300_coco_20200307-a92d2092.pth --output-file=ssd_300_coco.onnx --shape=300 --verify --show --mean 123.675 116.28 103.53 --std 1 1 1
285	-```
286	-### 3.2 onnx转om模型
287	-
288	-1.设置环境变量
289	-```shell
290	-export install_path=/usr/local/Ascend/ascend-toolkit/latest
291	-export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
292	-export PYTHONPATH=${install_path}/atc/python/site-packages:$PYTHONPATH
293	-export LD_LIBRARY_PATH=${install_path}/atc/lib64:${install_path}/acllib/lib64:$LD_LIBRARY_PATH
294	-export ASCEND_OPP_PATH=${install_path}/opp
295	-export ASCEND_AICPU_PATH=/usr/local/Ascend/ascend-toolkit/latest/
296	-```
297	-2.使用atc将onnx模型转换为om模型文件，工具使用方法可以参考[CANN V100R020C10 开发辅助工具指南 (推理) 01](https://support.huawei.com/enterprise/zh/doc/EDOC1100164868?idPath=23710424%7C251366513%7C22892968%7C251168373)，需要指定输出节点以去除无用输出，节点序号可能会因网络结构不同而不同，使用netron开源可视化工具查看具体的输出节点名：
298	-
299	-```shell
300	-/usr/local/Ascend/ascend-toolkit/latest/atc/bin/atc --model=ssd_300_coco.onnx --framework=5 --output=ssd_300_coco --input_format=NCHW --input_shape="input:1,3,300,300" --log=info --soc_version=Ascend310 --out_nodes="Concat_637:0;Reshape_639:0" --buffer_optimize=off_optimize --precision_mode allow_mix_precision
301	-```
302	-
303	-## 4 数据集预处理
304	-
305	-- [数据集获取](#41-数据集获取)
306	-
307	-- [数据集预处理](#42-数据集预处理)
308	-
309	-- [生成数据集信息文件](#43-生成数据集信息文件)
310	-
311	-### 4.1 数据集获取
312	-该模型使用[COCO官网](https://cocodataset.org/#download)的coco2017的5千张验证集进行测试，图片与标签分别存放在/root/datasets/coco/val2017/与/root/datasets/coco/annotations/instances_val2017.json。
313	-
314	-### 4.2 数据集预处理
315	-1.预处理脚本mmdetection_coco_preprocess.py
316	-
317	-```python
318	-# Copyright 2020 Huawei Technologies Co., Ltd
319	-#
320	-# Licensed under the Apache License, Version 2.0 (the "License");
321	-# you may not use this file except in compliance with the License.
322	-# You may obtain a copy of the License at
323	-#
324	-# http://www.apache.org/licenses/LICENSE-2.0
325	-#
326	-# Unless required by applicable law or agreed to in writing, software
327	-# distributed under the License is distributed on an "AS IS" BASIS,
328	-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
329	-# See the License for the specific language governing permissions and
330	-# limitations under the License.
331	-
332	-import numpy as np
333	-import os
334	-import cv2
335	-import argparse
336	-import mmcv
337	-import torch
338	-
339	-dataset_config = {
340	- 'resize': (300, 300),
341	- 'mean': [123.675, 116.28, 103.53],
342	- 'std': [1, 1, 1],
343	-}
344	-
345	-tensor_height = 300
346	-tensor_width = 300
347	-
348	-def coco_preprocess(input_image, output_bin_path):
349	- #define the output file name
350	- img_name = input_image.split('/')[-1]
351	- #print(img_name)
352	- bin_name = img_name.split('.')[0] + ".bin"
353	- bin_fl = os.path.join(output_bin_path, bin_name)
354	-
355	- one_img = mmcv.imread(os.path.join(input_image), backend='cv2')
356	- # one_img = mmcv.imrescale(one_img, (tensor_height, tensor_width))
357	- one_img = mmcv.imresize(one_img, (tensor_height, tensor_width))
358	- # calculate padding
359	- h = one_img.shape[0]
360	- w = one_img.shape[1]
361	- #print(h,w,tensor_height,tensor_width)
362	- pad_left = (tensor_width - w) // 2
363	- pad_top = (tensor_height - h) // 2
364	- pad_right = tensor_width - pad_left - w
365	- pad_bottom = tensor_height - pad_top - h
366	-
367	- mean = np.array(dataset_config['mean'], dtype=np.float32)
368	- std = np.array(dataset_config['std'], dtype=np.float32)
369	- one_img = mmcv.imnormalize(one_img, mean, std)
370	- # one_img = mmcv.impad(one_img, padding=(pad_left, pad_top, pad_right, pad_bottom), pad_val=0)
371	- one_img = one_img.transpose(2, 0, 1)
372	- one_img.tofile(bin_fl)
373	-
374	-if __name__ == "__main__":
375	- parser = argparse.ArgumentParser(description='preprocess of FasterRCNN pytorch model')
376	- parser.add_argument("--image_folder_path", default="./coco2014/", help='image of dataset')
377	- parser.add_argument("--bin_folder_path", default="./coco2014_bin/", help='Preprocessed image buffer')
378	- flags = parser.parse_args()
379	-
380	- if not os.path.exists(flags.bin_folder_path):
381	- os.makedirs(flags.bin_folder_path)
382	- images = os.listdir(flags.image_folder_path)
383	- for image_name in images:
384	- if not (image_name.endswith(".jpeg") or image_name.endswith(".JPEG") or image_name.endswith(".jpg")):
385	- continue
386	- #print("start to process image {}....".format(image_name))
387	- path_image = os.path.join(flags.image_folder_path, image_name)
388	- coco_preprocess(path_image, flags.bin_folder_path)
389	-
390	-```
391	-2.执行预处理脚本，生成数据集预处理后的bin文件
392	-```shell
393	-python3.7 mmdetection_coco_preprocess.py --image_folder_path /root/datasets/coco/val2017 --bin_folder_path val2017_ssd_bin
394	-```
395	-### 4.3 生成数据集信息文件
396	-1.生成数据集信息文件脚本get_info.py
397	-
398	-```python
399	-import os
400	-import sys
401	-import cv2
402	-from glob import glob
403	-
404	-
405	-def get_bin_info(file_path, info_name, width, height):
406	- bin_images = glob(os.path.join(file_path, '*.bin'))
407	- with open(info_name, 'w') as file:
408	- for index, img in enumerate(bin_images):
409	- content = ' '.join([str(index), img, width, height])
410	- file.write(content)
411	- file.write('\n')
412	-
413	-
414	-def get_jpg_info(file_path, info_name):
415	- extensions = ['jpg', 'jpeg', 'JPG', 'JPEG']
416	- image_names = []
417	- for extension in extensions:
418	- image_names.append(glob(os.path.join(file_path, '*.' + extension)))
419	- with open(info_name, 'w') as file:
420	- for image_name in image_names:
421	- if len(image_name) == 0:
422	- continue
423	- else:
424	- for index, img in enumerate(image_name):
425	- img_cv = cv2.imread(img)
426	- shape = img_cv.shape
427	- width, height = shape[1], shape[0]
428	- content = ' '.join([str(index), img, str(width), str(height)])
429	- file.write(content)
430	- file.write('\n')
431	-
432	-
433	-if __name__ == '__main__':
434	- file_type = sys.argv[1]
435	- file_path = sys.argv[2]
436	- info_name = sys.argv[3]
437	- if file_type == 'bin':
438	- width = sys.argv[4]
439	- height = sys.argv[5]
440	- assert len(sys.argv) == 6, 'The number of input parameters must be equal to 5'
441	- get_bin_info(file_path, info_name, width, height)
442	- elif file_type == 'jpg':
443	- assert len(sys.argv) == 4, 'The number of input parameters must be equal to 3'
444	- get_jpg_info(file_path, info_name)
445	-```
446	-2.执行生成数据集信息脚本，生成数据集信息文件
447	-```shell
448	-python3.7 get_info.py bin ./val2017_ssd_bin coco2017_ssd.info 300 300
449	-python3.7 get_info.py jpg /root/datasets/coco/val2017 coco2017_ssd_jpg.info
450	-```
451	-第一个参数为模型输入的类型，第二个参数为生成的bin文件路径，第三个为输出的info文件，后面为宽高信息
452	-## 5 离线推理
453	-
454	-- [benchmark工具概述](#51-benchmark工具概述)
455	-
456	-- [离线推理](#52-离线推理)
457	-
458	-### 5.1 benchmark工具概述
459	-
460	-benchmark工具为华为自研的模型推理工具，支持多种模型的离线推理，能够迅速统计出模型在Ascend310上的性能，支持真实数据和纯推理两种模式，配合后处理脚本，可以实现诸多模型的端到端过程，获取工具及使用方法可以参考[CANN V100R020C10 推理benchmark工具用户指南 01](https://support.huawei.com/enterprise/zh/doc/EDOC1100164874?idPath=23710424%7C251366513%7C22892968%7C251168373)
461	-### 5.2 离线推理
462	-1.设置环境变量
463	-```shell
464	-export install_path=/usr/local/Ascend/ascend-toolkit/latest
465	-export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
466	-export PYTHONPATH=${install_path}/atc/python/site-packages:$PYTHONPATH
467	-export LD_LIBRARY_PATH=${install_path}/atc/lib64:${install_path}/acllib/lib64:$LD_LIBRARY_PATH
468	-export ASCEND_OPP_PATH=${install_path}/opp
469	-export ASCEND_AICPU_PATH=/usr/local/Ascend/ascend-toolkit/latest/
470	-```
471	-2.执行离线推理
472	-```shell
473	-./benchmark.x86_64 -model_type=vision -batch_size=1 -device_id=0 -input_text_path=./coco2017_ssd.info -input_width=300 -input_height=300 -useDvpp=False -output_binary=true -om_path=ssd_300_coco.om
474	-```
475	- 注意：
476	-> label是int64，benchmark输出非二进制时会将float转为0
477	->
478	-
479	-输出结果默认保存在当前目录result/dumpOutput_device0，模型有两个输出，每个输入对应的输出对应两个_x.bin文件
480	-```
481	-输出 shape 数据类型数据含义
482	-output1 200 * 5 FP32 boxes and scores
483	-output2 200 * 1 INT64 labels
484	-```
485	-
486	-## 6 精度对比
487	-
488	-- [离线推理精度](#61-离线推理精度)
489	-- [开源精度](#62-开源精度)
490	-- [精度对比](#63-精度对比)
491	-
492	-### 6.1 离线推理精度统计
493	-
494	-后处理统计map精度
495	-```python
496	-# Copyright 2020 Huawei Technologies Co., Ltd
497	-#
498	-# Licensed under the Apache License, Version 2.0 (the "License");
499	-# you may not use this file except in compliance with the License.
500	-# You may obtain a copy of the License at
501	-#
502	-# http://www.apache.org/licenses/LICENSE-2.0
503	-#
504	-# Unless required by applicable law or agreed to in writing, software
505	-# distributed under the License is distributed on an "AS IS" BASIS,
506	-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
507	-# See the License for the specific language governing permissions and
508	-# limitations under the License.
509	-
510	-import os
511	-import numpy as np
512	-import argparse
513	-import cv2
514	-
515	-CLASSES = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
516	- 'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
517	- 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
518	- 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe',
519	- 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
520	- 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat',
521	- 'baseball glove', 'skateboard', 'surfboard', 'tennis racket',
522	- 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl',
523	- 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot',
524	- 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
525	- 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop',
526	- 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave',
527	- 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock',
528	- 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']
529	-
530	-def coco_postprocess(bbox: np.ndarray, image_size,
531	- net_input_width, net_input_height):
532	- """
533	- This function is postprocessing for FasterRCNN output.
534	-
535	- Before calling this function, reshape the raw output of FasterRCNN to
536	- following form
537	- numpy.ndarray:
538	- [x, y, width, height, confidence, probability of 80 classes]
539	- shape: (100,)
540	- The postprocessing restore the bounding rectangles of FasterRCNN output
541	- to origin scale and filter with non-maximum suppression.
542	-
543	- :param bbox: a numpy array of the FasterRCNN output
544	- :param image_path: a string of image path
545	- :return: three list for best bound, class and score
546	- """
547	- w = image_size[0]
548	- h = image_size[1]
549	- #print(w,h,net_input_width,net_input_height)
550	- scale_w = net_input_width/w
551	- scale_h = net_input_height/h
552	-
553	- # cal predict box on the image src
554	- pbox = bbox.copy()
555	- pbox[:, 0] = (bbox[:, 0]) / scale_w
556	- pbox[:, 1] = (bbox[:, 1]) / scale_h
557	- pbox[:, 2] = (bbox[:, 2]) / scale_w
558	- pbox[:, 3] = (bbox[:, 3]) / scale_h
559	- return pbox
560	-
561	-
562	-if __name__ == '__main__':
563	- parser = argparse.ArgumentParser()
564	- parser.add_argument("--bin_data_path", default="./result/dumpOutput_device0")
565	- parser.add_argument("--test_annotation", default="./coco2017_jpg.info")
566	- parser.add_argument("--det_results_path", default="./detection-results/")
567	- parser.add_argument("--net_out_num", default=2)
568	- parser.add_argument("--net_input_width", default=300)
569	- parser.add_argument("--net_input_height", default=300)
570	- parser.add_argument("--prob_thres", default=0.02)
571	- parser.add_argument("--ifShowDetObj", action="store_true", help="if input the para means True, neither False.")
572	- flags = parser.parse_args()
573	- # print(flags.ifShowDetObj, type(flags.ifShowDetObj))
574	- # generate dict according to annotation file for query resolution
575	- # load width and height of input images
576	- img_size_dict = dict()
577	- with open(flags.test_annotation)as f:
578	- for line in f.readlines():
579	- temp = line.split(" ")
580	- img_file_path = temp[1]
581	- img_name = temp[1].split("/")[-1].split(".")[0]
582	- img_width = int(temp[2])
583	- img_height = int(temp[3])
584	- img_size_dict[img_name] = (img_width, img_height, img_file_path)
585	-
586	- # read bin file for generate predict result
587	- bin_path = flags.bin_data_path
588	- det_results_path = flags.det_results_path
589	- os.makedirs(det_results_path, exist_ok=True)
590	- total_img = set([name[:name.rfind('_')]
591	- for name in os.listdir(bin_path) if "bin" in name])
592	- for bin_file in sorted(total_img):
593	- path_base = os.path.join(bin_path, bin_file)
594	- # load all detected output tensor
595	- res_buff = []
596	- for num in range(1, flags.net_out_num + 1):
597	- if os.path.exists(path_base + "_" + str(num) + ".bin"):
598	- if num == 1:
599	- buf = np.fromfile(path_base + "_" + str(num) + ".bin", dtype="float32")
600	- buf = np.reshape(buf, [200, 5])
601	- elif num == 2:
602	- buf = np.fromfile(path_base + "_" + str(num) + ".bin", dtype="int64")
603	- buf = np.reshape(buf, [200, 1])
604	- res_buff.append(buf)
605	- else:
606	- print("[ERROR] file not exist", path_base + "_" + str(num) + ".bin")
607	- res_tensor = np.concatenate(res_buff, axis=1)
608	- current_img_size = img_size_dict[bin_file]
609	- #print("[TEST]---------------------------concat{} imgsize{}".format(len(res_tensor), current_img_size))
610	- #print(res_tensor)
611	- predbox = coco_postprocess(res_tensor, current_img_size, flags.net_input_width, flags.net_input_height)
612	-
613	- if flags.ifShowDetObj == True:
614	- imgCur = cv2.imread(current_img_size[2])
615	-
616	- det_results_str = ''
617	- det_results = []
618	- for idx, class_ind in enumerate(predbox[:,5]):
619	- if float(predbox[idx][4]) < float(flags.prob_thres):
620	- continue
621	- # skip negative class index
622	- if class_ind < 0 or class_ind > 80:
623	- continue
624	-
625	- class_name = CLASSES[int(class_ind)]
626	- det_results.append([class_name, str(predbox[idx][4]), predbox[idx][0], predbox[idx][1], predbox[idx][2], predbox[idx][3]])
627	- det_results_str += "{} {} {} {} {} {}\n".format(class_name, str(predbox[idx][4]), predbox[idx][0],
628	- predbox[idx][1], predbox[idx][2], predbox[idx][3])
629	- if flags.ifShowDetObj == True:
630	- imgCur=cv2.rectangle(imgCur, (int(predbox[idx][0]), int(predbox[idx][1])),
631	- (int(predbox[idx][2]), int(predbox[idx][3])), (0,255,0), 1)
632	- imgCur = cv2.putText(imgCur, class_name+'\|'+str(predbox[idx][4]),
633	- (int(predbox[idx][0]), int(predbox[idx][1])), cv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 1)
634	- # 图像，文字内容，坐标，字体，大小，颜色，字体厚度
635	-
636	- if flags.ifShowDetObj == True:
637	- print(os.path.join(det_results_path, bin_file +'.jpg'))
638	- cv2.imwrite(os.path.join(det_results_path, bin_file +'.jpg'), imgCur, [int(cv2.IMWRITE_JPEG_QUALITY),70])
639	-
640	- det_results_file = os.path.join(det_results_path, bin_file + ".txt")
641	- with open(det_results_file, "w") as detf:
642	- detf.write(det_results_str)
643	-```
644	-txt文件转json
645	-
646	-```python
647	-import glob
648	-import os
649	-import sys
650	-import argparse
651	-import mmcv
652	-
653	-CLASSES = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
654	- 'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
655	- 'stop sign', 'parking meter', 'bench', 'bird', 'cat', 'dog',
656	- 'horse', 'sheep', 'cow', 'elephant', 'bear', 'zebra', 'giraffe',
657	- 'backpack', 'umbrella', 'handbag', 'tie', 'suitcase', 'frisbee',
658	- 'skis', 'snowboard', 'sports ball', 'kite', 'baseball bat',
659	- 'baseball glove', 'skateboard', 'surfboard', 'tennis racket',
660	- 'bottle', 'wine glass', 'cup', 'fork', 'knife', 'spoon', 'bowl',
661	- 'banana', 'apple', 'sandwich', 'orange', 'broccoli', 'carrot',
662	- 'hot dog', 'pizza', 'donut', 'cake', 'chair', 'couch',
663	- 'potted plant', 'bed', 'dining table', 'toilet', 'tv', 'laptop',
664	- 'mouse', 'remote', 'keyboard', 'cell phone', 'microwave',
665	- 'oven', 'toaster', 'sink', 'refrigerator', 'book', 'clock',
666	- 'vase', 'scissors', 'teddy bear', 'hair drier', 'toothbrush']
667	-
668	-cat_ids = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23,
669	-24, 25, 27, 28, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 46, 47,
670	-48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 67, 70,
671	-72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 84, 85, 86, 87, 88, 89, 90]
672	-
673	-'''
674	- 0,0 ------> x (width)
675	- \|
676	- \| (Left,Top)
677	- \| *_________
678	- \| \| \|
679	- \| \|
680	- y \|_________\|
681	- (height) *
682	- (Right,Bottom)
683	-'''
684	-
685	-def file_lines_to_list(path):
686	- # open txt file lines to a list
687	- with open(path) as f:
688	- content = f.readlines()
689	- # remove whitespace characters like `\n` at the end of each line
690	- content = [x.strip() for x in content]
691	- return content
692	-
693	-
694	-def error(msg):
695	- print(msg)
696	- sys.exit(0)
697	-
698	-
699	-def get_predict_list(file_path, gt_classes):
700	- dr_files_list = glob.glob(file_path + '/*.txt')
701	- dr_files_list.sort()
702	-
703	- bounding_boxes = []
704	- for txt_file in dr_files_list:
705	- file_id = txt_file.split(".txt", 1)[0]
706	- file_id = os.path.basename(os.path.normpath(file_id))
707	- lines = file_lines_to_list(txt_file)
708	- for line in lines:
709	- try:
710	- sl = line.split()
711	- if len(sl) > 6:
712	- class_name = sl[0] + ' ' + sl[1]
713	- scores, left, top, right, bottom = sl[2:]
714	- else:
715	- class_name, scores, left, top, right, bottom = sl
716	- if float(scores) < 0.02:
717	- continue
718	- except ValueError:
719	- error_msg = "Error: File " + txt_file + " wrong format.\n"
720	- error_msg += " Expected: <classname> <conf> <l> <t> <r> <b>\n"
721	- error_msg += " Received: " + line
722	- error(error_msg)
723	-
724	- # bbox = left + " " + top + " " + right + " " + bottom
725	- left = float(left)
726	- right = float(right)
727	- top = float(top)
728	- bottom = float(bottom)
729	- bbox = [left, top, right-left, bottom-top]
730	- #bounding_boxes.append({"image_id": int(file_id), "bbox": bbox,"score": float(scores), "category_id": 1+CLASSES.index(class_name)})
731	- bounding_boxes.append({"image_id": int(file_id), "bbox": bbox,"score": float(scores), "category_id": cat_ids[CLASSES.index(class_name)]})
732	- # sort detection-results by decreasing scores
733	- # bounding_boxes.sort(key=lambda x: float(x['score']), reverse=True)
734	- return bounding_boxes
735	-
736	-
737	-
738	-if __name__ == '__main__':
739	- parser = argparse.ArgumentParser('mAp calculate')
740	- parser.add_argument('--npu_txt_path', default="detection-results",
741	- help='the path of the predict result')
742	- parser.add_argument("--json_output_file", default="coco_detection_result")
743	- args = parser.parse_args()
744	-
745	- res_bbox = get_predict_list(args.npu_txt_path, CLASSES)
746	- mmcv.dump(res_bbox, args.json_output_file + '.json')
747	-```
748	-
749	-调用coco_eval.py评测map精度：
750	-
751	-```shell
752	-python3.7 mmdetection_coco_postprocess.py --bin_data_path=result/dumpOutput_device0 --prob_thres=0.02 --ifShowDetObj --det_results_path=detection-results --test_annotation=coco2017_ssd_jpg.info
753	-python3.7 txt_to_json.py
754	-python3.7 coco_eval.py --ground_truth /root/datasets/coco/annotations/instances_val2017.json
755	-```
756	-
757	-执行完后会打印出精度：
758	-
759	-```shell
760	-loading annotations into memory...
761	-Done (t=0.88s)
762	-creating index...
763	-index created!
764	-Loading and preparing results...
765	-DONE (t=9.06s)
766	-creating index...
767	-index created!
768	-Running per image evaluation...
769	-Evaluate annotation type bbox
770	-DONE (t=120.63s).
771	-Accumulating evaluation results...
772	-DONE (t=30.40s).
773	-Average Precision (AP) @[ IoU=0.50:0.95 \| area= all \| maxDets=100 ] = 0.255
774	-Average Precision (AP) @[ IoU=0.50 \| area= all \| maxDets=1000 ] = 0.438
775	-Average Precision (AP) @[ IoU=0.75 \| area= all \| maxDets=1000 ] = 0.263
776	-Average Precision (AP) @[ IoU=0.50:0.95 \| area= small \| maxDets=1000 ] = 0.070
777	-Average Precision (AP) @[ IoU=0.50:0.95 \| area=medium \| maxDets=1000 ] = 0.278
778	-Average Precision (AP) @[ IoU=0.50:0.95 \| area= large \| maxDets=1000 ] = 0.422
779	-Average Recall (AR) @[ IoU=0.50:0.95 \| area= all \| maxDets=100 ] = 0.375
780	-Average Recall (AR) @[ IoU=0.50:0.95 \| area= all \| maxDets=300 ] = 0.376
781	-Average Recall (AR) @[ IoU=0.50:0.95 \| area= all \| maxDets=1000 ] = 0.376
782	-Average Recall (AR) @[ IoU=0.50:0.95 \| area= small \| maxDets=1000 ] = 0.124
783	-Average Recall (AR) @[ IoU=0.50:0.95 \| area=medium \| maxDets=1000 ] = 0.417
784	-Average Recall (AR) @[ IoU=0.50:0.95 \| area= large \| maxDets=1000 ] = 0.586
785	-{'bbox_mAP': 0.255, 'bbox_mAP_50': 0.438, 'bbox_mAP_75': 0.263, 'bbox_mAP_s': 0.07, 'bbox_mAP_m': 0.278, 'bbox_mAP_l': 0.422, 'bbox_mAP_copypaste': '0.255 0.438 0.263 0.070 0.278 0.422'}
786	-```
787	-
788	-
789	-### 6.2 开源精度
790	-[官网精度](http://download.openmmlab.com/mmdetection/v2.0/ssd/ssd300_coco/ssd300_coco_20200307_174216.log.json)
791	-
792	-```
793	-{"mode": "val", "epoch": 24, "iter": 9162, "lr": 2e-05, "bbox_mAP": 0.256, "bbox_mAP_50": 0.438, "bbox_mAP_75": 0.263, "bbox_mAP_s": 0.068, "bbox_mAP_m": 0.278, "bbox_mAP_l": 0.422, "bbox_mAP_copypaste": "0.256 0.438 0.263 0.068 0.278 0.422"}
794	-```
795	-### 6.3 精度对比
796	-om推理box map精度为0.255，开源box map50精度为0.256，精度下降0.1%，精度达标
797	-
798	-## 7 性能对比
799	-
800	-- [npu性能数据](#71-npu性能数据)
801	-- [基准性能数据](#72-基准性能数据)
802	-- [性能对比](#73-性能对比)
803	-
804	-### 7.1 npu性能数据
805	-batch1的性能：
806	- 测试npu性能要确保device空闲，使用npu-smi info命令可查看device是否在运行其它推理任务
807	-
808	-在前面测试精度时，已经得到性能数据，运行
809	-```shell
810	-python3.7 test/parse.py result/perf_vision_batchsize_1_device_0.txt
811	-```
812	-parse.py文件为，
813	-```shell
814	-import sys
815	-import json
816	-import re
817	-
818	-if __name__ == '__main__':
819	- if sys.argv[1].endswith('.json'):
820	- result_json = sys.argv[1]
821	- with open(result_json, 'r') as f:
822	- content = f.read()
823	- tops = [i.get('value') for i in json.loads(content).get('value') if 'Top' in i.get('key')]
824	- print('om {} top1:{} top5:{}'.format(result_json.split('_')[1].split('.')[0], tops[0], tops[4]))
825	- elif sys.argv[1].endswith('.txt'):
826	- result_txt = sys.argv[1]
827	- with open(result_txt, 'r') as f:
828	- content = f.read()
829	- txt_data_list = [i.strip() for i in re.findall(r':(.*?),', content.replace('\n', ',') + ',')]
830	- fps = float(txt_data_list[7].replace('samples/s', '')) * 4
831	- print('310 bs{} fps:{}'.format(result_txt.split('_')[3], fps))
832	-```
833	-执行结果为
834	-```shell
835	-310 bs1 fps:56.9588
836	-```
837	-SSD mmdetection不支持多batch，故只在batch1上测试
838	-
839	-
840	-### 7.2 基准性能数据
841	-batch1性能：
842	-onnx包含自定义算子，因此不能使用开源TensorRT测试性能数据，故在基准机器上使用pth在线推理测试性能数据
843	-
844	-测评基准精度与性能：
845	-```shell
846	-wget http://download.openmmlab.com/mmdetection/v2.0/ssd/ssd300_coco/ssd300_coco_20200307-a92d2092.pth
847	-git clone https://github.com/open-mmlab/mmdetection.git
848	-cd mmdetection
849	-git reset --hard a21eb25535f31634cef332b09fc27d28956fb24b
850	-pip install -r requirements/build.txt
851	-pip install -v -e .
852	-mkdir data
853	-ln -s /root/datasets/coco data/coco
854	-python3 tools/test.py configs/ssd/ssd300_coco.py ../ssd300_coco_20200307-a92d2092.pth --eval bbox
855	-```
856	-```shell
857	-loading annotations into memory...
858	-Done (t=0.67s)
859	-creating index...
860	-index created!
861	-[>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 5000/5000, 37.6 task/s, elapsed: 133s, ETA: 0s
862	-Evaluating bbox...
863	-Loading and preparing results...
864	-DONE (t=8.12s)
865	-creating index...
866	-index created!
867	-```
868	-
869	-### 7.3 性能对比
870	-310单卡4个device，benchmark测试的是一个device。基准一个设备相当于4个device，测试的是整个设备。benchmark时延是吞吐率的倒数，基准时延是吞吐率的倒数乘以batch。对于batch1，56.9588 > 37.6，即npu性能超过基准性能
871	-对于batch1，npu性能高于基准性能1.2倍，该模型放在benchmark/cv/detection目录下
872		26
873		27
		28	+
		29	+- 参考实现：
		30	+
		31	+ ```
		32	+ url=https://github.com/open-mmlab/mmdetection.git
		33	+ branch=master
		34	+ commit_id=a21eb25535f31634cef332b09fc27d28956fb24b
		35	+ model_name=ssd
		36	+ ```
		37	+
		38	+
		39	+
		40	+ 通过Git获取对应commit\_id的代码方法如下：
		41	+
		42	+ ```
		43	+ git clone {repository_url} # 克隆仓库的代码
		44	+ cd {repository_name} # 切换到模型的代码仓目录
		45	+ git checkout {branch/tag} # 切换到对应分支
		46	+ git reset --hard {commit_id} # 代码设置到对应的commit_id（可选）
		47	+ cd {code_path} # 切换到模型代码所在路径，若仓库下只有该模型，则无需切换
		48	+ ```
		49	+
		50	+
		51	+## 输入输出数据<a name="section540883920406"></a>
		52	+
		53	+- 输入数据
		54	+
		55	+ \| 输入数据 \| 数据类型 \| 大小 \| 数据排布格式 \|
		56	+ \| -------- \| -------- \| ------------------------- \| ------------ \|
		57	+ \| input \| RGB_FP32 \| batchsize x 3 x 300 x 300 \| NCHW \|
		58	+
		59	+
		60	+- 输出数据
		61	+
		62	+ \| 输出数据 \| 大小 \| 数据类型 \| 数据排布格式 \|
		63	+ \| -------- \| -------- \| -------- \| ------------ \|
		64	+ \| boxes \| batchsize x 8732 x 4 \| FLOAT32 \| ND \|
		65	+ \| labels \| batchsize x 8732 x 80 \| FLOAT32 \| ND \|
		66	+
		67	+
		68	+
		69	+
		70	+
		71	+# 推理环境准备\[所有版本\]<a name="ZH-CN_TOPIC_0000001126281702"></a>
		72	+
		73	+- 该模型需要以下插件与驱动
		74	+
		75	+ 表 1 版本配套表
		76	+
		77	+\| 配套 \| 版本 \| 环境准备指导 \|
		78	+\| ------------------------------------------------------------ \| ------- \| ------------------------------------------------------------ \|
		79	+\| 固件与驱动 \| 22.0.2 \| [Pytorch框架推理环境准备](https://www.hiascend.com/document/detail/zh/ModelZoo/pytorchframework/pies) \|
		80	+\| CANN \| 5.1.RC2 \| - \|
		81	+\| Python \| 3.7.5 \| - \|
		82	+\| PyTorch \| 1.8.1 \| - \|
		83	+\| 说明：Atlas 300I Duo 推理卡请以CANN版本选择实际固件与驱动版本。 \| \ \| \ \|
		84	+
		85	+
		86	+# 快速上手<a name="ZH-CN_TOPIC_0000001126281700"></a>
		87	+
		88	+
		89	+## 准备环境<a name="section183221994411"></a>
		90	+
		91	+ 1. 环境安装
		92	+ ```
		93	+ pip install -r requirements.txt
		94	+ ```
		95	+
		96	+ 2. mmdetection源码安装。
		97	+ ```
		98	+ git clone https://github.com/open-mmlab/mmdetection.git
		99	+ cd mmdetection
		100	+ git reset --hard a21eb25535f31634cef332b09fc27d28956fb24b
		101	+ pip install -v -e .
		102	+ ```
		103	+
		104	+ 3. 通过打补丁的方式修改mmdetection。
		105	+ ```
		106	+ patch -p1 < ../ssd_mmdet.diff
		107	+ ```
		108	+
		109	+
		110	+
		111	+## 准备数据集<a name="section183221994411"></a>
		112	+
		113	+1. 获取原始数据集。（解压命令参考tar –xvf \.tar与 unzip \.zip）
		114	+
		115	+ 数据集名称：coco2017
		116	+
		117	+ 所用到的文件：推理数据集采用coco_val_2017
		118	+
		119	+ 下载链接：http://images.cocodataset.org
		120	+
		121	+ 存放路径：/root/datasets/
		122	+
		123	+ 目录结构：
		124	+
		125	+ ```
		126	+ ├── coco
		127	+ │ ├── val2017
		128	+ │ ├── annotations
		129	+ │ ├──instances_val2017.json
		130	+ ```
		131	+
		132	+2. 数据预处理。$请拆分sh脚本，将命令分开填写$
		133	+
		134	+ 将原始数据集转换为模型输入的二进制数据。执行mmdetection_coco_preprocess脚本。
		135	+
		136	+
		137	+ ```
		138	+ python mmdetection_coco_preprocess.py --image_folder_path /root/datasets/coco/val2017 --bin_folder_path val2017_ssd_bin
		139	+ ```
		140	+
		141	+ - 参数说明：
		142	+
		143	+ - --image_folder_path：原始数据验证集（.jpg）所在路径。
		144	+ - --bin_folder_path：输出的二进制文件（.bin）所在路径。
		145	+
		146	+
		147	+ 每个图像对应生成一个二进制文件。
		148	+
		149	+
		150	+3. 生成数据集info文件。
		151	+
		152	+ 运行get_info.py脚本，生成图片数据info文件。
		153	+ ```
		154	+ python get_info.py jpg /root/datasets/coco/val2017 coco2017_ssd_jpg.info
		155	+ ```
		156	+
		157	+ - 参数说明：
		158	+
		159	+ - 第一个参数：生成的数据集文件格式。
		160	+ - 第二个参数：预处理后的数据文件相对路径。
		161	+ - 第三个参数：生成的info文件名。
		162	+
		163	+ 运行成功后，在当前目录中生成coco2017_ssd_jpg.info。
		164	+
		165	+
		166	+## 模型推理<a name="section741711594517"></a>
		167	+
		168	+1. 模型转换。
		169	+
		170	+ 使用PyTorch将模型权重文件.pth转换为.onnx文件，再使用ATC工具将.onnx文件转为离线推理模型文件.om文件。
		171	+
		172	+ 1. 获取权重文件。
		173	+
		174	+ 获取经过训练的权重文件ssd300_coco_20200307-a92d2092.pth:
		175	+ ```
		176	+ wget http://download.openmmlab.com/mmdetection/v2.0/ssd/ssd300_coco/ssd300_coco_20200307-a92d2092.pth
		177	+ ```
		178	+
		179	+ 2. 导出onnx文件。
		180	+
		181	+ 使用pytorch2onnx.py导出onnx文件。
		182	+
		183	+ ```
		184	+ python mmdetection/tools/pytorch2onnx.py mmdetection/configs/ssd/ssd300_coco.py ./ssd300_coco_20200307-a92d2092.pth --output-file=ssd300_coco_dynamic_bs.onnx --shape=300 --show --mean 123.675 116.28 103.53 --std 1 1 1
		185	+ ```
		186	+
		187	+ - 参数说明：
		188	+
		189	+ - --output-file：为ONNX模型文件。
		190	+ - --shape：输入的图片大小。
		191	+ - --show：输出的OM模型。
		192	+ - --mean：输入数据的格式。
		193	+ - --std：输入数据的shape。
		194	+
		195	+ 获得ssd300_coco_dynamic_bs.onnx文件。
		196	+
		197	+
		198	+ 3. 使用ATC工具将ONNX模型转OM模型。
		199	+
		200	+ 1. 配置环境变量。
		201	+
		202	+ ```
		203	+ source /usr/local/Ascend/ascend-toolkit/set_env.sh
		204	+ ```
		205	+
		206	+ > 说明：
		207	+ >该脚本中环境变量仅供参考，请以实际安装环境配置环境变量。详细介绍请参见《[CANN 开发辅助工具指南 $推理$](https://support.huawei.com/enterprise/zh/ascend-computing/cann-pid-251168373?category=developer-documents&subcategory=auxiliary-development-tools)》。
		208	+
		209	+ 2. 执行命令查看芯片名称（$\{chip\_name\}）。
		210	+
		211	+ ```
		212	+ npu-smi info
		213	+ #该设备芯片名为Ascend310P3 （自行替换）
		214	+ 回显如下：
		215	+ +-------------------+-----------------+------------------------------------------------------+
		216	+ \| NPU Name \| Health \| Power(W) Temp(C) Hugepages-Usage(page) \|
		217	+ \| Chip Device \| Bus-Id \| AICore(%) Memory-Usage(MB) \|
		218	+ +===================+=================+======================================================+
		219	+ \| 0 310P3 \| OK \| 15.8 42 0 / 0 \|
		220	+ \| 0 0 \| 0000:82:00.0 \| 0 1074 / 21534 \|
		221	+ +===================+=================+======================================================+
		222	+ \| 1 310P3 \| OK \| 15.4 43 0 / 0 \|
		223	+ \| 0 1 \| 0000:89:00.0 \| 0 1070 / 21534 \|
		224	+ +===================+=================+======================================================+
		225	+ ```
		226	+
		227	+ 3. 执行ATC命令。
		228	+
		229	+ 设置环境变量：
		230	+ ```
		231	+ export LD_LIBRARY_PATH=/usr/local/Ascend/ascend-toolkit/latest/lib64:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/opskernel:/usr/local/Ascend/ascend-toolkit/latest/lib64/plugin/nnengine:/usr/local/Ascend/driver/lib64/driver:$LD_LIBRARY_PATH
		232	+ ```
		233	+
		234	+ 执行atc命令
		235	+ ```
		236	+ atc --model=ssd300_coco_dynamic_bs.onnx --framework=5 --output=${om_name} --input_format=NCHW --input_shape="input:${batchsize},3,300,300" --log=debug --soc_version=Ascend${chip_name} --buffer_optimize=off_optimize --precision_mode=allow_fp32_to_fp16
		237	+ ```
		238	+
		239	+ - 参数说明：
		240	+
		241	+ - --model：为ONNX模型文件。
		242	+ - --framework：5代表ONNX模型。
		243	+ - --output：输出的OM模型。
		244	+ - --input\_format：输入数据的格式。
		245	+ - --input\_shape：输入数据的shape。
		246	+ - --log：日志级别。
		247	+ - --soc\_version：处理器型号。
		248	+ - --buffer_optimize：
		249	+ - --precision_mode：
		250	+
		251	+ 运行成功后生成ssd300_coco_bs8.om模型文件。
		252	+
		253	+
		254	+
		255	+2. 开始推理验证。
		256	+
		257	+ 1. 使用ais-infer工具进行推理。
		258	+
		259	+ ais-infer工具获取及使用方式请点击查看[[ais_infer 推理工具使用文档](https://gitee.com/ascend/tools/tree/master/ais-bench_workload/tool/ais_infer)]
		260	+
		261	+ 2. 执行推理。
		262	+
		263	+
		264	+ ```
		265	+ python tools/ais-bench_workload/tool/ais_infer/ais_infer.py --model ${om_path} --input ./val2017_ssd_bin --batchsize ${batchsize} --output ${out_path}
		266	+ ```
		267	+
		268	+ - 参数说明：
		269	+
		270	+ - --model：为.OM模型文件的路径。
		271	+ - --input：转换之后的二进制数据集路径。
		272	+ - --batchsize：batch维度大小，与输入的.OM模型文件的batch维度一致。
		273	+ - --output：模型推理结果存放的路径。
		274	+
		275	+ 上述命令将会在 ${output} 所在目录创建一个以时间命名的文件夹来存放推理结果。
		276	+
		277	+ 2. 精度验证。
		278	+
		279	+ 调用coco_eval.py评测map精度：
		280	+
		281	+ ```
		282	+ python mmdetection_coco_postprocess.py --bin_data_path=${infer_result_path} --score_threshold=0.02 --test_annotation=coco2017_ssd_jpg.info --nms_pre 200 --det_results_path ${det_path}
		283	+ python txt_to_json.py --npu_txt_path ${det_path}
		284	+ python coco_eval.py --ground_truth /root/datasets/coco/annotations/instances_val2017.json
		285	+ ```
		286	+
		287	+ - 参数说明：
		288	+
		289	+ - --bin_data_path：为推理结果存放的路径。
		290	+ - --score_threshold：得分阈值。
		291	+ - --test_annotation：原始图片信息文件。
		292	+ - --nms_pre：每张图片获取框数量的阈值。
		293	+ - --det_results_path：后处理输出路径。
		294	+ - --npu_txt_path：后处理输出路径。
		295	+ - --ground_truth：instances_val2017.json文件路径。
		296	+
		297	+
		298	+
		299	+# 模型推理性能&精度<a name="ZH-CN_TOPIC_0000001172201573"></a>
		300	+
		301	+调用ACL接口推理计算，性能参考下列数据。
		302	+
		303	+
		304	+\| \| mAP \|
		305	+\|---\|---\|
		306	+\| 310精度 \| mAP=25.4 \|
		307	+\| 310P精度 \| mAP=25.4 \|
		308	+\| 性能 \| 337.01 \|
		309	+
		310	+
		311	+\| Throughput \| 310 \| 310P \| T4 \| 310P/310 \| 310P/T4 \|
		312	+\|---------\|----------\|----------\|----------\|-------------\|-------------\|
		313	+\| bs1 \| 179.194 \| 298.5514 \| 250.8491 \| 1.666079221 \| 1.190163329 \|
		314	+\| bs4 \| 207.596 \| 337.0112 \| 310.1569 \| 1.623399295 \| 1.086582952 \|
		315	+\| bs8 \| 211.7312 \| 323.5662 \| 332.0797 \| 1.528193294 \| 0.974363082 \|
		316	+\| bs16 \| 211.288 \| 318.1392 \| 352.4384 \| 1.505713528 \| 0.902680298 \|
		317	+\| bs32 \| 200.2948 \| 318.7303 \| 348.0656 \| 1.591305915 \| 0.915719048 \|
		318	+\| bs64 \| 196.4192 \| 313.0790 \| 370.7415 \| 1.593932772 \| 0.844467102 \|
		319	+\| 最优batch \| 211.7312 \| 337.0112 \| 370.7415 \| 1.591693619 \| 0.909019357 \|

MACL_PyTorch/contrib/cv/detection/SSD/mmdetection_coco_postprocess.py+174-40

	@@ -17,6 +17,15 @@ import os
17	import numpy as np	17	import numpy as np
18	import argparse	18	import argparse
19	import cv2	19	import cv2
		20	+import warnings
		21	+import torch
		22	+import time
		23	+try:
		24	+ from torch import npu_batch_nms as NMSOp
		25	+ NMS_ON_NPU = True
		26	+except:
		27	+ from torchvision.ops import batched_nms as NMSOp
		28	+ NMS_ON_NPU = False
20		29
21	CLASSES = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',	30	CLASSES = ['person', 'bicycle', 'car', 'motorcycle', 'airplane', 'bus',
22	'train', 'truck', 'boat', 'traffic light', 'fire hydrant',	31	'train', 'truck', 'boat', 'traffic light', 'fire hydrant',
	@@ -64,16 +73,37 @@ def coco_postprocess(bbox, image_size, net_input_width, net_input_height):
64	return pbox	73	return pbox
65		74
66		75
		76	+def np_clip_bbox(bboxes, max_shape):
		77	+ x1, y1, x2, y2 = bboxes[:, 0], bboxes[:, 1], bboxes[:, 2], bboxes[:, 3]
		78	+ h, w = max_shape
		79	+ x1 = x1.clip(min=0, max=w)
		80	+ y1 = y1.clip(min=0, max=h)
		81	+ x2 = x2.clip(min=0, max=w)
		82	+ y2 = y2.clip(min=0, max=h)
		83	+ bboxes = np.stack([x1, y1, x2, y2], axis=-1)
		84	+ return bboxes
		85	+
		86	+
67	if __name__ == '__main__':	87	if __name__ == '__main__':
68	parser = argparse.ArgumentParser()	88	parser = argparse.ArgumentParser()
69	parser.add_argument("--bin_data_path", default="./result/dumpOutput_device0")	89	parser.add_argument("--bin_data_path", default="./result/dumpOutput_device0")
70	parser.add_argument("--test_annotation", default="./coco2017_jpg.info")	90	parser.add_argument("--test_annotation", default="./coco2017_jpg.info")
71	parser.add_argument("--det_results_path", default="./detection-results/")	91	parser.add_argument("--det_results_path", default="./detection-results/")
72	- parser.add_argument("--net_out_num", default=2)	92	+ parser.add_argument("--net_out_num", default=2, type=int)
73	- parser.add_argument("--net_input_width", default=300)	93	+ parser.add_argument("--num_pred_box", default=8732, type=int)
74	- parser.add_argument("--net_input_height", default=300)	94	+ parser.add_argument("--nms_pre", default=-1, type=int)
75	- parser.add_argument("--prob_thres", default=0.02)	95	+ parser.add_argument("--net_input_width", default=300, type=int)
		96	+ parser.add_argument("--net_input_height", default=300, type=int)
		97	+ parser.add_argument("--min_bbox_size", default=0.01, type=float)
		98	+ parser.add_argument("--score_threshold", default=0.02, type=float)
		99	+ parser.add_argument("--nms", default=True, type=bool)
		100	+ parser.add_argument("--iou_threshold", default=0.45, type=float)
		101	+ parser.add_argument("--max_per_img", default=200, type=int)
76	parser.add_argument("--ifShowDetObj", action="store_true", help="if input the para means True, neither False.")	102	parser.add_argument("--ifShowDetObj", action="store_true", help="if input the para means True, neither False.")
		103	+ parser.add_argument("--start", default=0, type=float)
		104	+ parser.add_argument("--end", default=1, type=float)
		105	+ parser.add_argument("--device", default=0, type=int)
		106	+ parser.add_argument("--clear_cache", action='store_true')
77	flags = parser.parse_args()	107	flags = parser.parse_args()
78	# generate dict according to annotation file for query resolution	108	# generate dict according to annotation file for query resolution
79	# load width and height of input images	109	# load width and height of input images
	@@ -92,53 +122,157 @@ if __name__ == '__main__':
92	det_results_path = flags.det_results_path	122	det_results_path = flags.det_results_path
93	os.makedirs(det_results_path, exist_ok=True)	123	os.makedirs(det_results_path, exist_ok=True)
94	total_img = set([name[:name.rfind('_')] for name in os.listdir(bin_path) if "bin" in name])	124	total_img = set([name[:name.rfind('_')] for name in os.listdir(bin_path) if "bin" in name])
95	- for bin_file in sorted(total_img):	125	+ total_img = sorted(total_img)
96	- path_base = os.path.join(bin_path, bin_file)	126	+ num_img = len(total_img)
97	- # load all detected output tensor	127	+ start = int(flags.start * num_img)
98	- res_buff = []	128	+ end = int(flags.end * num_img)
99	- for num in range(1, flags.net_out_num + 1):	129	+ task_len = end - start + 1
100	- if os.path.exists(path_base + "_" + str(num) + ".bin"):
101	- if num == 1:
102	- buf = np.fromfile(path_base + "_" + str(num) + ".bin", dtype="float32")
103	- buf = np.reshape(buf, [200, 5])
104	- elif num == 2:
105	- buf = np.fromfile(path_base + "_" + str(num) + ".bin", dtype="int64")
106	- buf = np.reshape(buf, [200, 1])
107	- res_buff.append(buf)
108	- else:
109	- print("[ERROR] file not exist", path_base + "_" + str(num) + ".bin")
110	- res_tensor = np.concatenate(res_buff, axis=1)
111	- current_img_size = img_size_dict[bin_file]
112	- predbox = coco_postprocess(res_tensor, current_img_size, flags.net_input_width, flags.net_input_height)
113		130
		131	+ finished = 0
		132	+ time_start = time.time()
		133	+ for img_id in range(start, end):
		134	+ # for img_id, bin_file in enumerate(sorted(total_img)):
		135	+ bin_file = total_img[img_id]
		136	+ path_base = os.path.join(bin_path, bin_file)
		137	+ det_results_file = os.path.join(det_results_path, bin_file + ".txt")
		138	+ if os.path.exists(det_results_file) and not flags.clear_cache:
		139	+ continue
		140	+
		141	+ # load all detected output tensor
		142	+ bbox_file = path_base + "_" + str(0) + ".bin"
		143	+ score_file = path_base + "_" + str(1) + ".bin"
		144	+ assert os.path.exists(bbox_file), '[ERROR] file `{}` not exist'.format(bbox_file)
		145	+ assert os.path.exists(score_file), '[ERROR] file `{}` not exist'.format(score_file)
		146	+ bboxes = np.fromfile(bbox_file, dtype="float32").reshape(flags.num_pred_box, 4)
		147	+ scores = np.fromfile(score_file, dtype="float32").reshape(flags.num_pred_box, 80)
		148	+
		149	+ bboxes = torch.from_numpy(bboxes)
		150	+ scores = torch.from_numpy(scores)
		151	+ try:
		152	+ bboxes = bboxes.npu(flags.device)
		153	+ scores = scores.npu(flags.device)
		154	+ except:
		155	+ warnings.warn('npu is not available, running on cpu')
		156	+
		157	+ max_scores, _ = scores.max(-1)
		158	+ keep_inds = (max_scores > flags.score_threshold).nonzero(as_tuple=False).view(-1)
		159	+ bboxes = bboxes[keep_inds, :]
		160	+ scores = scores[keep_inds, :]
		161	+
		162	+ if flags.nms_pre > 0 and flags.nms_pre < bboxes.shape[0]:
		163	+ max_scores, _ = scores.max(-1)
		164	+ _, topk_inds = max_scores.topk(flags.nms_pre)
		165	+ bboxes = bboxes[topk_inds, :]
		166	+ scores = scores[topk_inds, :]
		167	+
		168	+ # clip bbox border
		169	+ bboxes[:, 0::2].clamp_(min=0, max=flags.net_input_width - 1)
		170	+ bboxes[:, 1::2].clamp_(min=0, max=flags.net_input_height - 1)
		171	+
		172	+ # remove small bbox
		173	+ bboxes_width_height = bboxes[:, 2:] - bboxes[:, :2]
		174	+ valid_bboxes = bboxes_width_height > flags.min_bbox_size
		175	+ keep_inds = (valid_bboxes[:, 0] & valid_bboxes[:, 1]
		176	+ ).nonzero(as_tuple=False).view(-1)
		177	+ bboxes = bboxes[keep_inds, :]
		178	+ scores = scores[keep_inds, :]
		179	+
		180	+ # rescale bbox to original image size
		181	+ original_img_info = img_size_dict[bin_file]
		182	+ rescale_factor = torch.tensor([
		183	+ original_img_info[0] / flags.net_input_width,
		184	+ original_img_info[1] / flags.net_input_height] * 2,
		185	+ dtype=bboxes.dtype, device=bboxes.device)
		186	+ bboxes *= rescale_factor
		187	+
		188	+ if flags.nms:
		189	+ if NMS_ON_NPU:
		190	+ # repeat bbox for each class
		191	+ # (N, 4) -> (B, N, 80, 4), where B = 1 is the batchsize
		192	+ bboxes = bboxes[None, :, None, :].repeat(1, 1, 80, 1)
		193	+ # (N, 80) -> (B, N, 80), where B = 1 is the batchsize
		194	+ scores = scores[None, :, :]
		195	+
		196	+ # bbox batched nms
		197	+ bboxes, scores, labels, num_total_bboxes = \
		198	+ NMSOp(
		199	+ bboxes.half(), scores.half(),
		200	+ score_threshold=flags.score_threshold,
		201	+ iou_threshold=flags.iou_threshold,
		202	+ max_size_per_class=flags.max_per_img,
		203	+ max_total_size=flags.max_per_img)
		204	+ bboxes = bboxes[0, :num_total_bboxes, :]
		205	+ scores = scores[0, :num_total_bboxes]
		206	+ class_idxs = labels[0, :num_total_bboxes]
		207	+ else:
		208	+ # repeat bbox and class idx for each class
		209	+ bboxes = bboxes[:, None, :].repeat(1, 80, 1) # (N, 4) -> (N, 80, 4)
		210	+ class_idxs = torch.arange(80, dtype=torch.long, device=bboxes.device
		211	+ )[None, :].repeat(bboxes.shape[0], 1) # (80) -> (N, 80)
		212	+
		213	+ # reshape bbox for torch nms
		214	+ bboxes = bboxes.view(-1, 4)
		215	+ scores = scores.view(-1)
		216	+ class_idxs = class_idxs.view(-1)
		217	+
		218	+ # bbox batched nms
		219	+ keep_inds = NMSOp(bboxes, scores, class_idxs, flags.iou_threshold)
		220	+ bboxes = bboxes[keep_inds]
		221	+ scores = scores[keep_inds]
		222	+ class_idxs = class_idxs[keep_inds]
		223	+ else:
		224	+ # repeat bbox and class idx for each class
		225	+ bboxes = bboxes[:, None, :].repeat(1, 80, 1) # (N, 4) -> (N, 80, 4)
		226	+ class_idxs = torch.arange(80, dtype=torch.long, device=bboxes.device
		227	+ )[None, :].repeat(bboxes.shape[0], 1) # (80) -> (N, 80)
		228	+
		229	+ # reshape bbox for torch nms
		230	+ bboxes = bboxes.view(-1, 4)
		231	+ scores = scores.view(-1)
		232	+ class_idxs = class_idxs.view(-1)
		233	+
		234	+ # keep topk max_per_img bbox
		235	+ if flags.max_per_img > 0 and flags.max_per_img < bboxes.shape[0]:
		236	+ _, topk_inds = scores.topk(flags.max_per_img)
		237	+ bboxes = bboxes[topk_inds, :]
		238	+ scores = scores[topk_inds]
		239	+ class_idxs = class_idxs[topk_inds]
		240	+
		241	+ # move to cpu if running on npu
		242	+ if bboxes.device != 'cpu':
		243	+ bboxes = bboxes.cpu()
		244	+ scores = scores.cpu()
		245	+ class_idxs = class_idxs.cpu()
		246	+
		247	+ # convert to numpy.ndarray
		248	+ bboxes = bboxes.numpy()
		249	+ scores = scores.numpy()
		250	+ class_idxs = class_idxs.numpy()
		251	+
		252	+ # make det result file
114	if flags.ifShowDetObj == True:	253	if flags.ifShowDetObj == True:
115	- imgCur = cv2.imread(current_img_size[2])	254	+ imgCur = cv2.imread(original_img_info[2])
116		255
117	det_results_str = ''	256	det_results_str = ''
118	- det_results = []	257	+ for idx in range(bboxes.shape[0]):
119	- for idx, class_ind in enumerate(predbox[:, 5]):	258	+ x1, y1, x2, y2 = bboxes[idx, :]
120	- if float(predbox[idx][4]) < float(flags.prob_thres):	259	+ predscore = scores[idx]
121	- continue	260	+ class_ind = class_idxs[idx]
122	- # skip negative class index
123	- if class_ind < 0 or class_ind > 80:
124	- continue
125		261
126	class_name = CLASSES[int(class_ind)]	262	class_name = CLASSES[int(class_ind)]
127	- det_results.append([class_name, str(predbox[idx][4]), predbox[idx][0], predbox[idx][1],	263	+ det_results_str += "{} {} {} {} {} {}\n".format(class_name, predscore, x1, y1, x2, y2)
128	- predbox[idx][2], predbox[idx][3]])
129	- det_results_str += "{} {} {} {} {} {}\n".format(class_name, str(predbox[idx][4]), predbox[idx][0],
130	- predbox[idx][1], predbox[idx][2], predbox[idx][3])
131	if flags.ifShowDetObj == True:	264	if flags.ifShowDetObj == True:
132	- imgCur=cv2.rectangle(imgCur, (int(predbox[idx][0]), int(predbox[idx][1])),	265	+ imgCur=cv2.rectangle(imgCur, (int(x1), int(y1)), (int(x2), int(y2)), (0, 255, 0), 1)
133	- (int(predbox[idx][2]), int(predbox[idx][3])), (0, 255, 0), 1)	266	+ imgCur = cv2.putText(imgCur, class_name + '\|' + str(predscore),
134	- imgCur = cv2.putText(imgCur, class_name + '\|' + str(predbox[idx][4]),	267	+ (int(x1), int(y1)), cv2.FONT_HERSHEY_SIMPLEX,
135	- (int(predbox[idx][0]), int(predbox[idx][1])), cv2.FONT_HERSHEY_SIMPLEX,
136	0.5, (0, 0, 255), 1)	268	0.5, (0, 0, 255), 1)
137		269
138	if flags.ifShowDetObj == True:	270	if flags.ifShowDetObj == True:
139	- print(os.path.join(det_results_path, bin_file + '.jpg'))
140	cv2.imwrite(os.path.join(det_results_path, bin_file + '.jpg'), imgCur, [int(cv2.IMWRITE_JPEG_QUALITY), 70])	271	cv2.imwrite(os.path.join(det_results_path, bin_file + '.jpg'), imgCur, [int(cv2.IMWRITE_JPEG_QUALITY), 70])
141		272
142	- det_results_file = os.path.join(det_results_path, bin_file + ".txt")
143	with open(det_results_file, "w") as detf:	273	with open(det_results_file, "w") as detf:
144	detf.write(det_results_str)	274	detf.write(det_results_str)
		275	+
		276	+ finished += 1
		277	+ speed = finished / (time.time() - time_start)
		278	+ print('processed {:5d}/{:<5d} images, speed: {:.2f}FPS'.format(finished, task_len, speed), end='\r')

MACL_PyTorch/contrib/cv/detection/SSD/requirements.txt+7-2

	@@ -1,8 +1,13 @@
1	onnx==1.7.0	1	onnx==1.7.0
2	torch==1.8.1	2	torch==1.8.1
3	torchvision==0.9.1	3	torchvision==0.9.1
4	-numpy==1.18.5	4	+numpy==1.21.6
		5	+Pillow==7.2.0
5	opencv-python==4.2.0.34	6	opencv-python==4.2.0.34
6	mmdet==2.8.0	7	mmdet==2.8.0
7	-mmcv-full==1.2.4	8	+mmcv-full==1.2.7
8	mmpycocotools==12.0.3	9	mmpycocotools==12.0.3
		10	+protobuf==3.20.0
		11	+onnxruntime==1.12.1
		12	+decorator==5.1.1
		13	+sympy==1.10.1

AACL_PyTorch/contrib/cv/detection/SSD/ssd_mmdet.diff+927-0

@@ -0,0 +1,927 @@
	1	+diff --git a/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py b/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py
	2	+index e9eb3579..1311b8e0 100644
	3	+--- a/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py
	4	++++ b/mmdet/core/bbox/coder/delta_xywh_bbox_coder.py
	5	+@@ -1,3 +1,7 @@
	6	++# Copyright (c) OpenMMLab. All rights reserved.
	7	++import warnings
	8	++
	9	++import mmcv
	10	+ import numpy as np
	11	+ import torch
	12	+
	13	+@@ -20,16 +24,25 @@ class DeltaXYWHBBoxCoder(BaseBBoxCoder):
	14	+ target for delta coordinates
	15	+ clip_border (bool, optional): Whether clip the objects outside the
	16	+ border of the image. Defaults to True.
	17	++ add_ctr_clamp (bool): Whether to add center clamp, when added, the
	18	++ predicted box is clamped is its center is too far away from
	19	++ the original anchor's center. Only used by YOLOF. Default False.
	20	++ ctr_clamp (int): the maximum pixel shift to clamp. Only used by YOLOF.
	21	++ Default 32.
	22	+ """
	23	+
	24	+ def __init__(self,
	25	+ target_means=(0., 0., 0., 0.),
	26	+ target_stds=(1., 1., 1., 1.),
	27	+- clip_border=True):
	28	++ clip_border=True,
	29	++ add_ctr_clamp=False,
	30	++ ctr_clamp=32):
	31	+ super(BaseBBoxCoder, self).__init__()
	32	+ self.means = target_means
	33	+ self.stds = target_stds
	34	+ self.clip_border = clip_border
	35	++ self.add_ctr_clamp = add_ctr_clamp
	36	++ self.ctr_clamp = ctr_clamp
	37	+
	38	+ def encode(self, bboxes, gt_bboxes):
	39	+ """Get box regression transformation deltas that can be used to
	40	+@@ -57,10 +70,16 @@ class DeltaXYWHBBoxCoder(BaseBBoxCoder):
	41	+ """Apply transformation `pred_bboxes` to `boxes`.
	42	+
	43	+ Args:
	44	+- boxes (torch.Tensor): Basic boxes.
	45	+- pred_bboxes (torch.Tensor): Encoded boxes with shape
	46	+- max_shape (tuple[int], optional): Maximum shape of boxes.
	47	+- Defaults to None.
	48	++ bboxes (torch.Tensor): Basic boxes. Shape (B, N, 4) or (N, 4)
	49	++ pred_bboxes (Tensor): Encoded offsets with respect to each roi.
	50	++ Has shape (B, N, num_classes * 4) or (B, N, 4) or
	51	++ (N, num_classes * 4) or (N, 4). Note N = num_anchors * W * H
	52	++ when rois is a grid of anchors.Offset encoding follows [1]_.
	53	++ max_shape (Sequence[int] or torch.Tensor or Sequence[
	54	++ Sequence[int]],optional): Maximum bounds for boxes, specifies
	55	++ (H, W, C) or (H, W). If bboxes shape is (B, N, 4), then
	56	++ the max_shape should be a Sequence[Sequence[int]]
	57	++ and the length of max_shape should also be B.
	58	+ wh_ratio_clip (float, optional): The allowed ratio between
	59	+ width and height.
	60	+
	61	+@@ -69,8 +88,28 @@ class DeltaXYWHBBoxCoder(BaseBBoxCoder):
	62	+ """
	63	+
	64	+ assert pred_bboxes.size(0) == bboxes.size(0)
	65	+- decoded_bboxes = delta2bbox(bboxes, pred_bboxes, self.means, self.stds,
	66	+- max_shape, wh_ratio_clip, self.clip_border)
	67	++ if pred_bboxes.ndim == 3:
	68	++ assert pred_bboxes.size(1) == bboxes.size(1)
	69	++
	70	++ if pred_bboxes.ndim == 2 and not torch.onnx.is_in_onnx_export():
	71	++ # single image decode
	72	++ decoded_bboxes = delta2bbox(bboxes, pred_bboxes, self.means,
	73	++ self.stds, max_shape, wh_ratio_clip,
	74	++ self.clip_border, self.add_ctr_clamp,
	75	++ self.ctr_clamp)
	76	++ else:
	77	++ if pred_bboxes.ndim == 3 and not torch.onnx.is_in_onnx_export():
	78	++ warnings.warn(
	79	++ 'DeprecationWarning: onnx_delta2bbox is deprecated '
	80	++ 'in the case of batch decoding and non-ONNX, '
	81	++ 'please use “delta2bbox” instead. In order to improve '
	82	++ 'the decoding speed, the batch function will no '
	83	++ 'longer be supported. ')
	84	++ decoded_bboxes = onnx_delta2bbox(bboxes, pred_bboxes, self.means,
	85	++ self.stds, max_shape,
	86	++ wh_ratio_clip, self.clip_border,
	87	++ self.add_ctr_clamp,
	88	++ self.ctr_clamp)
	89	+
	90	+ return decoded_bboxes
	91	+
	92	+@@ -126,7 +165,108 @@ def delta2bbox(rois,
	93	+ stds=(1., 1., 1., 1.),
	94	+ max_shape=None,
	95	+ wh_ratio_clip=16 / 1000,
	96	+- clip_border=True):
	97	++ clip_border=True,
	98	++ add_ctr_clamp=False,
	99	++ ctr_clamp=32):
	100	++ """Apply deltas to shift/scale base boxes.
	101	++
	102	++ Typically the rois are anchor or proposed bounding boxes and the deltas are
	103	++ network outputs used to shift/scale those boxes.
	104	++ This is the inverse function of :func:`bbox2delta`.
	105	++
	106	++ Args:
	107	++ rois (Tensor): Boxes to be transformed. Has shape (N, 4).
	108	++ deltas (Tensor): Encoded offsets relative to each roi.
	109	++ Has shape (N, num_classes * 4) or (N, 4). Note
	110	++ N = num_base_anchors * W * H, when rois is a grid of
	111	++ anchors. Offset encoding follows [1]_.
	112	++ means (Sequence[float]): Denormalizing means for delta coordinates.
	113	++ Default (0., 0., 0., 0.).
	114	++ stds (Sequence[float]): Denormalizing standard deviation for delta
	115	++ coordinates. Default (1., 1., 1., 1.).
	116	++ max_shape (tuple[int, int]): Maximum bounds for boxes, specifies
	117	++ (H, W). Default None.
	118	++ wh_ratio_clip (float): Maximum aspect ratio for boxes. Default
	119	++ 16 / 1000.
	120	++ clip_border (bool, optional): Whether clip the objects outside the
	121	++ border of the image. Default True.
	122	++ add_ctr_clamp (bool): Whether to add center clamp. When set to True,
	123	++ the center of the prediction bounding box will be clamped to
	124	++ avoid being too far away from the center of the anchor.
	125	++ Only used by YOLOF. Default False.
	126	++ ctr_clamp (int): the maximum pixel shift to clamp. Only used by YOLOF.
	127	++ Default 32.
	128	++
	129	++ Returns:
	130	++ Tensor: Boxes with shape (N, num_classes * 4) or (N, 4), where 4
	131	++ represent tl_x, tl_y, br_x, br_y.
	132	++
	133	++ References:
	134	++ .. [1] https://arxiv.org/abs/1311.2524
	135	++
	136	++ Example:
	137	++ >>> rois = torch.Tensor([[ 0., 0., 1., 1.],
	138	++ >>> [ 0., 0., 1., 1.],
	139	++ >>> [ 0., 0., 1., 1.],
	140	++ >>> [ 5., 5., 5., 5.]])
	141	++ >>> deltas = torch.Tensor([[ 0., 0., 0., 0.],
	142	++ >>> [ 1., 1., 1., 1.],
	143	++ >>> [ 0., 0., 2., -1.],
	144	++ >>> [ 0.7, -1.9, -0.5, 0.3]])
	145	++ >>> delta2bbox(rois, deltas, max_shape=(32, 32, 3))
	146	++ tensor([[0.0000, 0.0000, 1.0000, 1.0000],
	147	++ [0.1409, 0.1409, 2.8591, 2.8591],
	148	++ [0.0000, 0.3161, 4.1945, 0.6839],
	149	++ [5.0000, 5.0000, 5.0000, 5.0000]])
	150	++ """
	151	++ num_bboxes, num_classes = deltas.size(0), deltas.size(1) // 4
	152	++ if num_bboxes == 0:
	153	++ return deltas
	154	++
	155	++ deltas = deltas.reshape(-1, 4)
	156	++
	157	++ means = deltas.new_tensor(means).view(1, -1)
	158	++ stds = deltas.new_tensor(stds).view(1, -1)
	159	++ denorm_deltas = deltas * stds + means
	160	++
	161	++ dxy = denorm_deltas[:, :2]
	162	++ dwh = denorm_deltas[:, 2:]
	163	++
	164	++ # Compute width/height of each roi
	165	++ rois_ = rois.repeat(1, num_classes).reshape(-1, 4)
	166	++ pxy = ((rois_[:, :2] + rois_[:, 2:]) * 0.5)
	167	++ pwh = (rois_[:, 2:] - rois_[:, :2])
	168	++
	169	++ dxy_wh = pwh * dxy
	170	++
	171	++ max_ratio = np.abs(np.log(wh_ratio_clip))
	172	++ if add_ctr_clamp:
	173	++ dxy_wh = torch.clamp(dxy_wh, max=ctr_clamp, min=-ctr_clamp)
	174	++ dwh = torch.clamp(dwh, max=max_ratio)
	175	++ else:
	176	++ dwh = dwh.clamp(min=-max_ratio, max=max_ratio)
	177	++
	178	++ gxy = pxy + dxy_wh
	179	++ gwh = pwh * dwh.exp()
	180	++ x1y1 = gxy - (gwh * 0.5)
	181	++ x2y2 = gxy + (gwh * 0.5)
	182	++ bboxes = torch.cat([x1y1, x2y2], dim=-1)
	183	++ if clip_border and max_shape is not None:
	184	++ bboxes[..., 0::2].clamp_(min=0, max=max_shape[1])
	185	++ bboxes[..., 1::2].clamp_(min=0, max=max_shape[0])
	186	++ bboxes = bboxes.reshape(num_bboxes, -1)
	187	++ return bboxes
	188	++
	189	++
	190	++def onnx_delta2bbox(rois,
	191	++ deltas,
	192	++ means=(0., 0., 0., 0.),
	193	++ stds=(1., 1., 1., 1.),
	194	++ max_shape=None,
	195	++ wh_ratio_clip=16 / 1000,
	196	++ clip_border=True,
	197	++ add_ctr_clamp=False,
	198	++ ctr_clamp=32):
	199	+ """Apply deltas to shift/scale base boxes.
	200	+
	201	+ Typically the rois are anchor or proposed bounding boxes and the deltas are
	202	+@@ -134,21 +274,34 @@ def delta2bbox(rois,
	203	+ This is the inverse function of :func:`bbox2delta`.
	204	+
	205	+ Args:
	206	+- rois (Tensor): Boxes to be transformed. Has shape (N, 4)
	207	++ rois (Tensor): Boxes to be transformed. Has shape (N, 4) or (B, N, 4)
	208	+ deltas (Tensor): Encoded offsets with respect to each roi.
	209	+- Has shape (N, 4 * num_classes). Note N = num_anchors * W * H when
	210	+- rois is a grid of anchors. Offset encoding follows [1]_.
	211	+- means (Sequence[float]): Denormalizing means for delta coordinates
	212	++ Has shape (B, N, num_classes * 4) or (B, N, 4) or
	213	++ (N, num_classes * 4) or (N, 4). Note N = num_anchors * W * H
	214	++ when rois is a grid of anchors.Offset encoding follows [1]_.
	215	++ means (Sequence[float]): Denormalizing means for delta coordinates.
	216	++ Default (0., 0., 0., 0.).
	217	+ stds (Sequence[float]): Denormalizing standard deviation for delta
	218	+- coordinates
	219	+- max_shape (tuple[int, int]): Maximum bounds for boxes. specifies (H, W)
	220	++ coordinates. Default (1., 1., 1., 1.).
	221	++ max_shape (Sequence[int] or torch.Tensor or Sequence[
	222	++ Sequence[int]],optional): Maximum bounds for boxes, specifies
	223	++ (H, W, C) or (H, W). If rois shape is (B, N, 4), then
	224	++ the max_shape should be a Sequence[Sequence[int]]
	225	++ and the length of max_shape should also be B. Default None.
	226	+ wh_ratio_clip (float): Maximum aspect ratio for boxes.
	227	++ Default 16 / 1000.
	228	+ clip_border (bool, optional): Whether clip the objects outside the
	229	+- border of the image. Defaults to True.
	230	++ border of the image. Default True.
	231	++ add_ctr_clamp (bool): Whether to add center clamp, when added, the
	232	++ predicted box is clamped is its center is too far away from
	233	++ the original anchor's center. Only used by YOLOF. Default False.
	234	++ ctr_clamp (int): the maximum pixel shift to clamp. Only used by YOLOF.
	235	++ Default 32.
	236	+
	237	+ Returns:
	238	+- Tensor: Boxes with shape (N, 4), where columns represent
	239	+- tl_x, tl_y, br_x, br_y.
	240	++ Tensor: Boxes with shape (B, N, num_classes * 4) or (B, N, 4) or
	241	++ (N, num_classes * 4) or (N, 4), where 4 represent
	242	++ tl_x, tl_y, br_x, br_y.
	243	+
	244	+ References:
	245	+ .. [1] https://arxiv.org/abs/1311.2524
	246	+@@ -162,43 +315,76 @@ def delta2bbox(rois,
	247	+ >>> [ 1., 1., 1., 1.],
	248	+ >>> [ 0., 0., 2., -1.],
	249	+ >>> [ 0.7, -1.9, -0.5, 0.3]])
	250	+- >>> delta2bbox(rois, deltas, max_shape=(32, 32))
	251	++ >>> delta2bbox(rois, deltas, max_shape=(32, 32, 3))
	252	+ tensor([[0.0000, 0.0000, 1.0000, 1.0000],
	253	+ [0.1409, 0.1409, 2.8591, 2.8591],
	254	+ [0.0000, 0.3161, 4.1945, 0.6839],
	255	+ [5.0000, 5.0000, 5.0000, 5.0000]])
	256	+ """
	257	+- means = deltas.new_tensor(means).view(1, -1).repeat(1, deltas.size(1) // 4)
	258	+- stds = deltas.new_tensor(stds).view(1, -1).repeat(1, deltas.size(1) // 4)
	259	++ means = deltas.new_tensor(means).view(1,
	260	++ -1).repeat(1,
	261	++ deltas.size(-1) // 4)
	262	++ stds = deltas.new_tensor(stds).view(1, -1).repeat(1, deltas.size(-1) // 4)
	263	+ denorm_deltas = deltas * stds + means
	264	+- dx = denorm_deltas[:, 0::4]
	265	+- dy = denorm_deltas[:, 1::4]
	266	+- dw = denorm_deltas[:, 2::4]
	267	+- dh = denorm_deltas[:, 3::4]
	268	+- max_ratio = np.abs(np.log(wh_ratio_clip))
	269	+- dw = dw.clamp(min=-max_ratio, max=max_ratio)
	270	+- dh = dh.clamp(min=-max_ratio, max=max_ratio)
	271	++ dx = denorm_deltas[..., 0::4]
	272	++ dy = denorm_deltas[..., 1::4]
	273	++ dw = denorm_deltas[..., 2::4]
	274	++ dh = denorm_deltas[..., 3::4]
	275	++
	276	++ x1, y1 = rois[..., 0], rois[..., 1]
	277	++ x2, y2 = rois[..., 2], rois[..., 3]
	278	+ # Compute center of each roi
	279	+- px = ((rois[:, 0] + rois[:, 2]) * 0.5).unsqueeze(1).expand_as(dx)
	280	+- py = ((rois[:, 1] + rois[:, 3]) * 0.5).unsqueeze(1).expand_as(dy)
	281	++ px = ((x1 + x2) * 0.5).unsqueeze(-1).expand_as(dx)
	282	++ py = ((y1 + y2) * 0.5).unsqueeze(-1).expand_as(dy)
	283	+ # Compute width/height of each roi
	284	+- pw = (rois[:, 2] - rois[:, 0]).unsqueeze(1).expand_as(dw)
	285	+- ph = (rois[:, 3] - rois[:, 1]).unsqueeze(1).expand_as(dh)
	286	++ pw = (x2 - x1).unsqueeze(-1).expand_as(dw)
	287	++ ph = (y2 - y1).unsqueeze(-1).expand_as(dh)
	288	++
	289	++ dx_width = pw * dx
	290	++ dy_height = ph * dy
	291	++
	292	++ max_ratio = np.abs(np.log(wh_ratio_clip))
	293	++ if add_ctr_clamp:
	294	++ dx_width = torch.clamp(dx_width, max=ctr_clamp, min=-ctr_clamp)
	295	++ dy_height = torch.clamp(dy_height, max=ctr_clamp, min=-ctr_clamp)
	296	++ dw = torch.clamp(dw, max=max_ratio)
	297	++ dh = torch.clamp(dh, max=max_ratio)
	298	++ else:
	299	++ dw = dw.clamp(min=-max_ratio, max=max_ratio)
	300	++ dh = dh.clamp(min=-max_ratio, max=max_ratio)
	301	+ # Use exp(network energy) to enlarge/shrink each roi
	302	+ gw = pw * dw.exp()
	303	+ gh = ph * dh.exp()
	304	+ # Use network energy to shift the center of each roi
	305	+- gx = px + pw * dx
	306	+- gy = py + ph * dy
	307	++ gx = px + dx_width
	308	++ gy = py + dy_height
	309	+ # Convert center-xy/width/height to top-left, bottom-right
	310	+ x1 = gx - gw * 0.5
	311	+ y1 = gy - gh * 0.5
	312	+ x2 = gx + gw * 0.5
	313	+ y2 = gy + gh * 0.5
	314	+- if clip_border and max_shape is not None:
	315	+- x1 = x1.clamp(min=0, max=max_shape[1])
	316	+- y1 = y1.clamp(min=0, max=max_shape[0])
	317	+- x2 = x2.clamp(min=0, max=max_shape[1])
	318	+- y2 = y2.clamp(min=0, max=max_shape[0])
	319	++
	320	+ bboxes = torch.stack([x1, y1, x2, y2], dim=-1).view(deltas.size())
	321	++
	322	++ if clip_border and max_shape is not None:
	323	++ # clip bboxes with dynamic `min` and `max` for onnx
	324	++ if torch.onnx.is_in_onnx_export():
	325	++ from mmdet.core.export.onnx_helper import dynamic_clip_for_onnx
	326	++ x1, y1, x2, y2 = dynamic_clip_for_onnx(x1, y1, x2, y2, max_shape)
	327	++ bboxes = torch.stack([x1, y1, x2, y2], dim=-1).view(deltas.size())
	328	++ return bboxes
	329	++ if not isinstance(max_shape, torch.Tensor):
	330	++ max_shape = x1.new_tensor(max_shape)
	331	++ max_shape = max_shape[..., :2].type_as(x1)
	332	++ if max_shape.ndim == 2:
	333	++ assert bboxes.ndim == 3
	334	++ assert max_shape.size(0) == bboxes.size(0)
	335	++
	336	++ min_xy = x1.new_tensor(0)
	337	++ max_xy = torch.cat(
	338	++ [max_shape] * (deltas.size(-1) // 2),
	339	++ dim=-1).flip(-1).unsqueeze(-2)
	340	++ bboxes = torch.where(bboxes < min_xy, min_xy, bboxes)
	341	++ bboxes = torch.where(bboxes > max_xy, max_xy, bboxes)
	342	++
	343	+ return bboxes
	344	+diff --git a/mmdet/core/export/onnx_helper.py b/mmdet/core/export/onnx_helper.py
	345	+new file mode 100644
	346	+index 00000000..9abd220b
	347	+--- /dev/null
	348	++++ b/mmdet/core/export/onnx_helper.py
	349	+@@ -0,0 +1,245 @@
	350	++# Copyright (c) OpenMMLab. All rights reserved.
	351	++import os
	352	++
	353	++import torch
	354	++
	355	++
	356	++def dynamic_clip_for_onnx(x1, y1, x2, y2, max_shape):
	357	++ """Clip boxes dynamically for onnx.
	358	++
	359	++ Since torch.clamp cannot have dynamic `min` and `max`, we scale the
	360	++ boxes by 1/max_shape and clamp in the range [0, 1].
	361	++
	362	++ Args:
	363	++ x1 (Tensor): The x1 for bounding boxes.
	364	++ y1 (Tensor): The y1 for bounding boxes.
	365	++ x2 (Tensor): The x2 for bounding boxes.
	366	++ y2 (Tensor): The y2 for bounding boxes.
	367	++ max_shape (Tensor or torch.Size): The (H,W) of original image.
	368	++ Returns:
	369	++ tuple(Tensor): The clipped x1, y1, x2, y2.
	370	++ """
	371	++ # assert isinstance(
	372	++ # max_shape,
	373	++ # torch.Tensor), '`max_shape` should be tensor of (h,w) for onnx, got {}'.format(max_shape.__class__.__name__)
	374	++
	375	++ assert isinstance(max_shape, (torch.Tensor, torch.Size, list, tuple)), '`max_shape` should be ' + \
	376	++ 'torch.Tensor/torch.Size/list/tuple of (h, w) for onnx, got {}'.format(max_shape.__class__.__name__)
	377	++ if not isinstance(max_shape, torch.Tensor):
	378	++ max_shape = torch.tensor(max_shape, dtype=x1.dtype, device=x1.device)
	379	++ else:
	380	++ max_shape = max_shape.type_as(x1)
	381	++
	382	++ # scale by 1/max_shape
	383	++ x1 = x1 / max_shape[1]
	384	++ y1 = y1 / max_shape[0]
	385	++ x2 = x2 / max_shape[1]
	386	++ y2 = y2 / max_shape[0]
	387	++
	388	++ # clamp [0, 1]
	389	++ x1 = torch.clamp(x1, 0, 1)
	390	++ y1 = torch.clamp(y1, 0, 1)
	391	++ x2 = torch.clamp(x2, 0, 1)
	392	++ y2 = torch.clamp(y2, 0, 1)
	393	++
	394	++ # scale back
	395	++ x1 = x1 * max_shape[1]
	396	++ y1 = y1 * max_shape[0]
	397	++ x2 = x2 * max_shape[1]
	398	++ y2 = y2 * max_shape[0]
	399	++ return x1, y1, x2, y2
	400	++
	401	++
	402	++def get_k_for_topk(k, size):
	403	++ """Get k of TopK for onnx exporting.
	404	++
	405	++ The K of TopK in TensorRT should not be a Tensor, while in ONNX Runtime
	406	++ it could be a Tensor.Due to dynamic shape feature, we have to decide
	407	++ whether to do TopK and what K it should be while exporting to ONNX.
	408	++ If returned K is less than zero, it means we do not have to do
	409	++ TopK operation.
	410	++
	411	++ Args:
	412	++ k (int or Tensor): The set k value for nms from config file.
	413	++ size (Tensor or torch.Size): The number of elements of \
	414	++ TopK's input tensor
	415	++ Returns:
	416	++ tuple: (int or Tensor): The final K for TopK.
	417	++ """
	418	++ ret_k = -1
	419	++ if k <= 0 or size <= 0:
	420	++ return ret_k
	421	++ if torch.onnx.is_in_onnx_export():
	422	++ is_trt_backend = os.environ.get('ONNX_BACKEND') == 'MMCVTensorRT'
	423	++ if is_trt_backend:
	424	++ # TensorRT does not support dynamic K with TopK op
	425	++ if 0 < k < size:
	426	++ ret_k = k
	427	++ else:
	428	++ # Always keep topk op for dynamic input in onnx for ONNX Runtime
	429	++ ret_k = torch.where(k < size, k, size)
	430	++ elif k < size:
	431	++ ret_k = k
	432	++ else:
	433	++ # ret_k is -1
	434	++ pass
	435	++ return ret_k
	436	++
	437	++
	438	++def add_dummy_nms_for_onnx(boxes,
	439	++ scores,
	440	++ max_output_boxes_per_class=1000,
	441	++ iou_threshold=0.5,
	442	++ score_threshold=0.05,
	443	++ pre_top_k=-1,
	444	++ after_top_k=-1,
	445	++ labels=None):
	446	++ """Create a dummy onnx::NonMaxSuppression op while exporting to ONNX.
	447	++
	448	++ This function helps exporting to onnx with batch and multiclass NMS op.
	449	++ It only supports class-agnostic detection results. That is, the scores
	450	++ is of shape (N, num_bboxes, num_classes) and the boxes is of shape
	451	++ (N, num_boxes, 4).
	452	++
	453	++ Args:
	454	++ boxes (Tensor): The bounding boxes of shape [N, num_boxes, 4]
	455	++ scores (Tensor): The detection scores of shape
	456	++ [N, num_boxes, num_classes]
	457	++ max_output_boxes_per_class (int): Maximum number of output
	458	++ boxes per class of nms. Defaults to 1000.
	459	++ iou_threshold (float): IOU threshold of nms. Defaults to 0.5
	460	++ score_threshold (float): score threshold of nms.
	461	++ Defaults to 0.05.
	462	++ pre_top_k (bool): Number of top K boxes to keep before nms.
	463	++ Defaults to -1.
	464	++ after_top_k (int): Number of top K boxes to keep after nms.
	465	++ Defaults to -1.
	466	++ labels (Tensor, optional): It not None, explicit labels would be used.
	467	++ Otherwise, labels would be automatically generated using
	468	++ num_classed. Defaults to None.
	469	++
	470	++ Returns:
	471	++ tuple[Tensor, Tensor]: dets of shape [N, num_det, 5]
	472	++ and class labels of shape [N, num_det].
	473	++ """
	474	++ max_output_boxes_per_class = torch.LongTensor([max_output_boxes_per_class])
	475	++ iou_threshold = torch.tensor([iou_threshold], dtype=torch.float32)
	476	++ score_threshold = torch.tensor([score_threshold], dtype=torch.float32)
	477	++ batch_size = scores.shape[0]
	478	++ num_class = scores.shape[2]
	479	++
	480	++ if pre_top_k > 0:
	481	++ nms_pre = torch.tensor(pre_top_k, device=scores.device, dtype=torch.long)
	482	++ nms_pre = get_k_for_topk(nms_pre, boxes.shape[1])
	483	++
	484	++ if nms_pre > 0:
	485	++ max_scores, _ = scores.max(-1)
	486	++ _, topk_inds = max_scores.topk(nms_pre)
	487	++ batch_inds = torch.arange(batch_size).view(
	488	++ -1, 1).expand_as(topk_inds).long()
	489	++ # Avoid onnx2tensorrt issue in https://github.com/NVIDIA/TensorRT/issues/1134 # noqa: E501
	490	++ # transformed_inds = boxes.shape[1] * batch_inds + topk_inds
	491	++ transformed_inds = (boxes.shape[1] * batch_inds.int()) + topk_inds.int()
	492	++ transformed_inds = transformed_inds.long()
	493	++ boxes = boxes.reshape(-1, 4)[transformed_inds, :].reshape(
	494	++ batch_size, -1, 4)
	495	++ scores = scores.reshape(-1, num_class)[transformed_inds, :].reshape(
	496	++ batch_size, -1, num_class)
	497	++ if labels is not None:
	498	++ labels = labels.reshape(-1, 1)[transformed_inds].reshape(
	499	++ batch_size, -1)
	500	++
	501	++ scores = scores.permute(0, 2, 1)
	502	++ num_box = boxes.shape[1]
	503	++ # turn off tracing to create a dummy output of nms
	504	++ state = torch._C._get_tracing_state()
	505	++ # dummy indices of nms's output
	506	++ num_fake_det = 2
	507	++ batch_inds = torch.randint(batch_size, (num_fake_det, 1))
	508	++ cls_inds = torch.randint(num_class, (num_fake_det, 1))
	509	++ box_inds = torch.randint(num_box, (num_fake_det, 1))
	510	++ indices = torch.cat([batch_inds, cls_inds, box_inds], dim=1)
	511	++ output = indices
	512	++ setattr(DummyONNXNMSop, 'output', output)
	513	++
	514	++ # open tracing
	515	++ torch._C._set_tracing_state(state)
	516	++ selected_indices = DummyONNXNMSop.apply(boxes, scores,
	517	++ max_output_boxes_per_class,
	518	++ iou_threshold, score_threshold)
	519	++
	520	++ batch_inds, cls_inds = selected_indices[:, 0], selected_indices[:, 1]
	521	++ box_inds = selected_indices[:, 2]
	522	++ if labels is None:
	523	++ labels = torch.arange(num_class, dtype=torch.long).to(scores.device)
	524	++ labels = labels.view(1, num_class, 1).expand_as(scores)
	525	++ scores = scores.reshape(-1, 1)
	526	++ boxes = boxes.reshape(batch_size, -1).repeat(1, num_class).reshape(-1, 4)
	527	++ # pos_inds = (num_class * batch_inds + cls_inds) * num_box + box_inds # original
	528	++ pos_inds = (num_class * batch_inds.int()) + cls_inds.int()
	529	++ pos_inds = (pos_inds * num_box.int()) + box_inds.int()
	530	++ pos_inds = pos_inds.long()
	531	++ # pos_inds = (batch_inds.new_tensor(num_class) * batch_inds + cls_inds) * batch_inds.new_tensor(num_box) + box_inds
	532	++ mask = scores.new_zeros(scores.shape)
	533	++ # Avoid onnx2tensorrt issue in https://github.com/NVIDIA/TensorRT/issues/1134 # noqa: E501
	534	++ # PyTorch style code: mask[batch_inds, box_inds] += 1
	535	++ mask[pos_inds, :] += 1
	536	++ scores = scores * mask
	537	++ boxes = boxes * mask
	538	++
	539	++ scores = scores.reshape(batch_size, -1)
	540	++ boxes = boxes.reshape(batch_size, -1, 4)
	541	++ labels = labels.reshape(batch_size, -1)
	542	++
	543	++ if boxes.dtype != torch.float:
	544	++ boxes = boxes.float()
	545	++ scores = scores.float()
	546	++
	547	++ if after_top_k > 0:
	548	++ nms_after = torch.tensor(
	549	++ after_top_k, device=scores.device, dtype=torch.long)
	550	++ nms_after = get_k_for_topk(nms_after, num_box * num_class)
	551	++
	552	++ if nms_after > 0:
	553	++ _, topk_inds = scores.topk(nms_after)
	554	++ batch_inds = torch.arange(batch_size).view(-1, 1).expand_as(topk_inds).long()
	555	++ # Avoid onnx2tensorrt issue in https://github.com/NVIDIA/TensorRT/issues/1134 # noqa: E501
	556	++ batch_inds = scores.shape[1] * batch_inds
	557	++ # transformed_inds = batch_inds + topk_inds
	558	++ transformed_inds = batch_inds.int() + topk_inds.int()
	559	++ transformed_inds = transformed_inds.long()
	560	++ scores = scores.reshape(-1, 1)[transformed_inds, :].reshape(
	561	++ batch_size, -1)
	562	++ boxes = boxes.reshape(-1, 4)[transformed_inds, :].reshape(
	563	++ batch_size, -1, 4)
	564	++ labels = labels.reshape(-1, 1)[transformed_inds, :].reshape(
	565	++ batch_size, -1)
	566	++
	567	++ scores = scores.unsqueeze(2)
	568	++ dets = torch.cat([boxes, scores], dim=2)
	569	++ return dets, labels
	570	++
	571	++
	572	++class DummyONNXNMSop(torch.autograd.Function):
	573	++ """DummyONNXNMSop.
	574	++
	575	++ This class is only for creating onnx::NonMaxSuppression.
	576	++ """
	577	++
	578	++ @staticmethod
	579	++ def forward(ctx, boxes, scores, max_output_boxes_per_class, iou_threshold,
	580	++ score_threshold):
	581	++
	582	++ return DummyONNXNMSop.output
	583	++
	584	++ @staticmethod
	585	++ def symbolic(g, boxes, scores, max_output_boxes_per_class, iou_threshold,
	586	++ score_threshold):
	587	++ return g.op(
	588	++ 'NonMaxSuppression',
	589	++ boxes,
	590	++ scores,
	591	++ max_output_boxes_per_class,
	592	++ iou_threshold,
	593	++ score_threshold,
	594	++ outputs=1)
	595	+diff --git a/mmdet/core/export/pytorch2onnx.py b/mmdet/core/export/pytorch2onnx.py
	596	+index 8f9309df..b9f43d48 100644
	597	+--- a/mmdet/core/export/pytorch2onnx.py
	598	++++ b/mmdet/core/export/pytorch2onnx.py
	599	+@@ -39,6 +39,7 @@ def generate_inputs_and_wrap_model(config_path, checkpoint_path, input_config):
	600	+
	601	+ model = build_model_from_cfg(config_path, checkpoint_path)
	602	+ one_img, one_meta = preprocess_example_input(input_config)
	603	++ one_meta['img_shape_for_onnx'] = one_img.shape[-2:]
	604	+ tensor_data = [one_img]
	605	+ model.forward = partial(
	606	+ model.forward, img_metas=[[one_meta]], return_loss=False)
	607	+diff --git a/mmdet/models/backbones/ssd_vgg.py b/mmdet/models/backbones/ssd_vgg.py
	608	+index cbc4fbb2..4bb7e37a 100644
	609	+--- a/mmdet/models/backbones/ssd_vgg.py
	610	++++ b/mmdet/models/backbones/ssd_vgg.py
	611	+@@ -162,8 +162,14 @@ class L2Norm(nn.Module):
	612	+
	613	+ def forward(self, x):
	614	+ """Forward function."""
	615	+- # normalization layer convert to FP32 in FP16 training
	616	++ # # normalization layer convert to FP32 in FP16 training
	617	++ # x_float = x.float()
	618	++ # norm = x_float.pow(2).sum(1, keepdim=True).sqrt() + self.eps
	619	++ # return (self.weight[None, :, None, None].float().expand_as(x_float) *
	620	++ # x_float / norm).type_as(x)
	621	++
	622	+ x_float = x.float()
	623	+- norm = x_float.pow(2).sum(1, keepdim=True).sqrt() + self.eps
	624	++ x_mul = x_float * x_float
	625	++ norm = x_mul.sum(1, keepdim=True).sqrt() + self.eps
	626	+ return (self.weight[None, :, None, None].float().expand_as(x_float) *
	627	+ x_float / norm).type_as(x)
	628	+diff --git a/mmdet/models/dense_heads/anchor_head.py b/mmdet/models/dense_heads/anchor_head.py
	629	+index a5bb4137..1aef3804 100644
	630	+--- a/mmdet/models/dense_heads/anchor_head.py
	631	++++ b/mmdet/models/dense_heads/anchor_head.py
	632	+@@ -487,6 +487,162 @@ class AnchorHead(BaseDenseHead, BBoxTestMixin):
	633	+ num_total_samples=num_total_samples)
	634	+ return dict(loss_cls=losses_cls, loss_bbox=losses_bbox)
	635	+
	636	++ @force_fp32(apply_to=('cls_scores', 'bbox_preds'))
	637	++ def onnx_export(self,
	638	++ cls_scores,
	639	++ bbox_preds,
	640	++ score_factors=None,
	641	++ img_metas=None,
	642	++ with_nms=True):
	643	++ """Transform network output for a batch into bbox predictions.
	644	++
	645	++ Args:
	646	++ cls_scores (list[Tensor]): Box scores for each scale level
	647	++ with shape (N, num_points * num_classes, H, W).
	648	++ bbox_preds (list[Tensor]): Box energies / deltas for each scale
	649	++ level with shape (N, num_points * 4, H, W).
	650	++ score_factors (list[Tensor]): score_factors for each s
	651	++ cale level with shape (N, num_points * 1, H, W).
	652	++ Default: None.
	653	++ img_metas (list[dict]): Meta information of each image, e.g.,
	654	++ image size, scaling factor, etc. Default: None.
	655	++ with_nms (bool): Whether apply nms to the bboxes. Default: True.
	656	++
	657	++ Returns:
	658	++ tuple[Tensor, Tensor] \| list[tuple]: When `with_nms` is True,
	659	++ it is tuple[Tensor, Tensor], first tensor bboxes with shape
	660	++ [N, num_det, 5], 5 arrange as (x1, y1, x2, y2, score)
	661	++ and second element is class labels of shape [N, num_det].
	662	++ When `with_nms` is False, first tensor is bboxes with
	663	++ shape [N, num_det, 4], second tensor is raw score has
	664	++ shape [N, num_det, num_classes].
	665	++ """
	666	++ assert len(cls_scores) == len(bbox_preds)
	667	++
	668	++ num_levels = len(cls_scores)
	669	++
	670	++ featmap_sizes = [featmap.size()[-2:] for featmap in cls_scores]
	671	++
	672	++ mlvl_priors = self.anchor_generator.grid_anchors(
	673	++ featmap_sizes, device=bbox_preds[0].device)
	674	++
	675	++ mlvl_cls_scores = [cls_scores[i].detach() for i in range(num_levels)]
	676	++ mlvl_bbox_preds = [bbox_preds[i].detach() for i in range(num_levels)]
	677	++
	678	++ assert len(
	679	++ img_metas
	680	++ ) == 1, 'Only support one input image while in exporting to ONNX'
	681	++ img_shape = torch.tensor(
	682	++ img_metas[0]['img_shape_for_onnx'],
	683	++ dtype=torch.long,
	684	++ device=bbox_preds[0].device)
	685	++
	686	++ cfg = self.test_cfg
	687	++ assert len(cls_scores) == len(bbox_preds) == len(mlvl_priors)
	688	++ device = cls_scores[0].device
	689	++ batch_size = cls_scores[0].shape[0]
	690	++ # convert to tensor to keep tracing
	691	++ nms_pre_tensor = torch.tensor(
	692	++ cfg.get('nms_pre', -1), device=device, dtype=torch.long)
	693	++
	694	++ # e.g. Retina, FreeAnchor, etc.
	695	++ if score_factors is None:
	696	++ with_score_factors = False
	697	++ mlvl_score_factor = [None for _ in range(num_levels)]
	698	++ else:
	699	++ # e.g. FCOS, PAA, ATSS, etc.
	700	++ with_score_factors = True
	701	++ mlvl_score_factor = [
	702	++ score_factors[i].detach() for i in range(num_levels)
	703	++ ]
	704	++ mlvl_score_factors = []
	705	++
	706	++ mlvl_batch_bboxes = []
	707	++ mlvl_scores = []
	708	++
	709	++ for cls_score, bbox_pred, score_factors, priors in zip(
	710	++ mlvl_cls_scores, mlvl_bbox_preds, mlvl_score_factor,
	711	++ mlvl_priors):
	712	++ assert cls_score.size()[-2:] == bbox_pred.size()[-2:]
	713	++
	714	++ scores = cls_score.permute(0, 2, 3,
	715	++ 1).reshape(batch_size, -1,
	716	++ self.cls_out_channels)
	717	++ if self.use_sigmoid_cls:
	718	++ scores = scores.sigmoid()
	719	++ nms_pre_score = scores
	720	++ else:
	721	++ scores = scores.softmax(-1)
	722	++ nms_pre_score = scores
	723	++
	724	++ if with_score_factors:
	725	++ score_factors = score_factors.permute(0, 2, 3, 1).reshape(
	726	++ batch_size, -1).sigmoid()
	727	++ bbox_pred = bbox_pred.permute(0, 2, 3,
	728	++ 1).reshape(batch_size, -1, 4)
	729	++ priors = priors.expand(batch_size, -1, priors.size(-1))
	730	++ # Get top-k predictions
	731	++ from mmdet.core.export.onnx_helper import get_k_for_topk
	732	++ nms_pre = get_k_for_topk(nms_pre_tensor, bbox_pred.shape[1])
	733	++ if nms_pre > 0:
	734	++
	735	++ if with_score_factors:
	736	++ nms_pre_score = (nms_pre_score * score_factors[..., None])
	737	++ else:
	738	++ nms_pre_score = nms_pre_score
	739	++
	740	++ # Get maximum scores for foreground classes.
	741	++ if self.use_sigmoid_cls:
	742	++ max_scores, _ = nms_pre_score.max(-1)
	743	++ else:
	744	++ # remind that we set FG labels to [0, num_class-1]
	745	++ # since mmdet v2.0
	746	++ # BG cat_id: num_class
	747	++ max_scores, _ = nms_pre_score[..., :-1].max(-1)
	748	++ _, topk_inds = max_scores.topk(nms_pre)
	749	++
	750	++ batch_inds = torch.arange(
	751	++ batch_size, device=bbox_pred.device).view(
	752	++ -1, 1).expand_as(topk_inds).long()
	753	++ # Avoid onnx2tensorrt issue in https://github.com/NVIDIA/TensorRT/issues/1134 # noqa: E501
	754	++ # transformed_inds = bbox_pred.shape[1] * batch_inds + topk_inds
	755	++ transformed_inds = (bbox_pred.shape[1] * batch_inds).int() + topk_inds.int()
	756	++ transformed_inds = transformed_inds.long()
	757	++ priors = priors.reshape(
	758	++ -1, priors.size(-1))[transformed_inds, :].reshape(
	759	++ batch_size, -1, priors.size(-1))
	760	++ bbox_pred = bbox_pred.reshape(-1,
	761	++ 4)[transformed_inds, :].reshape(
	762	++ batch_size, -1, 4)
	763	++ scores = scores.reshape(
	764	++ -1, self.cls_out_channels)[transformed_inds, :].reshape(
	765	++ batch_size, -1, self.cls_out_channels)
	766	++ if with_score_factors:
	767	++ score_factors = score_factors.reshape(
	768	++ -1, 1)[transformed_inds].reshape(batch_size, -1)
	769	++
	770	++ bboxes = self.bbox_coder.decode(
	771	++ priors, bbox_pred, max_shape=img_shape)
	772	++
	773	++ mlvl_batch_bboxes.append(bboxes)
	774	++ mlvl_scores.append(scores)
	775	++ if with_score_factors:
	776	++ mlvl_score_factors.append(score_factors)
	777	++
	778	++ batch_bboxes = torch.cat(mlvl_batch_bboxes, dim=1)
	779	++ batch_scores = torch.cat(mlvl_scores, dim=1)
	780	++ if with_score_factors:
	781	++ batch_score_factors = torch.cat(mlvl_score_factors, dim=1)
	782	++
	783	++ if not self.use_sigmoid_cls:
	784	++ batch_scores = batch_scores[..., :self.num_classes]
	785	++
	786	++ if with_score_factors:
	787	++ batch_scores = batch_scores * (batch_score_factors.unsqueeze(2))
	788	++
	789	++ # directly return bboxes without NMS
	790	++ return batch_bboxes, batch_scores
	791	++
	792	+ @force_fp32(apply_to=('cls_scores', 'bbox_preds'))
	793	+ def get_bboxes(self,
	794	+ cls_scores,
	795	+@@ -545,38 +701,45 @@ class AnchorHead(BaseDenseHead, BBoxTestMixin):
	796	+ >>> assert det_bboxes.shape[1] == 5
	797	+ >>> assert len(det_bboxes) == len(det_labels) == cfg.max_per_img
	798	+ """
	799	+- assert len(cls_scores) == len(bbox_preds)
	800	+- num_levels = len(cls_scores)
	801	+-
	802	+- device = cls_scores[0].device
	803	+- featmap_sizes = [cls_scores[i].shape[-2:] for i in range(num_levels)]
	804	+- mlvl_anchors = self.anchor_generator.grid_anchors(
	805	+- featmap_sizes, device=device)
	806	+-
	807	+- result_list = []
	808	+- for img_id in range(len(img_metas)):
	809	+- cls_score_list = [
	810	+- cls_scores[i][img_id].detach() for i in range(num_levels)
	811	+- ]
	812	+- bbox_pred_list = [
	813	+- bbox_preds[i][img_id].detach() for i in range(num_levels)
	814	+- ]
	815	+- img_shape = img_metas[img_id]['img_shape']
	816	+- scale_factor = img_metas[img_id]['scale_factor']
	817	+- if with_nms:
	818	+- # some heads don't support with_nms argument
	819	+- proposals = self._get_bboxes_single(cls_score_list,
	820	+- bbox_pred_list,
	821	+- mlvl_anchors, img_shape,
	822	+- scale_factor, cfg, rescale)
	823	+- else:
	824	+- proposals = self._get_bboxes_single(cls_score_list,
	825	+- bbox_pred_list,
	826	+- mlvl_anchors, img_shape,
	827	+- scale_factor, cfg, rescale,
	828	+- with_nms)
	829	+- result_list.append(proposals)
	830	+- return result_list
	831	++ if torch.onnx.is_in_onnx_export():
	832	++ return self.onnx_export(cls_scores,
	833	++ bbox_preds,
	834	++ score_factors=None,
	835	++ img_metas=img_metas,
	836	++ with_nms=with_nms)
	837	++ else:
	838	++ assert len(cls_scores) == len(bbox_preds)
	839	++ num_levels = len(cls_scores)
	840	++
	841	++ device = cls_scores[0].device
	842	++ featmap_sizes = [cls_scores[i].shape[-2:] for i in range(num_levels)]
	843	++ mlvl_anchors = self.anchor_generator.grid_anchors(
	844	++ featmap_sizes, device=device)
	845	++
	846	++ result_list = []
	847	++ for img_id in range(len(img_metas)):
	848	++ cls_score_list = [
	849	++ cls_scores[i][img_id].detach() for i in range(num_levels)
	850	++ ]
	851	++ bbox_pred_list = [
	852	++ bbox_preds[i][img_id].detach() for i in range(num_levels)
	853	++ ]
	854	++ img_shape = img_metas[img_id]['img_shape']
	855	++ scale_factor = img_metas[img_id]['scale_factor']
	856	++ if with_nms:
	857	++ # some heads don't support with_nms argument
	858	++ proposals = self._get_bboxes_single(cls_score_list,
	859	++ bbox_pred_list,
	860	++ mlvl_anchors, img_shape,
	861	++ scale_factor, cfg, rescale)
	862	++ else:
	863	++ proposals = self._get_bboxes_single(cls_score_list,
	864	++ bbox_pred_list,
	865	++ mlvl_anchors, img_shape,
	866	++ scale_factor, cfg, rescale,
	867	++ with_nms)
	868	++ result_list.append(proposals)
	869	++ return result_list
	870	+
	871	+ def _get_bboxes_single(self,
	872	+ cls_score_list,
	873	+@@ -612,6 +775,7 @@ class AnchorHead(BaseDenseHead, BBoxTestMixin):
	874	+ are bounding box positions (tl_x, tl_y, br_x, br_y) and the
	875	+ 5-th column is a score between 0 and 1.
	876	+ """
	877	++ print('in _get_bboxes_single')
	878	+ cfg = self.test_cfg if cfg is None else cfg
	879	+ assert len(cls_score_list) == len(bbox_pred_list) == len(mlvl_anchors)
	880	+ mlvl_bboxes = []
	881	+diff --git a/tools/pytorch2onnx.py b/tools/pytorch2onnx.py
	882	+index a8e7487b..97ed2d09 100644
	883	+--- a/tools/pytorch2onnx.py
	884	++++ b/tools/pytorch2onnx.py
	885	+@@ -33,23 +33,32 @@ def pytorch2onnx(config_path,
	886	+ one_img, one_meta = preprocess_example_input(input_config)
	887	+ model, tensor_data = generate_inputs_and_wrap_model(
	888	+ config_path, checkpoint_path, input_config)
	889	++
	890	++ input_names = ['input']
	891	++ dynamic_axes = {'input': {0: 'batch', 2: 'height', 3: 'width'}}
	892	++
	893	+ output_names = ['boxes']
	894	++ dynamic_axes['boxes'] = {0: 'batch'}
	895	+ if model.with_bbox:
	896	+ output_names.append('labels')
	897	++ dynamic_axes['labels'] = {0: 'batch'}
	898	+ if model.with_mask:
	899	+ output_names.append('masks')
	900	++ dynamic_axes['masks'] = {0: 'batch'}
	901	+
	902	+ torch.onnx.export(
	903	+ model,
	904	+ tensor_data,
	905	+ output_file,
	906	+- input_names=['input'],
	907	++ input_names=input_names,
	908	+ output_names=output_names,
	909	++ dynamic_axes=dynamic_axes,
	910	+ export_params=True,
	911	+ keep_initializers_as_inputs=True,
	912	+ do_constant_folding=True,
	913	+ verbose=show,
	914	+- opset_version=opset_version)
	915	++ opset_version=opset_version,
	916	++ enable_onnx_checker=False)
	917	+
	918	+ model.forward = orig_model.forward
	919	+ print(f'Successfully exported ONNX model: {output_file}')
	920	+@@ -67,6 +76,7 @@ def pytorch2onnx(config_path,
	921	+ tensor_data = [one_img]
	922	+ # check the numerical value
	923	+ # get pytorch output
	924	++ one_meta['img_shape_for_onnx'] = one_img.shape[-2:]
	925	+ pytorch_results = model(tensor_data, [[one_meta]], return_loss=False)
	926	+ pytorch_results = pytorch_results[0]
	927	+ # get onnx output

DACL_PyTorch/contrib/cv/detection/SSD/test/README.md+0-24

	@@ -1,24 +0,0 @@
1	-环境准备：
2	-
3	-1.数据集路径
4	-通用的数据集统一放在/root/datasets/或/opt/npu/
5	-本模型数据集放在/root/datasets/
6	-
7	-2.进入工作目录
8	-cd SSD
9	-
10	-3.安装必要的依赖，测试环境可能已经安装其中的一些不同版本的库了，故手动测试时不推荐使用该命令安装
11	-pip3.7 install -r requirements.txt
12	-
13	-4.获取模型代码及权重文件
14	-bash test/prepare_env.sh
15	-
16	-5.获取benchmark工具
17	-将benchmark.x86_64 benchmark.aarch64放在当前目录
18	-
19	-6.310上执行，执行时确保device空闲
20	-bash test/pth2om.sh
21	-bash test/eval_acc_perf.sh --datasets_path=/root/datasets
22	-
23	-7.在基准环境上在线推理（onnx包含自定义算子，因此不能使用开源TensorRT测试性能数据，故在基准机器上使用pth在线推理测试性能数据）
24	-bash test/perf_benchmark.sh

DACL_PyTorch/contrib/cv/detection/SSD/test/eval_acc_perf.sh+0-66

	@@ -1,66 +0,0 @@
1	-#!/bin/bash
2	-
3	-datasets_path="/root/datasets/"
4	-
5	-for para in $*
6	-do
7	- if [[ $para == --datasets_path* ]]; then
8	- datasets_path=`echo ${para#*=}`
9	- fi
10	-done
11	-
12	-arch=`uname -m`
13	-rm -rf ./val2017_ssd_bin
14	-python3.7 mmdetection_coco_preprocess.py --image_folder_path ${datasets_path}/coco/val2017 --bin_folder_path val2017_ssd_bin
15	-if [ $? != 0 ]; then
16	- echo "fail!"
17	- exit -1
18	-fi
19	-echo "preprocess"
20	-
21	-python3.7 get_info.py bin ./val2017_ssd_bin coco2017_ssd.info 300 300
22	-python3.7 get_info.py jpg ${datasets_path}/coco/val2017 coco2017_ssd_jpg.info
23	-if [ $? != 0 ]; then
24	- echo "fail!"
25	- exit -1
26	-fi
27	-echo "get_info"
28	-
29	-source env.sh
30	-rm -rf result/dumpOutput_device0
31	-chmod u+x benchmark.x86_64
32	-./benchmark.x86_64 -model_type=vision -batch_size=1 -device_id=0 -input_text_path=./coco2017_ssd.info -input_width=300 -input_height=300 -useDvpp=False -output_binary=true -om_path=ssd_300_coco.om
33	-if [ $? != 0 ]; then
34	- echo "fail!"
35	- exit -1
36	-fi
37	-echo "benchmark.x86_64"
38	-
39	-python3.7 mmdetection_coco_postprocess.py --bin_data_path=result/dumpOutput_device0 --prob_thres=0.02 --ifShowDetObj --det_results_path=detection-results --test_annotation=coco2017_ssd_jpg.info
40	-if [ $? != 0 ]; then
41	- echo "fail!"
42	- exit -1
43	-fi
44	-echo "postprocess"
45	-
46	-python3.7 txt_to_json.py
47	-if [ $? != 0 ]; then
48	- echo "fail!"
49	- exit -1
50	-fi
51	-echo "txt_to_json"
52	-
53	-python3.7 coco_eval.py --ground_truth ${datasets_path}/coco/annotations/instances_val2017.json
54	-if [ $? != 0 ]; then
55	- echo "fail!"
56	- exit -1
57	-fi
58	-echo "coco_eval"
59	-
60	-echo "====performance data===="
61	-python3.7 test/parse.py result/perf_vision_batchsize_1_device_0.txt
62	-if [ $? != 0 ]; then
63	- echo "fail!"
64	- exit -1
65	-fi
66	-echo "success"

DACL_PyTorch/contrib/cv/detection/SSD/test/parse.py+0-31

	@@ -1,31 +0,0 @@
1	-# Copyright 2021 Huawei Technologies Co., Ltd
2	-#
3	-# Licensed under the Apache License, Version 2.0 (the "License");
4	-# you may not use this file except in compliance with the License.
5	-# You may obtain a copy of the License at
6	-#
7	-# http://www.apache.org/licenses/LICENSE-2.0
8	-#
9	-# Unless required by applicable law or agreed to in writing, software
10	-# distributed under the License is distributed on an "AS IS" BASIS,
11	-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12	-# See the License for the specific language governing permissions and
13	-# limitations under the License.
14	-import sys
15	-import json
16	-import re
17	-
18	-if __name__ == '__main__':
19	- if sys.argv[1].endswith('.json'):
20	- result_json = sys.argv[1]
21	- with open(result_json, 'r') as f:
22	- content = f.read()
23	- tops = [i.get('value') for i in json.loads(content).get('value') if 'Top' in i.get('key')]
24	- print('om {} top1:{} top5:{}'.format(result_json.split('_')[1].split('.')[0], tops[0], tops[4]))
25	- elif sys.argv[1].endswith('.txt'):
26	- result_txt = sys.argv[1]
27	- with open(result_txt, 'r') as f:
28	- content = f.read()
29	- txt_data_list = [i.strip() for i in re.findall(r':(.*?),', content.replace('\n', ',') + ',')]
30	- fps = float(txt_data_list[7].replace('samples/s', '')) * 4
31	- print('310 bs{} fps:{}'.format(result_txt.split('_')[3], fps))

DACL_PyTorch/contrib/cv/detection/SSD/test/perf_benchmark.sh+0-4

	@@ -1,4 +0,0 @@
1	-cd mmdetection
2	-mkdir data
3	-ln -s /root/datasets/coco data/coco
4	-python3 tools/test.py configs/ssd/ssd300_coco.py ../ssd300_coco_20200307-a92d2092.pth --eval bbox

DACL_PyTorch/contrib/cv/detection/SSD/test/prepare_env.sh+0-13

	@@ -1,13 +0,0 @@
1	-pip install mmdet==2.8.0
2	-pip install mmcv-full==1.2.4
3	-pip install mmpycocotools==12.0.3
4	-git clone https://github.com/open-mmlab/mmdetection.git
5	-cd mmdetection
6	-git reset --hard a21eb25535f31634cef332b09fc27d28956fb24b
7	-pip install -r requirements/build.txt
8	-pip install -v -e .
9	-patch -p1 < ../ssd_mmdetection.diff
10	-cd ..
11	-echo "install mmdetection successfully"
12	-wget http://download.openmmlab.com/mmdetection/v2.0/ssd/ssd300_coco/ssd300_coco_20200307-a92d2092.pth
13	-echo "download SSD300 pth successfully"

DACL_PyTorch/contrib/cv/detection/SSD/test/pth2om.sh+0-13

	@@ -1,13 +0,0 @@
1	-#!/bin/bash
2	-rm -rf ssd_300_coco.onnx
3	-python3.7 mmdetection/tools/pytorch2onnx.py mmdetection/configs/ssd/ssd300_coco.py ./ssd300_coco_20200307-a92d2092.pth --output-file=ssd_300_coco.onnx --shape=300 --verify --show --mean 123.675 116.28 103.53 --std 1 1 1
4	-echo "convert onnx"
5	-source env.sh
6	-rm -rf ssd_300_coco.om
7	-/usr/local/Ascend/ascend-toolkit/latest/atc/bin/atc --model=ssd_300_coco.onnx --framework=5 --output=ssd_300_coco --input_format=NCHW --input_shape="input:1,3,300,300" --log=info --soc_version=Ascend310 --out_nodes="Concat_637:0;Reshape_639:0" --buffer_optimize=off_optimize --precision_mode allow_mix_precision
8	-echo "convert om"
9	-if [ -f "ssd_300_coco.om" ]; then
10	- echo "success"
11	-else
12	- echo "fail!"
13	-fi