[重庆理工大学][高校贡献][PyTorch离线推理][Nested_UNet]-初次提交-ModelZoo-PyTorch-AtomGit

MACL_PyTorch/contrib/cv/segmentation/Nested_UNet/README.md+57-80

	@@ -6,6 +6,7 @@
6	- [2 环境说明](#2-环境说明)	6	- [2 环境说明](#2-环境说明)
7	- [2.1 深度学习框架](#21-深度学习框架)	7	- [2.1 深度学习框架](#21-深度学习框架)
8	- [2.2 python第三方库](#22-python第三方库)	8	- [2.2 python第三方库](#22-python第三方库)
		9	+ - [2.3 获取ais_infer工具](#23-获取ais_infer工具)
9	- [3 模型转换](#3-模型转换)	10	- [3 模型转换](#3-模型转换)
10	- [3.1 pth转onnx模型](#31-pth转onnx模型)	11	- [3.1 pth转onnx模型](#31-pth转onnx模型)
11	- [3.2 onnx转om模型](#32-onnx转om模型)	12	- [3.2 onnx转om模型](#32-onnx转om模型)
	@@ -14,7 +15,7 @@
14	- [4.2 数据集预处理](#42-数据集预处理)	15	- [4.2 数据集预处理](#42-数据集预处理)
15	- [4.3 生成数据集信息文件](#43-生成数据集信息文件)	16	- [4.3 生成数据集信息文件](#43-生成数据集信息文件)
16	- [5 离线推理](#5-离线推理)	17	- [5 离线推理](#5-离线推理)
17	- - [5.1 benchmark工具概述](#51-benchmark工具概述)	18	+ - [5.1 AisBench工具概述](#51-aisbench工具概述)
18	- [5.2 离线推理](#52-离线推理)	19	- [5.2 离线推理](#52-离线推理)
19	- [6 精度对比](#6-精度对比)	20	- [6 精度对比](#6-精度对比)
20	- [6.1 离线推理IoU精度](#61-离线推理iou精度)	21	- [6.1 离线推理IoU精度](#61-离线推理iou精度)
	@@ -22,6 +23,8 @@
22	- [6.3 精度对比](#63-精度对比)	23	- [6.3 精度对比](#63-精度对比)
23	- [7 性能对比](#7-性能对比)	24	- [7 性能对比](#7-性能对比)
24	- [7.1 npu性能数据](#71-npu性能数据)	25	- [7.1 npu性能数据](#71-npu性能数据)
		26	+ - [7.2 T4性能数据](#72-t4性能数据)
		27	+ - [7.3 性能对比](#73-性能对比)
25		28
26		29
27		30
	@@ -48,11 +51,12 @@ commit_id:557ea02f0b5d45ec171aae2282d2cd21562a633e
48		51
49	### 2.1 深度学习框架	52	### 2.1 深度学习框架
50	```	53	```
51	-CANN 5.0.1	54	+CANN 5.1.RC2
52	pytorch >= 1.5.0	55	pytorch >= 1.5.0
53	torchvision >= 0.6.0	56	torchvision >= 0.6.0
54	onnx >= 1.7.0	57	onnx >= 1.7.0
55	```	58	```
		59	+实测环境中Torch的版本为1.5.0
56		60
57	### 2.2 python第三方库	61	### 2.2 python第三方库
58		62
	@@ -68,6 +72,14 @@ albumentations == 0.5.2
68	>	72	>
69	> Arm架构：pytorch，torchvision和onnx可以通过源码编译安装，其它可以通过pip3.7 install 包名安装	73	> Arm架构：pytorch，torchvision和onnx可以通过源码编译安装，其它可以通过pip3.7 install 包名安装
70		74
		75	+
		76	+### 2.3 获取[ais_infer工具](https://gitee.com/ascend/tools/tree/master/ais-bench_workload/tool/ais_infer)
		77	+
		78	+将工具编译后的压缩包放置在当前目录；解压工具包，安装工具压缩包中的whl文件；
		79	+```
		80	+pip3 install aclruntime-0.01-cp37-cp37m-linux_xxx.whl
		81	+```
		82	+
71	## 3 模型转换	83	## 3 模型转换
72		84
73	- [pth转onnx模型](#31-pth转onnx模型)	85	- [pth转onnx模型](#31-pth转onnx模型)
	@@ -80,12 +92,10 @@ albumentations == 0.5.2
80	```	92	```
81	git clone https://github.com/4uiiurz1/pytorch-nested-unet	93	git clone https://github.com/4uiiurz1/pytorch-nested-unet
82	```	94	```
83	-2.原模型中resize采用双线性差值方法，影响模型在Ascend310上的推理性能，需要改为最近邻方法，并重新训练模型。将重新训练的模型移动到当前目录下，重命名为nested_unet.pth。	95	+2.原模型中resize采用双线性差值方法，影响模型在Ascend310P上的推理性能，需要改为最近邻方法。
84	```	96	```
85	cd pytorch-nested-unet	97	cd pytorch-nested-unet
86	patch -p1 < ../nested_unet.diff	98	patch -p1 < ../nested_unet.diff
		Liang Zhen2022年7月21日这个diff文件里存在很多不必要修改的地方，比如 `LovaszSoftmax.pytorch` 路径下的 `lovasz_losses.py` 文件的拷贝，另外需要判断一下新增的 `train_ids.txt` 和 `val_ids.txt` 是否有必要。如无必要，勿增实体 Liang Zhen2022年7月21日这个diff文件里存在很多不必要修改的地方，比如 `LovaszSoftmax.pytorch` 路径下的 `lovasz_losses.py` 文件的拷贝，另外需要判断一下新增的 `train_ids.txt` 和 `val_ids.txt` 是否有必要。如无必要，勿增实体 `train_ids.txt` 和 `val_ids.txt` 是否可以从数据集或者官网上下载，如果是网上下载的给出下载方式
87	-python3.7 train.py --dataset dsb2018_96 --arch NestedUNet --loss LovaszHingeLoss --epochs 200
88	-cp models/dsb2018_96_NestedUNet_woDS/model.pth ../nested_unet.pth
89	cd ..	99	cd ..
90	```	100	```
		Liang Zhen2022年7月21日使用最近邻代替双线性插值应该是不需要重新训练的，resize没有可学习参数
91		101
	@@ -104,13 +114,12 @@ python3.7 nested_unet_pth2onnx.py nested_unet.pth nested_unet.onnx
104		114
105	### 3.2 onnx转om模型	115	### 3.2 onnx转om模型
106		116
		Liang Zhen2022年7月21日这里没有复制完全，找支撑同事确认，把相关截图也复制过来
107	-1.设置环境变量	117	+1.使用atc将onnx模型转换为om模型文件，工具使用方法可以参考CANN 5.1.RC2 开发辅助工具指南 (推理) 01
		118	+`${chip_name}`可通过 `npu-smi info` 指令查看，例: 310P3
		119	+![Image](https://gitee.com/ascend/ModelZoo-PyTorch/raw/master/ACL_PyTorch/images/310P3.png)
		120	+
108	```	121	```
109	-source env.sh	122	+atc --framework=5 --model=./nested_unet.onnx --input_format=NCHW --input_shape="actual_input_1:16,3,96,96" --output=nested_unet_bs16 --log=debug --soc_version=Ascend${chip_name}
110	-```
111	-2.使用atc将onnx模型转换为om模型文件，工具使用方法可以参考CANN 5.0.1 开发辅助工具指南 (推理) 01
112	-```
113	-atc --framework=5 --model=./nested_unet.onnx --input_format=NCHW --input_shape="actual_input_1:16,3,96,96" --output=nested_unet_bs16 --log=debug --soc_version=Ascend310
114	```	123	```
115		124
116	## 4 数据集预处理	125	## 4 数据集预处理
	@@ -138,33 +147,27 @@ cd ..
138	```	147	```
139	python3.7 preprocess_nested_unet.py ./dsb2018_96/images ./prep_dataset ./val_ids.txt	148	python3.7 preprocess_nested_unet.py ./dsb2018_96/images ./prep_dataset ./val_ids.txt
140	```	149	```
141	-### 4.3 生成数据集信息文件
142	-1.生成数据集信息文件脚本gen_dataset_info.py
143		150
144	-2.执行生成数据集信息脚本，生成数据集信息文件
145	-```
146	-python3.7 gen_dataset_info.py bin ./prep_dataset ./nested_unet_prep_bin.info 96 96
147	-```
148	-第一个参数为模型输入的类型，第二个参数为生成的bin文件路径，第三个为输出的info文件，后面为宽高信息
149	## 5 离线推理	151	## 5 离线推理
150		152
151	-- [benchmark工具概述](#51-benchmark工具概述)	153	+- [AisBench工具概述](#51-AisBench工具概述)
152		154
153	- [离线推理](#52-离线推理)	155	- [离线推理](#52-离线推理)
154		156
155	-### 5.1 benchmark工具概述	157	+### 5.1 AisBench工具概述
156	-	158	+AisBench推理工具，该工具包含前端和后端两部分。后端基于c++开发，实现通用推理功能；前端基于python开发，实现用户界面功能。
157	-benchmark工具为华为自研的模型推理工具，支持多种模型的离线推理，能够迅速统计出模型在Ascend310上的性能，支持真实数据和纯推理两种模式，配合后处理脚本，可以实现诸多模型的端到端过程，获取工具及使用方法可以参考CANN 5.0.1 推理benchmark工具用户指南 01
158	### 5.2 离线推理	159	### 5.2 离线推理
159	-1.设置环境变量	160	+1.执行离线推理
160	```	161	```
161	-source env.sh	162	+python3.7 /path/to/tools/ais-bench_workload/tool/ais_infer/ais_infer.py --model ./nested_unet_bs1.om --input ./prep_dataset/ --output ./ais_results --outfmt BIN --batchsize=1
162	```	163	```
163	-2.执行离线推理	164	+--model：模型地址
164	-```	165	+--input：预处理完的数据集文件夹
165	-./benchmark.x86_64 -model_type=vision -device_id=0 -batch_size=1 -om_path=nested_unet_bs1.om -input_text_path=./nested_unet_prep_bin.info -input_width=96 -input_height=96 -output_binary=True -useDvpp=False	166	+--output：推理结果保存地址
166	-```	167	+--outfmt：推理结果保存格式
167	-输出结果默认保存在当前目录result/dumpOutput_deviceX(X为对应的device_id)，每个输入对应一个_X.bin文件的输出。	168	+--batchsize：模型batch size 默认为1 。当前推理模块根据模型输入和文件输出自动进行组batch。参数传递的 batchszie有且只用于结果吞吐率计算。请务必注意需要传入该值，以获取计算正确的吞吐率。
		169	+输出结果默认保存在当前目录ais_results/X(X为执行推理的时间)，每个输入对应一个_X.bin文件的输出。
		170	+
168		171
169	## 6 精度对比	172	## 6 精度对比
170		173
	@@ -178,9 +181,9 @@ source env.sh
178		181
179	调用postprocess_nested_unet.py脚本推理结果与语义分割真值进行比对，可以获得IoU精度数据。	182	调用postprocess_nested_unet.py脚本推理结果与语义分割真值进行比对，可以获得IoU精度数据。
180	```	183	```
181	-python3.7 postprocess_nested_unet.py result/dumpOutput_device0/ ./dsb2018_96/masks/0/	184	+python3.7 postprocess_nested_unet.py ./ais_results/2022_07_11-15_53_11/sumary.json ./dsb2018_96/masks/0/
182	```	185	```
183	-第一个为benchmark输出目录，第二个为真值所在目录。	186	+第一个为AisBench输出目录，第二个为真值所在目录。
184	查看输出结果：	187	查看输出结果：
185	```	188	```
186	IoU: 0.8385	189	IoU: 0.8385
	@@ -201,56 +204,30 @@ Nested U-Net 0.842
201	## 7 性能对比	204	## 7 性能对比
202		205
203	- [npu性能数据](#71-npu性能数据)	206	- [npu性能数据](#71-npu性能数据)
		207	+- [T4性能数据](#72-T4性能数据)
		208	+- [性能对比](#73-性能对比)
204		209
205	### 7.1 npu性能数据	210	### 7.1 npu性能数据
206	-benchmark工具在整个数据集上推理时也会统计性能数据，但是推理整个数据集较慢，如果这么测性能那么整个推理期间需要确保独占device，使用npu-smi info可以查看device是否空闲。也可以使用benchmark纯推理功能测得性能数据，但是由于随机数不能模拟数据分布，纯推理功能测的有些模型性能数据可能不太准，benchmark纯推理功能测性能仅为快速获取大概的性能数据以便调试优化使用，可初步确认benchmark工具在整个数据集上推理时由于device也被其它推理任务使用了导致的性能不准的问题。模型的性能以使用benchmark工具在整个数据集上推理得到bs1与bs16的性能数据为准，对于使用benchmark工具测试的batch4，8，32的性能数据在README.md中如下作记录即可。	211	+AisBench工具在整个数据集上推理时也会统计性能数据，但是推理整个数据集较慢，如果这么测性能那么整个推理期间需要确保独占device，使用npu-smi info可以查看device是否空闲。也可以使用AisBench纯推理功能测得性能数据，但是由于随机数不能模拟数据分布，纯推理功能测的有些模型性能数据可能不太准，AisBench纯推理功能测性能仅为快速获取大概的性能数据以便调试优化使用，可初步确认AisBench工具在整个数据集上推理时由于device也被其它推理任务使用了导致的性能不准的问题。模型的性能以使用AisBench工具在整个数据集上推理得到bs1与bs16的性能数据为准。
207	-1.benchmark工具在整个数据集上推理获得性能数据
208	-batch1的性能，benchmark工具在整个数据集上推理后生成result/perf_vision_batchsize_1_device_0.txt：
209	-```
210	-[e2e] throughputRate: 96.455, latency: 1389.25
211	-[data read] throughputRate: 1640.63, moduleLatency: 0.609522
212	-[preprocess] throughputRate: 1633.73, moduleLatency: 0.612097
213	-[infer] throughputRate: 356.828, Interface throughputRate: 428.223, moduleLatency: 2.78137
214	-[post] throughputRate: 356.654, moduleLatency: 2.80384
215	-```
216	-Interface throughputRate: 428.223，428.223x4=1712.892既是batch1 310单卡吞吐率
217	-batch16的性能，benchmark工具在整个数据集上推理后生成result/perf_vision_batchsize_16_device_1.txt：
218	-```
219	-[e2e] throughputRate: 95.2159, latency: 1407.33
220	-[data read] throughputRate: 12059, moduleLatency: 0.0829254
221	-[preprocess] throughputRate: 5134.3, moduleLatency: 0.194769
222	-[infer] throughputRate: 386.438, Interface throughputRate: 443.958, moduleLatency: 2.5702
223	-[post] throughputRate: 25.8257, moduleLatency: 38.7211
224	-```
225	-Interface throughputRate: 443.958，443.958x4=1775.832既是batch16 310单卡吞吐率
226	-batch4性能：
227	-```
228	-[e2e] throughputRate: 98.415, latency: 1361.58
229	-[data read] throughputRate: 12401.7, moduleLatency: 0.0806344
230	-[preprocess] throughputRate: 6240.4, moduleLatency: 0.160246
231	-[infer] throughputRate: 450.443, Interface throughputRate: 503.386, moduleLatency: 2.2088
232	-[post] throughputRate: 114.158, moduleLatency: 8.75977
233	-```
234	-batch4 310单卡吞吐率：503.386x4=2013.544fps
235	-batch8性能：
236	-```
237	-[e2e] throughputRate: 98.2831, latency: 1363.41
238	-[data read] throughputRate: 12465.1, moduleLatency: 0.0802239
239	-[preprocess] throughputRate: 5993.38, moduleLatency: 0.166851
240	-[infer] throughputRate: 439.33, Interface throughputRate: 484.033, moduleLatency: 2.2622
241	-[post] throughputRate: 55.5416, moduleLatency: 18.0045
242	-```
243	-batch8 310单卡吞吐率：484.033x4=1936.132fps
244	-batch32性能：
245	-```
246	-[e2e] throughputRate: 96.7847, latency: 1384.52
247	-[data read] throughputRate: 14351.5, moduleLatency: 0.0696791
248	-[preprocess] throughputRate: 6115.65, moduleLatency: 0.163515
249	-[infer] throughputRate: 352.844, Interface throughputRate: 386.865, moduleLatency: 2.8028
250	-[post] throughputRate: 13.1131, moduleLatency: 76.2594
251	-```
252	-batch32 310单卡吞吐率：386.865x4=1547.460fps
253		212
		213	+### 7.2 T4性能数据
		214	+在装有T4卡的服务器上测试gpu性能，测试过程请确保卡没有运行其他任务，TensorRT版本：7.2.3.4，cuda版本：11.0，cudnn版本：8.2
		215	+
		216	+### 7.3 性能对比
		217	+\| \| 310 \| 310P3 \| T4 \| 310P3/310 \| 310P3/T4 \|
		218	+\|---------\|-------------\|-------------\|-------------\|-------------\|--------------\|
		219	+\| bs1 \| 1674.950432 \| 1681.600281 \| 320.5865223 \| 1.003970177 \| 5.245386704 \|
		220	+\| bs4 \| 1868.409806 \| 2495.595994 \| 400.8416683 \| 1.335679135 \| 6.22588965 \|
		221	+\| bs8 \| 1845.83697 \| 1907.807965 \| 458.3922516 \| 1.033573385 \| 4.161955091 \|
		222	+\| bs16 \| 1757.895362 \| 1852.471243 \| 549.9608379 \| 1.053800632 \| 3.368369374 \|
		223	+\| bs32 \| 1724.390615 \| 1796.602154 \| 568.2170006 \| 1.041876555 \| 3.161824007 \|
		224	+\| \| \| \| \| \| \|
		225	+\| 最优Batch \| 1868.409806 \| 2495.595994 \| 568.2170006 \| 1.335679135 \| 4.391976993 \|
		226	+
		227	+
		228	+
		229	+310P单个device的吞吐率比310单卡的吞吐率大，故310P性能高于310性能，性能达标。
		230	+对于batch1与batch16，310P性能均高于310性能1.2倍，该模型放在Benchmark/cv/segmentation目录下。
254	性能优化：	231	性能优化：
255	->没有遇到性能不达标的问题，故不需要进行性能优化	232	+>以上在310P上的结果为AOE优化后的性能。
256	-	233	+因直接使用ATC导出模型已达标，所以不使用AOE进行性能优化。

DACL_PyTorch/contrib/cv/segmentation/Nested_UNet/env.sh+0-7

	@@ -1,7 +0,0 @@
1	-# 配置环境变量
2	-export install_path=/usr/local/Ascend/ascend-toolkit/latest
3	-export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
4	-export PYTHONPATH=${install_path}/atc/python/site-packages:$PYTHONPATH
5	-export LD_LIBRARY_PATH=${install_path}/atc/lib64:${install_path}/acllib/lib64:$LD_LIBRARY_PATH
6	-export ASCEND_OPP_PATH=${install_path}/opp
7	-export REPEAT_TUNE=true

DACL_PyTorch/contrib/cv/segmentation/Nested_UNet/gen_dataset_info.py+0-60

	@@ -1,60 +0,0 @@
1	-# Copyright 2020 Huawei Technologies Co., Ltd
2	-#
3	-# Licensed under the Apache License, Version 2.0 (the "License");
4	-# you may not use this file except in compliance with the License.
5	-# You may obtain a copy of the License at
6	-#
7	-# http://www.apache.org/licenses/LICENSE-2.0
8	-#
9	-# Unless required by applicable law or agreed to in writing, software
10	-# distributed under the License is distributed on an "AS IS" BASIS,
11	-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12	-# See the License for the specific language governing permissions and
13	-# limitations under the License.
14	-
15	-import os
16	-import sys
17	-import cv2
18	-from glob import glob
19	-
20	-
21	-def get_bin_info(file_path, info_name, width, height):
22	- bin_images = glob(os.path.join(file_path, '*.bin'))
23	- with open(info_name, 'w') as file:
24	- for index, img in enumerate(bin_images):
25	- content = ' '.join([str(index), img, width, height])
26	- file.write(content)
27	- file.write('\n')
28	-
29	-
30	-def get_jpg_info(file_path, info_name):
31	- extensions = ['jpg', 'jpeg', 'JPG', 'JPEG']
32	- image_names = []
33	- for extension in extensions:
34	- image_names.append(glob(os.path.join(file_path, '*.' + extension)))
35	- with open(info_name, 'w') as file:
36	- for image_name in image_names:
37	- if len(image_name) == 0:
38	- continue
39	- else:
40	- for index, img in enumerate(image_name):
41	- img_cv = cv2.imread(img)
42	- shape = img_cv.shape
43	- width, height = shape[1], shape[0]
44	- content = ' '.join([str(index), img, str(width), str(height)])
45	- file.write(content)
46	- file.write('\n')
47	-
48	-
49	-if __name__ == '__main__':
50	- file_type = sys.argv[1]
51	- file_path = sys.argv[2]
52	- info_name = sys.argv[3]
53	- if file_type == 'bin':
54	- width = sys.argv[4]
55	- height = sys.argv[5]
56	- assert len(sys.argv) == 6, 'The number of input parameters must be equal to 5'
57	- get_bin_info(file_path, info_name, width, height)
58	- elif file_type == 'jpg':
59	- assert len(sys.argv) == 4, 'The number of input parameters must be equal to 3'
60	- get_jpg_info(file_path, info_name)

MACL_PyTorch/contrib/cv/segmentation/Nested_UNet/nested_unet.diff+1-271

	@@ -11,276 +11,6 @@ index d24ae3a..203bec0 100644
11		11
12	self.conv0_0 = VGGBlock(input_channels, nb_filter[0], nb_filter[0])	12	self.conv0_0 = VGGBlock(input_channels, nb_filter[0], nb_filter[0])
13	self.conv1_0 = VGGBlock(nb_filter[0], nb_filter[1], nb_filter[1])	13	self.conv1_0 = VGGBlock(nb_filter[0], nb_filter[1], nb_filter[1])
14	-diff --git a/losses.py b/losses.py
15	-index 1a8f188..9d91fd3 100644
16	---- a/losses.py
17	-+++ b/losses.py
18	-@@ -3,7 +3,7 @@ import torch.nn as nn
19	- import torch.nn.functional as F
20	-
21	- try:
22	-- from LovaszSoftmax.pytorch.lovasz_losses import lovasz_hinge
23	-+ from lovasz_losses import lovasz_hinge
24	- except ImportError:
25	- pass
26	-
27	-diff --git a/lovasz_losses.py b/lovasz_losses.py
28	-new file mode 100644
29	-index 0000000..0162e66
30	---- /dev/null
31	-+++ b/lovasz_losses.py
32	-@@ -0,0 +1,250 @@
33	-+"""
34	-+Lovasz-Softmax and Jaccard hinge loss in PyTorch
35	-+Maxim Berman 2018 ESAT-PSI KU Leuven (MIT License)
36	-+"""
37	-+
38	-+from __future__ import print_function, division
39	-+
40	-+import torch
41	-+from torch.autograd import Variable
42	-+import torch.nn.functional as F
43	-+import numpy as np
44	-+try:
45	-+ from itertools import ifilterfalse
46	-+except ImportError: # py3k
47	-+ from itertools import filterfalse as ifilterfalse
48	-+
49	-+
50	-+def lovasz_grad(gt_sorted):
51	-+ """
52	-+ Computes gradient of the Lovasz extension w.r.t sorted errors
53	-+ See Alg. 1 in paper
54	-+ """
55	-+ p = len(gt_sorted)
56	-+ gts = gt_sorted.sum()
57	-+ intersection = gts - gt_sorted.float().cumsum(0)
58	-+ union = gts + (1 - gt_sorted).float().cumsum(0)
59	-+ jaccard = 1. - intersection / union
60	-+ if p > 1: # cover 1-pixel case
61	-+ jaccard[1:p] = jaccard[1:p] - jaccard[0:-1]
62	-+ return jaccard
63	-+
64	-+
65	-+def iou_binary(preds, labels, EMPTY=1., ignore=None, per_image=True):
66	-+ """
67	-+ IoU for foreground class
68	-+ binary: 1 foreground, 0 background
69	-+ """
70	-+ if not per_image:
71	-+ preds, labels = (preds,), (labels,)
72	-+ ious = []
73	-+ for pred, label in zip(preds, labels):
74	-+ intersection = ((label == 1) & (pred == 1)).sum()
75	-+ union = ((label == 1) \| ((pred == 1) & (label != ignore))).sum()
76	-+ if not union:
77	-+ iou = EMPTY
78	-+ else:
79	-+ iou = float(intersection) / float(union)
80	-+ ious.append(iou)
81	-+ iou = mean(ious) # mean accross images if per_image
82	-+ return 100 * iou
83	-+
84	-+
85	-+def iou(preds, labels, C, EMPTY=1., ignore=None, per_image=False):
86	-+ """
87	-+ Array of IoU for each (non ignored) class
88	-+ """
89	-+ if not per_image:
90	-+ preds, labels = (preds,), (labels,)
91	-+ ious = []
92	-+ for pred, label in zip(preds, labels):
93	-+ iou = []
94	-+ for i in range(C):
95	-+ if i != ignore: # The ignored label is sometimes among predicted classes (ENet - CityScapes)
96	-+ intersection = ((label == i) & (pred == i)).sum()
97	-+ union = ((label == i) \| ((pred == i) & (label != ignore))).sum()
98	-+ if not union:
99	-+ iou.append(EMPTY)
100	-+ else:
101	-+ iou.append(float(intersection) / float(union))
102	-+ ious.append(iou)
103	-+ ious = [mean(iou) for iou in zip(*ious)] # mean accross images if per_image
104	-+ return 100 * np.array(ious)
105	-+
106	-+
107	-+# --------------------------- BINARY LOSSES ---------------------------
108	-+
109	-+
110	-+def lovasz_hinge(logits, labels, per_image=True, ignore=None):
111	-+ """
112	-+ Binary Lovasz hinge loss
113	-+ logits: [B, H, W] Variable, logits at each pixel (between -\infty and +\infty)
114	-+ labels: [B, H, W] Tensor, binary ground truth masks (0 or 1)
115	-+ per_image: compute the loss per image instead of per batch
116	-+ ignore: void class id
117	-+ """
118	-+ if per_image:
119	-+ loss = mean(lovasz_hinge_flat(*flatten_binary_scores(log.unsqueeze(0), lab.unsqueeze(0), ignore))
120	-+ for log, lab in zip(logits, labels))
121	-+ else:
122	-+ loss = lovasz_hinge_flat(*flatten_binary_scores(logits, labels, ignore))
123	-+ return loss
124	-+
125	-+
126	-+def lovasz_hinge_flat(logits, labels):
127	-+ """
128	-+ Binary Lovasz hinge loss
129	-+ logits: [P] Variable, logits at each prediction (between -\infty and +\infty)
130	-+ labels: [P] Tensor, binary ground truth labels (0 or 1)
131	-+ ignore: label to ignore
132	-+ """
133	-+ if len(labels) == 0:
134	-+ # only void pixels, the gradients should be 0
135	-+ return logits.sum() * 0.
136	-+ signs = 2. * labels.float() - 1.
137	-+ errors = (1. - logits * Variable(signs))
138	-+ errors_sorted, perm = torch.sort(errors, dim=0, descending=True)
139	-+ perm = perm.data
140	-+ gt_sorted = labels[perm]
141	-+ grad = lovasz_grad(gt_sorted)
142	-+ loss = torch.dot(F.relu(errors_sorted), Variable(grad))
143	-+ return loss
144	-+
145	-+
146	-+def flatten_binary_scores(scores, labels, ignore=None):
147	-+ """
148	-+ Flattens predictions in the batch (binary case)
149	-+ Remove labels equal to 'ignore'
150	-+ """
151	-+ scores = scores.view(-1)
152	-+ labels = labels.view(-1)
153	-+ if ignore is None:
154	-+ return scores, labels
155	-+ valid = (labels != ignore)
156	-+ vscores = scores[valid]
157	-+ vlabels = labels[valid]
158	-+ return vscores, vlabels
159	-+
160	-+
161	-+class StableBCELoss(torch.nn.modules.Module):
162	-+ def __init__(self):
163	-+ super(StableBCELoss, self).__init__()
164	-+ def forward(self, input, target):
165	-+ neg_abs = - input.abs()
166	-+ loss = input.clamp(min=0) - input * target + (1 + neg_abs.exp()).log()
167	-+ return loss.mean()
168	-+
169	-+
170	-+def binary_xloss(logits, labels, ignore=None):
171	-+ """
172	-+ Binary Cross entropy loss
173	-+ logits: [B, H, W] Variable, logits at each pixel (between -\infty and +\infty)
174	-+ labels: [B, H, W] Tensor, binary ground truth masks (0 or 1)
175	-+ ignore: void class id
176	-+ """
177	-+ logits, labels = flatten_binary_scores(logits, labels, ignore)
178	-+ loss = StableBCELoss()(logits, Variable(labels.float()))
179	-+ return loss
180	-+
181	-+
182	-+# --------------------------- MULTICLASS LOSSES ---------------------------
183	-+
184	-+
185	-+def lovasz_softmax(probas, labels, classes='present', per_image=False, ignore=None):
186	-+ """
187	-+ Multi-class Lovasz-Softmax loss
188	-+ probas: [B, C, H, W] Variable, class probabilities at each prediction (between 0 and 1).
189	-+ Interpreted as binary (sigmoid) output with outputs of size [B, H, W].
190	-+ labels: [B, H, W] Tensor, ground truth labels (between 0 and C - 1)
191	-+ classes: 'all' for all, 'present' for classes present in labels, or a list of classes to average.
192	-+ per_image: compute the loss per image instead of per batch
193	-+ ignore: void class labels
194	-+ """
195	-+ if per_image:
196	-+ loss = mean(lovasz_softmax_flat(*flatten_probas(prob.unsqueeze(0), lab.unsqueeze(0), ignore), classes=classes)
197	-+ for prob, lab in zip(probas, labels))
198	-+ else:
199	-+ loss = lovasz_softmax_flat(*flatten_probas(probas, labels, ignore), classes=classes)
200	-+ return loss
201	-+
202	-+
203	-+def lovasz_softmax_flat(probas, labels, classes='present'):
204	-+ """
205	-+ Multi-class Lovasz-Softmax loss
206	-+ probas: [P, C] Variable, class probabilities at each prediction (between 0 and 1)
207	-+ labels: [P] Tensor, ground truth labels (between 0 and C - 1)
208	-+ classes: 'all' for all, 'present' for classes present in labels, or a list of classes to average.
209	-+ """
210	-+ if probas.numel() == 0:
211	-+ # only void pixels, the gradients should be 0
212	-+ return probas * 0.
213	-+ C = probas.size(1)
214	-+ losses = []
215	-+ class_to_sum = list(range(C)) if classes in ['all', 'present'] else classes
216	-+ for c in class_to_sum:
217	-+ fg = (labels == c).float() # foreground for class c
218	-+ if (classes is 'present' and fg.sum() == 0):
219	-+ continue
220	-+ if C == 1:
221	-+ if len(classes) > 1:
222	-+ raise ValueError('Sigmoid output possible only with 1 class')
223	-+ class_pred = probas[:, 0]
224	-+ else:
225	-+ class_pred = probas[:, c]
226	-+ errors = (Variable(fg) - class_pred).abs()
227	-+ errors_sorted, perm = torch.sort(errors, 0, descending=True)
228	-+ perm = perm.data
229	-+ fg_sorted = fg[perm]
230	-+ losses.append(torch.dot(errors_sorted, Variable(lovasz_grad(fg_sorted))))
231	-+ return mean(losses)
232	-+
233	-+
234	-+def flatten_probas(probas, labels, ignore=None):
235	-+ """
236	-+ Flattens predictions in the batch
237	-+ """
238	-+ if probas.dim() == 3:
239	-+ # assumes output of a sigmoid layer
240	-+ B, H, W = probas.size()
241	-+ probas = probas.view(B, 1, H, W)
242	-+ B, C, H, W = probas.size()
243	-+ probas = probas.permute(0, 2, 3, 1).contiguous().view(-1, C) # B * H * W, C = P, C
244	-+ labels = labels.view(-1)
245	-+ if ignore is None:
246	-+ return probas, labels
247	-+ valid = (labels != ignore)
248	-+ vprobas = probas[valid.nonzero().squeeze()]
249	-+ vlabels = labels[valid]
250	-+ return vprobas, vlabels
251	-+
252	-+def xloss(logits, labels, ignore=None):
253	-+ """
254	-+ Cross entropy loss
255	-+ """
256	-+ return F.cross_entropy(logits, Variable(labels), ignore_index=255)
257	-+
258	-+
259	-+# --------------------------- HELPER FUNCTIONS ---------------------------
260	-+def isnan(x):
261	-+ return x != x
262	-+
263	-+
264	-+def mean(l, ignore_nan=False, empty=0):
265	-+ """
266	-+ nanmean compatible with generators.
267	-+ """
268	-+ l = iter(l)
269	-+ if ignore_nan:
270	-+ l = ifilterfalse(isnan, l)
271	-+ try:
272	-+ n = 1
273	-+ acc = next(l)
274	-+ except StopIteration:
275	-+ if empty == 'raise':
276	-+ raise ValueError('Empty mean')
277	-+ return empty
278	-+ for n, v in enumerate(l, 2):
279	-+ acc += v
280	-+ if n == 1:
281	-+ return acc
282	-+ return acc / n
283	-
284	diff --git a/train.py b/train.py	14	diff --git a/train.py b/train.py
285	index dbbd1bd..2fb43f3 100644	15	index dbbd1bd..2fb43f3 100644
286	--- a/train.py	16	--- a/train.py
	@@ -1000,4 +730,4 @@ index 0000000..2921bb6
1000	+1a75e9f15481d11084fe66bc2a5afac6dc5bec20ed56a7351a6d65ef0fe8762b	730	+1a75e9f15481d11084fe66bc2a5afac6dc5bec20ed56a7351a6d65ef0fe8762b
1001	+08151b19806eebd58e5acec7e138dbfbb1761f41a1ab9620466584ecc7d5fada	731	+08151b19806eebd58e5acec7e138dbfbb1761f41a1ab9620466584ecc7d5fada
1002	+d827a7d80fc67487a3237135e0d43ae01b7bbcb135e1a167601fc974a8348c51	732	+d827a7d80fc67487a3237135e0d43ae01b7bbcb135e1a167601fc974a8348c51
1003	-+2869fad54664677e81bacbf00c2256e89a7b90b69d9688c9342e2c736ff5421c	733	++2869fad54664677e81bacbf00c2256e89a7b90b69d9688c9342e2c736ff5421c

MACL_PyTorch/contrib/cv/segmentation/Nested_UNet/nested_unet_pth2onnx.py+3-1

	@@ -25,13 +25,15 @@ def convert():
25	model.load_state_dict(checkpoint)	25	model.load_state_dict(checkpoint)
26		26
27	model.eval()	27	model.eval()
		28	+
28	input_names = ["actual_input_1"]	29	input_names = ["actual_input_1"]
29	output_names = ["output1"]	30	output_names = ["output1"]
30	dynamic_axes = {'actual_input_1': {0: '-1'}, 'output1': {0: '-1'}}	31	dynamic_axes = {'actual_input_1': {0: '-1'}, 'output1': {0: '-1'}}
31		32
32	dummy_input = torch.randn(1, 3, 96, 96)	33	dummy_input = torch.randn(1, 3, 96, 96)
33		34
34	- torch.onnx.export(model, dummy_input, output_file, input_names = input_names, dynamic_axes = dynamic_axes, output_names = output_names, opset_version=11)	35	+ torch.onnx.export(model, dummy_input, output_file, input_names = input_names, dynamic_axes = dynamic_axes,
		36	+ output_names = output_names, opset_version=11, verbose=True)
35		37
36	if __name__ == "__main__":	38	if __name__ == "__main__":
37	input_file = sys.argv[1]	39	input_file = sys.argv[1]

MACL_PyTorch/contrib/cv/segmentation/Nested_UNet/postprocess_nested_unet.py+31-9

	@@ -1,4 +1,4 @@
1	-# Copyright 2020 Huawei Technologies Co., Ltd	1	+# Copyright 2020-2022 Huawei Technologies Co., Ltd
2	#	2	#
3	# Licensed under the Apache License, Version 2.0 (the "License");	3	# Licensed under the Apache License, Version 2.0 (the "License");
4	# you may not use this file except in compliance with the License.	4	# you may not use this file except in compliance with the License.
	@@ -15,7 +15,7 @@
15	import os	15	import os
16	import sys	16	import sys
17	from glob import glob	17	from glob import glob
18	-	18	+import json
19	import cv2	19	import cv2
20	import numpy as np	20	import numpy as np
21		21
	@@ -46,18 +46,39 @@ def iou_score(output, target):
46		46
47	return (intersection + smooth) / (union + smooth)	47	return (intersection + smooth) / (union + smooth)
48		48
		49	+def read_info_from_json(json_path):
		50	+ '''
		51	+ 此函数用于读取inference_tools生成的json文件
		52	+ input: json文件地址
		53	+ output: dict结构: 为原始的json转换出来的结构
		54	+ '''
		55	+ if os.path.exists(json_path) is False:
		56	+ print(json_path, 'is not exist')
		57	+ with open(json_path, 'r') as f:
		58	+ load_data = json.load(f)
		59	+ file_info = load_data['filesinfo']
		60	+ return file_info
		61	+
49	def main():	62	def main():
50		63
51	- mask_ids = glob(os.path.join(result_dir, '*' + '.bin'))	64	+ file_info = read_info_from_json(result_json_path)
52	- mask_ids = [os.path.splitext(os.path.basename(p))[0] for p in mask_ids]
53		65
54	avg_meter = AverageMeter()	66	avg_meter = AverageMeter()
55		67
56	- for mask_id in mask_ids:	68	+ for i in file_info.items():
57	- result = np.fromfile(os.path.join(result_dir, mask_id + '.bin'), dtype='float32')	69	+ # 获取推理结果文件地址
		70	+ result_file_name = os.path.basename(i[1]['outfiles'][0])
		71	+ # 使用result_dir的路径作为结果文件的路径，可以使得运行该脚本的路径更通用
		72	+ res_path = os.path.join(result_dir_path, result_file_name)
		73	+ # 获取对应的标签
		74	+ label_id = os.path.splitext(os.path.basename(i[1]['infiles'][0]))[0]
		75	+
		76	+ result = np.fromfile(res_path, dtype='float32')
58	result = np.reshape(result, (1, 96, 96))	77	result = np.reshape(result, (1, 96, 96))
59	- mask = cv2.imread(os.path.join(mask_dir, mask_id.split('_')[0] + '.png'))	78	+
		79	+ mask = cv2.imread(os.path.join(mask_path, label_id + '.png'))
60	mask = mask.astype('float32') / 255	80	mask = mask.astype('float32') / 255
		81	+
61	mask = mask.transpose(2, 0, 1)[0]	82	mask = mask.transpose(2, 0, 1)[0]
62	iou = iou_score(result, mask)	83	iou = iou_score(result, mask)
63	avg_meter.update(iou)	84	avg_meter.update(iou)
	@@ -66,6 +87,7 @@ def main():
66		87
67		88
68	if __name__ == "__main__":	89	if __name__ == "__main__":
69	- result_dir = sys.argv[1]	90	+ result_json_path = sys.argv[1]
70	- mask_dir = sys.argv[2]	91	+ mask_path = sys.argv[2]
		92	+ result_dir_path = os.path.dirname(result_json_path)
71	main()	93	main()

DACL_PyTorch/contrib/cv/segmentation/Nested_UNet/test/README.md+0-49

	@@ -1,49 +0,0 @@
1	-环境准备：
2	-
3	-1.进入主目录
4	-```
5	-cd Nested_UNet
6	-```
7	-
8	-2.获取模型代码
9	-```
10	-git clone https://github.com/4uiiurz1/pytorch-nested-unet
11	-cd pytorch-nested-unet
12	-```
13	-
14	-3.安装必要的依赖，测试环境可能已经安装其中的一些不同版本的库了，故手动测试时不推荐使用该命令安装
15	-```
16	-pip3.7 install -r requirements.txt
17	-```
18	-
19	-4.修改模型代码
20	-```
21	-patch -p1 < ../nested_unet.diff
22	-```
23	-
24	-5.获取数据集
25	-从 https://www.kaggle.com/c/data-science-bowl-2018/data 下载stage1_train.zip到pytorch-nested-unet目录下，将其解压到指定目录。之后执行原代码仓自带的预处理脚本，并将处理好的数据集复制到主目录下。
26	-```
27	-mkdir -p inputs/data-science-bowl-2018/stage1_train/
28	-unzip -d inputs/data-science-bowl-2018/stage1_train/ stage1_train.zip
29	-python3.7 preprocess_dsb2018.py
30	-cp -r inputs/dsb2018_96/ ../
31	-cp val_ids.txt ../
32	-```
33	-
34	-6.获取权重文件
35	-由于原代码仓没有提供预训练的模型，因此需要自行在GPU环境下训练模型，之后将权重文件复制到主目录下。
36	-```
37	-python3.7 train.py --dataset dsb2018_96 --arch NestedUNet --loss LovaszHingeLoss --epochs 200
38	-cp models/dsb2018_96_NestedUNet_woDS/model.pth ../nested_unet.pth
39	-cd ../
40	-```
41	-
42	-7.获取benchmark工具
43	-将benchmark.x86_64 benchmark.aarch64放在当前目录
44	-
45	-8.310上执行，执行时确保device空闲
46	-```
47	-bash test/pth2om.sh
48	-bash test/eval_acc_perf.sh
49	-```

DACL_PyTorch/contrib/cv/segmentation/Nested_UNet/test/eval_acc_perf.sh+0-61

	@@ -1,61 +0,0 @@
1	-#!/bin/bash
2	-
3	-datasets_path="./dsb2018_96/"
4	-
5	-for para in $*
6	-do
7	- if [[ $para == --datasets_path* ]]; then
8	- datasets_path=`echo ${para#*=}`
9	- fi
10	-done
11	-
12	-arch=`uname -m`
13	-rm -rf ./prep_dataset
14	-python3.7 preprocess_nested_unet.py ${datasets_path}/images ./prep_dataset ./val_ids.txt
15	-if [ $? != 0 ]; then
16	- echo "fail!"
17	- exit -1
18	-fi
19	-python3.7 gen_dataset_info.py bin ./prep_dataset ./nested_unet_prep_bin.info 96 96
20	-if [ $? != 0 ]; then
21	- echo "fail!"
22	- exit -1
23	-fi
24	-source env.sh
25	-rm -rf result/dumpOutput_device0
26	-./benchmark.${arch} -model_type=vision -device_id=0 -batch_size=1 -om_path=nested_unet_bs1.om -input_text_path=./nested_unet_prep_bin.info -input_width=96 -input_height=96 -output_binary=True -useDvpp=False
27	-if [ $? != 0 ]; then
28	- echo "fail!"
29	- exit -1
30	-fi
31	-rm -rf result/dumpOutput_device1
32	-./benchmark.${arch} -model_type=vision -device_id=1 -batch_size=16 -om_path=nested_unet_bs16.om -input_text_path=./nested_unet_prep_bin.info -input_width=96 -input_height=96 -output_binary=True -useDvpp=False
33	-if [ $? != 0 ]; then
34	- echo "fail!"
35	- exit -1
36	-fi
37	-echo "====accuracy data===="
38	-echo "bs1"
39	-python3.7 postprocess_nested_unet.py result/dumpOutput_device0/ ${datasets_path}/masks/0/
40	-if [ $? != 0 ]; then
41	- echo "fail!"
42	- exit -1
43	-fi
44	-echo "bs16"
45	-python3.7 postprocess_nested_unet.py result/dumpOutput_device1/ ${datasets_path}/masks/0/
46	-if [ $? != 0 ]; then
47	- echo "fail!"
48	- exit -1
49	-fi
50	-echo "====performance data===="
51	-python3.7 test/parse.py result/perf_vision_batchsize_1_device_0.txt
52	-if [ $? != 0 ]; then
53	- echo "fail!"
54	- exit -1
55	-fi
56	-python3.7 test/parse.py result/perf_vision_batchsize_16_device_1.txt
57	-if [ $? != 0 ]; then
58	- echo "fail!"
59	- exit -1
60	-fi
61	-echo "success"

DACL_PyTorch/contrib/cv/segmentation/Nested_UNet/test/parse.py+0-32

	@@ -1,32 +0,0 @@
1	-# Copyright 2020 Huawei Technologies Co., Ltd
2	-#
3	-# Licensed under the Apache License, Version 2.0 (the "License");
4	-# you may not use this file except in compliance with the License.
5	-# You may obtain a copy of the License at
6	-#
7	-# http://www.apache.org/licenses/LICENSE-2.0
8	-#
9	-# Unless required by applicable law or agreed to in writing, software
10	-# distributed under the License is distributed on an "AS IS" BASIS,
11	-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12	-# See the License for the specific language governing permissions and
13	-# limitations under the License.
14	-
15	-import sys
16	-import json
17	-import re
18	-
19	-if __name__ == '__main__':
20	- if sys.argv[1].endswith('.json'):
21	- result_json = sys.argv[1]
22	- with open(result_json, 'r') as f:
23	- content = f.read()
24	- tops = [i.get('value') for i in json.loads(content).get('value') if 'Top' in i.get('key')]
25	- print('om {} top1:{} top5:{}'.format(result_json.split('_')[1].split('.')[0], tops[0], tops[4]))
26	- elif sys.argv[1].endswith('.txt'):
27	- result_txt = sys.argv[1]
28	- with open(result_txt, 'r') as f:
29	- content = f.read()
30	- txt_data_list = [i.strip() for i in re.findall(r':(.*?),', content.replace('\n', ',') + ',')]
31	- fps = float(txt_data_list[7].replace('samples/s', '')) * 4
32	- print('310 bs{} fps:{}'.format(result_txt.split('_')[3], fps))

DACL_PyTorch/contrib/cv/segmentation/Nested_UNet/test/pth2om.sh+0-13

	@@ -1,13 +0,0 @@
1	-#!/bin/bash
2	-
3	-rm -rf nested_unet.onnx
4	-python3.7 nested_unet_pth2onnx.py nested_unet.pth nested_unet.onnx
5	-source env.sh
6	-rm -rf nested_unet_bs1.om nested_unet_bs16.om
7	-atc --model=./nested_unet.onnx --framework=5 --auto_tune_mode="GA,RL" --output=nested_unet_bs1 --input_format=NCHW --input_shape="actual_input_1:1,3,96,96" --log=info --soc_version=Ascend310
8	-atc --model=./nested_unet.onnx --framework=5 --auto_tune_mode="GA,RL" --output=nested_unet_bs16 --input_format=NCHW --input_shape="actual_input_1:16,3,96,96" --log=info --soc_version=Ascend310
9	-if [ -f "nested_unet_bs1.om" ] && [ -f "nested_unet_bs16.om" ]; then
10	- echo "success"
11	-else
12	- echo "fail!"
13	-fi