7.1. 模型性能 Benchmark¶
7.1.1. 说明¶
测试条件：
测试开发板：x3sdbx3-samsung2G-3200。
测试核心数：latency 单核，fps 双核。
性能数据获取频率设置为：5 分钟时间内性能参数的平均值。
表头缩写：
C = 计算量，单位为 GOPs（十亿次运算/秒）。此数据通过
hb_perf工具获得。FPS = 每秒帧率。此数据在开发板单线程运行 ai_benchmark 示例包/script 路径下各模型子文件夹的 fps.sh 脚本获取，包含后处理。
ITC = 推理耗时，单位为 ms（毫秒）。此数据在开发板单线程运行 ai_benchmark 示例包/script 路径下各模型子文件夹的 latency.sh 脚本获取，不含后处理。
TCPP = 后处理耗时，单位为 ms（毫秒）。此数据在开发板单线程运行 ai_benchmark 示例包/script 路径下各模型子文件夹的 latency.sh 脚本获取。
RV = 单帧读取数据量，单位为 mb（兆比特）。此数据通过
hb_perf工具获得。WV = 单帧写入数据量，单位为 mb（兆比特）。此数据通过
hb_perf工具获得。
7.1.2. 模型重要性能数据¶
MODEL NAME |
INPUT SIZE |
C(GOPs) |
FPS |
ITC(ms) |
TCPP(ms) |
ACCURACY |
Dataset |
|---|---|---|---|---|---|---|---|
MobileNetv1 |
1x224x224x3 |
1.14 |
661.12 |
3.200 |
0.086 |
Top1: 0.7061(FLOAT) 0.7034(INT8) |
ImageNet |
MobileNetv2 |
1x224x224x3 |
0.86 |
869.68 |
2.461 |
0.086 |
Top1: 0.7167(FLOAT) 0.7122(INT8) |
ImageNet |
GoogleNet |
1x224x224x3 |
3.00 |
234.83 |
8.164 |
0.087 |
Top1: 0.7001(FLOAT) 0.6992(INT8) |
ImageNet |
Resnet18 |
1x224x224x3 |
3.65 |
228.31 |
8.754 |
0.086 |
Top1: 0.6836(FLOAT) 0.6830(INT8) |
ImageNet |
EfficientNet_Lite0 |
1x224x224x3 |
0.77 |
896.82 |
2.315 |
0.083 |
Top1: 0.7491(FLOAT) 0.7473(INT8) |
ImageNet |
EfficientNet_Lite1 |
1x240x240x3 |
1.20 |
624.36 |
3.196 |
0.081 |
Top1: 0.7647(FLOAT) 0.7625(INT8) |
ImageNet |
EfficientNet_Lite2 |
1x260x260x3 |
1.72 |
361.83 |
5.643 |
0.081 |
Top1: 0.7738(FLOAT) 0.7714(INT8) |
ImageNet |
EfficientNet_Lite3 |
1x280x280x3 |
2.77 |
229.13 |
8.550 |
0.080 |
Top1: 0.7922(FLOAT) 0.7901(INT8) |
ImageNet |
EfficientNet_Lite4 |
1x300x300x3 |
5.11 |
130.06 |
14.739 |
0.083 |
Top1: 0.8070(FLOAT) 0.8059(INT8) |
ImageNet |
YOLOv2_Darknet19 |
1x608x608x3 |
62.94 |
12.60 |
154.412 |
1.538 |
[IoU=0.50:0.95]= 0.2760(FLOAT) 0.2700(INT8) |
COCO |
YOLOv3_Darknet53 |
1x416x416x3 |
65.90 |
11.71 |
164.915 |
10.771 |
[IoU=0.50:0.95]= 0.3330(FLOAT) 0.3360(INT8) |
COCO |
YOLOv5s |
1x672x672x3 |
19.18 |
32.62 |
65.179 |
20.329 |
[IoU=0.50:0.95]= 0.3520(FLOAT) 0.3420(INT8) |
COCO |
SSD_MobileNetv1 |
1x300x300x3 |
2.30 |
277.29 |
6.914 |
1.369 |
mAP: 0.7342(FLOAT) 0.7274(INT8) |
VOC |
EfficientDetd0 |
1x512x512x3 |
4.93 |
78.31 |
17.080 |
20.202 |
[IoU=0.50:0.95]= 0.3240(FLOAT) 0.3150(INT8) |
COCO |
CenterNet_Resnet50 |
1x512x512x3 |
51.75 |
18.03 |
110.484 |
26.496 |
[IoU=0.50:0.95]= 0.3180(FLOAT) 0.3130(INT8) |
COCO |
FCOS_efficientnetb0(QAT) |
1x512x512x3 |
5.02 |
169.43 |
12.053 |
7.840 |
[IoU=0.50:0.95]= 0.3470(FLOAT) 0.3480(INT8) |
COCO |
Fcos_efficientnetb1 |
1x640x640x3 |
12.16 |
65.30 |
30.155 |
12.588 |
[IoU=0.50:0.95]= 0.4030(FLOAT) 0.4040(INT8) |
COCO |
FCOS_efficientnetb2 |
1x768x768x3 |
22.08 |
38.43 |
51.120 |
18.182 |
[IoU=0.50:0.95]= 0.4450(FLOAT) 0.4460(INT8) |
COCO |
FCOS_efficientnetb0(PTQ) |
1x512x512x3 |
5.02 |
174.14 |
13.360 |
3.969 |
[IoU=0.50:0.95]= 0.3630(FLOAT) 0.3480(INT8) |
COCO |
UNet_mobilenet |
1x1024x2048x3 |
7.37 |
57.16 |
27.690 |
21.501 |
mIoU: 0.6411(FLOAT) 0.6382(INT8) |
Cityscapes |
Deeplabv3plus_efficientnetb0 |
1x1024x2048x3 |
30.78 |
23.65 |
79.348 |
1.472 |
mIoU: 0.7630(FLOAT) 0.7567(INT8) |
Cityscapes |
Fastscnn_efficientnetb0 |
1x1024x2048x3 |
12.50 |
47.16 |
41.305 |
1.437 |
mIoU: 0.6997(FLOAT) 0.6927(INT8) |
Cityscapes |
7.1.3. 模型全部性能数据¶
7.1.3.1. MobileNetv1¶
INPUT SIZE: 1x224x224x3
C(GOPs): 1.14
FPS: 661.12
ITC(ms): 3.200
TCPP(ms): 0.086
RV(mb): 5.28
WV(mb): 0.83
Dataset: ImageNet
ACCURACY: Top1: 0.7061(FLOAT)/0.7034(INT8)
7.1.3.2. MobileNetv2¶
INPUT SIZE: 1x224x224x3
C(GOPs): 0.86
FPS: 869.68
ITC(ms): 2.461
TCPP(ms): 0.086
RV(mb): 3.99
WV(mb): 0.23
Dataset: ImageNet
ACCURACY: Top1: 0.7167(FLOAT)/0.7122(INT8)
7.1.3.3. GoogleNet¶
INPUT SIZE: 1x224x224x3
C(GOPs): 3.00
FPS: 234.83
ITC(ms): 8.164
TCPP(ms): 0.087
RV(mb): 10.31
WV(mb): 3.36
Dataset: ImageNet
ACCURACY: Top1: 0.7001(FLOAT)/0.6992(INT8)
LINKS: http://www.image-net.org
7.1.3.4. Resnet18¶
INPUT SIZE: 1x224x224x3
C(GOPs): 3.65
FPS: 228.31
ITC(ms): 8.754
TCPP(ms): 0.086
RV(mb): 13.26
WV(mb): 0.78
Dataset: ImageNet
ACCURACY: Top1: 0.6836(FLOAT)/0.6830(INT8)
LINKS: https://github.com/HolmesShuan/ResNet-18-Caffemodel-on-ImageNet
7.1.3.5. EfficientNet_Lite0¶
INPUT SIZE: 1x224x224x3
C(GOPs): 0.77
FPS: 896.82
ITC(ms): 2.315
TCPP(ms): 0.083
RV(mb): 5.31
WV(mb): 0.36
Dataset: ImageNet
ACCURACY: Top1: 0.7491(FLOAT)/0.7473(INT8)
LINKS: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite
7.1.3.6. EfficientNet_Lite1¶
INPUT SIZE: 1x240x240x3
C(GOPs): 1.20
FPS: 624.36
ITC(ms): 3.196
TCPP(ms): 0.081
RV(mb): 6.37
WV(mb): 0.60
Dataset: ImageNet
ACCURACY: Top1: 0.7647(FLOAT)/0.7625(INT8)
LINKS: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite
7.1.3.7. EfficientNet_Lite2¶
INPUT SIZE: 1x260x260x3
C(GOPs): 1.72
FPS: 361.83
ITC(ms): 5.643
TCPP(ms): 0.081
RV(mb): 6.61
WV(mb): 0.14
Dataset: ImageNet
ACCURACY: Top1: 0.7738(FLOAT)/0.7714(INT8)
LINKS: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite
7.1.3.8. EfficientNet_Lite3¶
INPUT SIZE: 1x280x280x3
C(GOPs): 2.77
FPS: 229.13
ITC(ms): 8.550
TCPP(ms): 0.080
RV(mb): 9.16
WV(mb): 0.42
Dataset: ImageNet
ACCURACY: Top1: 0.7922(FLOAT)/0.7901(INT8)
LINKS: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite
7.1.3.9. EfficientNet_Lite4¶
INPUT SIZE: 1x300x300x3
C(GOPs): 5.11
FPS: 130.06
ITC(ms): 14.739
TCPP(ms): 0.083
RV(mb): 14.96
WV(mb): 1.36
Dataset: ImageNet
ACCURACY: Top1: 0.8070(FLOAT)/0.8059(INT8)
LINKS: https://github.com/tensorflow/tpu/tree/master/models/official/efficientnet/lite
7.1.3.10. YOLOv2_Darknet19¶
INPUT SIZE: 1x608x608x3
C(GOPs): 62.94
FPS: 12.60
ITC(ms): 154.412
TCPP(ms): 1.538
RV(mb): 67.25
WV(mb): 15.25
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.2760(FLOAT)/0.2700(INT8)
7.1.3.11. YOLOv3_Darknet53¶
INPUT SIZE: 1x416x416x3
C(GOPs): 65.90
FPS: 11.71
ITC(ms): 164.915
TCPP(ms): 10.771
RV(mb): 103.57
WV(mb): 33.26
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.3330(FLOAT)/0.3360(INT8)
7.1.3.12. YOLOv5s¶
INPUT SIZE: 1x672x672x3
C(GOPs): 19.18
FPS: 32.62
ITC(ms): 65.179
TCPP(ms): 20.329
RV(mb): 42.32
WV(mb): 40.07
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.3520(FLOAT)/0.3420(INT8)
LINKS: https://github.com/ultralytics/yolov5/releases/tag/v2.0
7.1.3.13. SSD_MobileNetv1¶
INPUT SIZE: 1x300x300x3
C(GOPs): 2.30
FPS: 277.29
ITC(ms): 6.914
TCPP(ms): 1.369
RV(mb): 9.23
WV(mb): 2.96
Dataset: VOC
ACCURACY: mAP: 0.7342(FLOAT)/0.7274(INT8)
7.1.3.14. EfficientDetd0¶
INPUT SIZE: 1x512x512x3
C(GOPs): 4.93
FPS: 78.31
ITC(ms): 17.080
TCPP(ms): 20.202
RV(mb): 7.62
WV(mb): 19.09
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.3240(FLOAT)/0.3150(INT8)
LINKS: OE 发布包下载
7.1.3.15. CenterNet_Resnet50¶
INPUT SIZE: 1x512x512x3
C(GOPs): 51.75
FPS: 18.03
ITC(ms): 110.484
TCPP(ms): 26.496
RV(mb): 66.66
WV(mb): 33.88
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.3180(FLOAT)/0.3130(INT8)
LINKS: OE 发布包下载
7.1.3.16. FCOS_efficientnetb0(QAT)¶
INPUT SIZE: 1x512x512x3
C(GOPs): 5.02
FPS: 169.43
ITC(ms): 12.053
TCPP(ms): 7.840
RV(mb): 7.82
WV(mb): 4.77
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.3470(FLOAT)/0.3480(INT8)
LINKS: OE 发布包下载
7.1.3.17. Fcos_efficientnetb1¶
INPUT SIZE: 1x640x640x3
C(GOPs): 12.16
FPS: 65.30
ITC(ms): 30.155
TCPP(ms): 12.588
RV(mb): 29.70
WV(mb): 22.13
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.4030(FLOAT)/0.4040(INT8)
LINKS: OE 发布包下载
7.1.3.18. FCOS_efficientnetb2¶
INPUT SIZE: 1x768x768x3
C(GOPs): 22.08
FPS: 38.43
ITC(ms): 51.120
TCPP(ms): 18.182
RV(mb): 51.87
WV(mb): 39.74
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.4450(FLOAT)/0.4460(INT8)
LINKS: OE 发布包下载
7.1.3.19. FCOS_efficientnetb0(PTQ)¶
INPUT SIZE: 1x512x512x3
C(GOPs): 5.02
FPS: 174.14
ITC(ms): 13.360
TCPP(ms): 3.969
RV(mb): 8.06
WV(mb): 4.78
Dataset: COCO
ACCURACY: [IoU=0.50:0.95]= 0.3630(FLOAT)/0.3480(INT8)
LINKS: OE 发布包下载
7.1.3.20. UNet_mobilenet¶
INPUT SIZE: 1x1024x2048x3
C(GOPs): 7.37
FPS: 57.16
ITC(ms): 27.690
TCPP(ms): 21.501
RV(mb): 48.51
WV(mb): 49.91
Dataset: Cityscapes
ACCURACY: mIoU: 0.6411(FLOAT)/0.6382(INT8)
LINKS: OE 发布包下载
7.1.3.21. Deeplabv3plus_efficientnetb0¶
INPUT SIZE: 1x1024x2048x3
C(GOPs): 30.78
FPS: 23.65
ITC(ms): 79.348
TCPP(ms): 1.472
RV(mb): 84.31
WV(mb): 45.85
Dataset: Cityscapes
ACCURACY: mIoU: 0.7630(FLOAT)/0.7567(INT8)
LINKS: OE 发布包下载
7.1.3.22. Fastscnn_efficientnetb0¶
INPUT SIZE: 1x1024x2048x3
C(GOPs): 12.50
FPS: 47.16
ITC(ms): 41.305
TCPP(ms): 1.437
RV(mb): 47.00
WV(mb): 43.20
Dataset: Cityscapes
ACCURACY: mIoU: 0.6997(FLOAT)/0.6927(INT8)
LINKS: OE 发布包下载