11. 模型性能Benchmark

11.1. 说明

  • 测试条件：

    • 测试开发板：X5-DVB board。

    • 测试核心数：单核。

    • 性能数据获取频率设置为：5分钟时间内性能参数的平均值。

    • Python版本：Python3.10。

  • 缩写说明：

    • C = 计算量，单位为GOPs（十亿次运算/秒）。此数据通过 hb_perf 工具获得。

    • FPS = 每秒帧率。此数据在开发板单线程运行ai_benchmark示例包/script路径下各模型子文件夹的 fps.sh 脚本获取，包含后处理。

    • ITC = 推理耗时，单位为ms（毫秒）。此数据在开发板单线程运行ai_benchmark示例包/script路径下各模型子文件夹的 latency.sh 脚本获取，不含后处理。

    • TCPP = 后处理耗时，单位为ms（毫秒）。此数据在开发板单线程运行ai_benchmark示例包/script路径下各模型子文件夹的 latency.sh 脚本获取。

    • RV = 单帧读取数据量，单位为mb（兆比特）。此数据通过 hb_perf 工具获得。

    • WV = 单帧写入数据量，单位为mb（兆比特）。此数据通过 hb_perf 工具获得。

11.2. 模型重要性能数据

MODEL NAME

INPUT SIZE

C(GOPs)

FPS

ITC(ms)

TCPP(ms)

Dataset

MobileNetv1

1x224x224x3

1.14

1365.00

1.015

0.051

ImageNet

MobileNetv2

1x224x224x3

0.63

1492.78

0.892

0.068

ImageNet

GoogleNet

1x224x224x3

3.00

631.55

1.851

0.052

ImageNet

Resnet18

1x224x224x3

3.65

449.30

2.523

0.052

ImageNet

EfficientNet_Lite0

1x224x224x3

0.77

1014.45

1.272

0.052

ImageNet

EfficientNet_Lite1

1x240x240x3

1.20

828.00

1.508

0.052

ImageNet

EfficientNet_Lite2

1x260x260x3

1.72

627.00

1.887

0.052

ImageNet

EfficientNet_Lite3

1x280x280x3

2.77

455.70

2.490

0.051

ImageNet

EfficientNet_Lite4

1x300x300x3

5.11

285.15

3.804

0.052

ImageNet

Vargconvnet

1x224x224x3

9.06

310.87

3.488

0.052

ImageNet

Efficientnasnet_m

1x300x300x3

4.53

291.47

3.493

0.052

ImageNet

Efficientnasnet_s

1x280x280x3

1.44

777.49

1.559

0.052

ImageNet

YOLOv2_Darknet19

1x608x608x3

62.94

38.33

26.448

1.288

COCO

YOLOv3_Darknet53

1x416x416x3

65.90

31.28

32.377

7.582

COCO

YOLOv5x_v2.0

1x672x672x3

243.91

10.37

96.864

29.262

COCO

Ssd_mobilenetv1

1x300x300x3

2.30

618.90

1.896

0.916

VOC

Centernet_resnet101

1x512x512x3

90.54

26.38

38.314

3.764

COCO

YOLOv3_VargDarknet

1x416x416x3

42.82

51.73

19.791

7.567

COCO

Deeplabv3plus_efficientnetb0

1x1024x2048x3

30.78

35.84

28.170

0.900

Cityscapes

Fastscnn_efficientnetb0

1x1024x2048x3

12.49

70.02

14.460

0.822

Cityscapes

Deeplabv3plus_efficientnetm1

1x1024x2048x3

77.05

18.91

53.123

0.881

Cityscapes

Deeplabv3plus_efficientnetm2

1x1024x2048x3

124.16

12.48

80.466

0.840

Cityscapes

Resnet50

1x224x224x3

7.72

209.85

4.994

0.068

ImageNet

VargNetV2

1x224x224x3

0.72

1521.44

1.010

0.068

ImageNet

Swint

1x224x224x3

8.98

44.53

22.725

0.069

ImageNet

MixVarGENet

1x224x224x3

2.07

1278.23

1.010

0.068

ImageNet

Fcos_efficientnetb0

1x512x512x3

5.02

342.34

3.177

0.217

COCO

Fcos_efficientnetb2

1x768x768x3

22.08

69.26

14.980

5.738

COCO

Fcos_efficientnetb3

1x896x896x3

41.45

37.42

27.320

7.753

COCO

Pointpillars_kitti_car

1x1x150000x4

66.82

26.15

77.429

2.296

Kitti3d

RetinaNet_vargnetv2_fpn

1x1024x1024x3

301.27

9.83

102.106

4.858

COCO

Yolov3_mobilenetv1

1x416x416x3

20.58

95.67

10.878

1.275

VOC

Ganet_mixvargenet

1x320x800x3

10.74

290.82

3.687

0.795

CuLane

DETR_resnet50

1x800x1333x3

202.99

7.60

131.894

1.391

MS COCO

DETR_efficientnetb3

1x800x1333x3

67.31

10.69

393.779

1.399

MS COCO

FCOS3D_efficientnetb0

1x512x896x3

19.94

597.53

3.921

8.714

nuscenes

Centerpoint_pointpillar

300000x5

127.73

102.36

24.822

52.530

nuscenes

Keypoint_efficientnetb0

1x128x128x3

0.45

1345.10

0.968

0.296

carfusion

Unet_mobilenetv1

1x1024x2048x3

7.36

175.54

5.943

0.474

Cityscapes

Pwcnet_pwcnetneck

1x384x512x6

81.71

24.32

41.366

0.244

flyingchairs

Motr_efficientnetb3

image:

1x800x1422x3

track_query:

1x2x128x156

ref_points:

1x2x128x4

mask_query:

1x1x256x1

64.43

12.87

76.466

21.386

Mot17

Bev_lss_efficientnetb0_multitask

image:

6x256x704x3

points(0&1):

10x128x128x2

2.41

31.27

32.997

14.508

nuscenes

Bev_gkt_mixvargenet_multitask

image:

6x512x960x3

points(0-8):

6x64x64x2

34.49

10.76

94.288

14.636

nuscenes

Bev_ipm_efficientnetb0_multitask

image:

6x512x960x3

points:

6x128x128x2

8.83

25.15

40.900

14.543

nuscenes

Bev_ipm_4d_efficientnetb0_multitask

image:

6x512x960x3

points:

6x128x128x2

prev_feat:

1x128x128x64

prev_point:

1x128x128x2

8.93

23.12

45.353

14.780

nuscenes

Detr3d_efficientnetb3_nuscenes

coords(0-3):

6x4x256x2

image:

6x512x1408x3

masks:

1x4x256x24

37.55

4.87

206.104

2.009

nuscenes

Centerpoint_mixvargnet_multitask

300000x5

51.45

104.42

23.771

50.498

nuscenes

Stereonetplus_mixvargenet

2x544x960x3

24.29

253.98

6.477

15.395

SceneFlow

Densetnt_vectornet

goals_2d:

30x1x2048x2

goals_2d_mask:

30x1x2048x1

instance_mask:

30x1x96x1

lane_feat:

30x9x64x11

traj_feat:

30x19x32x9

0.42

86.81

26.482

10.476

Argoverse 1

11.3. 模型全部性能数据

11.3.1. MobileNetv1

11.3.2. MobileNetv2

  • INPUT SIZE: 1x224x224x3

  • C(GOPs): 0.63

  • FPS: 1492.78

  • ITC(ms): 0.892

  • TCPP(ms): 0.068

  • RV(mb): 3.03

  • WV(mb): 0.12

  • Dataset: ImageNet

11.3.3. GoogleNet

11.3.4. Resnet18

11.3.5. EfficientNet_Lite0

11.3.6. EfficientNet_Lite1

11.3.7. EfficientNet_Lite2

11.3.8. EfficientNet_Lite3

11.3.9. EfficientNet_Lite4

11.3.10. Vargconvnet

11.3.11. Efficientnasnet_m

11.3.12. Efficientnasnet_s

11.3.13. YOLOv2_Darknet19

  • INPUT SIZE: 1x608x608x3

  • C(GOPs): 62.94

  • FPS: 38.33

  • ITC(ms): 26.448

  • TCPP(ms): 1.288

  • RV(mb): 52.00

  • WV(mb): 7.31

  • Dataset: COCO

  • LINKS: https://pjreddie.com/darknet/yolo

11.3.14. YOLOv3_Darknet53

11.3.15. YOLOv5x_v2.0

11.3.16. Ssd_mobilenetv1

11.3.17. Centernet_resnet101

11.3.18. YOLOv3_VargDarknet

11.3.19. Deeplabv3plus_efficientnetb0

11.3.20. Fastscnn_efficientnetb0

11.3.21. Deeplabv3plus_efficientnetm1

11.3.22. Deeplabv3plus_efficientnetm2

11.3.23. Resnet50

  • INPUT SIZE: 1x224x224x3

  • C(GOPs): 7.72

  • FPS: 209.85

  • ITC(ms): 4.994

  • TCPP(ms): 0.068

  • RV(mb): 25.30

  • WV(mb): 1.77

  • Dataset: ImageNet

11.3.24. VargNetV2

  • INPUT SIZE: 1x224x224x3

  • C(GOPs): 0.72

  • FPS: 1521.44

  • ITC(ms): 1.010

  • TCPP(ms): 0.068

  • RV(mb): 3.66

  • WV(mb): 0.02

  • Dataset: ImageNet

11.3.25. Swint

  • INPUT SIZE: 1x224x224x3

  • C(GOPs): 8.98

  • FPS: 44.53

  • ITC(ms): 22.725

  • TCPP(ms): 0.069

  • RV(mb): 46.90

  • WV(mb): 11.00

  • Dataset: ImageNet

11.3.26. MixVarGENet

  • INPUT SIZE: 1x224x224x3

  • C(GOPs): 2.07

  • FPS: 1278.23

  • ITC(ms): 1.010

  • TCPP(ms): 0.068

  • RV(mb): 2.37

  • WV(mb): 0.12

  • Dataset: ImageNet

11.3.27. Fcos_efficientnetb0

  • INPUT SIZE: 1x512x512x3

  • C(GOPs): 5.02

  • FPS: 342.34

  • ITC(ms): 3.177

  • TCPP(ms): 0.217

  • RV(mb): 5.85

  • WV(mb): 1.28

  • Dataset: COCO

11.3.28. Fcos_efficientnetb2

11.3.29. Fcos_efficientnetb3

11.3.30. Pointpillars_kitti_car

  • INPUT SIZE: 1x1x150000x4

  • C(GOPs): 66.82

  • FPS: 26.15

  • ITC(ms): 77.429

  • TCPP(ms): 2.296

  • RV(mb): 72.90

  • WV(mb): 40.00

  • Dataset: Kitti3d

11.3.31. RetinaNet_vargnetv2_fpn

  • INPUT SIZE: 1x1024x1024x3

  • C(GOPs): 301.27

  • FPS: 9.83

  • ITC(ms): 102.106

  • TCPP(ms): 4.858

  • RV(mb): 145.00

  • WV(mb): 87.30

  • Dataset: COCO

11.3.32. Yolov3_mobilenetv1

  • INPUT SIZE: 1x416x416x3

  • C(GOPs): 20.58

  • FPS: 95.67

  • ITC(ms): 10.878

  • TCPP(ms): 1.275

  • RV(mb): 29.30

  • WV(mb): 4.82

  • Dataset: VOC

11.3.33. Ganet_mixvargenet

  • INPUT SIZE: 1x320x800x3

  • C(GOPs): 10.74

  • FPS: 290.82

  • ITC(ms): 3.687

  • TCPP(ms): 0.795

  • RV(mb): 5.23

  • WV(mb): 3.71

  • Dataset: CuLane

11.3.34. DETR_resnet50

  • INPUT SIZE: 1x800x1333x3

  • C(GOPs): 202.99

  • FPS: 7.60

  • ITC(ms): 131.894

  • TCPP(ms): 1.391

  • RV(mb): 384.00

  • WV(mb): 264.00

  • Dataset: MS COCO

11.3.35. DETR_efficientnetb3

  • INPUT SIZE: 1x800x1333x3

  • C(GOPs): 67.31

  • FPS: 10.69

  • ITC(ms): 93.779

  • TCPP(ms): 1.399

  • RV(mb): 250.00

  • WV(mb): 187.00

  • Dataset: MS COCO

11.3.36. FCOS3D_efficientnetb0

  • INPUT SIZE: 1x512x896x3

  • C(GOPs): 19.94

  • FPS: 597.53

  • ITC(ms): 3.921

  • TCPP(ms): 8.714

  • RV(mb): 11.55

  • WV(mb): 5.10

  • Dataset: nuscenes

11.3.37. Centerpoint_pointpillar

  • INPUT SIZE: 300000x5

  • C(GOPs): 127.73

  • FPS: 102.36

  • ITC(ms): 24.822

  • TCPP(ms): 52.530

  • RV(mb): 39.37

  • WV(mb): 19.04

  • Dataset: nuscenes

11.3.38. Keypoint_efficientnetb0

  • INPUT SIZE: 1x128x128x3

  • C(GOPs): 0.45

  • FPS: 1345.10

  • ITC(ms): 0.968

  • TCPP(ms): 0.296

  • RV(mb): 4.38

  • WV(mb): 0.01

  • Dataset: carfusion

11.3.39. Unet_mobilenetv1

  • INPUT SIZE: 1x1024x2048x3

  • C(GOPs): 7.36

  • FPS: 175.54

  • ITC(ms): 5.943

  • TCPP(ms): 0.474

  • RV(mb): 19.60

  • WV(mb): 14.70

  • Dataset: Cityscapes

11.3.40. Pwcnet_pwcnetneck

  • INPUT SIZE: 1x384x512x6

  • C(GOPs): 81.71

  • FPS: 24.32

  • ITC(ms): 41.366

  • TCPP(ms): 0.244

  • RV(mb): 66.60

  • WV(mb): 44.90

  • Dataset: flyingchairs

11.3.41. Motr_efficientnetb3

  • INPUT SIZE: image: 1x800x1422x3 track_query: 1x2x128x156 ref_points: 1x2x128x4 mask_query: 1x1x256x1

  • C(GOPs): 64.43

  • FPS: 12.87

  • ITC(ms): 76.466

  • TCPP(ms): 21.386

  • RV(mb): 157.00

  • WV(mb): 103.00

  • Dataset: Mot17

11.3.42. Bev_lss_efficientnetb0_multitask

  • INPUT SIZE: image: 6x256x704x3 points(0&1): 10x128x128x2

  • C(GOPs): 2.41

  • FPS: 31.27

  • ITC(ms): 32.997

  • TCPP(ms): 14.508

  • RV(mb): 6.03

  • WV(mb): 4.28

  • Dataset: nuscenes

11.3.43. Bev_gkt_mixvargenet_multitask

  • INPUT SIZE: image: 6x512x960x3 points(0-8): 6x64x64x2

  • C(GOPs): 34.49

  • FPS: 10.76

  • ITC(ms): 94.288

  • TCPP(ms): 14.636

  • RV(mb): 22.70

  • WV(mb): 12.50

  • Dataset: nuscenes

11.3.44. Bev_ipm_efficientnetb0_multitask

  • INPUT SIZE: image: 6x512x960x3 points: 6x128x128x2

  • C(GOPs): 8.83

  • FPS: 25.15

  • ITC(ms): 40.900

  • TCPP(ms): 14.543

  • RV(mb): 8.33

  • WV(mb): 6.45

  • Dataset: nuscenes

11.3.45. Bev_ipm_4d_efficientnetb0_multitask

  • INPUT SIZE: image: 6x512x960x3 points: 6x128x128x2 prev_feat: 1x128x128x64 prev_point: 1x128x128x2

  • C(GOPs): 8.93

  • FPS: 23.12

  • ITC(ms): 45.353

  • TCPP(ms): 14.780

  • RV(mb): 9.81

  • WV(mb): 7.70

  • Dataset: nuscenes

11.3.46. Detr3d_efficientnetb3_nuscenes

  • INPUT SIZE: coords(0-3): 6x4x256x2 image: 6x512x1408x3 masks: 1x4x256x24

  • C(GOPs): 37.55

  • FPS: 4.87

  • ITC(ms): 206.104

  • TCPP(ms): 2.009

  • RV(mb): 103.00

  • WV(mb): 77.20

  • Dataset: nuscenes

11.3.47. Centerpoint_mixvargnet_multitask

  • INPUT SIZE: 300000x5

  • C(GOPs): 51.45

  • FPS: 104.42

  • ITC(ms): 23.771

  • TCPP(ms): 50.498

  • RV(mb): 33.84

  • WV(mb): 14.45

  • Dataset: nuscenes

11.3.48. Stereonetplus_mixvargenet

  • INPUT SIZE: 2x544x960x3

  • C(GOPs): 24.29

  • FPS: 253.98

  • ITC(ms): 6.477

  • TCPP(ms): 15.395

  • RV(mb): 15.40

  • WV(mb): 10.90

  • Dataset: SceneFlow

11.3.49. Densetnt_vectornet

  • INPUT SIZE: goals_2d: 30x1x2048x2 goals_2d_mask: 30x1x2048x1 instance_mask: 30x1x96x1 lane_feat: 30x9x64x11 traj_feat: 30x19x32x9

  • C(GOPs): 0.42

  • FPS: 86.81

  • ITC(ms): 26.482

  • TCPP(ms): 10.476

  • RV(mb): 3.29

  • WV(mb): 2.92

  • Dataset: Argoverse 1