10.3.1. ARM Runtime API

10.3.1.1. General API Description

10.3.1.1.1. Data Types and Data Structures

10.3.1.1.1.1. HB_DSP_VERSION_MAJOR

#define HB_DSP_VERSION_MAJOR 0U

Major version number of DSP.

10.3.1.1.1.2. HB_DSP_VERSION_MINOR

#define HB_DSP_VERSION_MINOR 3U

Minor version number of DSP.

10.3.1.1.1.3. HB_DSP_VERSION_PATCH

#define HB_DSP_VERSION_PATCH 3U

Version number of DSP patch.

10.3.1.1.1.4. HB_DSP_INITIALIZE_RPC_CTRL_PARAM

#define HB_DSP_INITIALIZE_RPC_CTRL_PARAM(param) \
  {                                             \
    (param)->dspCoreId = HB_DSP_CORE_ANY;       \
    (param)->rpcCmd = HB_DSP_RPC_CMD_BUTT;      \
    (param)->priority = 0;                      \
    (param)->customId = 0;                      \
    (param)->reserved1 = 0;                     \
    (param)->reserved2 = 0;                     \
  }

Initialize control parameters.

10.3.1.1.2. hbDSPRpcCmd

typedef enum {
  // Naming rules: B means begin, E means end
  // [0x1, 0x400) reserved for framework
  HB_DSP_RPC_CMD_R_B = 0x0,
  HB_DSP_RPC_CMD_PING = HB_DSP_RPC_CMD_R_B,
  HB_DSP_RPC_CMD_R_E = 0x3ff,
  // [0x400, 0x800) for neural network operators
  HB_DSP_RPC_CMD_NN_B = 0x400,
  HB_DSP_RPC_CMD_NN_SOFTMAX = HB_DSP_RPC_CMD_NN_B,
  HB_DSP_RPC_CMD_NN_E = 0x7ff,
  // [0x800, 0xfff) for cv operators
  HB_DSP_RPC_CMD_CV_B = 0x800,
  HB_DSP_RPC_CMD_CV_E = 0xfff,
  // [0x1000, 0xffff] for custom purpose
  HB_DSP_RPC_CMD_BUTT = 0xffff
} hbDSPRpcCmd;

DSP RPC command.

  • Member

    Member Name

    Description

    HB_DSP_RPC_CMD_R_B

    Start of DSP reservation command.

    HB_DSP_RPC_CMD_PING

    DSP Ping command.

    HB_DSP_RPC_CMD_R_E

    End of DSP reservation command.

    HB_DSP_RPC_CMD_NN_B

    Start of DSP NN command.

    HB_DSP_RPC_CMD_NN_SOFTMAX

    DSP SoftMax command.

    HB_DSP_RPC_CMD_NN_E

    End of DSP NN command.

    HB_DSP_RPC_CMD_CV_B

    Start of DSP CV command.

    HB_DSP_RPC_CMD_CV_E

    End of DSP CV command.

    HB_DSP_RPC_CMD_BUTT

    DSP Max Command Code for registration command information.

10.3.1.1.2.1. hbDspRpcPriority

typedef enum {
  HB_DSP_RPC_PRIORITY_LOWEST = 0,
  HB_DSP_RPC_PRIORITY_HIGHEST = 255,
  HB_DSP_RPC_PRIORITY_PREEMP = HB_DSP_RPC_PRIORITY_HIGHEST
} hbDspRpcPriority;

DSP RPC task priority.

  • Member

    Member Name

    Description

    HB_DSP_RPC_PRIORITY_LOWEST

    DSP RPC lowest priority.

    HB_DSP_RPC_PRIORITY_HIGHEST

    DSP RPC highest priority.

    HB_DSP_RPC_PRIORITY_PREEMP

    DSP RPC priority of preemptive task.

10.3.1.1.2.2. hbDSPRpcCtrlParam

typedef struct {
  int32_t dspCoreId;
  int32_t rpcCmd;
  int32_t priority;
  int64_t customId;
  int32_t reserved1;
  int32_t reserved2;
} hbDSPRpcCtrlParam;

DSP RPC control parameters.

  • Member

    Member Name

    Description

    dspCoreId

    DSP Core ID; 0 indicates arbitrary core, 1 indicates core 0, 2 indicates core 1.

    rpcCmd

    DSP task command.

    priority

    Task priority. The first priority.

    customId

    Customized Id. The smaller the value, the higher the priority. The second priority.

    reserved1

    Reserved field 1.

    Reserved2

    Reserved field 2.

10.3.1.1.2.3. hbDSPTask_t

typedef void *hbDSPTask_t;

DSP task pointer.

10.3.1.1.3. API

10.3.1.1.3.1. hbDSPGetVersion

const char *hbDSPGetVersion();

Get version message about DSP.

  • Return value

    • Return message about DSP sample.

10.3.1.1.3.2. hbDSPRpc

int32_t hbDSPRpc(hbDSPTask_t *task,
                 hbSysMem *input,
                 hbSysMem *output,
                 hbDSPRpcCtrlParam *rpcCtrlParam);

Submit an rpc task to the DSP.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] input Indicates the input memory of the task.

    • [in/out] output Indicates the output memory of the task.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.1.3.3. hbDSPWaitTaskDone

int32_t hbDSPWaitTaskDone(hbDSPTask_t task,
                          int32_t timeout);

Wait for the task to finish.

  • Parameters

    • [in] task Task handle.

    • [in] timeout Task timeout time in ms.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

Note

  1. timeout > 0 Waiting time.

  2. timeout = 0 Wait until the task is completed.

10.3.1.1.3.4. hbDSPReleaseTask

int hbDSPReleaseTask(hbDSPTask_t task);

Release the task handle. It should to be called after hbDSPWaitTaskDone.

  • Parameters

    • [in] task task handle.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.2. Error Code

typedef enum {
  HB_DSP_SUCCESS = 0,                             // Execution succeeded
  HB_DSP_INVALID_ARGUMENT = -7000001,             // Invalid argument
  HB_DSP_MEM_ALLOC_FAIL = -7000002,               // Memory allocate failed
  HB_DSP_TASK_TIME_OUT = -7000003,                // Task timeout
  HB_DSP_CMD_UNAVAILABLE = -7000004,              // Unavailable CMD command
  HB_DSP_DSP_UNAVAILABLE = -7000005,              // DSP is unavailable
  HB_DSP_TASK_RUN_FAIL = -7000006,                // Task failed
  HB_DSP_OP_NOT_REGISTER = -7000007,              // DSP op not register
  HB_DSP_MMAP_FAIL = -7000008,                    // DSP mmap failed
  HB_DSP_XV_ALLOC_FAIL = -7000009,                // DSP xv allocate memory failed
  HB_DSP_XV_FREE_FAIL = -7000010,                 // DSP xv free memory failed
  HB_DSP_IDMA_COPY_FAIL = -7000011,               // DSP DMA copy fail
  HB_DSP_IDMA_BAD_INIT = -7000012,                // DSP DMA init failed
  HB_DSP_TASK_NUMBER_EXCEED_LIMIT = -7000013,     // The number of tasks exceeds the limit
  HB_DSP_INVALID_SCALE = -7000014,                // DSP invalid scale
} hbDSPStatus;

10.3.1.3. Configuration

Environment variable:

HB_DSP_LOG_LEVEL                // Set arm log level with values 1, 2, 3, 4, 5, 6, corresponding to the Verbose, Debug, Info, Warning, Error, Always levels respectively.
HB_DSP_VDSP_LOG_LEVEL           // Set dsp log level with values 1, 2, 3, 4, 5, corresponding to the Debug, Info, Warning, Error, Always levels respectively.
HB_DSP_ENABLE_DIRECT_MODE       // Enable RPC direct connection mode with values true, false, default value is false, i.e. relay mode.
HB_DSP_ENABLE_CORES             // Enable DSP core, 0 for all (default), 1 for vdsp0, 2 for vdsp1.
HB_DSP_RPMSG_RECV_TIMEOUT       // Set RPMSG timeout, default 0ms, which means timeout is not used.

10.3.1.4. RPC mode

RPC includes two modes: the direct connection mode and the relay mode.

  • The direct connection mode does not support multi-process, and communication overhead is about 200us.

  • The relay mode supports multi-process, and communication overhead is about 300us.

The direct connection mode configuration method is as follows:

  1. The direct connection mode conflicts with the relay mode, make sure there is no dsp_relay_server process in the background (if there is, you can kill dsp_relay_server && rm /var/run/vdsp_relay_*).

  2. export HB_DSP_ENABLE_DIRECT_MODE=true.

The relay mode configuration method is as follows:

  1. Unset HB_DSP_ENABLE_DIRECT_MODE.

  2. Export PATH=$PATH:{path of dsp_relay_server}. The dsp_relay_server process will be resident after startup.

When running the executable program in relay mode, make sure that the dsp_relay_server file exists under the PATH path. If an exception occurs when running the code, it means that the dsp_relay_server file does not have execute permissions, and a sample command to change the permissions is as follows:

chmod -R 777 execute_file

10.3.1.5. CV Side API Description

10.3.1.5.1. Preset Constraints

This section describes the preset parameters and property constraints to assist users in understanding the implicit properties of the interface and reduce the cost of trial and error.

10.3.1.5.1.1. Size constraints for input and output images

The input and output of some operators have corresponding scaling relations and preferentially obey the constraints within the operator.

Constraints Name

Value

Minimum value of image width

32

Minimum value of image height

16

Maximum value of image width

4096

Maximum value of image height

2160

10.3.1.5.1.2. Constraints on NV12 image format

Some of the current operators provide support for the NV12 format, and you can check the parameter constraints in the operators for details. Among them, the NV12 format images need to be guaranteed to follow the following constraints:

  1. The width and height of the image are even;

  2. Images using two pointers to data in the Y-area and UV-area;

  3. The height of the UV area is equal to half the height of the Y area in both pixel units and byte units;

  4. The width of the UV area is equal to half of the Y area in pixel units and to the width of the Y area in byte units.

If an NV12 picture does not follow the above constraints, it is considered to be an illegal NV12 format. Although the operator has internally checked the above constraints for errors, it may still lead to unintended situations during the execution of the operator. Please ensure the validity of the input and output adequately when using NV12 formats.

10.3.1.5.2. Data Types and Data Structures

10.3.1.5.2.1. hbDSPImageFormat

typedef enum {
  HB_DSP_IMAGE_FORMAT_Y = 0,
  HB_DSP_IMAGE_FORMAT_NV12,     /*YYYYYYYY... UVUV...*/
  HB_DSP_IMAGE_FORMAT_RGB_P,    /*RRRR...GGGG...BBBB...*/
  HB_DSP_IMAGE_FORMAT_RGB,      /*RGBRGBRGB...(C3) or RGB_RGB_RGB_...(C4)*/
  HB_DSP_IMAGE_FORMAT_BGR_P,    /*BBBB...GGGG...RRRR*/
  HB_DSP_IMAGE_FORMAT_BGR,      /*BGRBGRBGR...(C3) or BGR_BGR_BGR_...(C4)*/
  HB_DSP_IMAGE_FORMAT_YUV444,   /*YUVYUVYUV... or YUV_YUV_YUV_...*/
  HB_DSP_IMAGE_FORMAT_YUV444_P, /*YYYY...UUUU...VVVV...*/
  HB_DSP_IMAGE_FORMAT_YUV420    /*YYYY...U...V...*/
} hbDSPImageFormat;

Enum of image format.

  • Member

    Member Name

    Description

    HB_DSP_IMAGE_FORMAT_Y

    Grayscale format.

    HB_DSP_IMAGE_FORMAT_NV12

    NV12 format, channel layout is YYYYYYYY… UVUV….

    HB_DSP_IMAGE_FORMAT_RGB_P

    RGB_P format, channel layout is RRRR…GGGG…BBBB….

    HB_DSP_IMAGE_FORMAT_RGB

    RGB format, channel layout is RGBRGBRGB…(C3) or RGB*RGB*RGB*…(C4).

    HB_DSP_IMAGE_FORMAT_BGR_P

    BGR_P format, channel layout is BBB GGG RRR.

    HB_DSP_IMAGE_FORMAT_BGR

    BGR format, channel layout is BGRBGRBGR…(C3) or BGR*BGR*BGR*…(C4).

    HB_DSP_IMAGE_FORMAT_YUV444

    YUV444 format, channel layout is YUVYUVYUV… or YUV*YUV*YUV*….

    HB_DSP_IMAGE_FORMAT_YUV444_P

    YUV444_P format, channel layout is YYYY…UUUU…VVVV….

    HB_DSP_IMAGE_FORMAT_YUV420

    YUV420 format, channel layout is YYYY…U…V….

10.3.1.5.2.2. hbDSPImageType

typedef enum {
  HB_DSP_IMAGE_TYPE_U8C1 = 0,
  HB_DSP_IMAGE_TYPE_U8C3,
  HB_DSP_IMAGE_TYPE_U8C4,
  HB_DSP_IMAGE_TYPE_S16C1,
  HB_DSP_IMAGE_TYPE_S16C2,
  HB_DSP_IMAGE_TYPE_S32C1,
  HB_DSP_IMAGE_TYPE_F32C1
} hbDSPImageType;

Enum of image type. Types of image include depth and channels.

  • Member

    Member Name

    Description

    HB_DSP_IMAGE_TYPE_U8C1

    Single channel of type uint_8.

    HB_DSP_IMAGE_TYPE_U8C3

    Three channels of type uint_8.

    HB_DSP_IMAGE_TYPE_U8C4

    Four channels of type uint_8.

    HB_DSP_IMAGE_TYPE_S16C1

    Single channel of type short_16.

    HB_DSP_IMAGE_TYPE_S16C2

    Double channel of type short_16.

    HB_DSP_IMAGE_TYPE_S32C1

    Single channel of type int_32.

    HB_DSP_IMAGE_TYPE_F32C1

    Single channel of type float_32.

10.3.1.5.2.3. hbDSPImage

typedef struct {
  uint8_t imageFormat;
  uint8_t imageType;
  int32_t width;
  int32_t height;
  int32_t stride;
  void *dataVirAddr;
  uint64_t dataPhyAddr;
  void *uvVirAddr;
  uint64_t uvPhyAddr;
  int32_t uvStride;
} hbDSPImage;

Image structure. Only BPU memory usage is supported.

  • Member

    Member Name

    Description

    imageFormat

    Image format.

    imageType

    Image type.

    width

    Image width.

    height

    Image height.

    stride

    Image stride, measured in byte.

    dataVirAddr

    Logical address of the image on the arm side.

    dataPhyAddr

    Physical address of the image.

    uvVirAddr

    Logical address of uv data, only valid when the image format is NV12.

    uvPhyAddr

    Physical address of uv data, only valid when the image format is NV12.

    uvStride

    Stride of uv data, only valid when the image format is NV12, measured in byte.

10.3.1.5.2.4. hbDSPRoi

typedef struct {
  int32_t left;
  int32_t top;
  int32_t right;
  int32_t bottom;
} hbDSPRoi;

Image ROI area structure, the parameters are the subscripts of the four edges of the ROI area, numbered from 0. The width of the ROI area is equal to right - left + 1 and the height is equal to bottom - top + 1.

  • Member

    Member Name

    Description

    left

    The left subscript of the ROI area.

    top

    The top subscript of the ROI area.

    right

    The right subscript of the ROI area.

    bottom

    The bottom subscript of the ROI area.

10.3.1.5.2.5. hbDSPRpcCmd

typedef enum {
  HB_DSP_RPC_CMD_CV_RESIZE = 0x800,
  HB_DSP_RPC_CMD_CV_CVT_COLOR = 0x801,
  HB_DSP_RPC_CMD_CV_THRESHOLD = 0x802,
  HB_DSP_RPC_CMD_CV_BILATERAL_FILTER = 0x803,
  HB_DSP_RPC_CMD_CV_BOX_FILTER = 0x804,
  HB_DSP_RPC_CMD_CV_GAUSSIAN_BLUR = 0x805,
  HB_DSP_RPC_CMD_CV_MEDIAN_BLUR = 0x806,
  HB_DSP_RPC_CMD_CV_SOBEL = 0x807,
  HB_DSP_RPC_CMD_CV_ERODE = 0x808,
  HB_DSP_RPC_CMD_CV_PYR_UP = 0x809,
  HB_DSP_RPC_CMD_CV_EQUALIZE_HIST = 0x80a,
  HB_DSP_RPC_CMD_CV_FILTER2D = 0x80b,
  HB_DSP_RPC_CMD_CV_SEP_FILTER2D = 0x80c,
  HB_DSP_RPC_CMD_CV_TRANSPOSE = 0x80d,
  HB_DSP_RPC_CMD_CV_PYR_DOWN = 0x80e,
  HB_DSP_RPC_CMD_CV_WARP_AFFINE = 0x80f,
  HB_DSP_RPC_CMD_CV_WARP_PERSPECTIVE = 0x810,
  HB_DSP_RPC_CMD_CV_FLIP = 0x811,
  HB_DSP_RPC_CMD_CV_DILATE = 0x812,
  HB_DSP_RPC_CMD_CV_INTEGRAL = 0x813,
  HB_DSP_RPC_CMD_CV_CANNY = 0x814,
  HB_DSP_RPC_CMD_CV_ROTATE = 0x815,
  HB_DSP_RPC_CMD_CV_CORNER_HARRIS = 0x816,
  HB_DSP_RPC_CMD_CV_ROI_RESIZE = 0x817,
  HB_DSP_RPC_CMD_CV_REMAP = 0x818,
} hbDSPCvCmd;

RPC command used to call CV operators.

  • Member

    Member Name

    Description

    HB_DSP_RPC_CMD_CV_RESIZE

    DSP Resize command.

    HB_DSP_RPC_CMD_CV_CVT_COLOR

    DSP CvtColor command.

    HB_DSP_RPC_CMD_CV_THRESHOLD

    DSP Threshold command.

    HB_DSP_RPC_CMD_CV_BILATERAL_FILTER

    DSP BilateralFilter command.

    HB_DSP_RPC_CMD_CV_BOX_FILTER

    DSP Box Filter command.

    HB_DSP_RPC_CMD_CV_GAUSSIAN_BLUR

    DSP GaussianBlur command.

    HB_DSP_RPC_CMD_CV_MEDIAN_BLUR

    DSP MedianBlur command.

    HB_DSP_RPC_CMD_CV_SOBEL

    DSP Sobel command.

    HB_DSP_RPC_CMD_CV_ERODE

    DSP Erode command.

    HB_DSP_RPC_CMD_CV_PYR_UP

    DSP PyrUp command.

    HB_DSP_RPC_CMD_CV_EQUALIZE_HIST

    DSP Equalize command.

    HB_DSP_RPC_CMD_CV_FILTER2D

    DSP Filter2D command.

    HB_DSP_RPC_CMD_CV_SEP_FILTER2D

    DSP SepFilter2D command.

    HB_DSP_RPC_CMD_CV_TRANSPOSE

    DSP Transpose command.

    HB_DSP_RPC_CMD_CV_PYR_DOWN

    DSP PyrDown command.

    HB_DSP_RPC_CMD_CV_WARP_AFFINE

    DSP WarpAffine command.

    HB_DSP_RPC_CMD_CV_WARP_PERSPECTIVE

    DSP WarpPerspective command.

    HB_DSP_RPC_CMD_CV_FLIP

    DSP Flip command.

    HB_DSP_RPC_CMD_CV_DILATE

    DSP Dilate command.

    HB_DSP_RPC_CMD_CV_INTEGRAL

    DSP Intergral command.

    HB_DSP_RPC_CMD_CV_CANNY

    DSP Canny command.

    HB_DSP_RPC_CMD_CV_ROTATE

    DSP Rotate command.

    HB_DSP_RPC_CMD_CV_CORNER_HARRIS

    DSP CornerHarris command.

    HB_DSP_RPC_CMD_CV_ROI_RESIZE

    DSP RoiResize command.

    HB_DSP_RPC_CMD_CV_REMAP

    DSP Remap command.

10.3.1.5.2.6. hbDSPCVInterpolationFlags

typedef enum {
  HB_CV_INTER_NEAREST = 0,
  HB_CV_INTER_LINEAR = 1,
} hbDSPCVInterpolationFlags;

Enum of interpolation types.

  • Member

    Member Name

    Description

    HB_CV_INTER_NEAREST

    Nearest interpolation.

    HB_CV_INTER_LINEAR

    Bilinear interpolation.

10.3.1.5.2.7. hbDSPCVBorderTypes

typedef enum {
  HB_CV_BORDER_CONSTANT = 0,
  HB_CV_BORDER_REPLICATE = 1,
} hbDSPCVBorderTypes;

Enum of border types.

  • Member

    Member Name

    Description

    HB_CV_BORDER_CONSTANT

    Fixed value border, iiiiii|abcdefgh|iiiiiii.

    HB_CV_BORDER_REPLICATE

    Boundary value border, aaaaaa|abcdefgh|hhhhhhh.

10.3.1.5.2.8. hbDSPCVThresholdType

typedef enum
{
  HB_CV_THRESH_TOZERO = 3
} hbDSPCVThresholdType;

Enum of threshold types.

  • Member

    Member Name

    Description

    HB_CV_THRESH_TOZERO

    Threshold type, if(src > thresh) dst = src else dst = 0.

10.3.1.5.2.9. hbDSPCVCannyNorm

typedef enum {
  HB_CV_NORM_L1 = 1,
} hbDSPCVCannyNorm;

Enum of canny norm.

  • Member

    Member Name

    Description

    HB_CV_NORM_L1

    Canny argument, NORM_L1.

10.3.1.5.2.10. hbDSPRotateDegree

typedef enum {
  HB_CV_ROTATE_90_CLOCKWISE = 0,
  HB_CV_ROTATE_180_CLOCKWISE,
  HB_CV_ROTATE_90_COUNTERCLOCKWISE,
} hbDSPRotateDegree;

Enum of rotate angle.

  • Member

    Member Name

    Description

    HB_CV_ROTATE_90_CLOCKWISE

    Rotate argument, rotate 90° clockwise.

    HB_CV_ROTATE_180_CLOCKWISE

    Rotate argument, rotate 180° clockwise.

    HB_CV_ROTATE_90_COUNTERCLOCKWISE

    Rotate argument, rotate 90° counterclockwise.

10.3.1.5.2.11. hbDSPResizeParam

typedef struct { int8_t interpolation; } hbDSPResizeParam;

Resize argument.

  • Member

    Member Name

    Description

    interpolation

    Interpolation types in Resize, only those in hbDSPCVInterpolationFlags are supported.

10.3.1.5.2.12. hbDSPWarpAffineParam

typedef struct {
  float transformMatrix[6];
  int8_t interpolation;
  int8_t borderType;
  uint8_t borderValue;
} hbDSPWarpAffineParam;

WarpAffine argument.

  • Member

    Member Name

    Description

    transformMatrix[6]

    WarpAffine inverse transformation matrix (dst -> src), 2X3.

    interpolation

    Interpolation types in WarpAffine, only the types in hbDSPCVInterpolationFlags are supported.

    borderType

    Reserved field.

    borderValue

    Reserved field.

10.3.1.5.2.13. hbDSPWarpPerspectiveParam

typedef struct {
  float transformMatrix[9];
  int8_t interpolation;
  int8_t borderType;
  uint8_t borderValue;
} hbDSPWarpPerspectiveParam;

WarpPerspective argument.

  • Member

    Member Name

    Description

    transformMatrix[9]

    WarpPerspective transformation matrix, 3X3.

    interpolation

    Interpolation types in WarpPerspective, supporting the type in hbDSPCVInterpolationFlags.

    borderType

    Reserved field.

    borderValue

    Reserved field.

10.3.1.5.2.14. hbDSPFlipParam

typedef struct {
  uint8_t flipMode;
} hbDSPFlipParam;

Flip argument.

  • Member

    Member Name

    Description

    flipMode

    Flip argument. 0 means flipped around the X-axis, positive integers greater than 0 mean flipped around the Y-axis.

10.3.1.5.2.15. hbDSPThresholdParam

typedef struct {
  float64_t thresh;
  float64_t maxVal;
  int8_t type;
} hbDSPThresholdParam;

Threshold argument.

  • Member

    Member Name

    Description

    thresh

    Threshold, less than or equal to 255.

    maxVal

    Reserved field.

    type

    Threshold type, reserved field.

10.3.1.5.2.16. hbDSPBilateralFilterParam

typedef struct {
  float64_t sigmaColor;
  float64_t sigmaSpace;
  int8_t kernelSize;
  int8_t borderType;
} hbDSPBilateralFilterParam;

BilateralFilter argument.

  • Member

    Member Name

    Description

    sigmaColor

    Sigma argument of color space.

    sigmaSpace

    Sigma argument of coordinate space.

    kernelSize

    Size of kernel, only supporting 5 or 9.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.17. hbDSPBoxFilterParam

typedef struct {
  int8_t kernelHeight;
  int8_t kernelWidth;
  int8_t pointLocX;
  int8_t pointLocY;
  int8_t normalize;
  int8_t borderType;
} hbDSPBoxFilterParam;

BoxFilter argument.

  • Member

    Member Name

    Description

    kernelHeight

    Height of kernel, odd number with value [3,31), kernel height is equal to width.

    kernelWidth

    Width of kernel, odd number with value [3,31), kernel width is equal to height.

    pointLocX

    Reserved field.

    pointLocY

    Reserved field.

    normalize

    Reserved field.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.18. hbDSPGaussianBlurParam

typedef struct {
  float64_t sigmaX;
  float64_t sigmaY;
  int8_t kernelSize;
  int8_t borderType;
} hbDSPGaussianBlurParam;

GaussianBlur argument.

  • Member

    Member Name

    Description

    sigmaX

    Reserved field.

    sigmaY

    Reserved field.

    kernelSize

    Size of kernel, supporting 3X3 and 5X5.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.19. hbDSPMedianBlurParam

typedef struct { int8_t maskWidth; } hbDSPMedianBlurParam;

MedianBlur argument.

  • Member

    Member Name

    Description

    maskWidth

    Aperture linear size, supporting odd values greater than 1 and less than 9.

10.3.1.5.2.20. hbDSPSobelParam

typedef struct {
  float64_t scale;
  float64_t delta;
  int8_t dx;
  int8_t dy;
  int8_t kernelType;
  int8_t borderType;
} hbDSPSobelParam;

Sobel argument.

  • Member

    Member Name

    Description

    scale

    Reserved field.

    delta

    Reserved field.

    dx

    Order of the derivative x.

    dy

    Order of the derivative y.

    kernelType

    Size of the extended Sobel kernel, supporting 3 and 5 .

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.21. hbDSPErodeParam

typedef struct {
  int8_t pointLocX;
  int8_t pointLocY;
  int8_t iterations;
  int8_t borderType;
  uint8_t borderValue;
} hbDSPErodeParam;

Erode argument.

  • Member

    Member Name

    Description

    pointLocX

    Reserved field.

    pointLocY

    Reserved field.

    iterations

    Reserved field.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

    borderValue

    Reserved field.

10.3.1.5.2.22. hbDSPDilateParam

typedef struct {
  int8_t pointLocX;
  int8_t pointLocY;
  int8_t iterations;
  int8_t borderType;
  uint8_t borderValue;
} hbDSPDilateParam;

Dilate argument.

  • Member

    Member Name

    Description

    pointLocX

    Reserved field.

    pointLocY

    Reserved field.

    iterations

    Reserved field.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

    borderValue

    Reserved field.

10.3.1.5.2.23. hbDSPFilter2DParam

typedef struct {
  float64_t delta;
  int8_t pointLocX;
  int8_t pointLocY;
  int8_t borderType;
} hbDSPFilter2DParam;

Filter2D argument.

  • Member

    Member Name

    Description

    delta

    Reserved field.

    pointLocX

    Reserved field.

    pointLocY

    Reserved field.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.24. hbDSPSepFilter2DParam

typedef struct {
  int32_t delta;
  int8_t pointLocX;
  int8_t pointLocY;
  int8_t borderType;
} hbDSPSepFilter2DParam;

SepFilter2D argument.

  • Member

    Member Name

    Description

    delta

    Reserved field.

    pointLocX

    Reserved field.

    pointLocY

    Reserved field.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.25. hbDSPCannyParam

typedef struct {
    uint32_t     lowThresh;
    uint32_t     highThresh;
    int8_t       kernelSize;
    int8_t       norm;
    int8_t       overlap;
    int8_t       borderType;
} hbDSPCannyParam;

Canny argument.

  • Member

    Member Name

    Description

    threshold1

    First threshold for the hysteresis procedure.

    threshold2

    Second threshold for the hysteresis procedure.

    kernelSize

    Kernel size, supporting 3X3, 5X5 and 7x7.

    norm

    Enums of hbDSPCVCannyNorm are supported.

    overlap

    Reserved field.

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

10.3.1.5.2.26. hbDSPRotateParam

typedef struct {
    int8_t rotateCode;
} hbDSPRotateParam;

Rotate argument.

  • Member

    Member Name

    Description

    rotateCode

    Angle of rotate, only supporting enums of hbDSPRotateDegree.

10.3.1.5.2.27. hbDSPCornerHarrisParam

typedef struct {
    int8_t borderType;
    int8_t kernelSize;
    uint32_t blockSize;
    float sensitivity;
} hbDSPCornerHarrisParam;

CornerHarris argument.

  • Member

    Member Name

    Description

    borderType

    Border type, only supporting enums of hbDSPCVBorderTypes.

    kernelSize

    Kernel size, supporting 3X3, 5X5 and 7x7.

    blockSize

    Size of neighborhood, odd value and in the range of [3, 27].

    sensitivity

    Harris detector free parameter, recommended range of values is [0.04, 0.06].

10.3.1.5.2.28. hbDSPRoiResizeParam

typedef struct {
  int8_t interpolation;
  uint8_t paddingValue[4];
} hbDSPRoiResizeParam;

RoiResize argument.

  • Member

    Member Name

    Description

    interpolation

    Interpolation types, supporting the type in hbDSPCVInterpolationFlags.

    paddingValue[4]

    Padding values, each channel corresponds to one padding value, NV12 uses three channels.

10.3.1.5.2.29. hbDSPRemapParam

typedef struct {
  int8_t interpolation;
  int8_t borderType;
  uint8_t borderValue;
} hbDSPRemapParam;

Remap argument.

  • Member

    Member Name

    Description

    interpolation

    Interpolation types in Remap, supporting the type in hbDSPCVInterpolationFlags.

    borderType

    Reserved field.

    borderValue

    Reserved field.

10.3.1.5.3. CV API

10.3.1.5.3.1. hbDSPResize

int32_t hbDSPResize(hbDSPTask_t *task,
                    hbDSPImage *dstImg,
                    hbDSPImage *srcImg,
                    hbDSPResizeParam *resizeParam,
                    hbDSPRpcCtrlParam *ctrlParam);

Resize the API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type and format as srcImg. Its size can be calculated from srcImg size and scale.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y or nv12.

    • [in] resizeParam Operator parameter.

    • [in] ctrlParam Task control parameter, can be used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.2. hbDSPCvtColor

int32_t hbDSPCvtColor(hbDSPTask_t *task,
                      hbDSPImage *dstImg,
                      hbDSPImage *srcImg,
                      hbDSPRpcCtrlParam *ctrlParam);

Call the CvtColor API to support RGB to GRAY, RGB to nv12, BGR to nv12 conversion.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same size as srcImg. The image type supports U8C1, and the image format supports Y and nv12.

    • [in] srcImg Input image. The image type supports U8C3, and the image format supports RGB and BGR.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

  • Convert the support table

    srcFmtdstFmt

    gray

    nv12

    rgb_p

    rgb

    bgr_p

    bgr

    yuv_p

    yuv

    gray

    N

    N

    N

    N

    N

    N

    N

    N

    nv12

    N

    N

    N

    N

    N

    N

    N

    N

    rgb_p

    N

    N

    N

    N

    N

    N

    N

    N

    rgb

    Y

    Y

    N

    N

    N

    N

    N

    N

    bgr_p

    N

    N

    N

    N

    N

    N

    N

    N

    bgr

    N

    Y

    N

    N

    N

    N

    N

    N

    yuv_p

    N

    N

    N

    N

    N

    N

    N

    N

    yuv

    N

    N

    N

    N

    N

    N

    N

    N

10.3.1.5.3.3. hbDSPWarpAffine

int32_t hbDSPWarpAffine(hbDSPTask_t *task,
                        hbDSPImage *dstImg,
                        hbDSPImage *srcImg,
                        hbDSPWarpAffineParam *warpAffineParam,
                        hbDSPRpcCtrlParam *ctrlParam);

Call the WarpAffine API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type and format as srcImg. Its size can be calculated from transformation matrix.

    • [in] srcImg Input image, image type support U8C1, image format support Y and nv12.

    • [in] warpAffineParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.4. hbDSPWarpPerspective

int32_t hbDSPWarpPerspective(hbDSPTask_t *task,
                             hbDSPImage *dstImg,
                             hbDSPImage *srcImg,
                             hbDSPWarpPerspectiveParam *warpPerspectiveParam,
                             hbDSPRpcCtrlParam *ctrlParam);

Call the WarpPerspective API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format and size as srcImg.

    • [in] srcImg Input image, image type support U8C1, image format support Y and nv12.

    • [in] warpPerspectiveParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.5. hbDSPThreshold

int32_t hbDSPThreshold(hbDSPTask_t *task,
                       hbDSPImage *dstImg,
                       hbDSPImage *srcImg,
                       hbDSPThresholdParam *thresholdParam,
                       hbDSPRpcCtrlParam *ctrlParam);

Call the Threshold API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same size, type, and format as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] thresholdParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.6. hbDSPBilateralFilter

int32_t hbDSPBilateralFilter(hbDSPTask_t *task,
                             hbDSPImage *dstImg,
                             hbDSPImage *srcImg,
                             hbDSPBilateralFilterParam *bilateralParam,
                             hbDSPRpcCtrlParam *ctrlParam);

Call the bilateralFilter API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] bilateralParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.7. hbDSPBoxFilter

int32_t hbDSPBoxFilter(hbDSPTask_t *task,
                       hbDSPImage *dstImg,
                       hbDSPImage *srcImg,
                       hbDSPBoxFilterParam *boxParam,
                       hbDSPRpcCtrlParam *ctrlParam);

Call the boxFilter API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] boxParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.8. hbDSPGaussianBlur

int32_t hbDSPGaussianBlur(hbDSPTask_t *task,
                          hbDSPImage *dstImg,
                          hbDSPImage *srcImg,
                          hbDSPGaussianBlurParam *gaussianParam,
                          hbDSPRpcCtrlParam *ctrlParam);

Call the gaussianBlur API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] gaussianParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.9. hbDSPMedianBlur

int32_t hbDSPMedianBlur(hbDSPTask_t *task,
                        hbDSPImage *dstImg,
                        hbDSPImage *srcImg,
                        hbDSPMedianBlurParam *medianParam,
                        hbDSPRpcCtrlParam *ctrlParam);

Call the medianBlur API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] medianParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.10. hbDSPSobel

int32_t hbDSPSobel(hbDSPTask_t *task,
                   hbDSPImage *dstImg,
                   hbDSPImage *srcImg,
                   hbDSPSobelParam *sobelParam,
                   hbDSPRpcCtrlParam *ctrlParam);

Call the Sobel API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same format and size as srcImg. The image type supports S16C1.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] sobelParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.11. hbDSPErode

int32_t hbDSPErode(hbDSPTask_t *task,
                   hbDSPImage *dstImg,
                   hbDSPImage *srcImg,
                   hbDSPImage *krnImg,
                   hbDSPErodeParam *erodeParam,
                   hbDSPRpcCtrlParam *ctrlParam);

Call the Erode API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] krnImg Kernel of API. The image type supports U8C1, and the image format supports Y. Its kernel size needs to be odd and the maximum size is 9 x 9.

    • [in] erodeParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.12. hbDSPDilate

int32_t hbDSPDilate(hbDSPTask_t *task,
                    hbDSPImage *dstImg,
                    hbDSPImage *srcImg,
                    hbDSPImage *krnImg,
                    hbDSPDilateParam *dilateParam,
                    hbDSPRpcCtrlParam *ctrlParam);

Call the Dilate API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] krnImg Kernel of API. The image type supports U8C1, and the image format supports Y. Its kernel size needs to be odd and the maximum size is 9 x 9.

    • [in] dilateParam Operator parameter.

    • [in] ctrlParam Task control parameter, can be used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.13. hbDSPPyrUp

int32_t hbDSPPyrUp(hbDSPTask_t *task,
                   hbDSPImage *dstImg,
                   hbDSPImage *srcImg,
                   hbDSPRpcCtrlParam *ctrlParam);

Call the PyrUp API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type and format as srcImg. Its length and width are twice as long as the input image.

    • [in] srcImg Input image, image type support U8C1, image format support Y and nv12.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.14. hbDSPEqualizeHist

int32_t hbDSPEqualizeHist(hbDSPTask_t *task,
                          hbDSPImage *dstImg,
                          hbDSPImage *srcImg,
                          hbDSPRpcCtrlParam *ctrlParam);

Call the EqualizeHist API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.15. hbDSPFilter2D

int32_t hbDSPFilter2D(hbDSPTask_t *task,
                      hbDSPImage *dstImg,
                      hbDSPImage *srcImg,
                      hbDSPImage *krnImg,
                      hbDSPFilter2DParam *filter2DParam,
                      hbDSPRpcCtrlParam *ctrlParam);

Call the Filter2D API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] krnImg Kernel of API. The image type supports F32C1, and the image format supports Y. Its size is a positive odd value less than or equal to 9.

    • [in] filter2DParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.16. hbDSPSepFilter2D

int32_t hbDSPSepFilter2D(hbDSPTask_t *task,
                         hbDSPImage *dstImg,
                         hbDSPImage *srcImg,
                         hbDSPImage *krnXImg,
                         hbDSPImage *krnYImg,
                         hbDSPSepFilter2DParam *sepFilter2DParam,
                         hbDSPRpcCtrlParam *ctrlParam);

Call the SepFilter2D API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] krnXImg Filter coefficient per line of the operator. The image type supports F32C1, and the format supports Y. The size shall be 1xN and N is a positive odd value less than or equal to 9.

    • [in] krnYImg Filter coefficient per column of the operator. The image type supports F32C1. The size shall be Nx1, and N is a positive odd value less than or equal to 9.

    • [in] sepFilter2DParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.17. hbDSPTranspose

int32_t hbDSPTranspose(hbDSPTask_t *task,
                       hbDSPImage *dstImg,
                       hbDSPImage *srcImg,
                       hbDSPRpcCtrlParam *ctrlParam);

Call the Transpose API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type and format as srcImg. Its width is the same as the height of srcImg, and its height is the same as the width of srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y and nv12.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.18. hbDSPPyrDown

int32_t hbDSPPyrDown(hbDSPTask_t *task,
                     hbDSPImage *dstImg,
                     hbDSPImage *srcImg,
                     hbDSPRpcCtrlParam *ctrlParam);

Call the PyrDown API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type and format as srcImg. Its width is half of the width of srcImg (rounded up), and its height is half of the height of srcImg (rounded up).

    • [in] srcImg Input image, image type support U8C1, image format support Y and nv12.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.19. hbDSPFlip

int32_t hbDSPFlip(hbDSPTask_t *task,
                  hbDSPImage *dstImg,
                  hbDSPImage *srcImg,
                  hbDSPFlipParam *flipParam,
                  hbDSPRpcCtrlParam *ctrlParam);

Call the Flip API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type, format, and size as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y and nv12.

    • [in] flipParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.20. hbDSPIntegral

int32_t hbDSPIntegral(hbDSPTask_t *task,
                      hbDSPImage *dstImg,
                      hbDSPImage *srcImg,
                      hbDSPRpcCtrlParam *ctrlParam);

Call the Integral API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image. The image type supports S32C1, and the image format supports Y. Its width is the width of the input image +1, and its height is the height of the input image +1.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.21. hbDSPCanny

int32_t hbDSPCanny(hbDSPTask_t *task,
                   hbDSPImage *dstImg,
                   hbDSPImage *srcImg,
                   hbDSPCannyParam *cannyParam,
                   hbDSPRpcCtrlParam *ctrlParam);

Call the Canny API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same size, type, and format as srcImg.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] cannyParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.22. hbDSPRotate

int32_t hbDSPRotate(hbDSPTask_t *task,
                    hbDSPImage *dstImg,
                    hbDSPImage *srcImg,
                    hbDSPRotateParam *rotateParam,
                    hbDSPRpcCtrlParam *ctrlParam);

Call the Rotate API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same type and format as srcImg. Its size is determined by the rotation angle and the size of the input image.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y and nv12.

    • [in] rotateParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.23. hbDSPCornerHarris

int32_t hbDSPCornerHarris(hbDSPTask_t *task,
                          hbDSPImage *dstImg,
                          hbDSPImage *srcImg,
                          hbDSPCornerHarrisParam *cornerHarrisParam,
                          hbDSPRpcCtrlParam *ctrlParam);

Call the CornerHarris API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same format, size as srcImg and type supports S32C1.

    • [in] srcImg Input image. The image type supports U8C1, and the image format supports Y.

    • [in] cornerHarrisParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.24. hbDSPRoiResize

int32_t hbDSPRoiResize(hbDSPTask_t *task,
                       hbDSPImage *dstImg,
                       hbDSPImage *srcImg,
                       hbDSPRoi *roi,
                       hbDSPRoiResizeParam *roiResizeParam,
                       hbDSPRpcCtrlParam *ctrlParam);

Call the RoiResize API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same format and type as srcImg.

    • [in] srcImg Input image, the image type supports U8C1, and the image format supports Y and nv12.

    • [in] roi ROI area, the valid range takes the intersection of the ROI region and srcImg.

    • [in] roiResizeParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority, and the valid range needs to conform to the built-in constraints of the image.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.5.3.25. hbDSPRemap

int32_t hbDSPRemap(hbDSPTask_t *task,
                   hbDSPImage *dstImg,
                   hbDSPImage *srcImg,
                   hbDSPImage *map1,
                   hbDSPImage *map2,
                   hbDSPRemapParam *remapParam,
                   hbDSPRpcCtrlParam *ctrlParam);

Call the Remap API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstImg Output image, which has the same format, type as srcImg.

    • [in] srcImg Input image, the image type supports U8C1, and the image format supports Y and nv12.

    • [in] map1 map1 coordinates (x, y), type supports S16C2, the size is consistent with the output image, and the coordinates layout xyxy.

    • [in] map2 map2 coordinates (x, y), type supports S16C2, the size is consistent with the output image, and the coordinates layout xyxy. When the interpolation type is HB_CV_INTER_NEAREST, only map1 is used. When the interpolation type is HB_CV_INTER_LINEAR, the coordinate weight is stored, which can be customized or converted by the opencv interface convertMaps.

    • [in] remapParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.6. NN Side API Description

10.3.1.6.1. Data Types and Data Structure

10.3.1.6.1.1. hbDSPDataLayout

typedef enum {
  HB_DSP_LAYOUT_NONE = 0,
  HB_DSP_LAYOUT_NCHW = 1,
  HB_DSP_LAYOUT_NHWC = 2,
} hbDSPDataLayout;

Enum of DSP NN data layout types.

  • Member

    Member Name

    Description

    HB_DSP_LAYOUT_NONE

    DSP NN none laytou, use default layout type.

    HB_DSP_LAYOUT_NCHW

    DSP NN data layout is NCHW.

    HB_DSP_LAYOUT_NHWC

    DSP NN layout is NHWC.

10.3.1.6.1.2. hbDSPDataType

typedef enum {
  HB_DSP_TENSOR_TYPE_S8 = 0x01,
  HB_DSP_TENSOR_TYPE_S16 = 0x02,
  HB_DSP_TENSOR_TYPE_S32 = 0x03,
  HB_DSP_TENSOR_TYPE_F32 = 0x04,
} hbDSPDataType;

Enum of DSP NN data types.

  • Member

    Member Name

    Description

    HB_DSP_TENSOR_TYPE_S8

    DSP NN 8 bit int type.

    HB_DSP_TENSOR_TYPE_S16

    DSP NN 16 bit int type.

    HB_DSP_TENSOR_TYPE_S32

    DSP NN 32 bit int type.

    HB_DSP_TENSOR_TYPE_F32

    DSP NN float type.

10.3.1.6.1.3. hbDSPTensor

#define HB_DSP_TENSOR_MAX_DIMENSIONS 8

typedef struct {
  int32_t dimensionSize[HB_DSP_TENSOR_MAX_DIMENSIONS];
  int32_t numDimensions;
} hbDSPTensorShape;

typedef struct {
  hbDSPTensorShape dataShape;
  // Tensor data layout  NCHW | NHWC :  1 | 2
  uint8_t dataLayout;
  // Tensor data type  int8_t | int16_t | int32_t | float_t : 1 | 2 | 3 | 4
  uint8_t dataType;
  // Physical address for tensor data
  uint64_t dataPhyAddr;
} hbDSPTensor;

  • Member

    Member Name

    Description

    hbDSPTensorShape

    Tensor shape.

    dataLayout

    Data layout.

    dataType

    Data type.

    dataPhyAddr

    Tensor physical address.

10.3.1.6.1.4. hbDSPRpcCmd

typedef enum {
  // [0x400, 0x800) for neural network operators
  HB_DSP_RPC_CMD_NN_QUANTIZE = 0x401,
  HB_DSP_RPC_CMD_NN_DEQUANTIZE = 0x402,
} hbDSPNNCmd;

DSP calls the RPC command of the NN API.

  • Member

    Member Name

    Description

    HB_DSP_RPC_CMD_NN_QUANTIZE

    DSP NN Quantize command.

    HB_DSP_RPC_CMD_NN_DEQUANTIZE

    DSP NN Dequantize command.

10.3.1.6.1.5. hbDSPQuantizeParam

typedef struct {
  int32_t scaleChannel;
  int32_t zeroPointChannel;
  uint64_t scalePhyAddr;
  uint64_t zeroPointPhyAddr;
  int32_t min;
  int32_t max;
} hbDSPQuantizeParam;

DSP NN quantizes the operator parameter.

  • Member

    Member Name

    Description

    scaleChannel

    Number of scale channels.

    zeroPointChannel

    Number of zeroPoint channels.

    scalePhyAddr

    Physical address of scale data, data type must be float32_t.

    zeroPointPhyAddr

    Physical address of zeroPoint data, data type must be float32_t.

    min

    Minimum quantified value.

    max

    Maximum quantified value.

10.3.1.6.1.6. hbDSPDequantizeParam

typedef struct {
  int32_t scaleChannel;
  int32_t zeroPointChannel;
  uint64_t scalePhyAddr;
  uint64_t zeroPointPhyAddr;
} hbDSPDequantizeParam;

DSP NN dequantizes the operator parameter.

  • Member

    Member Name

    Description

    scaleChannel

    Number of scale channels.

    zeroPointChannel

    Number of zeroPoint channels.

    scalePhyAddr

    Physical address of scale data, data type must be float32_t.

    zeroPointPhyAddr

    Physical address of zeroPoint data, data type must be the same as input data type.

Note

  1. The value of scaleChannel should be 1 or the same as the value of the input channel;

  2. The value of zeroPointChannel should be 0 or 1 or same as the scaleChannel, and zeroPointChannel <= scaleChannel.

10.3.1.6.2. NN API

10.3.1.6.2.1. hbDSPQuantize

int32_t hbDSPQuantize(hbDSPTask_t *task,
                      hbDSPTensor *dstTs,
                      hbDSPTensor *srcTs,
                      hbDSPQuantizeParam *quantizeParam,
                      hbDSPRpcCtrlParam *ctrlParam);

Quantize API.

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstTs The output tensor of the task.

    • [in] srcTs The input tensor of the task.

    • [in] quantizeParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.6.2.2. hbDSPDequantize

int32_t hbDSPDequantize(hbDSPTask_t *task,
                        hbDSPTensor *dstTs,
                        hbDSPTensor *srcTs,
                        hbDSPDequantizeParam *dequantizeParam,
                        hbDSPRpcCtrlParam *ctrlParam);

  • Parameters

    • [out] task Task handle, which is used by the hbDSPWaitTaskDone interface to wait for the specified task to finish.

    • [in/out] dstTs The output tensor of the task.

    • [in] srcTs The input tensor of the task.

    • [in] dequantizeParam Operator parameter.

    • [in] ctrlParam Task control parameter, used to specify the task-running core and the task priority.

  • Return value

    • If 0 is returned, the API is executed successfully, otherwise it fails.

10.3.1.6.3. NN PLUGIN

NN PLUGIN is called as a plugin for libdnn, when using libdnn for model inference, the quantize or dequantize operator dispatched to the ARM can use the following environment variables to control whether the ARM offloads the task to the DSP.

export HB_DNN_PLUGIN_PATH=${ARM_PLUGIN} // The path to libhb_dsp_nn_plugin.so
export HB_DNN_ENABLE_DSP=1
// Current plugin policy selects whether to offloads the task to the DSP.
// The input data type of quantize is float, the output data type is int8_t, and the input data size is greater than or equal to 1x2^18
// The input data type of dequantize is int8_t/int32_t, the output data type is float, and the data size is greater than or equal to 1x2^20
export HB_DNN_DISABLE_ACC_AUTO_DEPLOY=1 // This switch controls whether all dimensions are offloaded to the DSP.