中文 EN

5.8. Video Processing - VSE

5.8.1. Module Description

VSE (Video Scale Engine) is used for image scaling.
It supports 6 scaling channels with the following characteristics:

  • Maximum processing capability: 3840x2160@60fps

  • Supports input cropping, independent horizontal and vertical scaling, and separate processing of chroma and luma components

  • 5 downscale channels

    • Maximum supported resolutions: 4K, 1080P, 1080P, 720P, 720P respectively

    • Minimum resolution: 64x64, supports arbitrary downscale factors

    • Supports frame rate control and OSD

  • 1 upscale channel

    • Maximum supported resolution: 4K, maximum scaling factor: 4X (scaling factor range: 0–4)

    • Supports frame rate control and OSD

Output Channel ID Type Channel Function
0 offline 4K Downscale
1 offline 1080P Downscale
2 offline 1080P Downscale
3 offline 720P Downscale
4 offline 720P Downscale
5 offline 4K Upscale

Input and output data formats:

  • Supported input data formats:

    • from DDR: YUV422sp(NV16), YUV422 interleave, YUV420sp(NV12)

    • from ISP: YUV420sp(NV12), YUV422sp(NV16)

  • Supported output data formats: YUV444, YUV422, YUV420, RGB888

Note:
Currently the software only supports the following combinations:
Input Format Output Format Notes
NV12 NV12 -
NV16 NV16 -
NV16 NV12 When input is NV16, output channel 5 can only be configured as NV16

OSD features:

  • Each OSD supports up to 4 overlay regions (4 ROIs non-overlapping in vertical direction) and 8 histogram statistic regions

  • For more information about OSD, please refer to Region Processing - OSD

5.8.2. Reference Example

  • VSE code examples can be found in the sample_vse section

5.8.3. API Reference

5.8.4. Interface Descriptions

5.8.4.1. hbn_vnode_open

【Function Declaration】

hobot_status hbn_vnode_open(hb_vnode_type vnode_type, uint32_t hw_id, int32_t ctx_id, hbn_vnode_handle_t *vnode_fd);

【Function Description】

Initialize and open the vnode device node, returning the vnode handle for this module.

【Parameter Description】

  • [IN] hb_vnode_type vnode_type : vnode type; for VSE, use HB_VSE

  • [IN] uint32_t hw_id : Hardware ID of the module; for VSE, hw_id is 0

  • [IN] int32_t ctx_id : Context ID of the module, a software concept; can be specified or set to AUTO_ALLOC_ID for automatic allocation by the HBN framework

  • [OUT] hbn_vnode_handle_t *vnode_fd : Returns the vnode handle for the module

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.2. hbn_vnode_close

【Function Declaration】

void hbn_vnode_close(hbn_vnode_handle_t vnode_fd);

【Function Description】

Close the module’s device node.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

【Return Value】

None

【Notes】

Must be used in pair with hbn_vnode_open.

【Compatibility】

Hardware: X5

5.8.4.3. hbn_vnode_set_attr

【Function Declaration】

hobot_status hbn_vnode_set_attr(hbn_vnode_handle_t vnode_fd, void *attr);

【Function Description】

Set basic attributes of the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] void *attr: Pointer to the attribute structure; for VSE, use vse_attr_t;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.4. hbn_vnode_set_ochn_attr

【Function Declaration】

hobot_status hbn_vnode_set_ochn_attr(hbn_vnode_handle_t vnode_fd, uint32_t ochn_id, void *attr);

【Function Description】

Set output channel attributes of the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ochn_id: Output channel ID; refer to VSE Channel Description;

  • [IN] void *attr: Pointer to the output channel attribute structure; for VSE, use vse_ochn_attr_t;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.5. hbn_vnode_get_ochn_attr

【Function Declaration】

hobot_status hbn_vnode_get_ochn_attr(hbn_vnode_handle_t vnode_fd, uint32_t ochn_id, void *attr);

【Function Description】

Get output channel attributes of the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ochn_id: Output channel ID; refer to VSE Channel Description;

  • [OUT] void *attr: Pointer to the output channel attribute structure; for VSE, use vse_ochn_attr_t;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.6. hbn_vnode_set_ichn_attr

【Function Declaration】

hobot_status hbn_vnode_set_ichn_attr(hbn_vnode_handle_t vnode_fd, uint32_t ichn_id, void *attr);

【Function Description】

Set input channel attributes of the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ichn_id: Input channel ID; refer to VSE Channel Description;

  • [IN] void *attr: Pointer to the input channel attribute structure; for VSE, use vse_ichn_attr_t;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.7. hbn_vnode_get_ichn_attr

【Function Declaration】

hobot_status hbn_vnode_get_ichn_attr(hbn_vnode_handle_t vnode_fd, uint32_t ichn_id, void *attr);

【Function Description】

Get input channel attributes of the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ichn_id: Input channel ID; for VSE, input channel is 0;

  • [OUT] void *attr: Pointer to the input channel attribute structure; for VSE, use vse_ichn_attr_t;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.8. hbn_vnode_set_ochn_buf_attr

【Function Declaration】

hobot_status hbn_vnode_set_ochn_buf_attr(hbn_vnode_handle_t vnode_fd, uint32_t ochn_id,
					 hbn_buf_alloc_attr_t *alloc_attr);

【Function Description】

Set output channel buffer attributes, including buffer count and whether buffer addresses are contiguous.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ochn_id: Output channel ID; refer to VSE Channel Description;

  • [IN] hbn_buf_alloc_attr_t *alloc_attr: Buffer allocation attributes;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.9. hbn_vnode_start

【Function Declaration】

hobot_status hbn_vnode_start(hbn_vnode_handle_t vnode_fd);

【Function Description】

Start the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

【Return Value】

【Notes】

Module must be opened before starting.

【Compatibility】

Hardware: X5

5.8.4.10. hbn_vnode_stop

【Function Declaration】

hobot_status hbn_vnode_stop(hbn_vnode_handle_t vnode_fd);

【Function Description】

Stop the module.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.11. hbn_vnode_getframe

【Function Declaration】

hobot_status hbn_vnode_getframe(hbn_vnode_handle_t vnode_fd, uint32_t ochn_id, uint32_t millisecondTimeout,
				hbn_vnode_image_t *out_img);

【Function Description】

Retrieve image from the module’s output channel; this is a blocking interface.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ochn_id: Output channel ID; refer to VSE Channel Description;

  • [IN] uint32_t millisecondTimeout: Timeout duration in milliseconds;

  • [OUT] hbn_vnode_image_t *out_img: Address of the output image buffer structure;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.12. hbn_vnode_sendframe

【Function Declaration】

hobot_status hbn_vnode_sendframe(hbn_vnode_handle_t vnode_fd, uint32_t ichn_id,
				 hbn_vnode_image_t *img);

【Function Description】

Send image to the module’s input channel, triggering processing. This is a blocking interface; waits for hardware processing to complete before returning. Default timeout is 1 second.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ichn_id: Input channel ID; for VSE, input channel is 0;

  • [IN] hbn_vnode_image_t *img: Address of the input image buffer;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.4.13. hbn_vnode_releaseframe

【Function Declaration】

hobot_status hbn_vnode_releaseframe(hbn_vnode_handle_t vnode_fd, uint32_t ochn_id, hbn_vnode_image_t *img);

【Function Description】

Release image buffer; the buffer is returned to the specified output channel.

【Parameter Description】

  • [IN] hbn_vnode_handle_t vnode_fd: The vnode handle of the module;

  • [IN] uint32_t ochn_id: Output channel ID; refer to VSE Channel Description;

  • [IN] hbn_vnode_image_t *img: Address of the image buffer structure;

【Return Value】

【Notes】

None

【Compatibility】

Hardware: X5

5.8.5. Data Structures

5.8.5.1. vse_cfg_t

Name Type Min Value Max Value Default Description
vse_attr vse_attr_t · · · VSE basic attributes
ichn_attr vse_ichn_attr_t · · · VSE input channel attributes
ochn_attr vse_ochn_attr_t · · · VSE output channel attributes

5.8.5.2. vse_attr_t

Name Type Min Value Max Value Default Description
fps frame_fps_ctrl_t · · 0 Currently only supports setting fps of input image src; used for VSE frame rate control

5.8.5.3. vse_ichn_attr_t

Name Type Min Value Max Value Default Description
tpg_en cam_bool_e 0 1 0 Enable TPG function (currently not supported)
width uint32_t 0 5432 0 Original width of input image
height uint32_t 0 3076 0 Original height of input image
fmt frame_format_e FRM_FMT_NV12 FRM_FMT_NV12 0 FRM_FMT_NULL - Invalid, FRM_FMT_RAW - RAW format, FRM_FMT_NV12 - NV12 format, FRM_FMT_UYVY - UYVY format; currently only NV12 is supported
bit_width uint32_t 8 8 0 Bit width of input image

5.8.5.4. vse_ochn_attr_t

Name Type Min Value Max Value Default Description
chn_en cam_bool_e 0 1 0 Enable channel
roi common_rect_t · · · Crop region for output image; all zeros means crop disabled
target_w uint32_t 0 Limited by channel capability 0 Output image width; 0 means scaling disabled
target_h uint32_t 0 Limited by channel capability 0 Output image height; 0 means scaling disabled
y_stride uint32_t 0 · 0 Y-plane stride of output image (currently not supported)
uv_stride uint32_t 0 · 0 UV-plane stride of output image (currently not supported)
fmt frame_format_e FRM_FMT_NV12 FRM_FMT_NV12 0 FRM_FMT_NULL - Invalid, FRM_FMT_RAW - RAW format, FRM_FMT_NV12 - NV12 format, FRM_FMT_UYVY - UYVY format; currently only NV12 is supported
bit_width uint32_t 8 8 0 Bit width of input image
fps frame_fps_ctrl_t · · 0 Currently only supports setting fps of output image dst; used for VSE frame rate control. When this value is 0, if input fps is 0, frame rate control is disabled; if input fps is non-zero, no data will be output

5.8.6. Return Value Description

Error Code Macro Definition Description Common Causes and Solutions
0 HBN_STATUS_SUCCESS Success
1 HBN_STATUS_INVALID_NODE Invalid vnode, corresponding vnode not found
2 HBN_STATUS_INVALID_NODETYPE Invalid vnode type, corresponding vnode not found For VSE, vnode type should be HB_VSE
3 HBN_STATUS_INVALID_HWID Invalid hardware module ID For VSE, hw_id should be 0
4 HBN_STATUS_INVALID_CTXID Invalid context ID Can be set to AUTO_ALLOC_ID for automatic allocation by HBN framework
5 HBN_STATUS_INVALID_OCHNID Invalid output channel ID VSE output channel range is [0,5]
6 HBN_STATUS_INVALID_ICHNID Invalid input channel ID VSE supports only one input channel
7 HBN_STATUS_INVALID_FORMAT Invalid format
8 HBN_STATUS_INVALID_NULL_PTR Null pointer
9 HBN_STATUS_INVALID_PARAMETER Invalid parameter, version check failed
10 HBN_STATUS_ILLEGAL_ATTR Invalid parameter
11 HBN_STATUS_INVALID_FLOW Invalid flow, corresponding flow not found
12 HBN_STATUS_FLOW_EXIST Flow already exists
13 HBN_STATUS_FLOW_UNEXIST Flow does not exist
14 HBN_STATUS_NODE_EXIST Node already exists
15 HBN_STATUS_NODE_UNEXIST Node does not exist
16 HBN_STATUS_NOT_CONFIG Reserved
17 HBN_STATUS_CHN_NOT_ENABLED Channel not enabled
18 HBN_STATUS_CHN_ALREADY_ENABLED Channel already enabled
19 HBN_STATUS_ALREADY_BINDED Node already bound
20 HBN_STATUS_NOT_BINDED Node not bound
21 HBN_STATUS_TIMEOUT Timeout
22 HBN_STATUS_NOT_INITIALIZED Not initialized
23 HBN_STATUS_NOT_SUPPORT Channel not supported or not activated
24 HBN_STATUS_NOT_PERM Operation not allowed
25 HBN_STATUS_NOMEM Insufficient memory
26 HBN_STATUS_INVALID_VNODE_FD Invalid node file descriptor
27 HBN_STATUS_INVALID_ICHNID_FD Invalid input channel file descriptor
28 HBN_STATUS_INVALID_OCHNID_FD Invalid output channel file descriptor
29 HBN_STATUS_OPEN_OCHN_FAIL Failed to open output channel
30 HBN_STATUS_OPEN_ICHN_FAIL Failed to open input channel
31 HBN_STATUS_JSON_PARSE_FAIL JSON parsing failed
32 HBN_STATUS_REQ_BUF_FAIL Failed to request buffer
33 HBN_STATUS_QUERY_BUF_FAIL Failed to query buffer information
34 HBN_STATUS_SET_CONTROL_FAIL Failed to set module control or adjustment parameters (e.g., ISP effect parameters)
35 HBN_STATUS_GET_CONTROL_FAIL Failed to get module control or adjustment parameters (e.g., ISP effect parameters)
36 HBN_STATUS_NODE_START_FAIL Failed to start node
37 HBN_STATUS_NODE_STOP_FAIL Failed to stop node
38 HBN_STATUS_NODE_POLL_ERROR Node channel poll error
39 HBN_STATUS_NODE_POLL_TIMEOUT Node channel poll timeout
40 HBN_STATUS_NODE_POLL_FRAME_DROP Frame drop occurred during node channel poll
41 HBN_STATUS_NODE_POLL_HUP File descriptor hung up during node channel poll
42 HBN_STATUS_NODE_ILLEGAL_EVENT Illegal event during node channel poll
43 HBN_STATUS_NODE_DEQUE_ERROR Node channel dequeue buffer error
44 HBN_STATUS_ILLEGAL_BUF_INDEX Invalid buffer index
45 HBN_STATUS_NODE_QUE_ERROR Node channel queue buffer error
46 HBN_STATUS_FLUSH_FRAME_ERROR Node channel frame flush error
47 HBN_STATUS_INIT_BIND_ERROR Error occurred during JSON parsing and binding
48 HBN_STATUS_ADD_NODE_FAIL Failed to add node to flow
49 HBN_STATUS_WRONG_CONFIG_ID System does not support the node ID
50 HBN_STATUS_BIND_NODE_FAIL Error occurred when binding node to flow
51 HBN_STATUS_INVALID_VERSION Version mismatch between lower driver and upper library
52 HBN_STATUS_GET_VERSION_ERROR Failed to get lower driver module version
53 HBN_STATUS_MEM_INIT_FAIL hbmem memory initialization failed
54 HBN_STATUS_MEM_IMPORT_FAIL hbmem memory import failed
55 HBN_STATUS_MEM_FREE_FAIL hbmem memory release failed
56 HBN_STATUS_SYSFS_OPEN_FAIL System file open failed
57 HBN_STATUS_STRUCT_SIZE_NOT_MATCH HAL layer structure size does not match kernel layer
58 HBN_STATUS_RGN_UNEXIST Cannot retrieve corresponding rgn data
59 HBN_STATUS_RGN_INVALID_OPERATION Invalid rgn operation
60 HBN_STATUS_RGN_OPEN_FILE_FAIL RGN module failed to open file
128 HBN_STATUS_ERR_UNKNOW Unknown error