中文 EN

4.4.6. ION System Debugging Guide

4.4.6.1. Overview

The ION system is a memory management subsystem in the Linux kernel designed for managing memory sharing between devices.
In the X5 BSP system, ION memory is primarily used in the following four scenarios:

  • BPU subsystem

  • HIFI subsystem (DSP)

  • Developers allocating ION memory via the hbmem interface

  • Multimedia system: including hardware acceleration units such as VIN, ISP, VSE, GDC, VPU, OSD, GPU, Display

ION Memory Regions

  • The ION memory management subsystem uses independent memory regions: reserved areas defined in the device tree, specifically allocated for the ION system. For details, see ION Region Adjustment Method.

  • General memory allocators in Linux cannot allocate memory from ION memory regions, such as kernel-space kmalloc, vmalloc, or user-space malloc.

  • The ION memory management subsystem assigns different names to each region: carveout, chunk, cma_reserved, ion_cma, system, system_contig.

    • In the X5 BSP, three main ION regions are used: cma_reserved, carveout, and ion_cma.

    • cma_reserved mainly provides memory for the multimedia system.

    • carveout mainly provides memory for the BPU subsystem.

    • ion_cma serves as a fallback region; when the above two regions are exhausted, the ION management subsystem automatically allocates memory from the ion_cma region.

Note: Explanation of ION region names:

  • An ION region is a dynamically allocated memory area. In software, such dynamic memory areas are commonly managed using a heap data structure. Therefore, each ION region in this document and in debug logs is also referred to as a heap region.

  • ION is an open-source memory management system implemented by Google in Android. For more information on ION region naming, refer to The Android ION memory allocator.

Characteristics of ION Memory

Feature Description
Memory Management ION provides a generic memory management mechanism that supports memory allocation and release from ION regions in both user and kernel space.
Contiguous Physical Memory Allocation Memory allocated by the ION memory management system is physically contiguous, enabling direct use by hardware accelerators (e.g., GPU, ISP, video decoders).
Inter-device Memory Sharing ION allows memory sharing between different devices (e.g., GPU, ISP, video decoders) through kernel-level communication and data exchange.
Inter-process Memory Sharing Supports sharing of the same ION memory region among multiple processes without memory copying.
Zero-copy Multiple processes can directly access shared memory regions without explicit memory copying, improving memory efficiency and overall system performance.

ION Region Allocation Rules

When allocating memory, the ION management subsystem must specify a specific ION region, which may result in:
Available free memory in the entire ION system, but allocation fails because the specified region is fully occupied.
Solution: The ION management subsystem supports automatic fallback allocation from other regions when the specified region is unavailable. The allocation logic is as follows:

  • When cma_reserved is specified and exhausted: prefer carveout, then ion_cma.

  • When carveout is specified and exhausted: prefer cma_reserved, then ion_cma.

  • When ion_cma is specified and exhausted: choose cma_reserved.

Note: The purpose of the ion_cma region is to serve as a fallback option for other regions.

ION Region Adjustment Method

  • The software frameworks of the multimedia system and BPU internally specify preferred ION regions, as shown in the table below:

Module Preferred ION Region
Multimedia System cma_reserved region
BPU System carveout region

Note:

  • The OSD module is special: although it belongs to the multimedia system’s hardware accelerators, it defaults to allocating memory from the carveout region.

  • Developers can specify the preferred ION region via the Flag parameter when allocating ION memory using the hbmem interface.

4.4.6.2. ION Usage Methods

ION Region Size Adjustment Method

ION regions are physical address spaces reserved in the device tree, located within DDR. When the DDR capacity varies, the reserved size of ION regions should also vary.
Therefore, the kernel device tree provides three default configurations based on DDR capacity:

  1. Default ION region configuration for 1GB DDR

  2. Default ION region configuration for 2GB DDR

  3. Default ION region configuration for DDR ≥ 4GB

One of these configurations is selected and applied during boot. The process is as follows:

  1. During miniboot startup, the current DDR capacity is detected and stored in registers.

  2. During uboot startup, based on the value stored by miniboot, the current DDR capacity is known. Then, the kernel device tree file is parsed to obtain the default ION memory configuration corresponding to the DDR capacity.

The above describes the process for applying default ION region configurations. The X5 BSP also supports modifying ION region configurations via the uboot command line.

Note: The ION region configuration saved via the uboot command line has higher priority than that configured in the Linux device tree.

Method to Adjust ION Regions in Linux Device Tree

The ION region configuration in the BSP source package file arch/arm64/boot/dts/hobot/x5-memory.dtsi is as follows:

/ {
	[...]
	reserved-memory {
		[...]

		/* UBoot will enable corresponding node when ddr size is confirmed */
		ion_reserved_1g: ion_reserved_1g@A4100000 {
			compatible = "ion-pool";
			reg = <0x0 0xA4100000 0x0 0x07000000>; /* 112MiB */
			status = "disabled";
		};

		ion_carveout_1g: ion_carveout_1g@AB100000 {
			compatible = "ion-carveout";
			[...]
		};

		ion_cma_1g: ion_cma_1g@B2100000 {
			compatible = "ion-cma";
			[...]
		};

		ion_reserved_2g: ion_reserved_2g@A4100000 {
			compatible = "ion-pool";
			[...]
		};

		ion_carveout_2g: ion_carveout_2g@C4100000 {
			compatible = "ion-carveout";
			[...]
		};

		ion_cma_2g: ion_cma_2g@E4100000 {
			compatible = "ion-cma";
			[...]
		};

		ion_reserved_ge4g: ion_reserved_ge4g@A4100000 {
			compatible = "ion-pool";
			[...]
		};

		ion_carveout_ge4g: ion_carveout_ge4g@E4100000 {
			compatible = "ion-carveout";
			[...]
		};

		ion_cma_ge4g: ion_cma_ge4g@124100000 {
			compatible = "ion-cma";
			[...]
		};
	};
};

Developers can modify the configuration according to their DDR size by referring to the following table:

DDR Capacity Device Tree Node for ION Region
1GB 1. cma_reserved region: ion_reserved_1g
2. carveout region: ion_carveout_1g
3. ion_cma region: ion_cma_1g
2GB 1. cma_reserved region: ion_reserved_2g
2. carveout region: ion_carveout_2g
3. ion_cma region: ion_cma_2g
≥4GB 1. cma_reserved region: ion_reserved_ge4g
2. carveout region: ion_carveout_ge4g
3. ion_cma region: ion_cma_ge4g

Method to Adjust ION Regions in U-Boot

Refer to: U-Boot Dynamic Configuration of Kernel ION Reserved Memory Size Usage Guide.

ION Debugging Methods

The X5 BSP provides two perspectives for viewing ION debugging information:

  • From the ION region perspective (memory provider):

    • Describes information about each ION region.

    • For example, total size, used size, and detailed information about each allocated memory block.

  • From the client perspective (memory consumer):

    • A client represents an identity of a memory user, such as process ID, VPU, JPU, etc.

    • Shows the total memory usage of the client and detailed allocation information.

View Statistics of Each ION Region

Commands:

  • View statistics of individual ION regions:

cat /sys/kernel/debug/ion/heaps/ion_cma
cat /sys/kernel/debug/ion/heaps/cma_reserved
cat /sys/kernel/debug/ion/heaps/carveout

  • View statistics of all ION regions:

cat /sys/kernel/debug/ion/heaps/all_heap_info

Here, ion_cma, cma_reserved, and carveout correspond to the three ION regions used in the X5 BSP (other regions are unused).

Example using /sys/kernel/debug/ion/heaps/cma_reserved:

root@buildroot:~# cat /sys/kernel/debug/ion/heaps/cma_reserved
-------------------------------------------------------------------------
the heap id is 6
-------------------------------------------------------------------------
-------------------------------------------------------------------------
    cma_reserved  heap total size        536870912
-------------------------------------------------------------------------
       heap name           client              pid             size
-------------------------------------------------------------------------
    cma_reserved            udevd              294          1134592
    cma_reserved            udevd              307          3145728
-------------------------------------------------------------------------
allocations (info is from last known client):
           udevd              294              gpu             4096 1
           udevd              294              gpu             4096 1
           udevd              294              gpu             4096 1
           udevd              294              gpu             8192 1
           udevd              294              gpu          1114112 1
           udevd              307              vpu          3145728 1
-------------------------------------------------------------------------
  total orphaned                0
          total           4280320
-------------------------------------------------------------------------

  • Part 1: heap id is the internal ID assigned by the ION subsystem to different ION region types. See enum ion_heap_type in the BSP source file kernel/include/uapi/ion.h.

  • Part 2: heap total size is the total reserved size of the current ION region in the system (Note: unit is bytes, decimal).

  • Part 3: Column meanings are as labeled.

    • Note: When client is udevd, it indicates the user-space process at allocation time was udevd, typically during driver initialization after automatic loading of .ko modules.

  • Part 4: Details per allocation:

    • Column 1: process name

    • Column 2: PID (Note: PID may vary across boots; shown values are for reference only)

    • Column 3: X5 ION system’s custom module identifier, derived from the Flag passed by the client during allocation

    • Column 4: allocation size (Note: unit is bytes, decimal)

    • Column 5: number of kernel references to the allocated memory

  • Part 5: Summary data:

    • total orphaned: size of memory not used by any client but not yet freed

    • total: total allocated memory size for the current heap (unit: bytes, decimal)

View Statistics of Clients Allocated ION Memory

The /sys/kernel/debug/ion/clients/ directory on X5 devices shows current clients that have allocated ION memory (excluding the BPU subsystem):
Note: The number of clients varies depending on module usage and is for reference only.

root@buildroot:~# ls /sys/kernel/debug/ion/clients/
1114-0  display-0  dsp0-0  galcore-0  jpu-0 vio_driver_ion-0  vpu-0  vsi_cam_drv_ion-0

Explanation of the listed clients:

Filename Client Description
1114-0 Process ID Process using ION, includes all module ION info used by the process
display-0 Display subsystem -
dsp0-0 DSP -
galcore-0 GPU ION memory used by 2DGPU and 3DGPU
jpu-0 JPU -
osd_driver_ion-0 OSD -
vio_driver_ion-0 Camera-related hardware accelerators (except OSD) Includes VIN, ISP, GDC, VSE
vpu-0 Video subsystem -
vsi_cam_drv_ion-0 Camera subsystem background service ION memory used by camera subsystem background service

Specific usage per client can be viewed using cat /sys/kernel/debug/ion/clients/<client name>. Example for “galcore-0”:

root@buildroot:~# cat /sys/kernel/debug/ion/clients/galcore-0
       heap_name:    size_in_bytes :  handle refcount :    handle import :       buffer ptr :  buffer refcount :  buffer share id : buffer share count
    cma_reserved:             1000 :                1 :                1 :         4302dede :                2:                2 :                1

    cma_reserved:             1000 :                1 :                1 :         417b1a04 :                2:                3 :                1

    cma_reserved:             1000 :                1 :                1 :         f8c13b57 :                2:                4 :                1

    cma_reserved:             2000 :                1 :                1 :         f2a1d186 :                2:                5 :                1

    cma_reserved:           110000 :                1 :                1 :         3b5dd773 :                2:                6 :                1

-------------------------------------------------------------------------
          total            115000
-------------------------------------------------------------------------

  • size in bytes: size of a single ION allocation (unit: bytes, hexadecimal)

  • buffer ptr: physical address of the allocated ION memory

  • total: total ION memory used by the current client (unit: bytes, hexadecimal)

ION Region Name Mapping Table

Correspondence between ION region names, device tree node names, and flags in the hbmem interface:

ION Region Name Device Tree Node Name Flag in hbmem Interface
cma_reserved ion_reserved_xg HB_MEM_USAGE_PRIV_HEAP_2_RESERVERD
HB_MEM_USAGE_PRIV_HEAP_2_RESERVED
carveout ion_carveout_xg HB_MEM_USAGE_PRIV_HEAP_RESERVERD
HB_MEM_USAGE_PRIV_HEAP_RESERVED
ion_cma ion_cma_xg HB_MEM_USAGE_PRIV_HEAP_DMA

  • In the table header, xg denotes the suffix of the ion reserved space node name in the Linux kernel device tree. Different definitions are made for different DDR capacities, so xg represents the capacity.

    • For example, ion_reserved_xg refers to device tree nodes: ion_reserved_1g, ion_reserved_2g, ion_reserved_ge4g

4.4.6.3. Adjusting ION Region Size Based on Application Scenarios

The default ION region configuration in X5 BSP is set for general use cases. Customers can adjust ION region configurations based on their specific scenarios. Below are the adjustment methods:

  • Theoretical calculation of ION resource usage

  • Run actual programs and verify theoretical values using ION debugging tools

  • Configure ION regions based on theoretical values

Theoretical Calculation of ION Resource Usage

VPU

H.264 Encoding
Function Usage Size Description
Custom SEI Information 16384 * 5 -
customMap 262144 -
vui 16384 -
Motion Vector Buffers 4K_ALIGN((64_ALIGN(length) * 64_ALIGN(width) / 32) * 2) + 4096 -
FBC Luma Table Buffer 4K_ALIGN((256_ALIGN(length) * 64_ALIGN(width) / 32) * 2) + 4096 -
FBC Chroma Table Buffer 4K_ALIGN((256_ALIGN(length/2) * 64_ALIGN(width)/32) * 2) + 4096 -
Sub-sampled Buffer for ME (Motion Estimation) 4K_ALIGN((Align32(length/4) * Align4(width/4)) * 2) + 4096 -
MediaCodec Output Buffer 4K_ALIGN(length * width * bit_depth (NV12: 1.5)) * output_buffer_count Example: 3112960 bytes for 1080P
MediaCodec Input Buffer 4K_ALIGN(length * width * bit_depth (NV12: 1.5)) * input_buffer_count Not needed in external memory mode
Task Buffer Complex calculation; refer to table -
Reconstructed and Reference Frames Complex calculation; refer to table -

  • task buffer and reconstructed and reference frames have complex logic, dependent only on image resolution. Refer to the following table. If resolution is not listed, estimate based on the table.

Resolution Example Camera Task Buffer Reconstructed & Reference Frames
3840*2160 imx415 61501440 12443648 * 2
1920*1080 sc230 16560128 3,112,960 * 2
1088*1280 sc132 16494592 2088960 * 2
1600*1200 sc202 16547840 2883584 * 2
640*480 sc035 16396288 462848 * 2
H.265 Encoding
Function Usage Size Description
Custom SEI Information 16384 * 5 -
vui 16384 -
Motion Vector Buffers 4K_ALIGN((64_ALIGN(length) * 64_ALIGN(width) / 32) * 2) + 4096 -
FBC Luma Table Buffer 4K_ALIGN((256_ALIGN(length) * 64_ALIGN(width) / 32) * 2) + 4096 -
FBC Chroma Table Buffer 4K_ALIGN((256_ALIGN(length/2) * 64_ALIGN(width)/32) * 2) + 4096 -
Sub-sampled Buffer for ME (Motion Estimation) 4K_ALIGN((Align32(length/4) * Align4(width/4)) * 2) + 4096 -
MediaCodec Output Buffer 4K_ALIGN(length * width * bit_depth (NV12: 1.5)) * output_buffer_count Example: 3112960 bytes for 1080P
MediaCodec Input Buffer 4K_ALIGN(length * width * bit_depth (NV12: 1.5)) * input_buffer_count Not needed in external memory mode
Task Buffer Complex calculation; refer to table -
Reconstructed and Reference Frames Complex calculation; refer to table -

  • task buffer and reconstructed and reference frames have complex logic, dependent only on image resolution. Refer to the following table. If resolution is not listed, estimate based on the table.

Resolution Example Camera Task Buffer Reconstructed & Reference Frames
3840*2160 imx415 60968960 12443648 * 2
1920*1080 sc230 16461824 3112960 * 2
1088*1280 sc132 16445440 2088960 * 2
1600*1200 sc202 16457728 2883584 * 2
640*480 sc035 16420864 462848 * 2

BPU

Function Usage Size Description
Fixed Usage 32768 * 2 Occupied once if BPU module is used
Model File 4K_ALIGN(Model File Size) + 4096 * 3 Occupied once
Model Input Buffer Size Determined by model input format, e.g., NV12:
(4K_ALIGN([input_width] * [input_height]) +
4K_ALIGN([input_width] * [input_height] / 2))		 Occupied upon calling hbSysAllocCachedMem
Model Output Buffer Size Determined by model output dimensions, e.g., yolov5:
4K_ALIGN(output[0].AlignedByteSize) +
4K_ALIGN(output[1].AlignedByteSize) +
4K_ALIGN(output[2].AlignedByteSize) +		 Occupied upon calling hbSysAllocCachedMem

  • Model Input Buffer Size: allocated via hbSysAllocCachedMem, stores input images

  • Model Output Buffer Size: allocated via hbSysAllocCachedMem, stores model output results

  • AlignedByteSize: member variable alignedByteSize in the hbDNNTensorProperties structure of the BPU software framework

Note: BPU ION usage has some special cases; thus, more accurate measurement requires running a model-only program:

  • Case 1: In certain models, the BPU software framework may allocate an additional heap region at runtime, e.g., yolov5.

  • Case 2: Some model operators require additional ION memory, e.g., yolov5.

Validating Theoretical Values

After running the actual or test program, use ION Debugging Methods to check actual ION memory usage per module.

As known from View Statistics of Clients Allocated ION Memory, most modules correspond to one client node, and total ION usage can be viewed in the client node.
However, some modules are special, so we discuss three cases:

  1. Camera modules (except OSD and background services) are grouped as one client; individual module usage cannot be viewed, requiring alternative methods.

  2. BPU module: no corresponding client; requires other methods.

  3. Other modules: VPU, GPU, OSD, etc., have corresponding client nodes.

Camera Modules

Camera modules (except OSD) are grouped as one client: vio_driver_ion-0, so only total usage is visible.
To view individual module usage:

  1. Specify different Flags in the HBN software interface to show different names in all_heap_info.

  2. View all_heap_info and filter by module names.

Example for VIN module (replace cimdma in command for other modules): Each line’s 4th column shows VIN’s ION usage size (decimal, bytes):

root@buildroot:~# cat /sys/kernel/debug/ion/heaps/all_heap_info |grep cimdma
  sunrise_camera             1070           cimdma             4096 1
  sunrise_camera             1070           cimdma          4149248 0
  sunrise_camera             1070           cimdma             4096 1
  sunrise_camera             1070           cimdma          4149248 0
  sunrise_camera             1070           cimdma             4096 1
  sunrise_camera             1070           cimdma          4149248 0
Module Identifier Description Flag in HBN Software Interface
VIN cimdma - HB_MEM_USAGE_HW_CIM
ISP isp_yuv - HB_MEM_USAGE_HW_ISP
VSE other(cma_reserved) - Not defined, uses default
GDC gdc - HB_MEM_USAGE_HW_GDC_OUT
GDC BIN File gdcfb - HB_MEM_USAGE_HW_GDC

BPU Module

Each line’s 4th column shows BPU’s ION usage (decimal, bytes). The sum of these values is the total BPU ION usage:

root@buildroot:~# cat /sys/kernel/debug/ion/heaps/all_heap_info |grep bpu
  sunrise_camera             1070              bpu            32768 0
  sunrise_camera             1070              bpu          9486336 0
  sunrise_camera             1070              bpu          4259840 0
  sunrise_camera             1070              bpu          7864320 0
  sunrise_camera             1070              bpu             4096 0
  sunrise_camera             1070              bpu          4259840 0
  sunrise_camera             1070              bpu             4096 0
  sunrise_camera             1070              bpu             4096 0
  sunrise_camera             1070              bpu            32768 0
  sunrise_camera             1070              bpu           229376 0
  sunrise_camera             1070              bpu           229376 0
  sunrise_camera             1070              bpu           454656 0
  sunrise_camera             1070              bpu           229376 0
  sunrise_camera             1070              bpu           454656 0
  sunrise_camera             1070              bpu           454656 0
  sunrise_camera             1070              bpu           454656 0
  sunrise_camera             1070              bpu           229376 0
  sunrise_camera             1070              bpu           454656 0
  sunrise_camera             1070              bpu           229376 0
  sunrise_camera             1095              bpu           450560 0
  sunrise_camera             1095              bpu           450560 0
  sunrise_camera             1095              bpu          1802240 0
  sunrise_camera             1095              bpu          7200768 0
  sunrise_camera             1095              bpu           450560 0
  sunrise_camera             1095              bpu          1802240 0
  sunrise_camera             1095              bpu          7200768 0
  sunrise_camera             1095              bpu           450560 0
  sunrise_camera             1095              bpu          1802240 0
  sunrise_camera             1095              bpu          7200768 0
  sunrise_camera             1095              bpu          7200768 0
  sunrise_camera             1095              bpu          1802240 0
  sunrise_camera             1095              bpu           450560 0
  sunrise_camera             1095              bpu          7200768 0
  sunrise_camera             1095              bpu          1802240 0

Other Modules

Example for Display module: Note: client node values are in hexadecimal. So total 5f0000 means Display uses 0x5f0000 bytes.

root@buildroot:~# cat /sys/kernel/debug/ion/clients/display-0 |grep "total"
          total            5f0000
Module Client Filename
VPU vpu-0
JPG/MJPEG jpu-0
GPU galcore-0
Display display-0
OSD osd_driver_ion-0
DSP dsp0-0
Camera Background Service vsi_cam_drv_ion-0

Modify and Apply ION Region Configuration

  1. Quick debugging method: configure via uboot command line. Refer to Method to Adjust ION Regions in U-Boot

  2. Formal method: modify device tree and recompile image. Refer to Method to Adjust ION Regions in Linux Device Tree