6.3.3. Model Operator Support List
6.3.3.1. Usage Limitations
This section mainly introduces the Caffe and ONNX operators supported by the X5 processor. Other operators not listed below are currently unsupported due to hardware limitations of the X5 processor’s BPU.
Terminology:
BPU Acceleration: Operators that can be accelerated on the X5 processor (under certain constraints). If constraints are not met, computation will fall back to the CPU.
CPU Computation: Operators already optimized on the ARM CPU of the X5 processor, supporting ONNX opset10 and opset11.
CPU Computation※: CPU operators not yet integrated.
Additional Notes:
All operators running on the X5 BPU follow general limitations:
Input and output dimensions must be 4D. For operators supporting non-4D cases, this will be explicitly indicated in the constraints.
Shape: H, W, C ∈ [1, 65536], N ≤ 4096.
N × C × H × W ≤ 1G bytes.
Supports
Caffe 1.0basic and commonly used extended operators, as well as ONNXopset10andopset11operators. Operators failing to meet BPU acceleration constraints will fall back to ARM CPU computation.Operators
Cast,Constant,Dropout,Reshape,Squeeze,Unsqueeze, andShapecannot run directly on the BPU, but in certain cases (e.g., constant folding), the algorithm toolchain may optimize them out, achieving indirect support.Operators marked as PyTorch are those not included in the official opset11. The X5 algorithm toolchain provides export scripts to convert these from PyTorch to X5 processor’s custom ONNX OPs.
Using the tensorflow-onnx (https://github.com/onnx/tensorflow-onnx) conversion tool, operators from
tensorflow1.*can be stably converted into ONNX models (opset6 to opset11). However,TensorFlow2.*support is currently experimental.Regarding active and passive quantization of OPs: An OP meeting the constraints in this chapter may still run on the CPU because it is classified as a passively quantized OP. The algorithm toolchain designs quantization logic considering multiple factors such as OP computation characteristics and BPU low-level logic. Current quantization logic is divided into: active quantization, passive quantization, and manual quantization. For more information, please refer to: Active and Passive Quantization Logic in Algorithm Toolchain.
6.3.3.2. X5 Supported Caffe Operators
| Caffe Operator Name | CPU Computation / BPU Acceleration | X5 BPU Constraints | CPU Constraints |
|---|---|---|---|
| Convolution | BPU Acceleration | Same constraints as ONNX Conv | Supports conv1d, conv2d, conv3d. Type constraints: float, int32, int8. auto_pad attribute not supported. pads attribute constraints: - conv1d: [Dstart, Dend], pads length equals 2, and Dstart = Dend. - conv2d: [Hstart, Wstart, Hend, Wend], pads length equals 4, and Hstart == Hend, Wstart == Wend. - conv3d: [Dstart, Hstart, Wstart, Dend, Hend, Wend], pads length equals 6, and Dstart = Dend, Hstart == Hend, Wstart == Wend. |
| Deconvolution | BPU Acceleration | Same constraints as ONNX ConvTranspose | Shape constraint: Only supports 4D tensor computation. Type constraint: Only supports float type. Attribute constraints: - Only supports dilations, group, output_padding, pads, strides attributes. - pads attribute constraint: [hstart, wstart, hend, wend] must satisfy (hstart==hend and wstart==wend). |
| MaxUnpool | CPU Computation | --- | from_type supports: - X: type constraint: only float type supported. - I: Tensor (int64). to_type supports: type constraint: only float type supported. |
| Pooling | BPU Acceleration | Four types of Pooling operators: MaxPooling, AveragePooling, GlobalMaxPooling, GlobalAveragePooling. Constraints for each: - MaxPooling: Supports int16 input/output. kernel <= 256; stride <= 256; padding <= 256. MaxPooling does not support dilation. - AveragePooling: Same constraints as ONNX AveragePool - GlobalAveragePooling: No constraints. - GlobalMaxPooling: H, W ∈ [1, 256]. |
None |
| SPP | CPU Computation | Not supported | Supports pyramid_height, 2^n pooling, n<7; pooling kernel <= 255; supports pool, configurable values {0, 1} |
| InnerProduct | BPU Acceleration | InnerProduct will be transformed into Conv; boundary constraints refer to Conv. axis attribute not supported. |
None |
| LRN | CPU Computation | Not supported | local_size supported. alpha supported. beta supported. norm_region supported, configurable values {ACROSS_CHANNELS, WITHIN_CHANNEL}. k supported. |
| MVN | CPU Computation | Not supported | normalize_variance supported, configurable values {0, 1}. across_channels supported, configurable values {0, 1}. Only supports Float32 computation. |
| BatchNorm | BPU Acceleration | No constraints | None |
| ELU | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions, highest dimension ∈ [1, 4096], other dimensions ∈ [1, 65536]. |
None |
| BNLL | CPU Computation | Not supported | None |
| PReLU | BPU Computation | 1. Only supports int8 input/output. 2. Only supports 4D input/output. |
- Type constraints: only float type supported. - from_type: X and slope. - to_type: Y. - X shape is data_shape, slope shape is slope_shape, shape constraints: - data_shape == slope_shape. - slope_shape.ProdSize() == 1. - X and slope only support 4D NCHW layout computation, and N, C dimensions must be equal. - HxW vs 1x1 (slope_shape). - HxW vs Hx1 (slope_shape). - HxW vs 1xW (slope_shape). - X is 4D && slope is 3D && data_shape[1] == slope_shape[0] && slope_shape[1] == 1 && slope_shape[2] == 1. |
| ReLU/LeakyRelu | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
None |
| Sigmoid | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
None |
| TanH | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
None |
| Eltwise | BPU Acceleration | Supported operations: Add, Sub, Mul. 1. Supports int16 input/output. 2. Input types support featuremap and constant, with at most one constant input. 3. Supports broadcasting across all dimensions, and mutual broadcasting between two inputs, e.g., NH1C and N1WC. 4. Input/output dimensions support 1-10D, size under general limit (see note). Supports different input dimensions. Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge; broadcast dimensions from different tensors cannot merge: [2,1,4,1,2] [1,5,1,5,1]. |
None |
| Bias | BPU Acceleration | Refer to Eltwise with Add operation | None |
| Scale | BPU Acceleration | Refer to Eltwise with Mul operation | None |
| AbsVal | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
None |
| Exp | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
None |
| Log | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
None |
| Power | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions 3. Second input only supports scalar. |
None |
| Threshold | CPU Computation | Not supported | None |
| Reduction | CPU Computation | Not supported | operation supports SUM, ASUM, SUMSQ, MEAN, Max, LogSum, Min, Prod; axis supported; Only supports Float32 computation. |
| Softmax | BPU Acceleration | 1. Supports int16 input/output. 2. Runs on CPU by default. When input is 4D and axis=1,2,3, use run_on_bpu to specify running on BPU. |
None |
| ArgMax | BPU Acceleration | 1. Only supports axis=1, c<=64. 2. top_k != 1 not supported. 3. Supports int16 input/output. |
None |
| Concat | BPU Acceleration | 1. Supports int16 input/output. 2. N-dimension concat not supported. |
None |
| Split | BPU Acceleration | 1. Supports int16 input/output. 2. Original input length must be multiple of each split tensor length. 3. Supports any dimension except N. 4. Split count must be divisible. 5. Supports non-4D input/output. |
None |
| Slice | BPU Acceleration | 1. Supports int16 input/output. 2. No constraints, supports non-4D input/output. |
None |
| Reshape | BPU Acceleration | 1. Supports int16 input/output. 2. Supports 1-10D input/output. |
shape supports [1,4] shape_dim configurations; axis supports [-4,3], default 0, follows Caffe rules; num_axes supports [-1,3], default -1 means transform all axes starting from axis |
| Flatten | CPU Computation | Not supported (can be fused in some scenarios) | axis range [-4,3], default 1, -4 same as 0. Only supports End_axis == -1. |
| Crop | CPU Computation | Not supported | None |
| Dropout | BPU Acceleration | No constraints | None |
| LSTM | BPU Acceleration | Only supports batch=1 | -- |
| Normalize | CPU Computation | Not supported | Type constraint: only float type supported. |
| PassThrough | BPU Acceleration | Supports mode=DCR and mode=CRD. Only supports rearrangement in H and W directions, and only blocksize=2 rearrangement. Example: NxCxHxW -> Nx(4C)x(H/2)x(W/2). |
Type constraint: only float type supported. |
| CReLU | CPU Computation | Not supported | Type constraint: only float type supported. |
| RReLU | CPU Computation | Not supported | None |
| Permute | BPU Acceleration | 1. Supports arbitrary input dimensions. 2. Supports transformation of any dimension except batch dimension (first dimension). |
- Supports nhwc2nchw, perm: [0, 3, 1, 2]. - Supports nchw2nhwc, perm: [0, 2, 3, 1]. - Supports specified perm dimension transformation, data types only support float, int8, int32. |
| MatMul | BPU Acceleration | C = MatMul(A, B), input A and B have the following dimension constraints: - Both A and B support non-4D input but must meet constraints: - A and B must have the same number of dimensions. - A and B's lowest two dimensions M, K ∈ [1, 8192], other higher dimensions ∈ [1, 4096]. Note: HDMK vs HDKN, MK/KN are the lowest two dimensions. - Supported broadcast must meet the following conditions: - For A and B, all dimensions except the lowest two must be all 1 or all non-broadcast values. - Example supported: HDMK vs H1KN - Counterexample not supported: H1MK vs 1DKN - For A, excluding the lowest two dimensions, other dimensions cannot mix broadcast and non-broadcast values. - Example supported: 11MK vs HDKN - Counterexample not supported: H1MK vs HDKN - For B, excluding the lowest two dimensions, if other dimensions mix broadcast and non-broadcast values, non-broadcast values must be in continuous higher dimensions. - Example supported: BHDMK vs B11KN - Counterexample not supported: BHDMK vs B1DKN Note: Broadcast vs non-broadcast values: - If A and B have values 1 and non-1 on a corresponding axis, then 1 is the broadcast value, non-1 is non-broadcast; - If A and B have equal values on a corresponding axis, both are non-broadcast values (e.g., HDMK vs H1KN, 1 is broadcast, H is non-broadcast). |
Type constraint: only float type supported. |
| Upsample | BPU Acceleration | Input featuremap must be 4D NCHW, and resize only supported on H and W dimensions; Scale factor cannot be simultaneously less than 2. |
None |
| ROIPooling | CPU Computation | Not supported | None |
| PSROIPooling | CPU Computation | Not supported | None |
6.3.3.3. X5 Supported ONNX Operators
| ONNX Operator Name | CPU Computation / BPU Acceleration | X5 BPU Constraints | CPU Constraints |
|---|---|---|---|
| Abs | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Acos | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraints: float and double supported. |
| Acosh | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraints: float and double supported. |
| Add | BPU Acceleration | 1. Supports int16 input/output. 2. Input types support featuremap and constant, with at most one constant input. 3. Supports broadcasting across all dimensions, and mutual broadcasting between two inputs, e.g., NH1C and N1WC. 4. Input/output dimensions support 1-10D, size under general limit (see note). Supports different input dimensions. Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge; broadcast dimensions from different tensors cannot merge: [2,1,4,1,2] [1,5,1,5,1]. 5. Add in resnet shortcut structure will be fused into the previous conv for acceleration. |
- Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| And | CPU Computation | -- | - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| ArgMax | BPU Acceleration | 1. Input dimension is 4D NCHW. 2. N ∈ [1, 4096], H,W ∈ [1, 65536], C ∈ [1, 8191]. 3. Supports int16 input/output. 4. Only supports argmax/argmin along C dimension, i.e., axis=1. |
Type constraint: only float type supported. |
| ArgMin | BPU Acceleration | 1. Input dimension is 4D NCHW. 2. N ∈ [1, 4096], H,W ∈ [1, 65536], C ∈ [1, 8191]. 3. Supports int16 input/output. 4. Only supports argmax/argmin along C dimension, i.e., axis=1. |
Type constraint: only float type supported. |
| Asin | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions. |
Type constraints: float and double supported. |
| Asinh | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions. |
Type constraints: float and double supported. |
| Atan | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions. |
Type constraints: float and double supported. |
| Atanh | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions. |
Type constraints: float and double supported. |
| AveragePool | BPU Acceleration | kernel: H,W ∈ [1, 256] H * W <= 8192, H * W > 1 stride: H,W ∈ [1, 256] padding: H,W ∈ [0, 255] |
Input/output supports 4D and 5D. |
| BatchNormalization | BPU Acceleration | No constraints. | Type constraint: only float type supported. Supports channel-first data layout computation. |
| BitShift | CPU Computation※ | -- | -- |
| Cast | CPU Computation | -- | from_type supports double, float, bool, int64, uint32, int32, uint16, int16, uint8, int8. to_type supports double, float, bool, int64, uint32, int32, uint16, int16, uint8, int8. |
| Ceil | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraints: double and float supported. |
| Clip | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
opset6: min, max as attributes, dtype only supports float; opset11: min, max as inputs, when only two inputs, second is min; dtype supports float, double. |
| Compress | CPU Computation※ | -- | -- |
| Concat | BPU Acceleration | 1. Supports int16 input/output. 2. N-dimension concat not supported. |
-- |
| ConcatFromSequence | CPU Computation※ | -- | -- |
| Constant | BPU Acceleration | Will be optimized via constant folding into value storage | Currently sparse_tensor attribute not supported. |
| ConstantOfShape | BPU Acceleration | Will be optimized via constant folding into value storage | Type constraints: float, int32, int8 supported. |
| Conv | BPU Acceleration | Supports 4D input (conv2d) and 5D input (conv3d). 4D input (conv2d): Kernel shape range: N,C ∈ [1, 8192]; H,W ∈ [1, 31]. C * H * W <= 32767. Input/output Channel range (one group) <= 8192; if Conv is last op in quantized subgraph, range <= 65536. Stride range: H,W ∈ [1, 256]; for Conv followed by Add (resnet shortcut), stride range: {1, 2}; for dilated>1 conv, stride only supports =1. Dilation range: H,W∈ [1, 16]; when H or W >1, only supports output int8, and input tensor H must be divisible by dilation H, input tensor W by dilation W. Padding range: H,W ∈ [0, 256]. 5D input (conv3d): Input size NCDHW: N ∈ [1, 128]; H,W,D,C ∈ [1, 65536]. Kernel size NCDHW: N,C ∈ [1, 65536]; H,W ∈ [1, 31], D ∈ [1, 8191]. Padding size DHW: H,W ∈ [0, 256], D ∈ [0, kernel_d/2]. Stride range: H, W both 1 or both 2. Group, dilation not supported. Size: 1G bytes; when D * C > 4096, H * alignCeil(W, 256) * D * C < 1G. weight D * input C <= 8192. |
Supports conv1d, conv2d, conv3d. Type constraints: float, int32, int8. auto_pad attribute not supported. pads attribute constraints: - conv1d: [Dstart, Dend], pads length equals 2, Dstart = Dend. - conv2d: [Hstart, Wstart, Hend, Wend], pads length equals 4, Hstart==Hend, Wstart==Wend. - conv3d: [Dstart, Hstart, Wstart, Dend, Hend, Wend], pads length equals 6, Dstart = Dend, Hstart==Hend, Wstart==Wend. |
| ConvInteger | CPU Computation※ | -- | -- |
| ConvTranspose | BPU Acceleration | Input/output featuremap size limits: N ∈ [1, 128]. H,W ∈ [1, 65536]. C ∈ [1, 2048]. Size: 1G bytes. Weight size limits: N,C ∈ [1, 2048]. H,W ∈ [1, 14] and HW not both 1. Size: psh = padding.h % stride.h; psw = padding.w % stride.w; ksh = (kernel.h - 1 + psh) / stride.h +1 ksw = (kernel.w - 1 + psw) / stride.w + 1 group_num = fout.c / kernel.c ksc = fin.c / group_num kernel_size = ksh * ksw * ksc kernel_size ∈ [1, 32767] Padding range: When stride odd, H,W ∈ [0, kernel / stride). When stride even, H,W ∈ [0, kernel / stride]. out_pad range: H,W ∈ {0,1}. stride >= 1 && stride <=14 but stride_h and stride_w cannot both be 1. Dilation ∈ {(1, 1)}. |
Shape constraint: only supports 4D tensor computation. Type constraint: only float type supported. Attribute constraints: - Only supports dilations, group, output_padding, pads, strides attributes. - pads constraint: [hstart, wstart, hend, wend] must satisfy (hstart==hend and wstart==wend). |
| Cos | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: float supported. |
| Cosh | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: float supported. |
| CumSum | CPU Computation | -- | axis: type constraint only int32 supported. |
| DepthToSpace | BPU Acceleration | Supports int16 input/output. Supports mode=DCR and mode=CRD. Only supports H and W direction rearrangement, and only blocksize=2 rearrangement. Example: NxCxHxW -> Nx(C/4)x(2H)x(2W), output channel must be multiple of 4. |
from_type supports: - Type constraint only float supported. - Only supports 4D tensor computation. to_type supports: - Type constraint only float supported. - Only supports 4D tensor computation. |
| DequantizeLinear | CPU Computation | -- | -- |
| Det | CPU Computation※ | -- | -- |
| Div | BPU Acceleration | Refer to Mul operator for input shape constraints. | - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| Dropout | BPU Acceleration | This operator is removed during inference and optimized out | -- |
| Einsum | CPU Computation※ | -- | -- |
| Elu | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Equal | BPU Acceleration | 1. Supports int16 input. 2. Supports broadcasting across all dimensions, supports broadcast for either fin0 or fin1, but not mutual broadcast. 3. Input/output dimensions support 1-10D, size under general limit (see note). Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge. 4. Runs on CPU by default, can be specified to run on BPU via run_on_bpu. |
- Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| Erf | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: float supported. |
| Exp | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Expand | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions, input and output differ in only one dimension. 3. Only one dimension can differ between input and output. |
-- |
| EyeLike | CPU Computation | -- | -- |
| Flatten | BPU Acceleration | Same constraints as Reshape. | -- |
| Floor | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| GRU | CPU Computation | -- | - direction attribute only supports forward. - Type constraint: only float type supported. |
| Gather | BPU Acceleration | 1. rank of input/output/indices must be ≤4. 2. indices support: - When indices is feature (output of other op), type constraint only int32. - When indices is weight (constant in model), type constraint supports int32 and int64. |
from_type supports: - input: type constraints: float,int64,int32,int8,uint64,uint32,uint8. - indices: type constraints: int32, int64. to_type supports: type constraints: float,int64,int32,int8,uint64,uint32,uint8. |
| GatherElements | BPU Acceleration | 1. Supports int16 input/output. 2. input/indices/output dimensions support 1-10D. 3. When input dimension i != axis, require indices.shape[i] <= input.shape[i]. |
-- |
| GatherND | CPU Computation | -- | from_type supports: - input: type constraints: float,int32,int8. - indices: tensor(int64). to_type supports: type constraints: float,int32,int8. |
| Gemm | BPU Acceleration | Gemm will be transformed into Conv; boundary constraints refer to Conv. | Type constraint: only float type supported. |
| GlobalAveragePool | BPU Acceleration | No constraints. | - Type constraint: only float type supported. - Only supports 4D tensor. |
| GlobalLpPool | CPU Computation | -- | - Type constraints: float and double supported. - Only supports 4D tensor computation. |
| GlobalMaxPool | BPU Acceleration | H, W ∈ [1, 256]. | - Type constraint only float supported. - Only supports 4D tensor. |
| Greater | BPU Acceleration | 1. Supports int16 input. 2. Supports broadcasting across all dimensions, supports broadcast for either fin0 or fin1, but not mutual broadcast. 3. Input/output dimensions support 1-10D, size under general limit (see note). Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge. 4. Runs on CPU by default, can be specified to run on BPU via run_on_bpu. |
- Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| HardSigmoid | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint only float supported. |
| Hardmax | CPU Computation※ | -- | -- |
| Identity | CPU Computation | -- | -- |
| If | CPU Computation※ | -- | -- |
| InstanceNormalization | CPU Computation | -- | - Type constraint only float supported. - Supports channel-first data layout computation. |
| IsInf | CPU Computation※ | -- | -- |
| IsNaN | CPU Computation※ | -- | -- |
| LRN | CPU Computation | -- | - Type constraint only float supported. - Only supports 4D tensor. |
| LSTM | BPU Acceleration | Only supports batch_size=1. For multi-batch, ensure batch=1 when exporting ONNX and set input_batch=1 in YAML. | - Type constraint only float supported. - Attribute constraint: direction only supports forward. - Input constraints: - Supports X, W, R inputs; - Supports X, W, R, B inputs (sequence_lens empty or default); - Supports X, W, R, B, sequence_lens, initial_h, initial_c, P inputs (sequence_lens empty or default). |
| LeakyRelu | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Less | BPU Acceleration | 1. Supports int16 input. 2. Supports broadcasting across all dimensions, supports broadcast for either fin0 or fin1, but not mutual broadcast. 3. Input/output dimensions support 1-10D, size under general limit (see note). Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge. 4. Runs on CPU by default, can be specified to run on BPU via run_on_bpu. |
- Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| LessOrEqual | BPU Acceleration | opset11 does not support single LessOrEqual operator, but supports split Greater+Not running on BPU, with constraints same as Greater. | - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| Log | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| LogSoftmax | CPU Computation | -- | Type constraint: only float type supported. |
| Loop | CPU Computation※ | -- | -- |
| LpNormalization | CPU Computation | -- | - p-norm only supports 1 or 2. - Type constraint supports float. |
| LpPool | CPU Computation | -- | - auto_pad attribute not supported. - Type constraint supports float. - Only supports 4D computation. |
| MatMulInteger | CPU Computation※ | -- | -- |
| MatMul | BPU Acceleration | C = MatMul(A, B), input A and B have the following dimension constraints: - Both A and B support non-4D input but must meet constraints: - A and B must have the same number of dimensions. - A and B's lowest two dimensions M, K ∈ [1, 8192], other higher dimensions ∈ [1, 4096]. Note: HDMK vs HDKN, MK/KN are the lowest two dimensions. - Supported broadcast must meet the following conditions: - For A and B, all dimensions except the lowest two must be all 1 or all non-broadcast values. - Example supported: HDMK vs H1KN - Counterexample not supported: H1MK vs 1DKN - For A, excluding the lowest two dimensions, other dimensions cannot mix broadcast and non-broadcast values. - Example supported: 11MK vs HDKN - Counterexample not supported: H1MK vs HDKN - For B, excluding the lowest two dimensions, if other dimensions mix broadcast and non-broadcast values, non-broadcast values must be in continuous higher dimensions. - Example supported: BHDMK vs B11KN - Counterexample not supported: BHDMK vs B1DKN Note: Broadcast vs non-broadcast values: - If A and B have values 1 and non-1 on a corresponding axis, 1 is broadcast, non-1 is non-broadcast; - If A and B have equal values on a corresponding axis, both are non-broadcast (e.g., HDMK vs H1KN, 1 is broadcast, H is non-broadcast) |
Type constraint: only float type supported. |
| Max | BPU Acceleration | 1. Supports int16 input/output. 2. Supports broadcasting across all dimensions, supports mutual broadcasting between two inputs, e.g., NH1C and N1WC. 3. Input/output dimensions support 1-10D, size under general limit (see note). Supports different input dimensions. Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge; broadcast dimensions from different tensors cannot merge: [2,1,4,1,2] [1,5,1,5,1]. |
- Supports 1 to ∞ inputs. - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| MaxPool | BPU Acceleration | Supports int16 input/output. kernel <= 256. stride <= 256. padding <= 256. MaxPool does not support dilation. |
1. Dilation only supports 1x1. 2. Only row-major storage supported. 3. auto_pad attribute not supported. 4. storage_order attribute not supported. 5. Input/output supports 4D and 5D. |
| MaxRoiPool | CPU Computation | -- | None |
| Mean | CPU Computation※ | -- | -- |
| Min | BPU Acceleration | 1. Supports int16 input/output. 2. Supports broadcasting across all dimensions, supports mutual broadcasting between two inputs, e.g., NH1C and N1WC. 3. Input/output dimensions support 1-10D, size under general limit (see note). Supports different input dimensions. Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge; broadcast dimensions from different tensors cannot merge: [2,1,4,1,2] [1,5,1,5,1]. 4. Runs on CPU by default, can be specified to run on BPU via run_on_bpu. |
- Supports 1 to ∞ inputs. - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| Mod | CPU Computation | -- | -- |
| Mul | BPU Acceleration | 1. Supports int16 input/output. 2. Input types support featuremap and constant, with at most one constant input. 3. Supports broadcasting across all dimensions, supports mutual broadcasting between two inputs, e.g., NH1C and N1WC. 4. Input/output dimensions support 1-10D, size under general limit (see note). Supports different input dimensions. Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge; broadcast dimensions from different tensors cannot merge: [2,1,4,1,2] [1,5,1,5,1]. |
- Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| Multinomial | CPU Computation※ | -- | -- |
| Neg | CPU Computation | -- | -- |
| Not | CPU Computation | -- | -- |
| OneHot | CPU Computation | -- | -- |
| Or | CPU Computation | -- | - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| PRelu | BPU Acceleration | 1. Only supports int8 input/output. 2. Only supports 4D input/output. |
- Type constraints: only float type supported. - from_type: X and slope. - to_type: Y. - X shape is data_shape, slope shape is slope_shape, shape constraints: - data_shape == slope_shape. - slope_shape.ProdSize() == 1. - X and slope only support 4D NCHW layout computation, and N, C dimensions must be equal. - HxW vs 1x1 (slope_shape). - HxW vs Hx1 (slope_shape). - HxW vs 1xW (slope_shape). - X is 4D && slope is 3D && data_shape[1] == slope_shape[0] && slope_shape[1] == 1 && slope_shape[2] == 1. |
| Pad | BPU Acceleration | 1. Supports int16 input/output. 2. Supports mode = Constant. 3. Supports padding in all dimensions. |
Pad-10: - Type constraint only float supported. - Only supports 4D NCHW layout tensor. - pads attribute constraints: - len(pads) == 8 && pads[i] >=0 && pads[0] == 0 && pads[1] == 0 && pads[4] == 0 && pads[5] == 0. Pad-11: - from_type supports: - data: type constraint only float supported. - pads: tensor(int64). - constant_value (optional): type constraint only float supported. - to_type supports: type constraint only float supported. - Input/output supports 4D, only supports 2/3 dimension padding. - Input/output supports 5D, only supports 2/3/4 dimension padding. |
| Pow | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions 3. Second input only supports scalar. |
- Type constraints: double, float, int64, int32 supported. - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 5. - Only supports X and Y same type. |
| QLinearConv | CPU Computation※ | -- | -- |
| QLinearMatMul | CPU Computation※ | -- | -- |
| QuantizeLinear | CPU Computation | -- | -- |
| RNN | CPU Computation | -- | - Type constraint: only float type supported. - Attribute constraint: direction only supports forward. - Input constraint: only supports X, W, R inputs, optional inputs B, sequence_lens, initial_h not supported. - Output constraint: only supports Y_h output, shape [num_directions, batch_size, hidden_size]. |
| RandomNormal | CPU Computation※ | -- | -- |
| RandomNormalLike | CPU Computation※ | -- | -- |
| RandomUniform | CPU Computation | -- | -- |
| RandomUniformLike | CPU Computation | -- | -- |
| Range | CPU Computation | -- | Type constraint supports: float,int64,int32,int16. |
| Reciprocal | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
-- |
| ReduceL1 | CPU Computation | -- | -- |
| ReduceL2 | CPU Computation | -- | -- |
| ReduceLogSum | CPU Computation | -- | -- |
| ReduceLogSumExp | CPU Computation | -- | Type constraints: float, double supported. |
| ReduceMax | BPU Acceleration | 1. Supports int16 input/output. 2. Input supports 2-5D, axes attribute must be specified, number of axes=1, does not support reduce along >1 dimension. 3. Size of reduced dimension ∈ [1, 8192]. 4. Only supports keepdims == 1. |
axes supports 0, 1 or equal to input data dimension |
| ReduceMean | BPU Acceleration | 1. Supports int16 input/output. 2. Input supports 2-5D, axes attribute must be specified, number of axes=1, does not support reduce along >1 dimension. 3. When reduce dimension=2, supports simultaneous reduce along HW dimensions. 4. Only supports keepdims == 1. |
axes supports 0, 1 or equal to input data dimension |
| ReduceMin | CPU Computation | -- | -- |
| ReduceProd | CPU Computation | -- | -- |
| ReduceSum | BPU Acceleration | 1. Supports int16 input/output. 2. Input supports 2-5D, axes attribute must be specified, number of axes=1, does not support reduce along >1 dimension. |
axes supports 0, 1 or equal to input data dimension |
| ReduceSumSquare | CPU Computation | -- | axes supports 0, 1 or equal to input data dimension |
| Relu | BPU Acceleration | No constraints | Type constraint: only float type supported. |
| Reshape | BPU Acceleration | 1. Supports int16 input/output. 2. Supports 1-10D input/output. |
-- |
| Resize | BPU Acceleration | 1. Input featuremap must be 4D NCHW, and resize only supported on H and W dimensions; onnx opset=11 supports roi input (models converted from PyTorch need manual modification to add roi input, roi only supports constant input), roi input only supports H and W dimensions, and only effective in tf_crop_and_resize mode. 2. mode attribute supports nearest and linear modes. 3. Supports upscaling and downscaling. 4. For mode=nearest, scale factor supports powers of 2 such as 2,4,8,16,32, etc.; H and W scale factors can differ but must satisfy H_factor <= W_factor. 5. For onnx opset=11, coordinate_transformation_mode supports half_pixel, pytorch_half_pixel, asymmetric, align_corners, and tf_crop_and_resize; when coordinate_transformation_mode=tf_crop_and_resize, boundary coordinates from roi input must be integers. |
resize-10 - When input count=2, use opset10. - Input data is 4D tensor. resize-11 - When input count>2, use opset11. - Input data is 4D tensor. - coordinate_transformation_mode supports half_pixel, asymmetric, align_corners, pytorch_half_pixel in nearest/linear modes, and only half_pixel in cubic mode. - extrapolation_value attribute not supported. |
| ReverseSequence | CPU Computation | -- | -- |
| RoiAlign | CPU Computation | -- | -- |
| Round | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions. |
-- |
| Scan | CPU Computation※ | -- | -- |
| Scatter (deprecated) | CPU Computation※ | -- | -- |
| ScatterElements | CPU Computation | -- | from_type supports: - data: type constraints: float,int32,int8. - indices: type constraint only int32. - updates: type constraints: float,int32,int8. to_type supports: type constraints: float,int32,int8. |
| ScatterND | CPU Computation | -- | from_type supports: - data: type constraints: float,int32,int8. - updates: type constraints: float,int32,int8. to_type supports: type constraints: float,int32,int8. |
| Selu | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions. |
Type constraint: only float type supported. |
| SequenceAt | CPU Computation※ | -- | -- |
| SequenceConstruct | CPU Computation※ | -- | -- |
| SequenceEmpty | CPU Computation※ | -- | -- |
| SequenceErase | CPU Computation※ | -- | -- |
| SequenceInsert | CPU Computation※ | -- | -- |
| SequenceLength | CPU Computation※ | -- | -- |
| Shape | BPU Acceleration | Will be optimized via constant folding into value storage | -- |
| Shrink | CPU Computation※ | -- | -- |
| Sigmoid | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Sign | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Sin | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraints: float and double supported. |
| Sinh | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: float supported. |
| Size | BPU Acceleration | Will be optimized via constant folding into value storage | -- |
| Slice | BPU Acceleration | 1. Supports int16 input/output. 2. No constraints, supports non-4D input/output. |
None |
| Softmax | BPU Acceleration | - Supports int16 input/output. - Runs on CPU by default. Due to differences between onnx::softmax and pytorch::softmax computation, two cases: 1. For onnx::softmax, when input is 4D and axis=3, use run_on_bpu to specify running on BPU. 2. For pytorch::softmax, when input is 4D and axis=1,2,3, use run_on_bpu to specify running on BPU. |
Type constraint: only float type supported. |
| Softplus | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Softsign | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| SpaceToDepth | BPU Acceleration | 1. Supports int16 input/output. 2. Supports mode=DCR and mode=CRD. Only supports H and W direction rearrangement, and only blocksize=2 rearrangement. Example: NxCxHxW -> Nx(4C)x(H/2)x(W/2) |
Type constraint: only float type supported. |
| Split | BPU Acceleration | 1. Supports int16 input/output. 2. Original input length must be multiple of each split tensor length. 3. Supports any dimension except N. 4. Split count must be divisible. 5. Supports non-4D input/output. |
Type constraint: only float type supported. |
| SplitToSequence | CPU Computation※ | -- | -- |
| Sqrt | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| Squeeze | BPU Acceleration | This op will be converted to Reshape, BPU constraints detailed in Reshape op. | -- |
| StringNormalizer | CPU Computation※ | -- | -- |
| Sub | BPU Acceleration | 1. Supports int16 input/output. 2. Input types support featuremap and constant, with at most one constant input. 3. Supports broadcasting across all dimensions, supports mutual broadcasting between two inputs, e.g., NH1C and N1WC. 4. Input/output dimensions support 1-10D, size under general limit (see note). Supports different input dimensions. Inputs >4D can be reduced to 4D by merging adjacent dimensions (including N), following rules: (1) Remove dimensions with output dim=1, e.g., [1,2,3,4] [1,2,1,4]->[1,2,3,4] can be viewed as [2,3,4],[2,1,4]->[2,3,4]. (2) Adjacent non-broadcast dimensions can be merged, e.g., [2,5,4,5,3] [2,5,1,5,3], 2,5 can be merged. (3) Adjacent broadcast dimensions from the same tensor can be merged: e.g., [2,5,4,5,2] [1,1,1,5,2], 2,5,4 can be merged. (4) Broadcast and non-broadcast dimensions cannot merge: e.g., [2,5,4,5,2] [2,1,4,1,2] cannot merge; broadcast dimensions from different tensors cannot merge: [2,1,4,1,2] [1,5,1,5,1]. |
- Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 8. |
| Sum | BPU Acceleration | Same constraints as Add | Type constraint: only float type supported. |
| Tan | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: float supported. |
| Tanh | BPU Acceleration | 1. Supports int16 input/output. 2. Input/output supports 1-10 dimensions |
Type constraint: only float type supported. |
| TfIdfVectorizer | CPU Computation※ | -- | -- |
| ThresholdedRelu | CPU Computation | -- | Type constraint: only float type supported. |
| Tile | BPU Acceleration | 1. Supports int16 input/output. 2. Input and output allowed to differ in only one dimension. |
Type constraint: only float,int64,int32,uint64,uint32 supported. |
| TopK | BPU Acceleration | 1. Supports int16 input/output. 2. input/indices/output dimensions support 1-10D. 3. indices type constraint supports int16/int32/int64. 4. Parameter sorted only supports true. |
- Type constraint: only float type supported. |
| Transpose | BPU Acceleration | 1. Supports int16 input/output. 2. Supports arbitrary input dimensions. |
- Supports nhwc2nchw, perm: [0, 3, 1, 2]. - Supports nchw2nhwc, perm: [0, 2, 3, 1]. - Supports specified perm dimension transformation, data types only support float, int8, int32. |
| Unique | CPU Computation※ | -- | -- |
| Unsqueeze | BPU Acceleration | This op will be converted to Reshape, BPU constraints detailed in Reshape op. | -- |
| Upsample (resize replacement) | BPU Acceleration | -- | Upsample-(resize-10) - When input count=2, use opset10. - Input data is 4D tensor. Upsample-(resize-11) - When input count>2, use opset11. - Input data is 4D tensor. - coordinate_transformation_mode supports half_pixel, asymmetric, align_corners, pytorch_half_pixel in nearest/linear modes, only half_pixel in cubic mode. - extrapolation_value attribute not supported. |
| Where | CPU Computation | -- | Type constraints: float and int64 supported. condition shape cond_shape, X shape x_shape, Y shape y_shape, output shape o_shape, shape constraints: - Only supports cond_shape == o_shape: - x_shape == o_shape broadcast. - y_shape == o_shape broadcast. - Only supports cond_shape.NDim() == 4 && o_shape.NDim() == 4 && N dimension equal && C dimension equal: - 1x1 (cond_shape) vs HxW (o_shape). - Hx1 (cond_shape) vs HxW (o_shape). - 1xW (cond_shape) vs HxW (o_shape). |
| Xor | CPU Computation※ | -- | -- |
| Function | CPU Computation※ | -- | -- |
| Celu | CPU Computation※ | -- | -- |
| DynamicQuantizeLinear | CPU Computation※ | -- | -- |
| GreaterOrEqual | BPU Acceleration | opset11 does not support single GreaterOrEqual operator, but supports split Less+Not running on BPU, with constraints same as Less. | - Supports same input shape computation. - Supports scalar input1 or input2. - Supports broadcast computation, max dimension 5. |
| MeanVarianceNormalization | CPU Computation※ | -- | -- |
| GridSample (PyTorch) | BPU Acceleration | 1. Input dimension only supports 4D, first input must satisfy N ∈ [1, 4096]; C ∈ [1, 65536]; H,W ∈ [1, 1024] and H * W <= 512 * 512. 2. mode only supports 'bilinear', 'nearest'. 3. padding_mode only supports 'zeros', 'border'. 4. This operator is an opset16 ONNX operator; to support opset11, the toolchain provides export via custom operator. Export ONNX models containing this operator using horizon_nn.torch.export_onnx instead of torch.onnx.export, with same parameters. Example: from horizon_nn.api import export_onnx ... export_onnx(...) |