6.1.7.4. hat.models¶

Models widely used in upper module in HAT.

6.1.7.4.1. Models¶

6.1.7.4.1.1. backbones¶

`EfficientNet`	A module of EfficientNet.
`efficientnet`	A module of efficientnet.
`efficientnet_lite`	A module of efficientnet_lite.
`MMDetBackboneAdaptor`	Adaptor of mmdetection backbones to use in HAT.
`MobileNetV1`	A module of mobilenetv1.
`MobileNetV2`	A module of mobilenetv2.
`ResNet18`	A module of resnet18.
`ResNet50`	A module of resnet50.
`VarGDarkNet53`
`ResNet50V2`	A module of resnet50V2.
`VargNASNet`	Module of VargNASNet.
`VargNetV2`	A module of vargnetv2.
`VargNetV2Stage2631`	VargNetV2 with 2631 stage setting.
`TinyVargNetV2`	A module of TinyVargNetv2.
`get_vargnetv2_stride2channels`	Get vargnet v2 stride to channel dict with giving channels and strides.
`CocktailVargNetV2`	CocktailVargNetV2.
`MixVarGENet`	Module of MixVarGENet.
`get_mixvargenet_stride2channels`	Get mixvargenet stride to channel dict with giving channels and strides.
`VargConvNet`	A module of vargconvnet.
`TimeSformer`	A module of TimeSformer.
`HorizonSwinTransformer`	A module of adjusted swin transformer, running faster on bpu.
`ZeroPad2DPatcher`	ZeroPad2D Patcher.
`IResNet100`	A module of resnet100.
`IResNet180`	A module of resnet180.
`SwinTransformer`	A module of swin transformer.
`VargNetV2ISP`	A module of vargnetv2 with NeuralISP module.
`WorkConditionResNet`	A ResNet class that is similar to torchvision's but contains the following changes.

6.1.7.4.1.2. losses¶

`CEWithLabelSmooth`	The losses of cross-entropy with label smooth.
`CrossEntropyLoss`	Calculate cross entropy loss of multi stride output.
`SoftTargetCrossEntropy`	The losses of cross-entropy with soft target.
`CEWithWeightMap`	Crossentropy loss with image-specfic class weighted map within batch.
`ElementwiseL1HingeLoss`	Elementwise L1 Hinge Loss.
`ElementwiseL2HingeLoss`	Elementwise L2 Hinge Loss.
`FocalLoss`	Sigmoid focal loss.
`FocalLossV2`	Focal Loss.
`SoftmaxFocalLoss`	Focal Loss.
`GaussianFocalLoss`	Guassian focal loss.
`GIoULoss`	Generalized Intersection over Union Loss.
`SegLoss`	Segmentation loss wrapper.
`SmoothL1Loss`	Smooth L1 Loss.
`WeightedSquaredHingeLoss`	Weighted Squared ElementWiseHingeLoss.
`YOLOV3Loss`	The loss module of YOLOv3.
`LidarFastFocalLoss`	Focal loss, exactly the same as the CornerNet version.
`LidarSmoothL1RegLoss`	Regression loss for an output tensor.
`LidarRegLoss`	Regression loss for an output tensor.
`AfdetFocalLoss`
`BoundaryLoss`
`MixSegLossMultipreds`	Calculate multi-losses with multi-preds and correspondence targets.
`MixSegLoss`	Calculate multi-losses with same prediction and target.
`LnNormLoss`	LnNorm loss.
`SigmoidFocalLoss`	Sigmoid focal cross entropy loss.
`WeightedSmoothL1Loss`	Smooth L1 localization loss function.
`WeightedSoftmaxClassificationLoss`	Softmax loss function.
`PointPillarsLoss`
`L1Loss`	Smooth L1 Loss.
`MultiStrideLosses`	Multiple Stride Losses.

6.1.7.4.1.3. necks¶

`BiFPN`	Weighted Bi-directional Feature Pyramid Network(BiFPN).
`DwUnet`	Unet segmentation neck structure.
`FPN`
`RetinaNetFPN`	FPN for RetinaNet.
`SequentialBottleNeck`
`Unet`	Unet neck module.
`YOLOV3Neck`	Necks module of yolov3.
`YoloGroupNeck`	Necks module of yolov3.
`PAFPN`	Path Aggregation Network for Instance Segmentation.
`FastSCNNNeck`	Upper neck module for segmentation.

6.1.7.4.1.4. structures¶

`Classifier`	The basic structure of classifier.
`EncoderDecoder`	The basic structure of encoder decoder.
`GraphModel`	A Module contains a graph to represent computational topology.
`Segmentor`	The basic structure of segmentor.
`SegmentorV2`	The basic structure of segmentor.

6.1.7.4.1.4.1. detectors¶

`RetinaNet`	The basic structure of retinanet.
`YOLOV3`	The basic structure of yolov3.
`FCOS`	The basic structure of fcos.
`PointPillarsDetector`	The basic structure of PointPillars.

6.1.7.4.1.4.2. opticalflow¶

PwcNet

The basic structure of PWCNet.

6.1.7.4.1.4.3. fcos¶

`FCOSDecoder`	param num_classes Number of categories excluding the background
`FCOSMultiStrideFilter`	Filter used for post-processing of FCOS.
`FCOSMultiStrideCatFilter`	A modified Filter used for post-processing of FCOS.
`FCOSHead`	Anchor-free head used in FCOS <https://arxiv.org/abs/1904.01355>.
`FCOSTarget`	Generate cls and reg targets for FCOS in training stage.
`DynamicFcosTarget`	Generate cls and reg targets for FCOS in training stage base on dynamic losses.
`multiclass_nms`	NMS for multi-class bboxes.
`get_points`	Generate points according to feat_sizes.
`distance2bbox`	Decode distance prediction to bounding box.
`BBoxUpscaler`	param strides A list contains the strides of fcos_head

6.1.7.4.1.4.4. lidar¶

`PillarFeatureNet`
`PointPillarScatter`
`RadFeatureExtractor`	Rad's Feature Extractor.
`RadScatter`
`AfdetHead`
`AfdetPredict`	Prediction procedure for anchor free det head.

6.1.7.4.1.4.5. retinanet¶

RetinaNetHead

An anchor-based head used in RetinaNet.

RetinaNetPostProcess

The postprocess of RetinaNet.

RetinanetMultiStrideFilter

6.1.7.4.1.4.6. seg¶

`SegDecoder`	Semantic Segmentation Decoder.
`VargNetSegDecoder`	Semantic Segmentation Decoder.
`SegHead`	Head Module for segmentation task.
`SegTarget`	Generate training targets for Seg task.
`FRCNNSegHead`	FRCNNSegHead module for segmentation task.

6.1.7.4.1.4.7. yolo¶

`YOLOV3AnchorGenerator`	Anchors generator for yolov3.
`YOLOv3Filter`	Filter used for post-processing of YOLOv3
`YOLOV3Head`	Heads module of yolov3.
`YOLOV3LabelEncoder`	Encode gt and matching results for yolov3.
`YOLOV3Matcher`	Bounding box classification label matcher by max iou.
`YOLOV3PostProcess`	The postprocess of YOLOv3.

6.1.7.4.1.4.8. deeplab¶

Deeplabv3plusHead

Head Module for FCN.

6.1.7.4.1.4.9. fcn¶

`FCNHead`	Head Module for FCN.
`DepthwiseSeparableFCNHead`
`FCNTarget`	Generate Target for FCN.
`FCNDecoder`	FCN Decoder.

6.1.7.4.1.4.10. pwcnet¶

`PwcNetHead`	A basic head of PWCNet.
`PwcNetNeck`	A extra features module of PWCNet.

6.1.7.4.1.4.11. pointpillars¶

`PointPillarHead`	Basic module of PointPillarHead.
`PointPillarsPostProcess`	PointPillars PostProcess Module.
`Anchor3DGeneratorStride`	3D Anchor Generator by stride.
`LidarTargetAssigner`	TargetAssigner for Lidar.
`PointPillarsPreProcess`

6.1.7.4.2. API Reference¶

class hat.models.backbones.CocktailVargNetV2(bn_kwargs: dict, model_type: str = 'VargNetV2', alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, disable_quanti_input: bool = False, flat_output: bool = True, input_channels: int = 3, head_factor: int = 1, input_resize_scale: int = None, top_layer: Optional[torch.nn.modules.module.Module] = None)¶

CocktailVargNetV2.

对 VargNetV2 进行了简单魔改. 主要是去掉对 num_classes 作为 args 的要求和支持 top_layer 自定义.

TODO(ziyang01.wang) 重构计划, 将相应的修改吸收到 VargNetV2 中.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.EfficientNet(model_type: str, coefficient_params: tuple, num_classes: int, in_channels: int = 3, bn_kwargs: Optional[dict] = None, bias: bool = False, drop_connect_rate: float = 0.2, depth_division: int = 8, activation: str = 'relu', use_se_block: bool = False, blocks_args: Sequence[Dict] = (BlockArgs(kernel_size=3, num_repeat=1, in_filters=32, out_filters=16, expand_ratio=1, id_skip=True, strides=1, se_ratio=0.25), BlockArgs(kernel_size=3, num_repeat=2, in_filters=16, out_filters=24, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=5, num_repeat=2, in_filters=24, out_filters=40, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=3, num_repeat=3, in_filters=40, out_filters=80, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=5, num_repeat=3, in_filters=80, out_filters=112, expand_ratio=6, id_skip=True, strides=1, se_ratio=0.25), BlockArgs(kernel_size=5, num_repeat=4, in_filters=112, out_filters=192, expand_ratio=6, id_skip=True, strides=2, se_ratio=0.25), BlockArgs(kernel_size=3, num_repeat=1, in_filters=192, out_filters=320, expand_ratio=6, id_skip=True, strides=1, se_ratio=0.25)), include_top: bool = True, flat_output: bool = True, resolution: int = 0, use_drop_connect: bool = False)¶

A module of EfficientNet.

参数

model_type (str) – Select to use which EfficientNet(B0-B7 or lite0-4), for EfficientNet model, model_type must be one of: [‘b0’, ‘b1’, ‘b2’, ‘b3’, ‘b4’, ‘b5’, ‘b6’, ‘b7’], for EfficientNet-lite model, model_type must be one of: [‘lite0’, ‘lite1’, ‘lite2’, ‘lite3’, ‘lite4’].
coefficient_params (tuple) – Parameter coefficients of EfficientNet, include: width_coefficient(float): scaling coefficient for net width. depth_coefficient(float): scaling coefficient for net depth. default_resolution(int): default input image size. dropout_rate(float): dropout rate for final classifier layer. num_classes (int): Num classes of output layer.
bn_kwargs (dict) – Dict for Bn layer.
bias (bool) – Whether to use bias in module.
drop_connect_rate (float) – Dropout rate at skip connections.
depth_division (int) – Depth division, Defaults to 8.
activation (str) – Activation layer, defaults to ‘relu’.
use_se_block (bool) – Whether to use SEBlock in module.
blocks_args (list) – A list of BlockArgs to MBConvBlock modules.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
use_drop_connect (bool) – Whether to use drop connect.

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.HorizonSwinTransformer(depth_list: List[int], num_heads: List[int], num_classes: int = 1000, patch_size: Union[int, Tuple[int, int]] = 4, in_channels: int = 3, embedding_dims: int = 96, window_size: int = 7, mlp_ratio: float = 4.0, qkv_bias: bool = True, qk_scale: Optional[float] = None, dropout_ratio: float = 0.0, attention_dropout_ratio: float = 0.0, drop_path_ratio: float = 0.0, patch_norm: bool = True, out_indices: Sequence[int] = (0, 1, 2, 3), frozen_stages: int = - 1, include_top: bool = True, flat_output: bool = True)¶

A module of adjusted swin transformer, running faster on bpu.

参数

depth_list – Depths of each Swin Transformer stage. for swin_T, the numbers could be [2, 2, 6, 2]. for swin_S, swin_B, or swin_L, the numbers could be [2, 2, 18, 2].
num_heads – Number of attention head of each stage. for swin_T or swin_S, the numbers could be [3, 6, 12, 24]. for swin_B, the numbers could be [4, 8, 16, 32]. for swin_L, the numbers could be [6, 12, 24, 48].
num_classes – Num classes of output layer.
patch_size – Patch size. Default: 4.
in_channels – Number of input image channels. Default: 3.
embedding_dims – Number of linear projection output channels. for swin_T or swin_S, the numbers could be 96. for swin_B, the number could be 128. for swin_L, the number could be 192.
window_size – Window size. Default: 7.
mlp_ratio – Ratio of mlp hidden dim to embedding dim. Default: 4.
qkv_bias – Whether to add a learnable bias to query, key, value. Default: True.
qk_scale – Override default qk scale of head_dim ** -0.5 if set.
dropout_ratio – Dropout rate. Default: 0.
attention_dropout_ratio – Attention dropout rate. Default: 0.
drop_path_ratio – Stochastic depth rate. Default: 0.
patch_norm – Whether to add normalization after patch embedding. Default: True.
out_indices – Output from which stages.
frozen_stages – Stages to be frozen (stop grad and set eval mode). Default: -1. -1 means not freezing any parameters.
include_top – Whether to include output layer. Default: True.
flat_output – Whether to view the output tensor. Default: True.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights(m)¶: Initialize the weights in backbone.

class hat.models.backbones.IResNet100(bn_kwargs: dict, bias: bool = True, act_type: str = 'relu', embedding_size: int = 256, dropout: float = 0.0, use_fp16: bool = False, inplace: bool = True)¶

A module of resnet100.

参数

bn_kwargs – Dict for BN layer.
bias – Whether to use bias in module.
act_type – Nonlinear method.
embedding_size – FaceID output embedding_size.
dropout – Dropout rate.
use_fp16 – Whether to use mixed precision training.
inplace – Whether to use inplace op.

class hat.models.backbones.IResNet180(bn_kwargs: dict, bias: bool = True, act_type: str = 'relu', embedding_size: int = 256, dropout: float = 0.0, use_fp16: bool = False, inplace: bool = True)¶

A module of resnet180.

参数

bn_kwargs – Dict for BN layer.
bias – Whether to use bias in module.
act_type – Nonlinear method.
embedding_size – FaceID output embedding_size.
dropout – Dropout rate.
use_fp16 – Whether to use mixed precision training.
inplace – Whether to use inplace op.

class hat.models.backbones.MMDetBackboneAdaptor(backbone: Union[torch.nn.modules.module.Module, collections.abc.Mapping])¶

Adaptor of mmdetection backbones to use in HAT.

参数

backbone – either a backbone module or a mmdet config dict
backbone. (that defines a) –
tensors. (The backbone takes a 4D image tensor and returns a list of) –

forward(x) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.MixVarGENet(net_config: List[hat.models.backbones.mixvargenet.MixVarGENetConfig], num_classes: int, bn_kwargs: dict, output_list: Union[List[int], Tuple[int]] = (), disable_quanti_input: bool = False, fc_filter: int = 1024, include_top: bool = True, flat_output: bool = True, bias: bool = False, input_channels: int = 3, input_sequence_length: int = 1, input_resize_scale: int = None)¶

Module of MixVarGENet.

参数

net_config (List[MixVarGENetConfig]) – network setting.
num_classes (int) – Num classes.
bn_kwargs (dict) – Kwargs of bn layer.
output_list (List[int]) – Output id of net_config blocks. The output of these block will be the output of this net. Set output_list as [] would export all block’s output.
disable_quanti_input (bool) – whether quanti input.
fc_filter (int) – the out_channels of the last_conv.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
bias (bool) – Whehter to use bias.
input_channels (int) – Input image channels, first conv input channels is input_channels times input_sequence_length.
input_resize_scale – This will resize the input image with the scale value.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

process_sequence_input(x: List) → torch.Tensor¶: Process sequence input with cat.

class hat.models.backbones.MobileNetV1(num_classes: int, bn_kwargs: dict, alpha: float = 1.0, bias: bool = True, dw_with_relu: bool = True, include_top: bool = True, flat_output: bool = True)¶

A module of mobilenetv1.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
alpha (float) – Alpha for mobilenetv1.
bias (bool) – Whether to use bias in module.
dw_with_relu (bool) – Whether to use relu in dw conv.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.MobileNetV2(num_classes, bn_kwargs: dict, alpha: float = 1.0, bias: bool = True, include_top: bool = True, flat_output: bool = True, use_dw_as_avgpool: bool = False)¶

A module of mobilenetv2.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
alpha (float) – Alpha for mobilenetv1.
bias (bool) – Whether to use bias in module.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
use_dw_as_avgpool (bool) – Whether to replace AvgPool with DepthWiseConv

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.ResNet18(num_classes: int, bn_kwargs: dict, bias: bool = True, include_top: bool = True, flat_output: bool = True)¶

A module of resnet18.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
bias (bool) – Whether to use bias in module.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.

class hat.models.backbones.ResNet50(num_classes: int, bn_kwargs: dict, bias: bool = True, include_top: bool = True, flat_output: bool = True)¶

A module of resnet50.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
bias (bool) – Whether to use bias in module.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.

class hat.models.backbones.ResNet50V2(num_classes: int, group_base: int, bn_kwargs: dict, bias: bool = True, extend_features: bool = False, include_top: bool = True, flat_output: bool = True)¶

A module of resnet50V2.

参数

num_classes – Num classes of output layer.
group_base – Group base for ExtendVarGNetFeatures.
bn_kwargs – Dict for BN layer.
bias – Whether to use bias in module.
extend_features – Whether to extend features.
include_top – Whether to include output layer.
flat_output – Whether to view the output tensor.

class hat.models.backbones.SwinTransformer(depth_list: List[int], num_heads: List[int], num_classes: int = 1000, patch_size: Union[int, Tuple[int, int]] = 4, in_channels: int = 3, embedding_dims: int = 96, window_size: int = 7, mlp_ratio: float = 4.0, qkv_bias: bool = True, qk_scale: Optional[float] = None, dropout_ratio: float = 0.0, attention_dropout_ratio: float = 0.0, drop_path_ratio: float = 0.0, patch_norm: bool = True, out_indices: Sequence[int] = (0, 1, 2, 3), frozen_stages: int = - 1, include_top: bool = True, flat_output: bool = True)¶

A module of swin transformer.

参数

depth_list – Depths of each Swin Transformer stage. for swin_T, the numbers could be [2, 2, 6, 2]. for swin_S, swin_B, or swin_L, the numbers could be [2, 2, 18, 2].
num_heads – Number of attention head of each stage. for swin_T or swin_S, the numbers could be [3, 6, 12, 24]. for swin_B, the numbers could be [4, 8, 16, 32]. for swin_L, the numbers could be [6, 12, 24, 48].
num_classes – Num classes of output layer.
patch_size – Patch size. Default: 4.
in_channels – Number of input image channels. Default: 3.
embedding_dims – Number of linear projection output channels. for swin_T or swin_S, the numbers could be 96. for swin_B, the number could be 128. for swin_L, the number could be 192.
window_size – Window size. Default: 7.
mlp_ratio – Ratio of mlp hidden dim to embedding dim. Default: 4.
qkv_bias – Whether to add a learnable bias to query, key, value. Default: True.
qk_scale – Override default qk scale of head_dim ** -0.5 if set.
dropout_ratio – Dropout rate. Default: 0.
attention_dropout_ratio – Attention dropout rate. Default: 0.
drop_path_ratio – Stochastic depth rate. Default: 0.0.
patch_norm – Whether to add normalization after patch embedding. Default: True.
out_indices – Output from which stages.
frozen_stages – Stages to be frozen (stop grad and set eval mode). Default: -1. -1 means not freezing any parameters.
include_top – Whether to include output layer. Default: True.
flat_output – Whether to view the output tensor. Default: True.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights(m)¶: Initialize the weights in backbone.

class hat.models.backbones.TimeSformer(img_size: int = 224, patch_size: int = 16, num_classes: int = 400, num_frames: int = 8, test_mode: bool = False, pretrained: bool = False, attention_type: str = 'divided_space_time', pretrained_model: str = '', **kwargs)¶

A module of TimeSformer.

参数

img_size – Image size. Default: 224.
patch_size – Patch size. Default: 16.
num_classes – Num classes of output layer.
num_frames – Number of frames. Default: 8.
test_mode – If true, use 3 spatial crops from clip and average the scores for these 3 crops. you should set UniformCropVideo transform in val_data_loader simultaneously.
pretrained – Wether to load pretrained model.
attention_type – Attention type, can be divided_space_time or space_only or joint_space_time.
pretrained_model – Path to pretrained model checkpoint.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.TinyVargNetV2(num_classes, bn_kwargs: dict, alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, extend_features: bool = False, disable_quanti_input: bool = False, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, input_sequence_length: int = 1, head_factor: int = 1, input_resize_scale: int = None)¶

A module of TinyVargNetv2.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
alpha (float) – Alpha for tinyvargnetv2.
group_base (int) – Group base for tinyvargnetv2.
factor (int) – Factor for channel expansion in basic block.
bias (bool) – Whether to use bias in module.
extend_features (bool) – Whether to extend features.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
input_channels (int) – Input channels of first conv.
input_sequence_length (int) – Length of input sequence.
head_factor (int) – Factor for channels expansion of stage1(mod2).
input_resize_scale (int) – Narrow_model need resize input 0.65 scale, While int_infer or visualize or eval

class hat.models.backbones.VarGDarkNet53(max_channels: int, bn_kwargs: dict, num_classes: int, include_top: bool, flat_output: bool)¶

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.VargConvNet(num_classes, bn_kwargs: dict, channels_list: list, repeats: list, group_list: int, factor_list: int, out_channels: int = 1024, bias: bool = True, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, deep_stem: bool = True)¶

A module of vargconvnet.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
channels_list (list) – List for output channels
repeats (list) – Depth of each stage.
group_list (list) – Group of each stage.
factor_list (list) – Factor for each stage.
out_channels (int) – Output channels.
bias (bool) – Whether to use bias in module.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
input_channels (int) – Input channels of first conv.
deep_stem (bool) – Whether use deep stem.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.VargNASNet(net_config: List[hat.models.backbones.vargnasnet.VargNASBlockConfig], num_classes: int, bn_kwargs: dict, disable_quanti_input: bool = False, fc_filter: int = 1024, include_top: bool = True, flat_output: bool = True, bias: bool = False, input_channels: int = 3, input_sequence_length: int = 1)¶

Module of VargNASNet.

参数

net_config (List[VargNASBlockConfig]) – network setting.
num_classes (int) – Num classes.
bn_kwargs (dict) – Kwargs of bn layer.
disable_quanti_input (bool) – whether quanti input.
fc_filter (int) – the out_channels of the last_conv.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
bias (bool) – Whehter to use bias.
input_channels (int) – Input image channels, first conv input channels is input_channels times input_sequence_length.
input_sequence_length (int) – Input sequence length, used in multiple input images case.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

process_sequence_input(x: List) → torch.Tensor¶: Process sequence input with cat.

class hat.models.backbones.VargNetV2(num_classes, bn_kwargs: dict, model_type: str = 'VargNetV2', alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, extend_features: bool = False, disable_quanti_input: bool = False, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, input_sequence_length: int = 1, head_factor: int = 1, input_resize_scale: int = None)¶

A module of vargnetv2.

参数

num_classes (int) – Num classes of output layer.
bn_kwargs (dict) – Dict for BN layer.
model_type (str) – Choose to use VargNetV2 or TinyVargNetV2.
alpha (float) – Alpha for vargnetv2.
group_base (int) – Group base for vargnetv2.
factor (int) – Factor for channel expansion in basic block.
bias (bool) – Whether to use bias in module.
extend_features (bool) – Whether to extend features.
include_top (bool) – Whether to include output layer.
flat_output (bool) – Whether to view the output tensor.
input_channels (int) – Input channels of first conv.
input_sequence_length (int) – Length of input sequence.
head_factor (int) – Factor for channels expansion of stage1(mod2).
input_resize_scale (int) – Narrow_model need resize input 0.65 scale, While int_infer or visualize or eval

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

process_sequence_input(x: List) → torch.Tensor¶: Process sequence input with cat.

class hat.models.backbones.VargNetV2ISP(num_classes: int, bn_kwargs: dict, model_type: str = 'VargNetV2', alpha: float = 1.0, group_base: int = 8, factor: int = 2, bias: bool = True, extend_features: bool = False, disable_quanti_input: bool = False, include_top: bool = True, flat_output: bool = True, input_channels: int = 3, input_sequence_length: int = 1, head_factor: int = 1, input_resize_scale: int = None)¶

A module of vargnetv2 with NeuralISP module.

参数

num_classes – Num classes of output layer.
bn_kwargs – Dict for BN layer.
model_type – Choose to use VargNetV2 or TinyVargNetV2.
alpha – Alpha for vargnetv2.
group_base – Group base for vargnetv2.
factor – Factor for channel expansion in basic block.
bias – Whether to use bias in module.
extend_features – Whether to extend features.
include_top – Whether to include output layer.
flat_output – Whether to view the output tensor.
input_channels – Input channels of first conv.
input_sequence_length – Length of input sequence.
head_factor – Factor for channels expansion of stage1(mod2).
input_resize_scale – Narrow_model need resize input 0.65 scale, While int_infer or visualize or eval

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.VargNetV2Stage2631(bn_kwargs, multiplier=0.5, group_base=8, last_channels=1024, num_classes=1000, stages=(1, 2, 3, 4, 5, 6), use_bias=False, input_channels: int = 3, include_top=False, extend_features=False, flat_output=True)¶

VargNetV2 with 2631 stage setting.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.WorkConditionResNet(layers, output_dim, heads, channel_list, bn_kwargs, input_resolution=(224, 224), use_attnpool=False)¶

A ResNet class that is similar to torchvision’s but contains the following changes.

There are now 3 “stem” convolutions as opposed to 1, with an average pool instead of a max pool.
Performs anti-aliasing strided convolutions, where an avgpool is prepended to convolutions with stride > 1
The final pooling layer is a QKV attention instead of an average pool

参数

layers – Number of layers for workconditionresnet.
output_dim – Out dimension of attention module, useful when use_attnpool is True.
heads – Number of attention head, useful when use_attnpool is True.
channel_list – Channel dimension of each stage.
bn_kwargs – Dict for BN layer.
input_resolution – Input size.
use_attnpool – Whether to use attention module.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.backbones.ZeroPad2DPatcher(backbone: torch.nn.modules.module.Module, input_padding: Tuple[int])¶

ZeroPad2D Patcher.

参数

backbone – Backbone module.
input_padding – Input padding, in (w_left, w_right, h_top, h_bottom) format. # noqa E501

hat.models.backbones.efficientnet(model_type, **kwargs)¶: A module of efficientnet.

hat.models.backbones.efficientnet_lite(model_type, **kwargs)¶: A module of efficientnet_lite.

hat.models.backbones.get_mixvargenet_stride2channels(net_config: Union[List[List[hat.models.backbones.mixvargenet.MixVarGENetConfig]], List[List[hat.models.backbones.mixvargenet.IdentityConfig]]], strides: Optional[List[int]] = None) → Dict¶

Get mixvargenet stride to channel dict with giving channels and strides.

参数

net_config – network setting
strides – stride list corresponding to channels.

Returns: strides2channels: a stride to channel dict.

hat.models.backbones.get_vargnetv2_stride2channels(alpha: float, channels: Optional[List[int]] = None, strides: Optional[List[int]] = None) → Dict¶

Get vargnet v2 stride to channel dict with giving channels and strides.

参数

alpha – channel multipler.
channels – base channel of each stride.
strides – stride list corresponding to channels.

Returns: strides2channels: a stride to channel dict.

class hat.models.losses.AfdetFocalLoss¶

forward(pred: torch.Tensor, target: torch.Tensor, ignore_mask: Optional[torch.Tensor] = None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.BoundaryLoss(theta0: float = 3, theta: float = 5)¶

forward(pred: torch.Tensor, gt: torch.Tensor) → torch.Tensor¶

Forward.

参数

pred – the output from model (before softmax) shape (N, C, H, W).
gt – ground truth map shape (N, H, w).

返回

boundary loss, averaged over mini-bathc

one_hot(label: torch.Tensor, n_classes: int, requires_grad=True) → torch.Tensor¶

Return one Hot Label.

参数

label – origin label with type class.
n_classes – num classes.
requires_grad – requires grad or not.

返回

one hot label from ori label.

返回类型

one_hot_label

class hat.models.losses.CEWithLabelSmooth(smooth_alpha=0.1)¶

The losses of cross-entropy with label smooth.

参数: smooth_alpha (float) – Alpha of label smooth.

forward(input, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.CEWithWeightMap(weight_min=0.5, **kwargs)¶

Crossentropy loss with image-specfic class weighted map within batch.

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.CrossEntropyLoss(use_sigmoid: bool = False, reduction: str = 'mean', class_weight: Optional[List[float]] = None, loss_weight: float = 1.0, ignore_index: int = - 1, loss_name: Optional[str] = None, auto_class_weight: Optional[bool] = False, weight_min: Optional[float] = None, weight_noobj: Optional[float] = None, num_class: int = 0)¶

Calculate cross entropy loss of multi stride output.

参数

use_sigmoid (bool) – Whether logits tensor is converted to probability through sigmoid, Defaults to False. If True, use F.binary_cross_entropy_with_logits. If False, use F.cross_entropy.
reduction (str) – The method used to reduce the loss. Options are [none, mean, sum].
class_weight (list[float]) – Weight of each class. Defaults is None.
loss_weight (float) – Global weight of loss. Defaults is 1.
ignore_index (int) – Only works when using cross_entropy.
loss_name (str) – The key of loss in return dict. If None, return loss directly.

返回

cross entropy loss

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.ElementwiseL1HingeLoss(loss_bound_l1: float = 0.0, pos_label: int = 1, neg_label: int = 0, norm_type: str = 'positive_label_elt', loss_weight: float = 1.0, reduction: Optional[str] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

Elementwise L1 Hinge Loss.

参数

loss_bound_l1 – Upper bound of l1 loss value in each entry.
pos_label – Value in label that represents positive entries.
neg_label – Value in label that represents negative entries.
norm_type – Normalization method, can be “positive_label_elt”, in which normalization factor is the number of positive elements, or “positive_loss_elt”, the number of positive losses.
loss_weight – Global weight of loss. Defaults is 1.0.
reduction – The method used to reduce the loss. Options are [none, mean, sum]. By default and recommended to be ‘mean’.
hard_neg_mining_cfg – hard negative mining config. Please refer to LossHardNegativeMining.

返回

loss value

返回类型

torch.Tensor

class hat.models.losses.ElementwiseL2HingeLoss(loss_bound_l1: float = 0.0, pos_label: int = 1, neg_label: int = 0, norm_type: str = 'positive_label_elt', loss_weight: float = 1.0, reduction: Optional[str] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

Elementwise L2 Hinge Loss.

参数

loss_bound_l1 – Upper bound of l1 loss value in each entry.
pos_label – Value in label that represents positive entries.
neg_label – Value in label that represents negative entries.
norm_type – Normalization method, can be “positive_label_elt”, in which normalization factor is the number of positive elements, or “positive_loss_elt”, the number of positive losses.
loss_weight – Global weight of loss. Defaults is 1.0.
reduction – The method used to reduce the loss. Options are [none, mean, sum]. By default and recommended to be ‘mean’.
hard_neg_mining_cfg – hard negative mining config. Please refer to LossHardNegativeMining.

返回

loss value

返回类型

torch.Tensor

class hat.models.losses.FocalLoss(loss_name, num_classes, alpha=0.25, gamma=2.0, loss_weight=1.0, eps=1e-12, reduction='mean')¶

Sigmoid focal loss.

参数

loss_name (str) – The key of loss in return dict.
num_classes (int) – Num_classes including background, C+1, C is number of foreground categories.
alpha (float) – A weighting factor for pos-sample, (1-alpha) is for neg-sample.
gamma (float) – Gamma used in focal loss to compress the contribution of easy examples.
loss_weight (float) – Global weight of loss. Defaults is 1.0.
eps (float) – A small value to avoid zero denominator.
reduction (str) – The method used to reduce the loss. Options are [none, mean, sum].

返回

A dict containing the calculated loss, the key of loss is loss_name.

返回类型

dict

forward(pred, target, weight=None, avg_factor=None, points_per_strides=None, valid_classes_list=None)¶

Forward method.

参数

pred (Tensor) – Cls pred, with shape(N, C), C is num_classes of foreground.
target (Tensor) – Cls target, with shape(N,), values in [0, C-1] represent the foreground, C or negative value represent the background.
weight (Tensor) – The weight of loss for each prediction. Default is None.
avg_factor (float) – Normalized factor.

class hat.models.losses.FocalLossV2(alpha: float = 0.25, gamma: float = 2.0, eps: float = 1e-12, from_logits: bool = True, reduction: str = 'mean')¶

Focal Loss.

参数

alpha – A weighting factor for pos-sample, (1-alpha) is for neg-sample.
gamma – Gamma used in focal loss to compress the contribution of easy examples.
eps – A small value to avoid zero denominator.
from_logits – Whether the input prediction is logits (before sigmoid).
reduction – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None)¶

Forward method.

参数

pred – cls pred, with shape (B, N, C), C is num_classes of foreground.
target – cls target, with shape (B, N, C), C is num_classes of foreground.
weight – The weight of loss for each prediction. It is mainly used to filter the ignored box. Default is None.
avg_factor – Normalized factor.

class hat.models.losses.GIoULoss(loss_name, loss_weight=1.0, eps=1e-06, reduction='mean')¶

Generalized Intersection over Union Loss.

参数

loss_name (str) – The key of loss in return dict.
loss_weight (float) – Global weight of loss. Defaults is 1.0.
eps (float) – A small value to avoid zero denominator.
reduction (str) – The method used to reduce the loss. Options are [none, mean, sum].

返回

A dict containing the calculated loss, the key of loss is loss_name.

返回类型

dict

forward(pred, target, weight=None, avg_factor=None)¶

Forward method.

参数

pred (torch.Tensor) – Predicted bboxes of format (x1, y1, x2, y2), represent upper-left and lower-right point, with shape(N, 4).
target (torch.Tensor) – Corresponding gt_boxes, the same shape as pred.
weight (torch.Tensor) – Element-wise weight loss weight, with shape(N,).
avg_factor (float) – Average factor that is used to average the loss.

class hat.models.losses.GaussianFocalLoss(alpha: float = 2.0, gamma: float = 4.0, loss_weight: float = 1.0)¶

Guassian focal loss.

参数

alpha – A weighting factor for positive sample.
gamma – Used in focal loss to balance contribution of easy examples and hard examples.
loss_weight – Weight factor for guassian focal loss.

forward(pred, target, weight=None, grad_tensor=None)¶

Forward function.

参数

pred – The prediction.
target – The learning target of the prediction in gaussian distribution.
weight – The weight for class.

class hat.models.losses.L1Loss(beta: float = 1.0, reduction: str = 'mean', loss_weight: Optional[float] = None)¶

Smooth L1 Loss.

参数

beta – The threshold in the piecewise function. Defaults to 1.0.
reduction – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.
loss_weight – Loss weight.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None)¶

Forward function.

参数

pred – The prediction.
target – The learning target of the prediction.
weight – The weight of loss for each prediction. Defaults to None.
avg_factor – Normalized factor.

class hat.models.losses.LidarFastFocalLoss¶

Focal loss, exactly the same as the CornerNet version.

Faster and costs much less memory.

forward(out: torch.Tensor, target: torch.Tensor, ind: torch.Tensor, mask: torch.Tensor, cat: torch.Tensor) → float¶

Forward calculation.

参数

out (torch.Tensor) – B x C x H x W tensor for output of the model.
target (torch.Tensor) – B x C x H x W tensor for training target.
ind (torch.Tensor) – B x M index tensor for tensor gathering.
mask (torch.Tensor) – B x M mask tensor for tensor masking.
cat (torch.Tensor) – B x M tensor indicating category id for peaks.

返回

loss value.

返回类型

[float]

class hat.models.losses.LidarRegLoss¶

Regression loss for an output tensor.

forward(output: torch.Tensor, mask: torch.Tensor, ind: torch.Tensor, target: torch.Tensor)¶

Forward calculation.

参数

output (torch.Tensor) – (batch x dim x h x w)
mask (torch.Tensor) – (batch x max_objects)
ind (torch.Tensor) – (batch x max_objects)
target (torch.Tensor) – (batch x max_objects x dim)

class hat.models.losses.LidarSmoothL1RegLoss¶

Regression loss for an output tensor.

forward(output: torch.Tensor, mask: torch.Tensor, ind: torch.Tensor, target: torch.Tensor)¶

Forward calculation.

参数

output (torch.Tensor) – B x C x H x W tensor for output of model.
mask (torch.Tensor) – B x M mask tensor for tensor masking.
ind (torch.Tensor) – B x M index tensor for tensor gathering.
target (torch.Tensor) – B x C x H x W tensor for training target.

返回

loss value.

返回类型

[float]

class hat.models.losses.LnNormLoss(norm_order: int = 2, epsilon: float = 0.0, power: float = 1.0, reduction: Optional[str] = None, loss_weight: Optional[float] = None)¶

LnNorm loss.

Different from torch.nn.L1Loss, the loss function uses Ln norm to calculate the distance of two feature maps.

参数

norm_order – The order of norm.
epsilon – A small constant for finetune.
power – A power num of norm + epsilon of loss.
reduction – Reduction mode.
loss_weight – If present, it will be used to weight the output.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None) → torch.Tensor¶

Forward method.

参数

pred (Tensor) – Optical flow pred, with shape(N, 2, H, W).
target (Tensor) – Optical flow target, with shape(N, 2, H, W),
sampling. (which obtained by ground truth) –
weight (Tensor) – The weight of loss for each prediction. Default is None.
avg_factor (float) – Normalized factor.

class hat.models.losses.MixSegLoss(losses: List[torch.nn.modules.module.Module], losses_weight: List[float] = None)¶

Calculate multi-losses with same prediction and target.

参数

losses – List of losses with the same input pred and target.
losses_weight – List of weights used for loss calculation. Default: None

forward(pred, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.MixSegLossMultipreds(losses: List[torch.nn.modules.module.Module], losses_weight: List[float] = None)¶

Calculate multi-losses with multi-preds and correspondence targets.

参数

losses – List of losses with different prediction and target.
losses_weight – List of weights used for loss calculation. Default: None

forward(pred, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.MultiStrideLosses(num_classes: int, out_strides: List[int], loss: torch.nn.modules.module.Module, loss_weights: Optional[List[float]] = None)¶

Multiple Stride Losses.

Apply the same loss function with different loss weights to multiple outputs.

参数

num_classes – Number of classes.
out_strides – strides of output feature maps
loss – Loss module.
loss_weights – Loss weight.

forward(preds: List[torch.Tensor], targets: List[torch.Tensor]) → Dict[str, torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.PointPillarsLoss(num_classes, loss_cls: Optional[torch.nn.modules.module.Module] = None, loss_bbox: Optional[torch.nn.modules.module.Module] = None, loss_dir: Optional[torch.nn.modules.module.Module] = None, pos_cls_weight: float = 1.0, neg_cls_weight: float = 1.0, num_direction_bins: int = 2, direction_offset: float = 0.0)¶

forward(anchors, box_cls_labels, reg_targets, box_preds, cls_preds, dir_preds)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SegLoss(loss: List[torch.nn.modules.module.Module])¶

Segmentation loss wrapper.

参数: loss (dict) – loss config.

注解

This class is not universe. Make sure you know this class limit before using it.

forward(pred: Any, target: List[Dict]) → Dict¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SigmoidFocalLoss(gamma=2.0, alpha=0.25, reduction='mean', loss_weight=1.0)¶

Sigmoid focal cross entropy loss.

Focal loss down-weights well classified examples and focusses on the hard examples. See https://arxiv.org/pdf/1708.02002.pdf for the loss definition.

参数

gamma – exponent of the modulating factor.
alpha – optional alpha weighting factor to balance positives vs negatives.
all_zero_negative – bool. if True, will treat all zero as background. else, will treat first label as background. only affect alpha.

forward(prediction_tensor, target_tensor, weights=None, class_indices=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SmoothL1Loss(beta: float = 1.0, reduction: str = 'mean', loss_weight: Optional[float] = None, hard_neg_mining_cfg: Optional[Dict] = None)¶

Smooth L1 Loss.

参数

beta – The threshold in the piecewise function. Defaults to 1.0.
reduction – The method to reduce the loss. Options are “none”, “mean” and “sum”. Defaults to “mean”.
loss_weight – Loss weight.
hard_neg_mining_cfg – Hard negative mining cfg.

forward(pred: torch.Tensor, target: torch.Tensor, weight: Optional[torch.Tensor] = None, avg_factor: Optional[Union[float, torch.Tensor]] = None)¶

Forward function.

参数

pred – The prediction.
target – The learning target of the prediction.
weight – The weight of loss for each prediction. Defaults to None.
avg_factor – Normalized factor.

class hat.models.losses.SoftTargetCrossEntropy(loss_name=None)¶

The losses of cross-entropy with soft target.

参数: loss_name (str) – The name of returned losses.

forward(input, target)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.SoftmaxFocalLoss(loss_name: str, num_classes: int, alpha: float = 0.25, gamma: float = 2.0, reduction: str = 'mean', weight: Union[float, Sequence] = 1.0)¶

Focal Loss.

参数

loss_name (str) – The key of loss in return dict.
num_classes (int) – Class number.
alpha (float, optional) – Alpha. Defaults to 0.25.
gamma (float, optional) – Gamma. Defaults to 2.0.
reduction (str, optional) – Specifies the reduction to apply to the output: 'mean' | 'sum'. Defaults to 'mean'.
weight (Union[float, Sequence], optional) – Weight to be applied to the loss of each input. Defaults to 1.0.

forward(logits, labels)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.WeightedSmoothL1Loss(sigma=3.0, reduction='mean', code_weights=None, codewise=True, loss_weight=1.0)¶

Smooth L1 localization loss function.

forward(prediction_tensor, target_tensor, weights=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.WeightedSoftmaxClassificationLoss(logit_scale=1.0, loss_weight=1.0, name='')¶

Softmax loss function.

参数: logit_scale – When this value is high, the prediction is “diffused” and when this value is low, the prediction is made peakier. (default 1.0)

forward(prediction_tensor, target_tensor, weights)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.WeightedSquaredHingeLoss(reduction: str, loss_weight: float = 1.0, weight_low_thr: float = 0.1, weight_high_thr: float = 1.0, hard_neg_mining_cfg: Optional[Dict] = None)¶

Weighted Squared ElementWiseHingeLoss.

参数

reduction (str) – Possible values are {‘mean’, ‘sum’, ‘sum_mean’, ‘none’}
loss_weight (float) – by default 1.0
weight_low_thr (float) – Lower threshold for elementwise weight, by default 0.1
weight_high_thr (float) – Upper threshold for pixel-wise weight, by default 1.0
hard_neg_mining_cfg (dict) – Hard negative mining cfg

forward(pred, target, weight=None, avg_factor=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.losses.YOLOV3Loss(num_classes: int, anchors: list, strides: list, ignore_thresh: float, loss_xy: dict, loss_wh: dict, loss_conf: dict, loss_cls: dict, lambda_loss: list)¶

The loss module of YOLOv3.

参数

num_classes (int) – Num classes of class branch.
anchors (list) – The anchors of YOLOv3.
strides (list) – The strides of feature maps.
ignore_thresh (float) – Ignore thresh of target.
loss_xy (dict) – Losses of xy.
loss_wh (dict) – Losses of wh.
loss_conf (dict) – Losses of conf.
loss_cls (dict) – Losses of cls.
lambda_loss (list) – The list of weighted losses.

forward(input, target=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.BiFPN(in_strides, out_strides, stride2channels, out_channels, num_outs, stack=3, start_level=0, end_level=- 1, fpn_name='bifpn_sum')¶

Weighted Bi-directional Feature Pyramid Network(BiFPN).

This is an implementation of - EfficientDet: Scalable and Efficient Object Detection (https://arxiv.org/abs/1911.09070)

参数

in_strides (list[int]) – Stride of input feature map
out_strides (int) – Stride of output feature map
stride2channels (dict) – The key:value is stride:channel , the channles have been multipified by alpha
out_channels (int|dict) – Channel number of output layer, the key:value is stride:channel.
num_outs (int) – Number of BifpnLayer’s input, the value is must 5, because the bifpn layer is fixed
stack (int) – Number of BifpnLayer
start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, means the last level.
fpn_name (str) – the value is mutst between with ‘bifpn_sum’, ‘bifpn_fa’

forward(inputs)¶

Forward features.

参数: inputs (list[tensor]) – Input tensors

Returns (list[tensor]): Output tensors

class hat.models.necks.DwUnet(base_channels: int, bn_kwargs: Dict = None, act_type: torch.nn.modules.module.Module = <class 'torch.nn.modules.activation.ReLU'>, use_deconv: bool = False, dw_with_act: bool = False, output_scales: Sequence = (4, 8, 16, 32, 64))¶

Unet segmentation neck structure.

Built with separable convolution layers.

参数

base_channels (int) – Output channel number of the output layer of scale 1.
bn_kwargs (Dict, optional) – Keyword arguments for BN layer. Defaults to {}.
use_deconv (bool, optional) – Whether user deconv for upsampling layer. Defaults to False.
dw_with_act (bool, optional) – Whether user relu after the depthwise conv in SeparableConv. Defaults to False.
output_scales (Sequence, optional) – The scale of each output layer. Defaults to (4, 8, 16, 32, 64).

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.FPN(in_strides: List[int], in_channels: List[int], out_strides: List[int], out_channels: List[int], fix_out_channel: Optional[int] = None, bn_kwargs: Optional[Dict] = None)¶

forward(features: List[torch.Tensor]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.FastSCNNNeck(in_channels: List[int], feat_channels: List[int], indexes: List[int], bn_kwargs: Optional[Dict] = None, scale_factor: int = 4)¶

Upper neck module for segmentation.

参数

in_channels – channels of each input feature map
feat_channels – channels for featture maps.
indexes – indexes of inputs.
bn_kwargs – Dict for Bn layer.
scale_factor – scale factor for fusion.

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.PAFPN(in_channels, out_channels, out_strides, num_outs, start_level=0, end_level=- 1, add_extra_convs=False, relu_before_extra_convs=False)¶

Path Aggregation Network for Instance Segmentation.

This is an implementation of the PAFPN in Path Aggregation Network <https://arxiv.org/abs/1803.01534>.

参数

in_channels (List[int]) – Number of input channels per scale.
out_channels (int | Dict) – Output channels of each scale
out_strides (List[int]) – Stride of output feature map
num_outs (int) – Number of output scales.
start_level (int) – Index of the start input backbone level used to build the feature pyramid. Default: 0.
end_level (int) – Index of the end input backbone level (exclusive) to build the feature pyramid. Default: -1, which means the last level.
add_extra_convs (bool | str) –
If bool, it decides whether to add conv layers on top of the original feature maps. Default to False. If True, it is equivalent to add_extra_convs=’on_input’. If str, it specifies the source feature map of the extra convs. Only the following options are allowed:
- ’on_input’: Last feat map of neck inputs (i.e. backbone feature).
- ’on_lateral’: Last feature map after lateral convs.
- ’on_output’: The last output feature map after fpn convs.
relu_before_extra_convs (bool) – Whether to apply relu before the extra conv. Default: False.

forward(inputs)¶: Forward function.

class hat.models.necks.RetinaNetFPN(in_strides: List[int], in_channels: List[int], out_strides: List[int], out_channels: List[int], fix_out_channel: Optional[int] = None)¶

FPN for RetinaNet.

The difference with FPN is that RetinaNetFPN has two extra convs correspond to stride 64 and stride 128 except the lateral convs.

参数

in_strides (list) – strides of each input feature map
in_channels (list) – channels of each input feature map, the length of in_channels should be equal to in_strides
out_strides (list) – strides of each output feature map, should be a subset of in_strides, and continuous (any subsequence of 2, 4, 8, 16, 32, 64 …). The largest stride in in_strides and out_strides should be equal
out_channels (list) – channels of each output feature maps the length of out_channels should be equal to out_strides
fix_out_channel (int, optional) – if set, there will be a 1x1 conv following each output feature map so that each final output has fix_out_channel channels

forward(features: List[torch.Tensor]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()¶: Initialize the weights of FPN module.

class hat.models.necks.SequentialBottleNeck(layer_nums: List[int], ds_layer_strides: List[int], ds_num_filters: List[int], us_layer_strides: List[int], us_num_filters: List[int], num_input_features: int, bn_kwargs: Optional[Dict[str, Any]] = None, quantize: bool = False, return_stride_features: bool = True, use_relu6: bool = True, dilation_kernel: int = 1, conv_kernel: int = 3, ds_maxpool: bool = False, ds_avgpool: bool = False, use_iden: bool = False, use_scnet: bool = False, use_res2net: bool = False, use_repvgg: bool = False, deploy: bool = False, use_tconv: bool = False, use_secnet: bool = False)¶

forward(x: torch.Tensor)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.Unet(in_strides: List[int], out_strides: List[int], stride2channels: Dict[int, int], out_stride2channels: Dict[int, int] = None, factor: int = 2, use_bias: bool = False, bn_kwargs: Optional[Dict] = None, group_base: int = 8)¶

Unet neck module.

参数

in_strides – contains the strides of feature maps from backbone.
out_strides – contains the strides of feature maps the neck output.
out_stride2channels – output stride to channel dict.
stride2channels – input stride to channel dict.

forward(features)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.YOLOV3Neck(backbone_idx: list, in_channels_list: list, out_channels_list: list, bn_kwargs: dict, bias: bool = True)¶

Necks module of yolov3.

参数

backbone_idx (list) – Index of backbone output for necks.
in_channels_list (list) – List of input channels.
out_channels_list (list) – List of output channels.
bn_kwargs (dict) – Config dict for BN layer.
bias (bool) – Whether to use bias in module.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.necks.YoloGroupNeck(backbone_idx: list, in_channels_list: list, out_channels_list: list, bn_kwargs: dict, bias: bool = True, head_group: bool = True)¶

Necks module of yolov3.

参数

backbone_idx (list) – Index of backbone output for necks.
in_channels_list (list) – List of input channels.
out_channels_list (list) – List of output channels.
bn_kwargs (dict) – Config dict for BN layer.
bias (bool) – Whether to use bias in module.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.Classifier(backbone, losses=None)¶

The basic structure of classifier.

参数

backbone (torch.nn.Module) – Backbone module.
losses (torch.nn.Module) – Losses module.

forward(data)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.EncoderDecoder(backbone: torch.nn.modules.module.Module, decode_head: torch.nn.modules.module.Module, target: object = None, loss: Optional[torch.nn.modules.module.Module] = None, neck: Optional[torch.nn.modules.module.Module] = None, auxiliary_heads: Optional[List[Dict]] = None, decode: object = None, with_target: Optional[torch.nn.modules.module.Module] = False)¶

The basic structure of encoder decoder.

参数

backbone – Backbone module.
decode_head – Decode head module.
target – Target module for decode head. Default: None.
loss – Loss module for decode head. Default: None.
neck – Neck module. Default: None.
auxiliary_heads – List of auxiliary head modules which contains of “head”, “target”, “loss”. Default: None.
decode – decode. Defualt: None.
with_target – Whether return target during inference.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.GraphModel(nodes: Dict[str, Union[dict, Callable]], inputs: Dict[str, Any], topology_builder: Callable[[Dict[str, Callable], Dict[str, hatbc.workflow.proxy.Variable]], Dict[str, hatbc.workflow.proxy.WorkflowVariable]], lazy_forward: Optional[bool] = True, output_names2flat_conditions: Dict[str, Callable[Any, bool]] = None)¶

A Module contains a graph to represent computational topology.

Each graph consist of multiple nodes, and each node wraps an operator. We make nn.Module instance, e.g. backbone, neck, head, as one of the graph nodes.

The purpose we introduce a graph object:

In some cases, we just want to forward part (not all) of the task specific heads of a multitask model, which may have some parameters shared by multiple tasks.

2 steps to accomplish this purpose:

Get a sub graph by specific a subset of graph output nodes.
Forward it, which means to forward part of the GraphModel.

An example:

# make nn.Module traceable
GraphTracer.register_class_traceable_under_scope(torch.nn.Module)
GraphTracer.register_basic_types(torch.Tensor)

def build_topo(nodes, inputs):
    #
    # Computational Topology:
    #
    # (img) -> backbone -> neck -> head1 -> (task1_out)
    #                         \ -> head2 -> (task2_out)
    #                         \ -> head3 -> (task3_out)
    #
    name2out = OrderedDict()
    bone_feats = nodes['backbone'](inputs['img'])
    neck_feats = nodes['neck'](bone_feats)
    name2out['task1_out'] = nodes['head1'](neck_feats)
    name2out['task2_out'] = nodes['head2'](neck_feats)
    name2out['task3_out'] = nodes['head3'](neck_feats)
    return name2out

gm = GraphModel(
    nodes=dict(
        backbone=ResNet(...),
        neck=FPN(...),
        head1=FCOSHead(...),
        head2=FCOSHead(...),
        head3=FCOSHead(...),
    ),
    inputs=dict(img=None),
    topology_builder=build_topo,
    lazy_forward=True,
)

batch = dict(img=torch.randn((1, 3, 600, 800)))

# Case1, forward the whole model (full graph). Topology:
#
# (img) -> backbone -> neck -> head1 -> (task1_out)
#                         \ -> head2 -> (task2_out)
#                         \ -> head3 -> (task3_out)
#
gm(batch, out_names=None)

# Case2, forward task1 (sub graph). Topology:
#
# (img) -> backbone -> neck -> head1 -> (task1_out)
#
gm(batch, out_names='task1_out')

# Case3, forward task2 and task3 (sub graph). Topology:
#
# (img) -> backbone -> neck -> head2 -> (task2_out)
#                         \ -> head3 -> (task3_out)
#
gm(batch, out_names=['task2_out', 'task3_out'])

参数

nodes – Graph nodes, can be any callable as well as traceable (by workflow.GraphTracer) object.
inputs – {input name: value} pairs, used to build the graph’s input variables, i.e. placeholders. Value can be None (unset) when lazy_forward is True.
topology_builder –
A callable func, with already build nodes and inputs as input arguments, used to build computational topology under with GraphTracer() scope.

It returns an OrderedDict object to specific the ordered graph outputs (a collection of WorkflowVariable objects, and the WorkflowVariable.idata attr contains model output Tensors).

This func will be called only once to get traced graph before formal training, i.e. the traced graph is a static graph.
lazy_forward –
Whether forward graph nodes when topology building. If True, means not immediately forward, delay to GraphModel.forward() time when topology building is done. At this case, inputs can be (str, None) pairs, i.e. you don’t have to provide input values, just provide input names to create placeholders.

If False, means immediately forward, i.e. forward to build a graph, then you achieve node outputs immediately, of course, you have to provide input values. This is useful for debugging the topology before formal training.

注解

No matter True or False, finally a static traced graph is build, i.e. will not change among training.

Refer GraphTracer.imperative for more.
output_names2flat_conditions – Dict of function with values or (values, key) as input, return True/False means whether flat this values or not, used when flatting graph output results. See to_flat_ordered_dict for more.

forward(inputs: Dict[str, Any], out_names: Optional[Union[str, Sequence[str]]] = None) → collections.namedtuple¶

Provide graph output names and necessary input data to forward full or sub graph.

参数

out_names –
Graph output names, should be a subset of self._output_names , i.e. should keep accordance with the keys of self._name2output which is return from self.topology_builder .

If None, means to forward the whole graph.

If not None, we will use it to get a sub graph then forward.
inputs –
A dict of (input name, data), should be a subset of self.inputs , providing necessary input data to forward the full or sub graph.

注解

Only provide reliable inputs used in graph forward, extra inputs will cause error.

返回

A namedtuple consists of output Tensors in order of out_names .

注解

Return dict is forbidden by torch.jit.trace(), so return namedtuple, or tuple (list is not recommended).

Example:

gm = GraphModel(nodes, inputs, topology_builder)
inputs = dict(img=torch.randn((1, 3, 600, 800)))
results = gm(inputs, out_names=['person', 'vehicle'])

print(results._fields)
# (
#     'person_cls_pred_0_', 'person_cls_pred_1_', ... ,
#     'vehicle_cls_pred_0_', 'vehicle_cls_pred_1_', ... ,
#     ...
# )

results[0] == results.person_cls_pred_0_
# True

results[1] == results.person_cls_pred_1_
# True

property graph¶: Full graph which represents GraphModel’s computational topology.

named_buffers_by_outname(out_names: Tuple[str], prefix: str = '') → Tuple[str, Any]¶: Get all named buffers that contained by sub-graph of outname.

named_modules_by_outname(out_names: Tuple[str], prefix: str = '') → Tuple[str, Any]¶: Get all named modules that contained by sub-graph of outname.

named_parameters_by_outname(out_names: Tuple[str], prefix: str = '') → Tuple[str, Any]¶: Get all named parameters that contained by sub-graph of outname.

property output_names¶: Names of graph output variables.

class hat.models.structures.Segmentor(backbone, neck, head, losses=None)¶

The basic structure of segmentor.

参数

backbone (torch.nn.Module) – Backbone module.
neck (torch.nn.Module) – Neck module.
head (torch.nn.Module) – Head module.
losses (torch.nn.Module) – Losses module.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.SegmentorV2(backbone: torch.nn.modules.module.Module, neck: torch.nn.modules.module.Module, head: torch.nn.modules.module.Module, target: Optional[torch.nn.modules.module.Module] = None, loss: Optional[torch.nn.modules.module.Module] = None, desc: Optional[torch.nn.modules.module.Module] = None, postprocess: Optional[torch.nn.modules.module.Module] = None)¶

The basic structure of segmentor.

参数

backbone – Backbone module.
neck – Neck module.
head – Head module.
loss – Loss module.
desc – Desc module
postprocess – Postprocess module.

forward(data: dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.FCOS(backbone: torch.nn.modules.module.Module, neck: torch.nn.modules.module.Module = None, head: torch.nn.modules.module.Module = None, targets: torch.nn.modules.module.Module = None, post_process: torch.nn.modules.module.Module = None, loss_cls: torch.nn.modules.module.Module = None, loss_reg: torch.nn.modules.module.Module = None, loss_centerness: torch.nn.modules.module.Module = None)¶

The basic structure of fcos.

参数

backbone – Backbone module.
neck – Neck module.
head – Head module.
targets – Target module.
loss_cls – Classification loss module.
loss_reg – Regiression loss module.
loss_centerness – Centerness loss module
postprocess – Postprocess module.

extract_feat(img)¶: Directly extract features from the backbone + neck.

forward(data: Dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.PointPillarsDetector(feature_map_shape: List[int], pre_process: Optional[torch.nn.modules.module.Module] = None, reader: Optional[torch.nn.modules.module.Module] = None, backbone: Optional[torch.nn.modules.module.Module] = None, neck: Optional[torch.nn.modules.module.Module] = None, head: Optional[torch.nn.modules.module.Module] = None, anchor_generator: Optional[torch.nn.modules.module.Module] = None, targets: Optional[torch.nn.modules.module.Module] = None, loss: Optional[torch.nn.modules.module.Module] = None, postprocess: Optional[torch.nn.modules.module.Module] = None, quant_begin_neck: bool = False, is_deploy: bool = False)¶

The basic structure of PointPillars.

参数

feature_map_shape – Feature map shape, in (W, H, 1) format.
out_size_factor – Downsample factor.
reader (torch.nn.Module) – Reader module.
backbone (torch.nn.Module) – Backbone module.
neck (torch.nn.Module) – Neck module.
head (torch.nn.Module) – Head module.
anchor_generator (torch.nn.Module) – Anchor generator module.
targets (torch.nn.Module) – Target generator module.
loss (torch.nn.Module) – Loss module.
postprocess (torch.nn.Module) – Postprocess module.

forward(example)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.RetinaNet(backbone: torch.nn.modules.module.Module, neck: Optional[torch.nn.modules.module.Module] = None, head: Optional[torch.nn.modules.module.Module] = None, filter_module: Optional[torch.nn.modules.module.Module] = None, anchors: Optional[torch.nn.modules.module.Module] = None, targets: Optional[torch.nn.modules.module.Module] = None, post_process: Optional[torch.nn.modules.module.Module] = None, loss_cls: Optional[torch.nn.modules.module.Module] = None, loss_reg: Optional[torch.nn.modules.module.Module] = None)¶

The basic structure of retinanet.

参数

backbone – backbone module or dict for building backbone module.
neck – neck module or dict for building neck module.
head – head module or dict for building head module.
anchors – anchors module or dict for building anchors module.
targets – targets module or dict for building target module.
post_process – post_process module or dict for building post_process module.
loss_cls – loss_cls module or dict for building loss_cls module.
loss_reg – loss_reg module or dict for building loss_reg module.

forward(data: Dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.detectors.YOLOV3(backbone: Optional[dict] = None, neck: Optional[dict] = None, head: Optional[dict] = None, filter_module: Optional[dict] = None, anchor_generator: Optional[dict] = None, target_generator: Optional[dict] = None, loss: Optional[dict] = None, postprocess: Optional[dict] = None)¶

The basic structure of yolov3.

参数

backbone (torch.nn.Module) – Backbone module.
neck (torch.nn.Module) – Neck module.
head (torch.nn.Module) – Head module.
anchor_generator (torch.nn.Module) – Anchor generator module.
target_generator (torch.nn.Module) – Target generator module.
loss (torch.nn.Module) – Loss module.
postprocess (torch.nn.Module) – Postprocess module.

forward(data)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.structures.opticalflow.PwcNet(backbone: torch.nn.modules.module.Module, neck: Optional[torch.nn.modules.module.Module] = None, head: Optional[torch.nn.modules.module.Module] = None, loss: Optional[torch.nn.modules.module.Module] = None, loss_weights: Optional[List[float]] = None)¶

The basic structure of PWCNet.

参数

backbone – backbone module or dict for building backbone module.
neck – neck module or dict for building neck module.
head – head module or dict for building head module.
loss – loss module or dict for building loss module.
loss_weights – loss weights for each feature.

forward(data: Dict)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.BBoxUpscaler(strides, bbox_upscale=False, nhwc_output=False)¶

参数

strides (Sequence[int]) – A list contains the strides of fcos_head output.
bbox_upscale (bool) – mul bbox with strides.
nhwc_output (bool) – transpose output layout to nhwc.

forward(pred: Sequence[torch.Tensor], meta_data: Dict[str, Any])¶

Do post process for model predictions.

参数

pred – Prediction tensors.
meta_data – Meta data used in post processor, e.g. image width, height.

class hat.models.task_modules.fcos.DynamicFcosTarget(strides, topK, loss_cls, loss_reg, cls_out_channels, background_label, bbox_relu=False)¶

Generate cls and reg targets for FCOS in training stage base on dynamic losses.

参数

strides (Sequence[int]) – Strides of points in multiple feature levels.
topK (int) – Number of postive sample for each ground trouth to keep.
cls_out_channels (int) – Out_channels of cls_score.
background_label (int) – Label ID of background, set as num_classes.
loss_cls (nn.Module) – Loss for cls to choose positive target.
loss_reg (nn.Module) – Loss for reg to choose positive target.
bbox_relu (bool, optional) – Whether apply relu to bbox preds.

forward(label, pred, *args)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSDecoder(num_classes, strides, transforms=None, inverse_transform_key=None, nms_use_centerness=True, nms_sqrt=True, rescale=True, test_cfg=None, input_resize_scale=None, truncate_bbox=True, filter_score_mul_centerness=False, upscale_bbox_pred=False, bbox_relu=False)¶

参数

num_classes (int) – Number of categories excluding the background category.
strides (Sequence[int]) – A list contains the strides of fcos_head output.
transforms (Sequence[dict]) – A list contains the transform config.
inverse_transform_key (Sequence[str]) – A list contains the inverse transform info key.
nms_use_centerness (bool, optional) – If True, use centerness as a factor in nms post-processing.
nms_sqrt (bool, optional) – If True, sqrt(score_thr * score_factors).
rescale (bool, optional) – Whether to map the prediction result to the orig img.
test_cfg (dict, optional) – Cfg dict, including some configurations of nms.
truncate_bbox (bool, optional) – If True, truncate the predictive bbox out of image boundary. Default True.
filter_score_mul_centerness (bool, optional) – If True, filter out bbox by score multiply centerness, else filter out bbox by score. Default False.
bbox_relu (bool, optional) – Whether apply relu to bbox preds.

forward(pred: Sequence[torch.Tensor], meta_data: Dict[str, Any])¶

Do post process for model predictions.

参数

pred – Prediction tensors.
meta_data – Meta data used in post processor, e.g. image width, height.

class hat.models.task_modules.fcos.FCOSHead(num_classes, in_strides, out_strides, stride2channels, upscale_bbox_pred, feat_channels=256, stacked_convs=4, use_sigmoid=True, share_bn=False, dequant_output=True, int8_output=True, int16_output=False, share_conv=True, deepcopy_share_conv=False, bbox_relu=True, set_int16_qconfig=False)¶

Anchor-free head used in FCOS <https://arxiv.org/abs/1904.01355>.

参数

num_classes (int) – Number of categories excluding the background category.
in_strides (Sequence[int]) – A list contains the strides of feature maps from backbone or neck.
out_strides (Sequence[int]) – A list contains the strides of this head will output.
stride2channels (dict) – A stride to channel dict.
feat_channels (int) – Number of hidden channels.
stacked_convs (int) – Number of stacking convs of the head.
use_sigmoid (bool) – Whether the classification output is obtained using sigmoid.
share_bn (bool) – Whether to share bn between multiple levels, default is share_bn.
upscale_bbox_pred (bool) – If true, upscale bbox pred by FPN strides.
dequant_output (bool) – Whether to dequant output. Default: True
int8_output (bool) – If True, output int8, otherwise output int32. Default: True
int16_output (bool) – If True, output int16, otherwise output int32. Default: True
share_conv (bool) – Only the number of all stride channels is the same, share_conv can be True, branches share conv, otherwise not. Default: True
deepcopy_share_conv (bool) – Whether use deepcopy for share conv. Default: False
bbox_relu (bool) – Whether use relu for bbox. Default: True

forward(feats)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_single(x, i, stride)¶

Forward features of a single scale levle.

参数

x (Tensor) – FPN feature maps of the specified stride.
i (int) – Index of feature level.
stride (int) – The corresponding stride for feature maps, only used to upscale bbox pred when self.upscale_bbox_pred is True.

class hat.models.task_modules.fcos.FCOSMultiStrideCatFilter(strides: Sequence[int], threshold: float, task_strides: Sequence[Sequence[int]], idx_range: Optional[Tuple[int, int]] = None)¶

A modified Filter used for post-processing of FCOS.

In each stride, concatenate the scores of each task as the first input of FilterModule, which can reduce latency in BPU.

参数

strides (Sequence[int]) – A list contains the strides of feature maps.
idx_range (Optional[Tuple[int, int]], optional) – The index range of values counted in compare of the first input. Defaults to None which means use all the values.
threshold (float) – The lower bound of output.
task_strides (Sequence[Sequence[int]]) – A list of out_stirdes of each task.

forward(preds: Sequence[torch.Tensor], **kwargs) → Sequence[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSMultiStrideFilter(strides: Sequence[int], threshold: float, idx_range: Optional[Tuple[int, int]] = None, for_compile: bool = False, score_threshold: float = 0.05, iou_threshold: float = 0.6, max_shape: Tuple[int, int] = (512, 512))¶

Filter used for post-processing of FCOS.

参数

strides (Sequence[int]) – A list contains the strides of feature maps.
idx_range (Optional[Tuple[int, int]], optional) – The index range of values counted in compare of the first input. Defaults to None which means use all the values.
threshold (float) – The lower bound of output.
for_compile (bool) – Whether used for compile. if true, should not include postprocess.

forward(preds: Sequence[torch.Tensor], meta_and_label: Optional[Dict] = None, **kwargs) → Sequence[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcos.FCOSTarget(strides, regress_ranges, cls_out_channels, background_label, norm_on_bbox=True, center_sampling=True, center_sample_radius=1.5, use_iou_replace_ctrness=False, task_batch_list=None)¶

Generate cls and reg targets for FCOS in training stage.

参数

strides (Sequence[int]) – Strides of points in multiple feature levels.
regress_ranges (tuple[tuple[int, int]]) – Regress range of multiple level points.
cls_out_channels (int) – Out_channels of cls_score.
background_label (int) – Label ID of background, set as num_classes.
center_sampling (bool) – If true, use center sampling.
center_sample_radius – Radius of center sampling. Default: 1.5.
norm_on_bbox (bool) – If true, normalize the regression targets with FPN strides.
use_iou_replace_ctrness (bool) – If true, use iou as box quality assessment method, else use ctrness. Default: false.
task_batch_list ([int, int]) – two datasets use same head, so we generate mask

hat.models.task_modules.fcos.distance2bbox(points, distance, max_shape=None)¶

Decode distance prediction to bounding box.

参数

points (torch.Tensor) – Shape (n, 2), [x, y].
distance (torch.Tensor) – Distance from the given point to 4 boundaries (left, top, right, bottom).
max_shape (tuple, optional) – Shape of the image, used to clamp decoded bbox in max_shape range.

返回

Decoded bbox, with shape (n, 4).

返回类型

torch.Tensor

hat.models.task_modules.fcos.get_points(feat_sizes, strides, dtype, device, flatten=False)¶

Generate points according to feat_sizes.

参数

feat_sizes (list[tuple]) – Multi-level feature map sizes, the value is the HW of a certain layer.
dtype (torch.dtype) – Type of points should be.
device (torch.device) – Device of points should be.
flatten (bool) – Whether to flatten 2D coordinates into 1D dimension.

返回

Points of multiple levels belong to each image,: the value in mlvl_points is [Tensor(H1W1, 2), Tensor(H2W2, 2), …]

返回类型

list[torch.Tensor]

hat.models.task_modules.fcos.multiclass_nms(multi_bboxes, multi_scores, score_thr, nms, iou_threshold, max_per_img=- 1, score_factors=None, nms_sqrt=False, filter_score_mul_centerness=False)¶

NMS for multi-class bboxes.

参数

multi_bboxes (Tensor) – shape (n, #class*4) or (n, 4)
multi_scores (Tensor) – shape (n, #class), where the last column contains scores of the background class, but this will be ignored.
score_thr (float) – bbox threshold, bboxes with scores lower than it will not be considered.
nms (str) – nms type, candidate values are [‘nms’, ‘soft_nms’].
iou_threshold (float) – NMS IoU threshold
max_per_img (int) – if there are more than max_num bboxes after NMS, only top max_num will be kept.
score_factors (Tensor) – The factors multiplied to scores before applying NMS
nms_sqrt (bool) – If True, sqrt(score_thr * score_factors)

class hat.models.task_modules.lidar.AfdetHead(in_channels: List[int] = None, bn_kwargs: Dict[str, Any] = None, tasks: List[Dict[str, Any]] = None, common_heads=None, init_bias: float = - 2.19, use_share_conv: bool = True, share_conv_channel: int = 64, num_hm_conv: int = 2, input_idx: int = 0, hm_key: str = 'hm', quantize: bool = False, use_stride_feats: bool = False, channel_factor: int = 1, use_relu6: bool = False, us_factor: int = 1, split_hm_head: bool = False)¶

forward(input_tensor: torch.Tensor, stride_feats: List[torch.Tensor]) → List[Dict]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.lidar.AfdetPredict(num_classes: List[int], maxpool_dict: Dict, return_box_preds_all: bool = False)¶

Prediction procedure for anchor free det head.

参数

num_classes – object class number.
return_box_preds_all – return origin pred box.

forward(example: Dict, preds_dicts: Dict, post_center_limit_range: List[float], pc_range: List[float], out_size_factor: int, voxel_size: List[float], alpha: List[float], fb_head_cfg: Optional[Dict] = None, per_class_nms: bool = False, split_hm_head: bool = False)¶

Decode, nms, then return the detection result.

Additionaly support double flip testing.

class hat.models.task_modules.lidar.PillarFeatureNet(num_input_features: int, num_filters: Tuple[int, ...] = (64,), with_distance: bool = False, voxel_size: Tuple[float, float, int] = (0.2, 0.2, 4), pc_range: Tuple[float, ...] = (0.0, - 40.0, - 3.0, 70.4, 40.0, 1.0), bn_kwargs: dict = None, quantize: bool = True, use_4dim: bool = False, use_conv: bool = True, pool_size: Tuple[int, int] = (1, 1))¶

forward(features: torch.Tensor, coors: torch.Tensor, num_voxels: Optional[torch.Tensor] = None, horizon_preprocess: bool = True)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.lidar.PointPillarScatter(num_input_features: int, use_horizon_pillar_scatter: bool = False, quantize=True, **kwargs)¶

forward(voxel_features: torch.Tensor, coords: torch.Tensor, batch_size: int, input_shape: torch.Tensor)¶

Forward pass of the scatter module.

Note: batch_size has to be passed in additionally, because voxel features are concatenated on the M-channel since the number of voxels in each frame differs and there is no easy way we concat them same as image (CHW -> NCHW). M-channel concatenation would require another tensor to record number of voxels per frame, which indicates batch_size consequently.

参数

voxel_features (torch.Tensor) – MxC tensor of pillar features, where M is number of pillars, C is each pillar’s feature dim.
coords (torch.Tensor) – each pillar’s original BEV coordinate.
batch_size (int) – batch size of the feature.
input_shape (torch.Tensor) – shape of the expected BEC map. Derived from point-cloud range and voxel size.

返回

a BEV view feature tensor with point features: scattered on it.

返回类型

[torch.Tensor]

class hat.models.task_modules.lidar.RadFeatureExtractor(num_input_features: int = 4)¶

Rad’s Feature Extractor.

Extractor points feature with some op. You can read paper <Realtime and Accurate 3D Object Detection>.

Args: num_input_features: Number of input features.

forward(features, num_voxels, coors=None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.lidar.RadScatter(intensity_range: List[int], density_range: List[int], add_intensity=True, add_density: bool = False, add_location=False, occupancy_range: List[float] = None, add_pp_features: bool = False, add_occupancy: bool = True, use_occupancy_density: bool = False, num_input_features: int = 64, max_points_in_voxel: int = 5, max_points_in_pillar: int = 30, add_maxz: bool = False, maxz_range: List[float] = None, add_timestamp: bool = False)¶

forward(batch_size: int, input_shape: numpy.ndarray, voxel_features: Optional[torch.Tensor] = None, coords: Optional[numpy.ndarray] = None, num_in_voxels: Optional[numpy.ndarray] = None, voxel_features_pillars: Optional[torch.Tensor] = None, coords_pillars: Optional[numpy.ndarray] = None, num_in_pillars: Optional[numpy.ndarray] = None)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_density(voxel_features: torch.Tensor, coords: numpy.ndarray, num_in_pillars: numpy.ndarray, batch_size: int, input_shape: numpy.ndarray)¶: Make density tensor.

forward_features(voxel_features: torch.Tensor, coords: numpy.ndarray, num_in_pillars: numpy.ndarray, batch_size: int, input_shape: numpy.ndarray, dimension: Any, mode: str = 'max')¶

Make feature tensor.

Gather feature to concat tensor in channel.

forward_location(voxel_features: torch.Tensor, coords: numpy.ndarray, num_in_pillars: numpy.ndarray, batch_size: int, input_shape: numpy.ndarray)¶: Make location tensor.

forward_occupancy(voxel_features: torch.Tensor, coords: numpy.ndarray, num_in_voxels: numpy.ndarray, batch_size: int, input_shape: numpy.ndarray, use_density: bool = False)¶: Make occupancy tensor.

class hat.models.task_modules.retinanet.RetinaNetHead(num_classes: int, num_anchors: int, in_channels: int, feat_channels: int, stacked_convs: int = 4, int16_output: bool = True, dequant_output: bool = True)¶

An anchor-based head used in RetinaNet.

The head contains two subnetworks. The first classifies anchor boxes and the second regresses deltas for the anchors.

参数

num_classes (int) – Number of categories excluding the background category.
num_anchors (int) – Number of anchors for each pixel.
in_channels (int) – Number of channels in the input feature map.
feat_channels (int) – Number of hidden channels.
stacked_convs (int) – Number of convs before cls and reg.
int16_output (bool) – If True, output int16, otherwise output int32. Default: True

forward(features)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_single(x)¶

Forward feature of a single scale level.

参数

x (Tensor) – Feature of a single scale level.

返回

cls_score (Tensor): Cls scores for a single scale level: the channels number is num_anchors * num_classes.
bbox_pred (Tensor): Box energies / deltas for a single scale: level, the channels number is num_anchors * 4.

返回类型

tuple

init_weights()¶: Initialize weights of the head.

class hat.models.task_modules.retinanet.RetinaNetPostProcess(score_thresh: float, nms_thresh: float, detections_per_img: int, topk_candidates: int = 1000)¶

The postprocess of RetinaNet.

参数

score_thresh (float) – Filter boxes whose score is lower than this.
nms_thresh (float) – thresh for nms.
detections_per_img (int) – Get top n boxes by score after nms.
topk_candidates (int) – Get top n boxes by score after decode.

forward(boxes, preds, image_shapes)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.retinanet.RetinanetMultiStrideFilter(strides: Sequence[int], threshold: float)¶

forward(cls_scores, bbox_preds)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.seg.FRCNNSegHead(group_base: int, in_strides: List, in_channels: List, out_strides: List, out_channels: List, bn_kwargs: Dict, proj_channel_multiplier: float = 1.0, with_extra_conv: bool = False, use_bias: bool = True, linear_out: bool = True, argmax_output: bool = False, with_score: bool = False, rle_label: bool = False, dequant_output: bool = True, int8_output: bool = False, no_upscale_infer: bool = False)¶

FRCNNSegHead module for segmentation task.

参数

group_base – Group base of group conv
in_strides – The strides corresponding to the inputs of seg_head, the inputs usually come from backbone or neck.
in_channels – Number of channels of each input stride.
out_strides – List of output strides.
out_channels – Number of channels of each output stride.
bn_kwargs – Extra keyword arguments for bn layers.
proj_channel_multiplier – Multiplier of channels of pw conv in block.
with_extra_conv – Whether to use extra conv module.
use_bias – Whether to use bias in conv module.
linear_out – Whether NOT to use to act of pw.
argmax_output – Whether conduct argmax on output.
with_score – Whether to keep score in argmax operation.
rle_label – Whether to calculate rle representation of label output.
dequant_output – Whether to dequant output.
int8_output – If True, output int8, otherwise output int32.
no_upscale_infer – Load params from x2 scale if True.

forward(x: List[torch.Tensor]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.seg.SegDecoder(out_strides: List[int], decode_strides: List[int], upscale_times: Optional[List[int]] = None, transforms: Optional[List[dict]] = None, inverse_transform_key: Optional[List[str]] = None, output_names: Optional[str] = 'pred_seg')¶

Semantic Segmentation Decoder.

参数

out_strides – List of output strides, represents the strides of the output from seg_head.
output_names – Keys of returned results dict.
decode_strides – Strides that need to be decoded, should be a subset of out_strides.
upscale_times – Bilinear upscale times for each decode stride, default to None, which means same as decode stride.
transforms – A list contains the transform config.
inverse_transform_key – A list contains the inverse transform info key.

class hat.models.task_modules.seg.SegHead(num_classes, in_strides, out_strides, stride2channels, feat_channels=256, stride_loss_weights=None, stacked_convs=1, argmax_output=False, dequant_output=True, int8_output=True, upscale=False, upscale_stride=4, output_with_bn=False, bn_kwargs=None, upsample_output_scale=None)¶

Head Module for segmentation task.

参数

num_classes (int) – Number of classes.
in_strides (list[int]) – The strides corresponding to the inputs of seg_head, the inputs usually come from backbone or neck.
out_strides (list[int]) – List of output strides.
stride2channels (dict) – A stride to channel dict.
feat_channels (int or list[int]) – Number of hidden channels (of each output stride).
stride_loss_weights (list[int]) – loss weight of each stride.
stacked_convs (int) – Number of stacking convs of head.
argmax_output (bool) – Whether conduct argmax on output. Default: False
dequant_output (bool) – Whether to dequant output. Default: True
int8_output (bool) – If True, output int8, otherwise output int32. Default: True
upscale (bool) – If True, stride{x}’s feature map is upsampled by 2x, then the upsampled feature is adding supervisory signal. Default is False.
upscale_stride (int) – Specify which stride’s feature need to be upsampled when upscale is True.
output_with_bn (bool) – Whether add bn layer to the output conv.
bn_kwargs (dict) – Extra keyword arguments for bn layers.
upsample_output_scale (int) – Output upsample scale, only used in qat model, default is None.

forward(feats)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

forward_single(x, stride_index=0)¶

Forward features of a single scale level.

参数

x (Tensor) – feature maps of the specified stride.
stride_index (int) – stride index of input feature map.

返回

seg predictions of input feature maps.

返回类型

tuple

class hat.models.task_modules.seg.SegTarget(ignore_index=255, label_name='gt_seg')¶

Generate training targets for Seg task.

参数

ignore_index (int, optional) – Index of ignore class.
label_name (str, optional) – The key corresponding to the gt seg in label.

class hat.models.task_modules.seg.VargNetSegDecoder(out_strides: List[int], input_padding: Sequence[int] = (0, 0, 0, 0))¶

Semantic Segmentation Decoder.

参数

out_strides (list[int]) – List of output strides, represents the strides of the output from seg_head.
output_names (str or list[str]) – Keys of returned results dict.
decode_strides (int or list[int]) – Strides that need to be decoded, should be a subset of out_strides.
transforms (Sequence[dict]) – A list contains the transform config.
inverse_transform_key (Sequence[str]) – A list contains the inverse transform info key.

forward(pred: Sequence[torch.Tensor])¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.yolo.YOLOV3AnchorGenerator(anchors: List, strides: List, image_size: List)¶

Anchors generator for yolov3.

参数

anchors (List) – list if anchor size.
strides (List) – strides of feature map for anchors.
image_size (List) – Input size of image.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.yolo.YOLOV3Head(in_channels_list: list, feature_idx: list, num_classes: int, anchors: list, bn_kwargs: dict, bias: bool = True, reverse_feature: bool = True, int16_output: bool = True, dequant_output: bool = True)¶

Heads module of yolov3.

shared convs -> conv head (include all objs)

参数

in_channels_list (list) – List of input channels.
feature_idx (list) – Index of feature for head.
num_classes (int) – Num classes of detection object.
anchors (list) – Anchors for all feature maps.
bn_kwargs (dict) – Config dict for BN layer.
bias (bool) – Whether to use bias in module.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.yolo.YOLOV3LabelEncoder(class_encoder: torch.nn.modules.module.Module)¶

Encode gt and matching results for yolov3.

参数: class_encoder (torch.nn.Module) – config of class label encoder

forward(boxes: torch.Tensor, gt_boxes: torch.Tensor, match_pos_flag: torch.Tensor, match_gt_id: torch.Tensor, ig_flag: Optional[torch.Tensor] = None) → Dict[str, torch.Tensor]¶

Forward method.

参数

boxes (torch.Tensor) – (B, N, 4), batched predicted boxes
gt_boxes (torch.Tensor) – (B, M, 5+), batched ground truth boxes, might be padded.
match_pos_flag (torch.Tensor) – (B, N) matched result of each predicted box
match_gt_id (torch.Tensor) – (B, M) matched gt box index of each predicted box
ig_flag (torch.Tensor) – (B, N) ignore matched result of each predicted box

class hat.models.task_modules.yolo.YOLOV3Matcher(ignore_thresh: float)¶

Bounding box classification label matcher by max iou.

Different rule and return condition with MaxIoUMatcher. YOLOv3MaxIoUMatcher will be merged with MaxIoUMatcher in future.

参数: ignore_thresh (float) – Boxes whose IOU larger than ignore_thresh is regarded as ignore samples for losses.

forward(boxes: torch.Tensor, gt_boxes: torch.Tensor, gt_boxes_num: torch.Tensor, im_hw: Optional[torch.Tensor] = None) → Tuple[torch.Tensor, torch.Tensor]¶

Forward Method.

参数

boxes (torch.Tensor) – Box tensor with shape (B, N, 4) or (N, 4) when boxes are identical in the whole batch.
gt_boxes (torch.Tensor) – GT box tensor with shape (B, M, 5+). In one sample, if the number of gt boxes is less than M, the first M entries should be filled with real data, and others padded with arbitrary values.
gt_boxes_num (torch.Tensor) – GT box num tensor with shape (B). The actual number of gt boxes for each sample. Cannot be greater than M.

返回

tuple contains:

flag (torch.Tensor): flag tensor with shape (B, N). Entries
with value 1 represents ignore, 0 for neg.

matched_pred_id (torch.Tensor): matched_pred_id tensor in
(B, M). The best matched of gt_boxes.

返回类型

(tuple)

class hat.models.task_modules.yolo.YOLOV3PostProcess(anchors: list, strides: list, num_classes: int, score_thresh: float = 0.01, nms_thresh: float = 0.45, topK: int = 200)¶

The postprocess of YOLOv3.

参数

anchors (list) – The anchors of yolov3.
num_classes (int) – The num classes of class branch.
score_thresh (float) – Score thresh of postprocess before nms.
nms_thresh (float) – Nms thresh.
topK (int) – The output num of bboxes after postprocess.

forward(inputs: Sequence[torch.Tensor])¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.yolo.YOLOv3Filter(strides: Sequence[int], threshold: float, idx_range: Optional[Tuple[int, int]] = None, last_channels: float = 75)¶

Filter used for post-processing of YOLOv3

参数

strides (Sequence[int]) – A list contains the strides of feature maps.
idx_range (Optional[Tuple[int, int]], optional) – The index range of values counted in compare of the first input. Defaults to None which means use all the values.
threshold (float) – The lower bound of output.
last_channels (float) – Last channels.

forward(preds: Sequence[torch.Tensor], meta_and_label: Optional[Dict] = None, **kwargs) → Sequence[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.deeplab.Deeplabv3plusHead(in_channels: int, c1_index: int, c1_in_channels: int, feat_channels: int, num_classes: int, dilations: List[int], num_repeats: List[int], argmax_output: Optional[bool] = False, dequant_output: Optional[bool] = True, int8_output: Optional[bool] = True, bn_kwargs: Optional[Dict] = None, dropout_ratio: Optional[float] = 0.1, upsample_output_scale: Optional[int] = None, upsample_decode_scale: Optional[int] = 4, bias=True)¶

Head Module for FCN.

参数

in_channels – Input channels.
c1_index – Index for c1 input.
c1_in_channels – In channels of c1.
feat_channels – Channels for the module.
num_classes – Number of classes.
dilations – List of dilations for aspp.
num_repeats – List of repeat for each branch of ASPP.
argmax_output – Whether conduct argmax on output. Default: False.
dequant_output – Whether to dequant output. Default: True
int8_output – If True, output int8, otherwise output int32. Default: False.
bn_kwargs – Extra keyword arguments for bn layers. Default: None.
dropout_ratio – Ratio for dropout during training. Default: 0.1.
upsample_decode_scale – upsample scale to c1. Default is 4.
upsample_output_scale – Output upsample scale, only used in qat model, default is None.
bias – Whether has bias. Default: True.

forward(inputs)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcn.DepthwiseSeparableFCNHead(in_channels, feat_channels, num_convs=1, **kwargs)¶

class hat.models.task_modules.fcn.FCNDecoder(upsample_output_scale=8, use_bce=False, bg_cls=0, bg_threshold=0.25)¶

FCN Decoder.

参数: upsample_output_scale – Output upsample scale. Default: 8.

forward(pred)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcn.FCNHead(input_index: int, in_channels: int, feat_channels: int, num_classes: int, dropout_ratio: Optional[float] = 0.1, int8_output: Optional[bool] = False, argmax_output: Optional[bool] = False, dequant_output: Optional[bool] = True, upsample_output_scale: Optional[int] = None, num_convs: Optional[int] = 2, bn_kwargs: Optional[Dict] = None)¶

Head Module for FCN.

参数

input_index – Index of inputs.
in_channels – Input channels.
feat_channels – Channels for the module.
num_classes – Number of classes.
dropout_ratio – Ratio for dropout during training. Default: 0.1.
int8_output – If True, output int8, otherwise output int32. Default: False.
argmax_output – Whether conduct argmax on output. Default: False.
dequant_output – Whether to dequant output. Default: True.
upsample_output_scale – Output upsample scale. Default: None.
num_convs – number of convs in head. Default: 2.
bn_kwargs – Extra keyword arguments for bn layers. Default: None.

forward(inputs: List[torch.Tensor])¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.fcn.FCNTarget¶

Generate Target for FCN.

参数: num_classes – Number of classes. Defualt: 19.

forward(label: torch.Tensor, pred: torch.Tensor) → dict¶

参数

label – data Tenser.(n, h, w)
pred – Output Tenser. (n, c, h, w).

返回

Loss inputs.

返回类型

dict

class hat.models.task_modules.pwcnet.PwcNetHead(in_channels: List[int], bn_kwargs: dict, use_bn: bool = False, md: int = 4, use_res: bool = True, use_dense: bool = True, flow_pred_lvl: int = 2, pyr_lvls: int = 6, bias: bool = True, act_type=None)¶

A basic head of PWCNet.

参数

in_channels – Number of channels in the input feature map.
bn_kwargs – Dict for BN layer.
use_bn – Whether to use BN in module.
md – search range of Correlation module.
use_res – Whether to use residual connections.
use_dense – Whether to use dense connections.
flow_pred_lvl – Which level to upsample to generate the final optical flow prediction.
pyr_lvls – Number of feature levels in the flow pyramid.
bias – Whether to use bias in module.
act_type – Activation layer.

forward(features: List[List[torch.Tensor]]) → List[torch.Tensor]¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()¶: Initialize the weights of head module.

warp(x: torch.Tensor, flo: torch.Tensor) → torch.Tensor¶

Warp an image/tensor (im2) back to im1, according to the optical flow.

参数

x – [B, C, H, W] (im2)
flo – [B, 2, H, W] flow

class hat.models.task_modules.pwcnet.PwcNetNeck(out_channels: list, use_bn: bool, bn_kwargs: dict, bias: bool = True, pyr_lvls: int = 6, flow_pred_lvl: int = 2, act_type=None)¶

A extra features module of PWCNet.

参数

out_channels – Channels for each block.
use_bn – Whether to use BN in module.
bn_kwargs – Dict for BN layer.
bias – Whether to use bias in module.
pyr_lvls – Number of feature levels in the flow pyramid.
flow_pred_lvl – Which level to upsample to generate the final optical flow prediction.
act_type – Activation layer.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

init_weights()¶: Initialize the weights of pwcnet module.

class hat.models.task_modules.pointpillars.Anchor3DGeneratorStride(anchor_sizes: List[List[float]] = [[1.6, 3.9, 1.56]], anchor_strides: List[List[float]] = [[0.4, 0.4, 1.0]], anchor_offsets: List[List[float]] = [[0.2, - 39.8, - 1.78]], rotations: List[List[float]] = [[0, 1.5707963267948966]], class_names: List[str] = ['Car'], match_thresholds: List[float] = [- 1], unmatch_thresholds: List[float] = [- 1], dtype: Any = torch.float32)¶

3D Anchor Generator by stride.

参数

anchor_sizes (list[list[float]]) – 3D sizes of anchors.
anchor_strides (list[list[float]]) – Strides of anchors.
anchor_offsets (list[list[float]]) – Offsets of anchors.
rotations (list[list[float]]) – Rotations of anchors in a feature grid.
class_names (list[str]) – Class names of data.
match_thresholds (list[float]) – Match thresholds of IoU.
unmatch_thresholds (list[float]) – Unmatch thresholds of IoU.

forward(feature_map_size, device)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.LidarTargetAssigner(box_coder: hat.core.box_coders.BoxCoder, class_names: List[str], region_similarity_calculator: Optional[hat.core.region_similarity.RegionSimilarityCalculator], positive_fraction: int = None, sample_size: int = 512)¶

TargetAssigner for Lidar.

参数

box_coder – BoxCoder.
class_names – Class names.
region_similarity_calculator – Region similarity calculator.
positive_fraction – Positive fraction.
sample_size – Sample size.

create_targets_single(all_anchors, gt_boxes, similarity_fn, box_encoding_fn, gt_classes=None, matched_threshold=0.6, unmatched_threshold=0.45, positive_fraction=None, sample_size=300, norm_by_num_examples=False, box_code_size=7)¶

Modify from FAIR detectron.

参数

all_anchors – [num_of_anchors, box_ndim] float tensor.
gt_boxes – [num_gt_boxes, box_ndim] float tensor.
similarity_fn – a function, accept anchors and gt_boxes, return similarity matrix(such as IoU).
box_encoding_fn – a function, accept gt_boxes and anchors, return box encodings(offsets).
prune_anchor_fn – a function, accept anchors, return indices that indicate valid anchors.
gt_classes – [num_gt_boxes] int tensor. indicate gt classes, must start with 1.
matched_threshold – float, iou greater than matched_threshold will be treated as positives.
unmatched_threshold – float, iou smaller than unmatched_threshold will be treated as negatives.
positive_fraction – [0-1] float or None. if not None, we will try to keep ratio of pos/neg equal to positive_fraction when sample. if there is not enough positives, it fills the rest with negatives.
rpn_batch_size – int. sample size.
norm_by_num_examples – bool. norm box_weight by number of examples.

返回

labels, bbox_targets, bbox_outside_weights

forward(anchors_list, matched_thresholds, unmatched_thresholds, annos, device)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.PointPillarHead(in_channels: int = 128, num_classes: int = 1, anchors_num_per_class: int = 2, use_direction_classifier: bool = True, num_direction_bins: int = 2, box_code_size: int = 7)¶

Basic module of PointPillarHead.

参数

in_channels (int) – Channel number of input feature.
num_classes (int) – Number of class.
anchors_num_per_class (int) – Anchor number for per class.
use_direction_classifier (bool) – Whether to use direction.
num_direction_bins (int) – Number of direction bins.
box_code_size (int) – BoxCoder size.

forward(x)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.PointPillarsPostProcess(num_classes: int, box_coder: int, use_direction_classifier: bool = True, num_direction_bins: int = 2, direction_offset: float = 0.0, use_rotate_nms: bool = False, nms_pre_max_size: int = 1000, nms_post_max_size: int = 300, nms_iou_threshold: float = 0.5, score_threshold: float = 0.05, post_center_limit_range: List[float] = [0, - 39.68, - 5, 69.12, 39.68, 5], max_per_img: int = 100)¶

PointPillars PostProcess Module.

参数

num_classes (int) – Number of classes.
box_coder (int) – BoxCeder module.
use_direction_classifier (bool) – Whether to use direction.
num_direction_bins (int) – _description_. Defaults to 2.
direction_offset (float) – _description_. Defaults to 0.0.
use_rotate_nms (bool) – Whether to use ratated nms.
nms_pre_max_size (int) – Max size of nms preprocess.
nms_post_max_size (int) – Max size of nms postprocess.
nms_iou_threshold (float) – IoU threshold of nms.
score_threshold (float) – Score threshold.
post_center_limit_range (list[float]) – PointCloud range.
max_per_img (int) – Max number of object per image.

forward(box_preds, cls_preds, dir_preds, anchors, meta_data)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

class hat.models.task_modules.pointpillars.PointPillarsPreProcess(pc_range: List[float], voxel_size: List[float], max_voxels_num=20000, max_points_in_voxel=30)¶

forward(points_lst, is_deploy=False)¶

Defines the computation performed at every call.

Should be overridden by all subclasses.

注解

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.