Casting uint8 to int32 fails with an internal assertion failure

Description

class Traversability(nn.Module):
    def forward(self, untraversable: torch.Tensor) -> torch.Tensor:
       return untraversable.to(torch.int32)

...
    dummy_input = torch.randint(0, 4, (3, 224, 224), dtype=torch.uint8)
...

results in

[04/08/2024-15:58:18] [TRT] [E] ModelImporter.cpp:726: While parsing node number 0 [Cast -> "output"]:
[04/08/2024-15:58:18] [TRT] [E] ModelImporter.cpp:727: --- Begin node ---
[04/08/2024-15:58:18] [TRT] [E] ModelImporter.cpp:728: input: "input"
output: "output"
name: "/Cast"
op_type: "Cast"
attribute {
  name: "to"
  i: 6
  type: INT
}

[04/08/2024-15:58:18] [TRT] [E] ModelImporter.cpp:729: --- End node ---
[04/08/2024-15:58:18] [TRT] [E] ModelImporter.cpp:732: ERROR: builtin_op_importers.cpp:319 In function importCast:
[8] Assertion failed: newType == DataType::kFLOAT || newType == DataType::kHALF

Environment

TensorRT Version: 8.5
GPU Type: GeForce RTX 3060
Nvidia Driver Version: 550.54.15
CUDA Version: 11.8
CUDNN Version: 8.9.7
Operating System + Version: Ubuntu 20.04
Python Version (if applicable): 3.8
PyTorch Version (if applicable): 2.2.2
Baremetal or Container (if container which image + tag): nvidia/cuda:11.4.3-devel-ubuntu20.04

Relevant Files

Steps To Reproduce

From that branch (bug4)

./docker/build.sh
./docker/run.sh
python3 trav.py save # Creates the onnx file
python3 convert.py