TensorRT Engine Creation Fails with “tensor volume exceeds 2147483648” on MobileNet SSD v2 in DeepStream 7.1 (Jetson Orin Nano)

AI & Data Science Deep Learning (Training & Inference) TensorRT
, ,
1

Subject: TensorRT Engine Creation Fails with “tensor volume exceeds 2147483648” on MobileNet SSD v2 in DeepStream 7.1 (Jetson Orin Nano)

Hello NVIDIA Developer Forum,

I’m attempting to deploy the TensorFlow ssd_mobilenet_v2_coco_2018_03_29 model on a Jetson Orin Nano 8GB using DeepStream 7.1 and TensorRT, but the engine build fails with a tensor-volume overflow. I’d appreciate any guidance or known-working variants.

1. Platform & Software Versions

  • Hardware: Jetson Orin Nano 8GB
  • OS: Ubuntu 22.04 (JetPack 6.2)
  • DeepStream: v7.1
  • TensorRT: 10.3
  • CUDA: 12.6
  • cuDNN: 9.3
  • ONNX opset: 13

2. Model Conversion

# Download from TF Model Zoo
# Convert to ONNX
python3 -m tf2onnx.convert \
  --saved-model ssd_mobilenet_v2_coco_2018_03_29/saved_model \
  --output ssd_mobilenet_v2.onnx \
  --opset 13 \
  --inputs input_tensor:0[1,300,300,3] \
  --outputs detection_boxes,detection_scores,detection_classes,num_detections

Verified ONNX input:

name: "image_tensor:0"
shape: [1, 300, 300, 3]

3. DeepStream Config Snippet

[infer_config]
infer-dims=3;300;300
output-blob-names=detection_boxes,detection_scores,detection_classes,num_detections
parse-bbox-func-name=NvDsInferParseCustomSSD
custom-lib-path=/opt/nvidia/deepstream/deepstream-7.1/lib/libnvds_infercustomparser.so
# Alternatively tried nvdsinfer_custom_impl_ssd/libnvdsinfer_custom_impl_ssd.so

4. Error Message

When running deepstream-app -c dstest1_pgie_config.txt, I get:

ITensor::getDimensions: Error Code 4: API Usage Error 
(Conv__6197_output: tensor volume exceeds 2147483648, dimensions are [1024,96,150,150])

It seems the convolutional tensor [1024×96×150×150] exceeds the 2 GB limit during parsing.

5. Questions / Requests

  1. Workarounds or config tweaks to prevent tensor-volume overflows in DeepStream/TensorRT?
  2. Model modifications (e.g., pruning, layer fusion, reshape) you could recommend for MobileNet SSD v2/v3 on 8 GB Jetson?
  3. Known MobileNet SSD variants (or custom SSD models) confirmed to build and run cleanly with DeepStream 7.1 on Orin Nano.
  4. Any reference config files or sample projects for a MobileNet SSD in DeepStream would be extremely helpful.

Thank you in advance for your insights!

—
Tagging: deepstream tensorrt onnx #JetsonOrin #MobileNetSSD

3

Hi @soumya777prusty ,
Would recommend you to please reach out to Deepstream Forum.

Thanks

4

Hi @Soumya — that error is TensorRT’s way of saying an intermediate tensor is larger than the 2 GB int32 indexing limit. The tensor `[1024, 96, 150, 150]` has ~2.2 billion elements, which is just over the line.

**Why it’s happening**

MobileNet SSD v2 has a feature extractor that produces large intermediate activations. The default input resolution of 300x300 combined with the depthwise separable convolutions can produce tensors whose total volume exceeds `2^31 - 1` elements during the TRT parser’s static shape analysis.

**Practical fixes**

1. **Reduce input resolution.** Try 224x224 or 192x192 instead of 300x300. This is usually the quickest fix and often has minimal accuracy impact for Jetson deployments.

```bash

–inputs input_tensor:0[1,224,224,3]

```

2. **Use a smaller backbone.** Consider SSD-MobileNet v3 or EfficientDet-Lite instead of v2. They’re designed for edge devices and have smaller activations.

3. **Switch to FP16 explicitly.** Sometimes the parser is more aggressive with tensor layout in FP32. Building with `–fp16` from the start can avoid the large intermediate.

```bash

/usr/src/tensorrt/bin/trtexec --onnx=ssd_mobilenet_v2.onnx --fp16 --saveEngine=model.plan

```

4. **Check the ONNX graph for unnecessary transposes/reshapes.** Some tf2onnx exports produce ops that expand tensors unnecessarily. Run the model through `onnxsim` first:

```bash

python3 -m onnxsim ssd_mobilenet_v2.onnx ssd_mobilenet_v2_sim.onnx

```

5. **For DeepStream specifically**, make sure your `infer-dims` matches the ONNX input exactly, and consider using `output-tensor-meta=1` with a custom parser instead of relying on the SSD parser if the output names don’t align.

**Reference configs**

The DeepStream sample apps include SSD configs under `/opt/nvidia/deepstream/deepstream-7.1/sources/objectDetector_SSD/`. Those are tuned for TensorRT-friendly variants and are a better starting point than the raw TF model zoo export.

Good luck — Jetson Orin Nano 8GB is tight for SSD v2 at 300x300, so dropping the resolution is probably your fastest path.