Deploying spconv-based 3D detection model (PillarNet) with TensorRT on aarch64 GB10 — libspconv SM120 missing

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Hi,

I am trying to deploy a PillarNet-18 3D object detection model (trained with OpenPCDet on Waymo) using TensorRT on an aarch64 GB10 (Blackwell, SM121) system.

Current Approach

Since spconv (SparseConv2d/SubMConv2d) is not natively supported by ONNX/TensorRT, I split the pipeline into two parts:

  1. VFE + Scatter → Custom CUDA kernels
  2. Backbone + Neck + CenterHead → ONNX → TensorRT engine

For part 2, I replaced spconv layers with standard Conv2d (weights reused via permute(0,3,1,2), no retraining needed) and successfully exported to ONNX and built TensorRT engines (FP32/FP16/INT8).

Problem

The dense Conv2d replacement loses sparsity, processing 100% of the BEV grid (1504×1504 = 2.26M cells) instead of only the ~8-10% active pillars. This results in significantly slower inference compared to PyTorch + spconv:

Method Latency
PyTorch + spconv ~23 ms
TensorRT FP16 (dense Conv2d) ~50 ms
TensorRT INT8 (dense Conv2d) ~35 ms

The proper solution is NVIDIA’s libspconv (from Lidar_AI_Solution), which runs sparse convolution natively in CUDA. However, the aarch64 prebuilt binaries do not include SM120 kernels, while the x86_64 version does:

Binary SM120
x86_64_cuda13.0 (53 MB) Yes (sm_80/86/90/110/120)
aarch64_cuda12.8 (27.8 MB) No (sm_101a only)
aarch64_cuda13.0 (24.3 MB) No (sm_110 only)

Questions

  1. Is there a plan to release an aarch64 libspconv binary with SM120 support?
  2. Are there any alternative approaches to deploy spconv-based models with TensorRT on GB10 that preserve sparsity?

Environment

  • GPU: NVIDIA GB10 (Blackwell, compute capability 12.1)
  • Platform: aarch64 (ARM 64-bit)
  • OS: Ubuntu 24.04.4 LTS
  • CUDA: 12.8, TensorRT: 10.15.1
  • Driver: 590.48.01

Related

Thank you!

4

Hi @kkurangg,
I see the original request for the sm_120 binary was fulfilled over in GitHub issue #361, but it looks like you’ve since run into the 128-channel segfault bug (#363) which is currently open. Since the active debugging is happening on GitHub, is there anything else we can help you with here on the forums right now?

5

Hi @athkumar,

Thank you for the SM120 binary update via #361 — that resolved the missing kernel issue.

However, the 128-channel segfault described in this issue remains a blocker for production
deployment. The PillarNet-18 model we are currently developing requires 256-channel sparse
convolutions in stage 4, and we’re forced to fall back to dense TensorRT convolution for
that stage, which negates much of the sparsity benefit.

Could you provide an update on:

  1. Is this bug being tracked internally?
  2. Is there an estimated timeline for a fix in libspconv?

We are happy to provide any additional logs, test cases, or model files that would help
reproduce and debug this issue.

Thanks in advance.

6

Hi @athkumar,

Following up on my last post (April 2). It’s been over two weeks with no
response on the 128-channel segfault issue
(NVIDIA-AI-IOT/Lidar_AI_Solution#363).

To summarize the situation:

  • spconv::load_engine_from_onnx() segfaults at engine.cu:2596 when any
    SparseConvolution layer exceeds 128 channels
  • This blocks deployment of PillarNet-18, CenterPoint, BEVFusion and other
    production models that require 256-channel sparse convolutions on aarch64 (GB10)
  • A minimal reproduction script is already provided in the GitHub issue

Could you please check with the internal team:

  1. Is this bug being tracked internally?
  2. Is there any estimated timeline for a fix?

I’m happy to provide additional logs, test cases, or model files to help
reproduce and debug.

Thank you,
Jaeho

7

Friendly follow-up — it has been almost 4 weeks since the last response
from @athkumar (March 31), and ~11 days since my previous follow-up.

To recap the current blocker:

  • sm_120 aarch64 libspconv binaries are now available — thank you again.
  • However, GitHub issue #363 (engine build segfault when SparseConvolution
    channels exceed 128) is still open and unresolved as of today.
  • This prevents us from running PillarNet-18 (which requires 256-channel
    sparse convolutions in conv4) on the GB10 platform.

Could someone from the libspconv team please share:

  1. Whether issue #363 is actively being investigated.
  2. A rough timeline for a fix (or a public-facing workaround).
  3. Any way we can assist — additional logs, reproduction environments,
    etc.

We have been waiting since early March and the silence is making it hard
to plan around this for our deployment schedule. Any update at all would
be greatly appreciated.

8

Hi @kkurangg,

Really sorry for the long silence on this one, that’s on me. Thank you for the patience and for keeping the thread updated with clear context each time, it makes a real difference on our end.

Quick update on where things stand:

I’m escalating issue #363 internally to the libspconv team today and asking for confirmation on whether it’s being tracked, plus any rough sense of timeline. I’ll post back here as soon as I hear something concrete, even if the answer is “no ETA yet.” You shouldn’t have to keep guessing whether this is on someone’s radar.

In the meantime, the minimal repro you’ve already attached to #363 is exactly what the team will need, so no additional logs required from your side right now. If that changes once I get a response, I’ll let you know what would help.

I’ll aim to have something back to you within the next few days. Appreciate you sticking with us on this.

Best, Atharva