Pointpillars training error

dataset_convert and data split is no error
when I use my own data for training, error happen bellow:,

File “/opt/conda/lib/python3.8/site-packages/pointcloud/pointpillars/pcdet/models/init.py”, line 142, in model_func
ret_dict, tb_dict, disp_dict = model(batch_dict)
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1110, in _call_impl
return forward_call(*input, **kwargs)
File “/home/jenkins/agent/workspace/tlt-pytorch-main-nightly/pointcloud/pointpillars/pcdet/models/detectors/pointpillar.py”, line 16, in forward
File “/opt/conda/lib/python3.8/site-packages/torch/nn/modules/module.py”, line 1110, in _call_impl
return forward_call(*input, **kwargs)
File “/home/jenkins/agent/workspace/tlt-pytorch-main-nightly/pointcloud/pointpillars/pcdet/models/backbones_2d/base_bev_backbone.py”, line 107, in forward
RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 127 but got size 128 for tensor number 1 in the list.

Can you share the training spec file? If possible, could you please share the smallest part of dataset which can easily reproduce the error? Thanks.

Still checking, will update to you if any. Thanks.

BTW, did you run the default notebook successfully?


use kitti data ,no problem

OK. Indeed, with your dataset, I can also reproduce now. Will check further.

It is related to POINT_CLOUD_RANGE.
The point cloud coordinates range in [xmin, ymin, zmin, xmax, ymax, zmax] format .
Please set compatible range according to your dataset.

After checking, below can work with your attached dataset.
For example,
POINT_CLOUD_RANGE: [0, -39.68, -3, 69.12, 39.68, 1]
POINT_CLOUD_RANGE: [0, -30.68, -2, 50.00, 30.68, 1]

How to set this range?is lidar sensor range? my dataset range is 360. new problem happen

For CUDNN error, please exit the container and retry again.

Thanks for the reply, how do we get the compatible coordinates range of dataset [xmin, ymin, zmin, xmax, ymax, zmax], do we need to iterate through the dataset and find these values?

another problem when I retrain.

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.

Please check nvidia-smi.
You can share its result here.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.