Program crashing when kernel blocksize is increased

ps_nithin · March 28, 2025, 7:15am

Hi,
I was trying to improve performance of my program using nsight compute. For a particular kernel it recommends to increase the blocksize in multiples of 32 between 128 and 256. When i run the program with blocksize 32 the program runs without issues. But when i increase the blocksize to 256 the program crashes after few iterations. How can i resolve this issue.

Thanks,

ps_nithin · March 28, 2025, 7:19am

This is the program i am running.

github.com/ps-nithin/pyrebel

demo/pyrebel_main.py

main

# Copyright (C) 2024-2025 Nithin PS.
# This file is part of Pyrebel.
#
# Pyrebel is free software: you can redistribute it and/or modify it under the terms of 
# the GNU General Public License as published by the Free Software Foundation, either 
# version 3 of the License, or (at your option) any later version.
#
# Pyrebel is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; 
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 
# PURPOSE. See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with Pyrebel.
# If not, see <https://www.gnu.org/licenses/>.
#

import numpy as np
from PIL import Image
import math,argparse,time
from pyrebel.preprocess import Preprocess
from pyrebel.abstract import Abstract

This file has been truncated. show original

I removed the break at the end of the program.

The kernel i was trying to improve is find_ba_max_pd in do_abstract_all in

github.com/ps-nithin/pyrebel

src/pyrebel/abstract.py

main

# Copyright (C) 2024-2025 Nithin PS.
# This file is part of Pyrebel.
#
# Pyrebel is free software: you can redistribute it and/or modify it under the terms of 
# the GNU General Public License as published by the Free Software Foundation, either 
# version 3 of the License, or (at your option) any later version.
#
# Pyrebel is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; 
# without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR 
# PURPOSE. See the GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along with Pyrebel.
# If not, see <https://www.gnu.org/licenses/>.
#

from pyrebel.getnonzeros import *
import numpy as np
from numba import cuda
import cmath,math

This file has been truncated. show original

When i launch with blocksize 32
find_ba_max_pd[math.ceil(len(nz_ba_pre_hor)/32),32](nz_ba_pre_hor_d,ba_size_pre_hor_d,bound_data_ordered_d,ba_max_pd_pre_d,shape_d)
it doesn’t crash.

But when i launch with blocksize 256
find_ba_max_pd[math.ceil(len(nz_ba_pre_hor)/256),256](nz_ba_pre_hor_d,ba_size_pre_hor_d,bound_data_ordered_d,ba_max_pd_pre_d,shape_d)
it crashes.

Thanks,

ps_nithin · March 28, 2025, 8:26am

Also, this program crashing happens on an orin nano but does not happen on jetson nano.

Thanks,

Topic		Replies	Views
GPU requirements CUDA Programming and Performance jetson	28	173	February 15, 2025
Getting the best performance CUDA Programming and Performance	1	245	November 27, 2023
Odd behavior. Bug in opencl implementation? CUDA Programming and Performance	5	6328	May 12, 2010
CUDA kernels keep on crashing CUDA Programming and Performance	6	3662	October 27, 2008
kernel panic when using Bumblebee Linux	2	1633	April 30, 2013
code that crashes unpredictably CUDA Programming and Performance	15	12653	April 28, 2010
unspecified launch failure problem CUDA Programming and Performance	1	8325	February 24, 2010
Can a Kernel be too big?? CUDA_ERROR_NO_BINARY_FOR_GPU error 209 CUDA Programming and Performance	11	3072	November 13, 2017
Wrong output when adding blocks what am I doing wrong? CUDA Programming and Performance	13	11993	December 4, 2007
Kernel panic due to incompatibility between freeCudaHost and free_pages Jetson AGX Orin cuda , nvbugs	8	60	May 12, 2025

Program crashing when kernel blocksize is increased

Related topics