Scenario 1: Custom CUDA Kernel, cuBLAS errors, memory allocation issues, stream context corruption (gate_up_silu 3-in-1 fused Kernel cuBLAS status=13)

Accelerated Computing CUDA CUDA Programming and Performance
, , ,
1

Full Post Body

Environment

  • GPU: Tesla V100 32GB (SM70)
  • Stack: Custom LLM inference engine xLLM, Q4_K quantized FFN fused CUDA kernels
  • Role: Independent Researcher, self-maintained inference framework with 70+ repos & 900+ commits

1. Experiment Result: Discontinued ffn_down raw Q4_K fusion patch

I tested standalone raw-Q4_K caching for ffn_down and rolled back all related patches, as the overall latency regresses against the existing gate-up-SiLU 3-in-1 fused kernel.

  • Standalone ffn_down optimization: reduces single ffn_down runtime from ~153ms → 137ms
  • Major downside: enabling ffn_down raw cache disables gate-up-SiLU fusion; ffn_gate_up latency jumps from ~206ms → 318ms
  • Full split sweep best config: g=26 d_start=16, TPOT = 12.19ms
  • Baseline stable config (gate_up_silu with raw cache cap=64): TPOT = 11.96ms

Conclusion: Under the 64 raw tensor safety cache limit for V100 32GB VRAM, the minor speedup of ffn_down quantization cannot offset severe performance loss on gate/up layers. We will not retain this optimization path.

2. Low-overhead performance fix: Cache static cudaGetDeviceProperties value

Profiling via Nsight Systems identified redundant device property lookup inside nxt_paged_attention_v2():

  • Hot call count: 1152 invocations (36 transformer layers × 32 decode steps)
  • Root cause: cudaGetDeviceProperties(&prop, 0) executed on every kernel launch
  • Fix: Cache sharedMemPerBlock as a static global value once at initialization; all numeric computation paths remain unchanged.

Full Validation Results

  1. Build & code quality

    • cmake --build build_cuda -j4: Pass
    • git diff --check: Pass

  2. Functional correctness

    • test_q4k_gate_up_silu 5: Pass
    • test_engine_correctness: 4/4 unit tests passed
    • compute-sanitizer --tool memcheck: ERROR SUMMARY = 0
    • Deterministic batch text output comparison: fully identical pre/post patch

  3. Benchmark performance

    • Before patch: 71.34 tok/s, TPOT = 9.66 ms
    • After static cache patch: 73.03 tok/s, TPOT = 9.66 ms

  4. Profiling confirmation

    Nsight Systems CUDA API summary verifies cudaGetDeviceProperties calls are completely eliminated from decode hot paths.

Code Change Scope

Only single-file modification pending commit:

operators/page_attention_v2.cu | 16 lines modified (+/-)