# Fp8/fp16 accumulation on ada RTX 4090

**URL:** <https://forums.developer.nvidia.com/t/fp8-fp16-accumulation-on-ada-rtx-4090/294400>\
**Category:** GPU-Accelerated Libraries\
**Tags:** cublas, cuda\
**Created:** [May 29, 2024, 4:13am UTC](https://forums.developer.nvidia.com/t/fp8-fp16-accumulation-on-ada-rtx-4090/294400 "2024-05-29T04:13:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![mustafak6483](https://developer.download.nvidia.com/images/forums/profile-default-devtalk-84.png) [@mustafak6483](https://forums.developer.nvidia.com/u/mustafak6483)\
**Post date:** [May 29, 2024, 4:13am UTC](https://forums.developer.nvidia.com/t/fp8-fp16-accumulation-on-ada-rtx-4090/294400/1 "2024-05-29T04:13:36Z")

</div>

This [old post](https://forums.developer.nvidia.com/t/ada-geforce-rtx-4090-fp8-cublaslt-performance/250737) has been resolved with the [updated Whitepaper](https://images.nvidia.com/aem-dam/Solutions/geforce/ada/nvidia-ada-gpu-architecture.pdf?_gl=1*wgnr05*_gcl_au*MTExNDUwMjc2Mi4xNzE0OTExNzU0), confirming fp32 accumulate is half-rate on geforce, but does cublas (cublasLtMatmul()) support fp8 with fp16 accumulate now? The [fp8 example](https://github.com/NVIDIA/CUDALibrarySamples/blob/master/cuBLASLt/LtFp8Matmul/sample_cublasLt_LtFp8Matmul.cu) breaks when I change CUBLAS\_COMPUTE\_32F to CUBLAS\_COMPUTE\_16F (Line 71), returning “cuBLAS API failed with status 15”, so I assume it’s not supported, but I want to make sure.

---

<div class="post-metadata">

**Author:** ![mnicely](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mnicely/32/14047_2.png) [@mnicely](https://forums.developer.nvidia.com/u/mnicely)\
**Post date:** [June 5, 2024, 4:28pm UTC](https://forums.developer.nvidia.com/t/fp8-fp16-accumulation-on-ada-rtx-4090/294400/2 "2024-06-05T16:28:27Z")

</div>

From [cublasLtMatmul()](https://docs.nvidia.com/cuda/cublas/index.html#cublasltmatmul)

> To use FP8 kernels, the following set of requirements must be satisfied:
> 
> - All matrix pointers must be 16-byte aligned.
> - A must be transposed and B non-transposed (The “TN” format).
> - **The compute type must be CUBLAS\_COMPUTE\_32F.**
> - The scale type must be CUDA\_R\_32F.

---

<div class="post-metadata">

**Author:** ![system](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/system/32/68080_2.png) [@system](https://forums.developer.nvidia.com/u/system)\
**Post date:** [June 19, 2024, 4:29pm UTC](https://forums.developer.nvidia.com/t/fp8-fp16-accumulation-on-ada-rtx-4090/294400/3 "2024-06-19T16:29:05Z")

</div>

Closed on June 19, 2024, 4:29pm UTC

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.