Hi.
I’m trying to use device memory in ConnectX-6.
When I query MEMIC_SIZE_LIMIT, it shows 256KB.
Is it the maximum device memory size I can allocate?
Also, what does MEMIC_BAR_SIZE mean?
I want to allocate more than 256KB on ConnectX-6 memory for partial DMA.
It is known to have 4MiB for its SRAM, but I do not know why MEMIC_SIZE_LIMIT cannot be set larger than 256KB.
Can you tell me how to allocate large device memory in ConnectX-6?
Thank you for posting your inquiry to the NVIDIA Developer Forums.
Description for these firmware configuration elements can be found via mlxconfig (included with Mellanox Firmware Tools - MFT).
For example, using the latest MFT (available at this link), and assuming your device is mlx5_0:
mlxconfig -d mlx5_0 show_confs
The MEMIC CONF section displays the following details:
MEMIC_BAR_SIZE=<NUM> The amount of BAR size assigned for MEMIC. The size in bytes is memic_size_limit*2^log_memic_bar_size.
When activating MEMIC Atomic access by MEMIC_ATOMIC, the allocated BAR size for MEMIC is doubled.
MEMIC_SIZE_LIMIT=<DISABLED|_1024KB|_256KB|_512KB>The maximum amount of internal device memory that can be consumed by the MEMIC application.
0x0: DISABLED
0x1: _256KB
0x2: _512KB
0x3: _1024KB
If you have any additional questions about this, and have valid support entitlement, please open a support ticket with Enterprise Support.