Amount of memory required for ibv_reg_mr on Mellanox card

Hello,

I am new with this so pardon my ignorance but I have a question. I want to register large amount(at least a few hundred GBs) of memory using ibv_reg_mr. The ibv_reg_mr maps the memory so it must be creating some kind of page table right?

I want to calculate the size of the page table created by ibv_reg_mr so that I can calculate the total amount of memory that would be required to store the mr of a particular amount of memory.

Can someone explain the logic of calculating such required amount of memory to store the mr on a Mellanox card? or is there a blog or source code available somewhere which I can look through to understand the logic? Only RAM will be used to store the page table or there will be some of the stuff stored on the device too? Note that I am talking about creating a single mr for all the memory to be registered and not doing this in chunk of small memory blocks.

Hello Kartik,

Thank you for posting your question on the Mellanox Community.

Please review the following community post which provides the formula on how-to calculate the pagepool for ibv_mem_reg → https://community.mellanox.com/s/article/howto-increase-memory-size-used-by-mellanox-adapters

If you need more assistance regarding to this subject, please open a NVIDIA/Mellanox Support Ticket (valid support contract required) by sending an email to support@mellanox.com

Thank you,

~NVIDIA/Mellanox Technical Support

Thanks Martijn for the answer. I saw the post and looks like it explains the calculation for mlx4 driver. Can you please confirm if the calculation is same for mlx5 driver as well? because I do not think that mlx5 driver contain parameters like log_num_mtt as mentioned in the post. Also, does this calculation apply to all the new Mellanox cards such as ConnectX-4 LX or newer versions?

Thanks in advance!