Here is a good starting page: http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_2_4-1_0_0.pdf http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_Release_Notes_2_4-1_0_0.pdf
in the release notes you will also find the supported operating systems and kernel versions.
once you installed the OFED package (which includes the drivers and all tools), you will need to run the opensm daemon to get the network going. start that daemon on one or two servers. all other machines will not need it.
You didn’t mention what is out there beyond the servers. anyways, good luck. we are here!