we have 35 HP G9 and 40 HP G8 server. Help us for solution to create one or 2 HPCs.
Those are basic steps:
Hardware need to be the same, this is basic HPC requirement, so using two different servers is not recommended and you, probably, need some kind of job scheduler - SLURM, Torque, LSF, etc. If you are not using identical hardware, you most likely will have an issue related to performance.
Use the same network adapter ( onboard Ethernet, HighSpeed Adapter aka HCA). See item 1.
Install same OS, drivers
Is using IB, be sure to run OpenSM
If you are using Mellanox hardware, use HPC-X toolkit - http://www.mellanox.com/page/products_dyn?product_family=189&mtag=hpc-x http://www.mellanox.com/page/products_dyn?product_family=189&mtag=hpc-x
Your simple question, is is compilated subject that included almost everything - fabric design, networking, performance tuning including BIOS, OS and driver ( for reference you may check Mellanox Tuning Guide) and it is definitely should be splitted to different subjects.
Take, for example, this article - http://hpcugent.github.io/vsc_user_docs/pdf/intro-HPC-windows-gent.pdf http://hpcugent.github.io/vsc_user_docs/pdf/intro-HPC-windows-gent.pdf (135 pages)
I would suggest to start building and run jobs using onboard Ethernet adapter ( for HPC you can use any communication channel that exists on the host). When this phase is over, add Mellanox and you’ll get much better performance.