FlexBoot linux IBoIP

Dear community.

Trying to figure out how to use FlexBoot for our IB only server nodes.

Reading the documents

http://www.mellanox.com/related-docs/prod_software/Mellanox_PXE_User_Guide.pdf http://www.mellanox.com/related-docs/prod_software/Mellanox_PXE_User_Guide.pdf

http://www.mellanox.com/related-docs/prod_software/FlexBoot_User_Manual_v1.4.pdf http://www.mellanox.com/related-docs/prod_software/FlexBoot_User_Manual_v1.4.pdf

and using the download area

Mellanox Products: Mellanox FlexBoot http://www.mellanox.com/page/products_dyn?&product_family=34&mtag=flexboot

So basically I find for download: customized versions of initrd and vmlinuz to be used in the PXE server

to netboot the nodes to be installed.

RHEL 5.6 and Centos 5.6 are the latest versions to be downloaded.

Question: What do I have to do in order to remote install a later version like RH/Centos 6.4 or 6.5 ?

  1. are there any plans to produce newer version of initrd and vmlinuz for these later version

  2. is it possible to modify the initrd following the 7.1.1 example adding IB driver to initrd (linux)” in the document FlexBoot_User_Manual, but if so, what needs to be done with the vmlinuz file?

Regards, Ingvar

I am also trying to boot via PXE on Infiniband with ConnectIB card and I have found out, there are available Driver Update disc for Ethernet on FlexBoot site: DUD MLNX_EN for RHEL7.1 Driver http://www.mellanox.com/downloads/Drivers/DUD/dd-rhel7.1-mlnx-en-3.1-1.0.0-x86_64.iso.gz

Is there anything similar for PXE over Infiniband on CentOS 7.2?

Thank you for your answer

Can you elaborate which configurations got lost ?

For example is ISCSI configuration is done thorough DHCP , they are

persistent.

Regarding our ability to reserve configurations - let’s review

our configurations setting options and their persistency:

Option 1 : DHCP server - configurations are stored in the server.

Option 2 : External Tool - which configures the adapter:

  1. UEFI - HII (Mellanox Driver) :

NIC configurations , Boot configurations are persistent and apply to

legacy OptionRom ( FlecBoot).

  1. OS - user level application -

configuration tool - being defined these days (to configure FlexBoot)

Option3 : FlexBoot GUI - for configuring the Adapter (invoked by pressing

hot key) - NIC configurations , Boot configurations are

persistent. Currently this GUI is at beta level and should be

available around August.

Thanks!

For IB support you will need to install the MellanoxOFED package (that includes drivers for IB and Eth)

Mellanox Products: Mellanox OpenFabrics Enterprise Distribution for Linux (MLNX_OFED) http://www.mellanox.com/page/products_dyn?product_family=26&mtag=linux_sw_drivers

Hi ingvar_j,

“Question: What do I have to do in order to remote install a later version like RH/Centos 6.4 or 6.5 ?”

Nothing! support is built in.

in any case - you can find RH/CentOS 6.4 at :

http://www.mellanox.com/downloads/Drivers/DUD/DUD-mlnx_en-2.0-3.0.0-rhel6.4.tgz http://www.mellanox.com/downloads/Drivers/DUD/DUD-mlnx_en-2.0-3.0.0-rhel6.4.tgz

Hi!

I was tried option 3.

iSCSI boot over IPoIB and SRP boot.

Both option was works well.

ESXi installer can install on boot LUN 0.

This LUNs isolated each ESXi host.

Installation finished and reboot the host and host showed a message of can’t find boot device.

Sure!

My system BIOS support IB boot sequence.

I was check configuration via press hot key.

There isn’t any configuration and clean status.

I’m also tried reconfigure and boot with LUN 0 that have a ESXI boot image.

Boot process work perfectly in both iSCSI, SRP targets.

This test proceed in last year this time.

But I was give up that configuration.

Because there is any command that save configuration permernantly.

And…

I’m also read FlexBoot release note.

That shows me a one message that can’t support QSFP cable with latest ConnectX-2 firmware.

It seems only support ConnectX-3 with QSFP cable.

Is there any solution?

will try it, although it states “mlnx_en_xxxxx_rhel6.4” and I’m about to run it over IB.

Can I use the same image?

This install is currently in my lab servers only, but its in my master plan to disconnect all ethernet cables in favour for qdr or fdr .

Regards

I was very disappointed with FlexBoot.

I think OptionROM has a unique room for reserve any configurations (SRP, iSCSI,PXE) like CNA’s from

Intel, Chelsio, Emulex and so on.

If you reboot the server then all configuration will be go away…

IPoIB installation fails as RedHat/CentOS doesn’t support it. Even if the IPoIB modules are integrated into the RAMFS, installation will not recognize ipoib device as suitable for installation