打开Mellanox官网下载驱动 Linux InfiniBand Drivers
点击下载链接跳转至
Tgz解压缩执行 ./mlnxofedinstall发现缺少模块
# ./mlnxofedinstall
Logs dir: /tmp/MLNX_OFED_LINUX.11337.logs
General log file: /tmp/MLNX_OFED_LINUX.11337.logs/general.log
Verifying KMP rpms compatibility with target kernel...
Error: One or more required packages for installing MLNX_OFED_LINUX are missing.
Please install the missing packages using your Linux distribution Package Management tool.
Run:
yum install kernel-modules-extra tk先安装 kernel modules
# yum install kernel-modules-extra tk
再安装OFED
# ./mlnxofedinstall
Logs dir: /tmp/MLNX_OFED_LINUX.33933.logs
General log file: /tmp/MLNX_OFED_LINUX.33933.logs/general.log
Verifying KMP rpms compatibility with target kernel...
This program will install the MLNX_OFED_LINUX package on your machine.
Note that all other Mellanox, OEM, OFED, RDMA or Distribution IB packages will be removed.
Those packages are removed due to conflicts with MLNX_OFED_LINUX, do not reinstall them.
Do you want to continue?[y/N]:y
Uninstalling MLNX_EN driver
rpm --nosignature -e --allmatches --nodeps libibverbs libibverbs-utils libibumad ibacm librdmacm opensm opensm-libs infiniband-diags opensm opensm-libs libibumad ibacm libibverbs libibverbs-utils librdmacm infiniband-diags
Starting MLNX_OFED_LINUX-23.10-1.1.9.0 installation ...
Verifying... ########################################
Preparing... ########################################
Verifying... ########################################
Preparing... ########################################
Installing mlnx-ofa_kernel RPM
Verifying... ########################################
Preparing... ########################################
Updating / installing...
mlnx-ofa_kernel-23.10-OFED.23.10.1.1.9########################################
Configured /etc/security/limits.conf
Installing kmod-mlnx-ofa_kernel 23.10 RPM
Verifying... ########################################
Preparing... ########################################
Installing mlnx-ofa_kernel-devel RPM
Verifying... ########################################
Preparing... ########################################
Updating / installing...
mlnx-ofa_kernel-devel-23.10-OFED.23.10########################################
Installing mlnx-ofa_kernel-source RPM
Verifying... ########################################
Preparing... ########################################
Updating / installing...
mlnx-ofa_kernel-source-23.10-OFED.23.1########################################
Installing kmod-kernel-mft-mlnx 4.26.1 RPM
Verifying... ########################################
Preparing... ########################################
Installing knem RPM
Verifying... ########################################
Preparing... ########################################
Updating / installing...
knem-1.1.4.90mlnx3-OFED.23.10.0.2.1.1.########################################
Installing kmod-knem 1.1.4.90mlnx3 RPM
Verifying... ########################################
Preparing... ########################################
Installing xpmem RPM
Verifying... ########################################
Preparing... ########################################
Updating / installing...
xpmem-2.7.3-1.2310055.rhel8u1 ########################################
Installing kmod-xpmem 2.7.3 RPM
Verifying... ########################################
Preparing... ########################################
Installing kmod-iser 23.10 RPM
Verifying... ########################################
Preparing... ########################################
Installing kmod-srp 23.10 RPM
Verifying... ########################################
Preparing... ########################################
Installing kmod-isert 23.10 RPM
Verifying... ########################################
Preparing... ########################################
Installing libxpmem 2.7.3 RPM
Verifying... ########################################
Preparing... ########################################
Updating / installing...
libxpmem-2.7.3-1.2310055.rhel8u1 ########################################
Installing user level RPMs:
Verifying... ########################################
Preparing... ########################################
Device (0000:2f:00.0):
0000:2f:00.0 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
Link Width: x16
PCI Link Speed: 8GT/s
Device (0000:2f:00.1):
0000:2f:00.1 Ethernet controller: Mellanox Technologies MT28908 Family [ConnectX-6]
Link Width: x16
PCI Link Speed: 8GT/s
Installation finished successfully.
Verifying... ################################# [100%]
Preparing... ################################# [100%]
Updating / installing...
1:mlnx-fw-updater-23.10-1.1.9.0 ################################# [100%]
Added 'RUN_FW_UPDATER_ONBOOT=no to /etc/infiniband/openib.conf
Initializing...
Attempting to perform Firmware update...
The firmware for this device is not distributed inside Mellanox driver: 0000:2f:00.0 (PSID: LNV0000000017)
To obtain firmware for this device, please contact your HW vendor.
Failed to update Firmware.
See /tmp/MLNX_OFED_LINUX.33933.logs/fw_update.log
To load the new driver, run:
/etc/init.d/openibd restart
# /etc/init.d/openibd restart
Unloading rdma_cm [FAILED]
rmmod: ERROR: Module rdma_cm is in use by: rpcrdma ib_srpt ib_isert
# modprobe -r rdma_cm
modprobe: FATAL: Module rdma_cm is in use.
# rmmod rdma_cm
rmmod: ERROR: Module rdma_cm is in use by: rpcrdma ib_srpt ib_isert
# cat /tmp/MLNX_OFED_LINUX.33933.logs/fw_update.log
The firmware for this device is not distributed inside Mellanox driver: 0000:2f:00.0 (PSID: LNV0000000017)
To obtain firmware for this device, please contact your HW vendor.
EXIT_STATUS: 2
继续安装OFED，但是kernel不支持，先增加kernel模块，缺少装啥即可
# yum install kernel-devel-4.18.0-305.19.1.el8_4.x86_64 kernel-rpm-macros
# bash mlnx_add_kernel_support.sh -m ./
Note: This program will create MLNX_OFED_LINUX TGZ for rhel8.1 under /tmp directory.
Do you want to continue?[y/N]:y
See log file /tmp/mlnx_iso.20469_logs/mlnx_ofed_iso.20469.log
Checking if all needed packages are installed...
/lib/modules/4.18.0-305.19.1.el8_4.x86_64/build//scripts is required to build mlnx-ofa_kernel-modules RPM.
Please install the corresponding kernel-devel RPM.
Error: One or more required packages for installing OFED-internal are missing.
Please install the missing packages using your Linux distribution Package Management tool.
Run:
yum install kernel-devel-4.18.0-305.19.1.el8_4.x86_64 kernel-rpm-macros
再继续安装
./mlnxofedinstall --add-kernel-support驱动需要重启Server次阿能生效，注意IB的VF路径与RDMA卡是不同的文章来源：https://www.toymoban.com/news/detail-809012.html
# echo 6 > /sys/class/infiniband/mlx5_0/device/mlx5_num_vfs
VF生成成功了文章来源地址https://www.toymoban.com/news/detail-809012.html
# lspci -nn|grep Mellanox
0000:2f:00.0 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6] [15b3:101b]
0000:2f:00.1 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6] [15b3:101b]
0000:2f:00.2 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6 Virtual Function] [15b3:101c]
0000:2f:00.3 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6 Virtual Function] [15b3:101c]
0000:2f:00.4 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6 Virtual Function] [15b3:101c]
0000:2f:00.5 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6 Virtual Function] [15b3:101c]
0000:2f:00.6 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6 Virtual Function] [15b3:101c]
0000:2f:00.7 Ethernet controller [0200]: Mellanox Technologies MT28908 Family [ConnectX-6 Virtual Function] [15b3:101c]
# ip link
link/ether b4:96:91:bf:94:94 brd ff:ff:ff:ff:ff:ff
16: ens1f0np0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether b8:3f:d2:d3:e4:c6 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off
vf 1 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off
vf 2 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off
vf 3 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off
vf 4 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off
vf 5 MAC 00:00:00:00:00:00, spoof checking off, link-state auto, trust off, query_rss off
到了这里，关于CentOS上安装Mellanox OFED的文章就介绍完了。如果您还想了解更多内容，请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章，希望大家以后多多支持TOY模板网！
