RHEL7: Large block read performance issue with XFS+LVM with Intel NVMe
https://access.redhat.com/solutions/3406851
SOLUTION 已验证 - 已更新 2019年二月28日23:06 -
环境
- Red Hat Enterprise Linux 7
XFS
on aLVM
Logical Volume- NVMe storage
问题
- Large block reads from a
XFS
file system backed by aLVM
Logical Volume on a Intel NVMe device are under performing. - This behavior is not observed on a
XFS
file system directly on the NVMe device. - Reads from a
EXT4
file system on a Logical Volume do not exhibit this behavior.
决议
Red Hat Enterprise Linux 7
- This issue has been resolved with the errata RHSA-2018:3083 for the package(s)
kernel-3.10.0-957.el7
or later.
根源
- File systems were not being created with optimal strip width at time of creation. This is because
# mkfs.xfs
was not receiving accurate information from opt_io hint. - chunk_sectors are now properly reported to io_min and io_opt.
- This now resolves the performance degradation.
诊断步骤
- Results from testing
XFS+LVM:
READ: bw=319MiB/s (335MB/s), 319MiB/s-319MiB/s (335MB/s-335MB/s), io=37.4GiB (40.2GB), run=120001-120001msec
READ: bw=321MiB/s (336MB/s), 321MiB/s-321MiB/s (336MB/s-336MB/s), io=37.6GiB (40.3GB), run=120001-120001msec
XFS+LVM+MQ
# cat /sys/module/dm_mod/parameters/use_blk_mq
Y
READ: bw=321MiB/s (337MB/s), 321MiB/s-321MiB/s (337MB/s-337MB/s), io=37.6GiB (40.4GB), run=120001-120001msec
XFS: Without LVM
READ: bw=1178MiB/s (1235MB/s), 1178MiB/s-1178MiB/s (1235MB/s-1235MB/s), io=138GiB (148GB), run=120001-120001msec
READ: bw=1067MiB/s (1119MB/s), 1067MiB/s-1067MiB/s (1119MB/s-1119MB/s), io=125GiB (134GB), run=120001-120001msec
EXT4: Without LVM
READ: bw=1156MiB/s (1212MB/s), 1156MiB/s-1156MiB/s (1212MB/s-1212MB/s), io=135GiB (145GB), run=120001-120001msec
EXT4+LVM:
READ: bw=1026MiB/s (1076MB/s), 1026MiB/s-1026MiB/s (1076MB/s-1076MB/s), io=120GiB (129GB), run=120002-120002msec