作者：石文
时间：2018-11-10
fio测试工具：
fio -name=test -bs=4k -rw=randwrite -filename=/data0/to-test1 -size=5g -group_reporting -iodepth=128 -numjobs=16 -runtime=60 -time_based
图示：
测试结果：
test: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=128
...
fio-3.1
Starting 16 processes
test: Laying out IO file (1 file / 5120MiB)
Jobs: 16 (f=16): [w(16)][100.0%][r=0KiB/s,w=686MiB/s][r=0,w=176k IOPS][eta 00m:00s]
test: (groupid=0, jobs=16): err= 0: pid=328286: Wed Dec 5 16:29:23 2018
write: IOPS=162k, BW=633MiB/s (664MB/s)(37.1GiB/60003msec)
clat (usec): min=3, max=294546, avg=98.06, stdev=1138.24
lat (usec): min=3, max=294546, avg=98.15, stdev=1138.24
clat percentiles (usec):
| 1.00th=[ 6], 5.00th=[ 7], 10.00th=[ 8], 20.00th=[ 16],
| 30.00th=[ 57], 40.00th=[ 85], 50.00th=[ 87], 60.00th=[ 89],
| 70.00th=[ 91], 80.00th=[ 93], 90.00th=[ 96], 95.00th=[ 99],
| 99.00th=[ 105], 99.50th=[ 110], 99.90th=[10028], 99.95th=[20579],
| 99.99th=[43254]
bw ( KiB/s): min= 1464, max=149720, per=6.25%, avg=40506.43, stdev=14148.69, samples=1912
iops : min= 366, max=37432, avg=10126.50, stdev=3537.17, samples=1912
lat (usec) : 4=0.01%, 10=14.26%, 20=8.89%, 50=6.09%, 100=66.97%
lat (usec) : 250=3.56%, 500=0.03%, 750=0.01%, 1000=0.01%
lat (msec) : 2=0.01%, 4=0.02%, 10=0.05%, 20=0.05%, 50=0.05%
lat (msec) : 100=0.01%, 250=0.01%, 500=0.01%
cpu : usr=1.02%, sys=68.60%, ctx=3527234, majf=0, minf=17587
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,9720295,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
WRITE: bw=633MiB/s (664MB/s), 633MiB/s-633MiB/s (664MB/s-664MB/s), io=37.1GiB (39.8GB), run=60003-60003msec
Disk stats (read/write):
sda: ios=328/1228067, merge=1/68643, ticks=4317/195420544, in_queue=195424286, util=28.40%
测试结论：
不加这个参数会使用buffer。-direct=1。
fio -name=test2 -direct=1 -ioengine=psync -bs=4k -rw=randwrite -filename=/data0/io-test2 -size=5g -group_reporting -iodepth=128 -numjobs=16 -runtime=60 -time_based
图解：
结果：
test2: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=128
...
fio-3.1
Starting 16 processes
test2: Laying out IO file (1 file / 5120MiB)
Jobs: 16 (f=16): [w(16)][100.0%][r=0KiB/s,w=37.3MiB/s][r=0,w=9536 IOPS][eta 00m:00s]
test2: (groupid=0, jobs=16): err= 0: pid=332920: Wed Dec 5 16:47:13 2018
write: IOPS=9333, BW=36.5MiB/s (38.2MB/s)(2190MiB/60072msec)
clat (usec): min=29, max=163714, avg=1711.97, stdev=10250.08
lat (usec): min=29, max=163714, avg=1712.14, stdev=10250.08
clat percentiles (usec):
| 1.00th=[ 36], 5.00th=[ 40], 10.00th=[ 46], 20.00th=[ 50],
| 30.00th=[ 56], 40.00th=[ 63], 50.00th=[ 71], 60.00th=[ 82],
| 70.00th=[ 100], 80.00th=[ 128], 90.00th=[ 215], 95.00th=[ 490],
| 99.00th=[ 66847], 99.50th=[ 83362], 99.90th=[102237], 99.95th=[107480],
| 99.99th=[117965]
bw ( KiB/s): min= 296, max= 9744, per=6.26%, avg=2337.71, stdev=1103.31, samples=1920
iops : min= 74, max= 2436, avg=584.27, stdev=275.84, samples=1920
lat (usec) : 50=19.69%, 100=50.36%, 250=21.00%, 500=4.03%, 750=1.46%
lat (usec) : 1000=0.17%
lat (msec) : 2=0.06%, 4=0.08%, 10=0.20%, 20=0.35%, 50=1.03%
lat (msec) : 100=1.44%, 250=0.13%
cpu : usr=0.09%, sys=2.52%, ctx=719614, majf=0, minf=1342
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwt: total=0,560697,0, short=0,0,0, dropped=0,0,0
latency : target=0, window=0, percentile=100.00%, depth=128
Run status group 0 (all jobs):
WRITE: bw=36.5MiB/s (38.2MB/s), 36.5MiB/s-36.5MiB/s (38.2MB/s-38.2MB/s), io=2190MiB (2297MB), run=60072-60072msec
Disk stats (read/write):
sda: ios=306/560135, merge=0/5, ticks=6201/929465, in_queue=936253, util=99.19%
IO引擎：
psync 通synx
sync 最正常的IO操作逻辑（先到缓存区就恢复OK，在一段时间内调用sync(1)冲刷块缓存区结束）
fsync 在真正持久化到磁盘后再返回结果！（每次写磁盘要进行2次IO）
fdatasync在真正持久化到磁盘后再返回结果！（实时的更新metadata,比fsync快一些）
https://blog.csdn.net/cywosp/article/details/8767327?utm_source=blogxgwz0
mmap
libaio 异步IO
磁盘写操作方法：
各进行100次操作并计算写入速度的均值
dd命令的方式：
877MB/s
for m in seq 1 100;do dd if=/dev/zero of=/data0/dd-tests7-$m iflag=nocache oflag=sync count=300 bs=1M ;done
核心参数：
iflag=nocache 读禁用缓存
oflag=sync 写操作当数据down到磁盘
iozone软件：
842MB/s
for m in seq 1 10;do /export/servers/iozone/iozone3_484/src/current/iozone -i 0 -t 1 -F /data0/iozone-test4-$m -w -r 1M -s 300M -R -o
核心参数：
-o 每次将数据写入磁盘后再计算写入时间
-t 单线程的方式进行写入
-i 写操作
-r 块大小
-s 文件大小
-R 输出Exacl的格式
IOZone 磁盘性能测试工具
./iozone -Rac
This tells Iozone to include the close() in the measurement. This may be needed if the client is running NFS version 3. Including the close() helps to reduce the client side cache effects of NFS version 3. If you use a file size that is larger than the amount of memory in the client then the ‘c’ flag is not needed.
-f
指定文件位置
-w
保留测试文件
-b
测试报告输出位置 output.xks
-o
写操作是同步写到磁盘的。 (O_SYNC). Iozone 会以O_SYNC 标志打开文件。这强制所有写操作完全写入磁盘后才返回测试。
-t
指定线程数量
-r
指定块大小
-s
指定文件大小
-i
指定 测试内容 1 写测试 2读测试
-D
对mmap文件使用msync(MS_ASYNC) 。这告诉操作系统在mmap空间的所有数据
需要被异步地写到磁盘上。
-B
使用mmap()文件。这将使用mmap()接口来创建并访问所有测试用的临时文件。一
些应用程序倾向于将文件当作内存的一块来看待。这些应用程序对文件执行mmap()
调用，然后就可以以读写内存的方式访问那个块来完成文件I/O。
-S
指定高速缓存的大小
-c
计算时间时将close()包括进来。This is useful only if you suspect that close() is
broken in the operating system currently under test. 对于NFS版本3测试而言这将会
很有用，同时它也能帮助我们识别nfs3_commit 是否正常工作。
Include close() in the timing calculations
-R
产生Excel到标准输出
-b
指定输出到指定文件上. 比如 -Rb ttt.xls
-I
对所有文件操作使用VxFS VX_DIRECT 。告诉VXFS 文件系统所有对文件的操作将跨
过缓存直接在磁盘上进行。
-p
在每个文件操作之前清除处理器缓存
-e
Include flush (fsync,fflush) in the timing calculations、
ab 域名并发压力测试
https://www.cnblogs.com/myvic/p/7703973.html
wrk
locust
基础知识：
Bytes 和bit 为字节和比特 1字节等于8比特
dd测试方法：
iflag=FLAG[,FLAG] 使用FLAG参数指定的输入方式
oflag=FLAG[,FLAG] 使用FLAG参数指定的输出方式
