Linux下Python脚本的编写解析fio测试log(四)

这篇具有很好参考价值的文章主要介绍了Linux下Python脚本的编写解析fio测试log(四)。希望对大家有所帮助。如果存在错误或未考虑完全的地方,请大家不吝赐教,您也可以点击"举报违法"按钮提交疑问。

在我们平常的测试盘的过程中,fio是一款业界最为通用的工具,通过Fio可以只知道一个盘具体的带宽,iops,延迟等等.比如下面这段fio的log
Linux下Python脚本的编写解析fio测试log(四)
Linux下Python脚本的编写解析fio测试log(四)

这里边需要拿出log里的一些参数 做成表格 如下:
Linux下Python脚本的编写解析fio测试log(四)
下面说下如何用Python解析log并做成表格
1 同样是用到Python中文件函数open,调用四个模块sys,os,re和argparse模块 如下:

#!/usr/bin/python
import argparse
import  sys
import os
import re

sys.argv[n]是传递Python脚本的第几个参数,这里我主要用到三个参数:
inputfile :代表要打开的fio log文件
resultfile:代表要输出的解析结果
rwm: 代表read.write,mix三种读写模式
os 模块:调用Linux指令
re模块: 筛选字符串所用
这里因为要考虑两种读写模式:混合读写和非混合读写模式
因为两种log的格式不一样
2 第二步就是把里边的值筛选出来了
iops = IOPS
bw = 带宽
avg_lat = lat平均延迟
max_lat = lat最大延迟
clat99 = 代表clat99%的点延迟
clat999 = 代表clat99.9%的点延迟
clat9999 =代表clat99.99%的点延迟
clat_unit = clat延迟的单位
(1)IOPS和BW在同一行可以用 ","分割来求出,BW的话需要过滤掉括号,并且按照MB单位进行换算

#!/usr/bin/python
import argparse
import  sys
import os
import re
inputfile = sys.argv[1]
resultfile = sys.argv[3]
rwm = sys.argv[2]
def not_mix_fio():
    datastr = open(inputfile).read()
    data=datastr.split(",")
    datalat=datastr.split("\n")
    iops = ""
    bw = ""
    avg_lat = ""
    max_lat = ""
    clat99 = ""
    clat999 = ""
    clat9999 = ""
    clat_unit = ""
    for i in data:
        if "IOPS" in i:
          iops = i.split("IOPS=")[1].strip()
        if "BW" in i:
            bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', bw)
            bw=bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               bw=float(bw)/1000  
            if unit == "b/s" or unit == "B/s":
               bw=float(bw)/1000000 
    print("iops=",iops)
    print("bw=",bw)

运行如下:
Linux下Python脚本的编写解析fio测试log(四)
(2)筛选出lat的最大值和平均值 并且单位以ms为准
以换行符分割较为简单

#!/usr/bin/python
import argparse
import  sys
import os
import re
inputfile = sys.argv[1]
resultfile = sys.argv[3]
rwm = sys.argv[2]
def not_mix_fio():
    datastr = open(inputfile).read()
    data=datastr.split(",")
    datalat=datastr.split("\n")
    iops = ""
    bw = ""
    avg_lat = ""
    max_lat = ""
    clat99 = ""
    clat999 = ""
    clat9999 = ""
    clat_unit = ""
    for i in data:
        if "IOPS" in i:
          iops = i.split("IOPS=")[1].strip()
        if "BW" in i:
          
            temp=re.findall(r'[(](.*?)[)]', i)
            bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', bw)
            bw=bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               bw=float(bw)/1000  
            if unit == "b/s" or unit == "B/s":
               bw=float(bw)/1000000 
    print("iops=",iops)
    print("bw=",bw)
    for line in datalat:
        if "lat" in line and "clat" not in line and "slat" not in line and "max" in line and "avg" in line:
            if "max" in line:
                max_lat = line.split("max=")[1].split(",")[0].strip()
            if "K" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000)
            if "M" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000000)
           
            if "usec" in line:
                    max_lat = str(float(max_lat)/1000)
            if "nsec" in line:
                    max_lat = str(float(max_lat)/1000000)

            if "avg" in line:
                avg_lat = line.split("avg=")[1].split(",")[0].strip()
            if "K" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000)
            if "M" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000000)
           
            if "usec" in line:
                    avg_lat = str(float(avg_lat)/1000)
            if "nsec" in line:
                    avg_lat = str(float(avg_lat)/1000000)
            print("avg_lat=",avg_lat)
            print("max_lat=",max_lat)
            sys.exit(1)

打印如下:
Linux下Python脚本的编写解析fio测试log(四)(3)求出99 999 和999的值 单位仍然以ms为主

#!/usr/bin/python
import argparse
import  sys
import os
import re
inputfile = sys.argv[1]
resultfile = sys.argv[3]
rwm = sys.argv[2]
def not_mix_fio():
    datastr = open(inputfile).read()
    data=datastr.split(",")
    datalat=datastr.split("\n")
    iops = ""
    bw = ""
    avg_lat = ""
    max_lat = ""
    clat99 = ""
    clat999 = ""
    clat9999 = ""
    clat_unit = ""
    for i in data:
        if "IOPS" in i:
          iops = i.split("IOPS=")[1].strip()
        if "BW" in i:
            temp=re.findall(r'[(](.*?)[)]', i)
            bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', bw)
            bw=bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               bw=float(bw)/1000  
            if unit == "b/s" or unit == "B/s":
               bw=float(bw)/1000000 
    print("iops=",iops)
    print("bw=",bw)
    for line in datalat:
        if "lat" in line and "clat" not in line and "slat" not in line and "max" in line and "avg" in line:
            if "max" in line:
                max_lat = line.split("max=")[1].split(",")[0].strip()
            if "K" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000)
            if "M" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000000)
            if "usec" in line:
                    max_lat = str(float(max_lat)/1000)
            if "nsec" in line:
                    max_lat = str(float(max_lat)/1000000)

            if "avg" in line:
                avg_lat = line.split("avg=")[1].split(",")[0].strip()
            if "K" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000)
            if "M" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000000)
            if "usec" in line:
                    avg_lat = str(float(avg_lat)/1000)
            if "nsec" in line:
                    avg_lat = str(float(avg_lat)/1000000)
            print("avg_lat=",avg_lat)
            print("max_lat=",max_lat)
            #sys.exit(1)
        if "clat percentiles" in line and "usec" in line:
            clat_unit = "usec"
        if "clat percentiles" in line and "nsec" in line:
            clat_unit = "nsec"
        if "99.00th" in line:
            clat99 = line.split("99.00th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
            clat999 = line.split("99.90th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
        if "99.99th" in line:
            clat9999=line.split("99.99th=")[1].split("[")[1].split("]")[0].strip()

    if clat_unit == "usec":
            clat9999 = str(float(clat9999)/1000)
            clat999 = str(float(clat999) / 1000)
            clat99 = str(float(clat99) / 1000)
    if clat_unit == "nsec":
            clat9999 = str(float(clat9999)/1000000)
            clat999 = str(float(clat999) / 1000000)
            clat99 = str(float(clat99) / 1000000)
    print("clat99=",clat99)
    print("clat999=",clat999)
    print("clat9999=",clat9999)
    sys.exit()

打印如下:
Linux下Python脚本的编写解析fio测试log(四)
(4)接下来就是把这些值放在CSV里,还是调用open函数

#!/usr/bin/python
import argparse
import  sys
import os
import re
inputfile = sys.argv[1]
resultfile = sys.argv[3]
rwm = sys.argv[2]
def not_mix_fio():
    datastr = open(inputfile).read()
    data=datastr.split(",")
    datalat=datastr.split("\n")
    iops = ""
    bw = ""
    avg_lat = ""
    max_lat = ""
    clat99 = ""
    clat999 = ""
    clat9999 = ""
    clat_unit = ""
    for i in data:
        if "IOPS" in i:
          iops = i.split("IOPS=")[1].strip()
        if "BW" in i:
            temp=re.findall(r'[(](.*?)[)]', i)
            bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', bw)
            bw=bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               bw=float(bw)/1000  
            if unit == "b/s" or unit == "B/s":
               bw=float(bw)/1000000 
    print("iops=",iops)
    print("bw=",bw)
    for line in datalat:
        if "lat" in line and "clat" not in line and "slat" not in line and "max" in line and "avg" in line:
            if "max" in line:
                max_lat = line.split("max=")[1].split(",")[0].strip()
            if "K" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000)
            if "M" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000000)
            if "usec" in line:
                    max_lat = str(float(max_lat)/1000)
            if "nsec" in line:
                    max_lat = str(float(max_lat)/1000000)

            if "avg" in line:
                avg_lat = line.split("avg=")[1].split(",")[0].strip()
            if "K" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000)
            if "M" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000000)
            if "usec" in line:
                    avg_lat = str(float(avg_lat)/1000)
            if "nsec" in line:
                    avg_lat = str(float(avg_lat)/1000000)
            print("avg_lat=",avg_lat)
            print("max_lat=",max_lat)
            #sys.exit(1)
        if "clat percentiles" in line and "usec" in line:
            clat_unit = "usec"
        if "clat percentiles" in line and "nsec" in line:
            clat_unit = "nsec"
        if "99.00th" in line:
            clat99 = line.split("99.00th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
            clat999 = line.split("99.90th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
        if "99.99th" in line:
            clat9999=line.split("99.99th=")[1].split("[")[1].split("]")[0].strip()

    if clat_unit == "usec":
            clat9999 = str(float(clat9999)/1000)
            clat999 = str(float(clat999) / 1000)
            clat99 = str(float(clat99) / 1000)
    if clat_unit == "nsec":
            clat9999 = str(float(clat9999)/1000000)
            clat999 = str(float(clat999) / 1000000)
            clat99 = str(float(clat99) / 1000000)
    print("clat99=",clat99)
    print("clat999=",clat999)
    print("clat9999=",clat9999)
    #sys.exit()
    write_line = "filename,bandwidth(MB/s),iops,avg_latency(msec),p99_lat(msec),p999_lat(msec),p9999_lat(msec),max_latency(msec)\n"
    fd = open(resultfile + ".csv","a")
    if not os.path.getsize(resultfile + ".csv"):
        fd.write(write_line)
    fd.close()
    write_line = "%s,%s,%s,%s,%s,%s,%s,%s\n" %(inputfile.split("/")[-1],bw,iops,avg_lat,clat99,clat999,clat9999,max_lat)
    fd=open(resultfile + ".csv","a")
    fd.write(write_line)
    fd.close()

表格如下:
Linux下Python脚本的编写解析fio测试log(四)
3 后面把混合读写解析方法补上几乎类似.这里我是用split(“write:”)来分割read和write


#!/usr/bin/python
#Autor :xiao xuetao
import argparse
import  sys
import os
import re
inputfile = sys.argv[1]
resultfile = sys.argv[3]
rwm = sys.argv[2]
def not_mix_fio():
    datastr = open(inputfile).read()
    data=datastr.split(",")
    datalat=datastr.split("\n")
    iops = ""
    bw = ""
    avg_lat = ""
    max_lat = ""
    clat99 = ""
    clat999 = ""
    clat9999 = ""
    clat_unit = ""
    for i in data:
        if "IOPS" in i:
          iops = i.split("IOPS=")[1].strip()
        if "BW" in i:
            temp=re.findall(r'[(](.*?)[)]', i)
            bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', bw)
            bw=bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               bw=float(bw)/1000  
            if unit == "b/s" or unit == "B/s":
               bw=float(bw)/1000000  
    for line in datalat:
        if "lat" in line and "clat" not in line and "slat" not in line and "max" in line and "avg" in line:
            if "max" in line:
                max_lat = line.split("max=")[1].split(",")[0].strip()
            if "K" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000)
            if "M" in max_lat.upper():
                max_lat = str(float(max_lat[:-1])*1000000)
            if "usec" in line:
                    max_lat = str(float(max_lat)/1000)
            if "nsec" in line:
                    max_lat = str(float(max_lat)/1000000)

            if "avg" in line:
                avg_lat = line.split("avg=")[1].split(",")[0].strip()
            if "K" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000)
            if "M" in avg_lat.upper():
                avg_lat = str(float(avg_lat[:-1])*1000000)
            if "usec" in line:
                    avg_lat = str(float(avg_lat)/1000)
            if "nsec" in line:
                    avg_lat = str(float(avg_lat)/1000000)
            
        if "clat percentiles" in line and "usec" in line:
            clat_unit = "usec"
        if "clat percentiles" in line and "nsec" in line:
            clat_unit = "nsec"
        if "99.00th" in line:
            clat99 = line.split("99.00th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
            clat999 = line.split("99.90th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
        if "99.99th" in line:
            clat9999=line.split("99.99th=")[1].split("[")[1].split("]")[0].strip()

    if clat_unit == "usec":
            clat9999 = str(float(clat9999)/1000)
            clat999 = str(float(clat999) / 1000)
            clat99 = str(float(clat99) / 1000)
    if clat_unit == "nsec":
            clat9999 = str(float(clat9999)/1000000)
            clat999 = str(float(clat999) / 1000000)
            clat99 = str(float(clat99) / 1000000)

    write_line = "filename,bandwidth(MB/S),iops,avg_latency(msec),p99_lat(msec),p999_lat(msec),p9999_lat(msec),max_latency(msec)\n"
    fd = open(resultfile + ".csv","a")
    if not os.path.getsize(resultfile + ".csv"):
        fd.write(write_line)
    fd.close()
    write_line = "%s,%s,%s,%s,%s,%s,%s,%s\n" %(inputfile.split("/")[-1],bw,iops,avg_lat,clat99,clat999,clat9999,max_lat)
    fd=open(resultfile + ".csv","a")
    fd.write(write_line)
    fd.close()

def mix_fio():
    datastr = open(inputfile).read()
    data = datastr.split("write:")
    read_data = data[0].split("\n")
    write_data = data[1].split("\n")
    read_iops = ""
    read_bw = ""
    read_avg_lat = ""
    read_max_lat = ""
    read_clat99 = ""
    read_clat999 = ""
    read_clat9999 = ""
    read_clat_unit = ""
    write_iops = ""
    write_bw = ""
    write_avg_lat = ""
    write_max_lat = ""
    write_clat99 = ""
    write_clat999 = ""
    write_clat9999 = ""
    write_clat_unit = ""
    for i in data[0].split(","):
        if "IOPS" in i:
            read_iops = i.split("IOPS=")[1].strip()
        if "BW" in i:           
            temp=re.findall(r'[(](.*?)[)]', i)
            read_bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', read_bw)
            read_bw=read_bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               read_bw=float(read_bw)/1000  
            if unit == "b/s" or unit == "B/s":
               read_bw=float(read_bw)/1000000  

    for line in read_data:
        if "lat" in line and "clat" not in line and "slat" not in line and "max" in line and "avg" in line:
            if "max" in line:
                read_max_lat = line.split("max=")[1].split(",")[0].strip()
            if "K" in read_max_lat.upper():
                read_max_lat = str(float(read_max_lat[:-1]) * 1000)
            if "M" in read_max_lat.upper():
                read_max_lat = str(float(read_max_lat[:-1]) * 1000000)
            
            if "usec" in line:
                read_max_lat = str(float(read_max_lat) / 1000)
            if "nsec" in line:
                read_max_lat = str(float(read_max_lat) / 1000000)

            if "avg" in line:
                read_avg_lat = line.split("avg=")[1].split(",")[0].strip()
            if "K" in read_avg_lat.upper():
                read_avg_lat = str(float(read_avg_lat[:-1]) * 1000)
            if "M" in read_avg_lat.upper():
                read_avg_lat = str(float(read_avg_lat[:-1]) * 1000000)
            
            if "usec" in line:
                read_avg_lat = str(float(read_avg_lat) / 1000)
            if "nsec" in line:
                read_avg_lat = str(float(read_avg_lat) / 1000000)
        if "clat percentiles" in line and "usec" in line:
            read_clat_unit = "usec"
        if "clat percentiles" in line and "nsec" in line:
            read_clat_unit = "nsec"
        if "99.00th" in line:
            read_clat99 = line.split("99.00th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
            read_clat999 = line.split("99.90th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
        if "99.99th" in line:
            read_clat9999 = line.split("99.99th=")[1].split("[")[1].split("]")[0].strip()

    if read_clat_unit == "usec":
        read_clat9999 = str(float(read_clat9999) / 1000)
        read_clat999 = str(float(read_clat999) / 1000)
        read_clat99 = str(float(read_clat99) / 1000)
    if read_clat_unit == "nsec":
        read_clat9999 = str(float(read_clat9999) / 1000000)
        read_clat999 = str(float(read_clat999) / 1000000)
        read_clat99 = str(float(read_clat99) / 1000000)

    for i in data[1].split(","):
        if "IOPS" in i:
            write_iops = i.split("IOPS=")[1].strip()
        if "BW" in i:
            temp=re.findall(r'[(](.*?)[)]', i)
            write_bw=temp[0]
            unit= re.sub(r'[0-9.]+', '', write_bw)
            write_bw=write_bw.split(unit)[0].strip()
            if unit == "kB/s" or unit == "KB/s":
               write_bw=float(write_bw)/1000  
            if unit == "b/s" or unit == "B/s":
               write_bw=float(write_bw)/1000000 
    for line in write_data:
        if "lat" in line and "clat" not in line and "slat" not in line and "max" in line and "avg" in line:
            if "max" in line:
                write_max_lat = line.split("max=")[1].split(",")[0].strip()
            if "K" in write_max_lat.upper():
                write_max_lat = str(float(write_max_lat[:-1]) * 1000)
            if "M" in write_max_lat.upper():
                write_max_lat = str(float(write_max_lat[:-1]) * 1000000)
            
            if "usec" in line:
                write_max_lat = str(float(write_max_lat) / 1000)
            if "nsec" in line:
                write_max_lat = str(float(write_max_lat) / 1000000)

            if "avg" in line:
                write_avg_lat = line.split("avg=")[1].split(",")[0].strip()
            if "K" in write_avg_lat.upper():
                write_avg_lat = str(float(write_avg_lat[:-1]) * 1000)
            if "M" in write_avg_lat.upper():
                write_avg_lat = str(float(write_avg_lat[:-1]) * 1000000)
            
            if "usec" in line:
                write_avg_lat = str(float(write_avg_lat) / 1000)
            if "nsec" in line:
                write_avg_lat = str(float(write_avg_lat) / 1000000)
        if "clat percentiles" in line and "usec" in line:
            write_clat_unit = "usec"
        if "clat percentiles" in line and "nsec" in line:
            write_clat_unit = "nsec"
        if "99.00th" in line:
            write_clat99 = line.split("99.00th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
            write_clat999 = line.split("99.90th=")[1].split(",")[0].split("[")[1].split("]")[0].strip()
        if "99.99th" in line:
            write_clat9999 = line.split("99.99th=")[1].split("[")[1].split("]")[0].strip()

    if write_clat_unit == "usec":
        write_clat9999 = str(float(write_clat9999) / 1000)
        write_clat999 = str(float(write_clat999) / 1000)
        write_clat99 = str(float(write_clat99) / 1000)
    if write_clat_unit == "nsec":
        write_clat9999 = str(float(write_clat9999) / 1000000)
        write_clat999 = str(float(write_clat999) / 1000000)
        write_clat99 = str(float(write_clat99) / 1000000)
    write_line = "filename,read,bandwidth(MB/S),iops,avg_latency(msec),p99_lat(msec),p999_lat(msec),p9999_lat(msec),max_latency(msec),write,bandwidth(MB/s),iops,avg_latency(msec),p99_lat(msec),p999_lat(msec),p9999_lat(msec),max_latency(msec)\n"
    fd = open(resultfile + ".csv", "a")
    if not os.path.getsize(resultfile + ".csv"):
        fd.write(write_line)
    fd.close()
    write_line = "%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s\n" % (
    inputfile.split("/")[-1], "read", read_bw, read_iops, read_avg_lat, read_clat99, read_clat999, read_clat9999, read_max_lat,
    "write", write_bw, write_iops, write_avg_lat, write_clat99, write_clat999, write_clat9999, write_max_lat)
    fd = open(resultfile + ".csv", "a")
    fd.write(write_line)
    fd.close()

if rwm == "mix":
    mix_fio()
else:
    not_mix_fio()


表格如下:Linux下Python脚本的编写解析fio测试log(四)
4 Python部分已经完成,接下来可以写一个shell小脚本跑下fio, 以Intel的p5510 nvme 为例子

#!/bin/bash
DEV=$1 TIME=$2

if [ $# != 2 ]; then
	echo "You must input 2 arg: 1st is device name                           
	                            2nd is the time of each testcase (unit is second)"

	exit -1
fi

echo "Testing device: $DEV"
echo "Run time of every testcase is ${TIME}s"
nvme format /dev/$DEV
mkdir "$DEV"
mkdir ./"$DEV"/test_log
mkdir ./"$DEV"/test_data

dmidecode | grep "System Information" -A9 | egrep "Manufacturer|Product|Serial" > ./"$DEV"/test_log/server_info.log
lscpu > ./"$DEV"/test_log/cpu_info.log
dmidecode -t memory > ./"$DEV"/test_log/memory_info.log
uname -a > ./"$DEV"/test_log/os_info.log
lspci | grep -i non > ./"$DEV"/test_log/lspci_info.log
nvme id-ctrl /dev/$DEV > ./"$DEV"/test_log/"$DEV"_id_info.log
nvme smart-log /dev/$DEV > ./"$DEV"/test_log/"$DEV"_smart_info_before.log

#************************One thread sequential write Precondition*************************#
echo "One thread sequential write predonditon"
date
#fio --name=Precondition --filename=/dev/$DEV --ioengine=libaio --direct=1 --thread=1 --numjobs=1 --iodepth=128 --rw=write --bs=128k --loops=2 --size=100% --group_reporting  >> "$DEV"_one_thread_seq_wr_precondition.log

#**************************One thread sequential Test Start*************************#
echo "One thread sequential write test"
date
for blk_size in 1024 256 128 
do
	for jobs in 1
	do
		for queue_depth in 64
		do
			job_name="${blk_size}kB_seq_WR_${jobs}job_QD${queue_depth}"
			echo "$job_name"
			fio --name=${job_name} --filename=/dev/$DEV --ioengine=libaio --direct=1 --thread=1 --numjobs=${jobs} --iodepth=${queue_depth} --rw=write --bs=${blk_size}k --runtime=$TIME --time_based=1 --size=100% --group_reporting  > "$DEV"_${blk_size}kB_seq_WR_${jobs}job_QD${queue_depth}.log
      python xuetao.py "$DEV"_${blk_size}kB_seq_WR_${jobs}job_QD${queue_depth}.log write ${DEV}_table
		done
	done
done



echo "One thread sequential read test"
date
for blk_size in 1024 256 128 
do
	for jobs in 1
	do
		for queue_depth in 64
		do
			job_name="${blk_size}kB_seq_RD_${jobs}job_QD${queue_depth}"
			echo "$job_name"
			fio --name=${job_name} --filename=/dev/$DEV --ioengine=libaio --direct=1 --thread=1 --numjobs=${jobs} --iodepth=${queue_depth} --rw=read --bs=${blk_size}k --runtime=$TIME --time_based=1 --size=100% --group_reporting  >> "$DEV"_${blk_size}kB_seq_RD_${jobs}job_QD${queue_depth}.log
      python xuetao.py "$DEV"_${blk_size}kB_seq_RD_${jobs}job_QD${queue_depth}.log read ${DEV}_table
		done
	done
done

echo "One thread sequential rw test"
date
for blk_size in 1024 256 128 
do
	for jobs in 1
	do
		for queue_depth in 64
		do
			job_name="${blk_size}kB_rw_${jobs}job_QD${queue_depth}"
			echo "$job_name"
			fio --name=${job_name} --filename=/dev/$DEV --ioengine=libaio --direct=1 --thread=1 --numjobs=${jobs} --iodepth=${queue_depth} --rw=rw --bs=${blk_size}k --runtime=$TIME --time_based=1 --size=100% --group_reporting  >> "$DEV"_${blk_size}kB_rw_${jobs}job_QD${queue_depth}.log
      python xuetao.py "$DEV"_${blk_size}kB_rw_${jobs}job_QD${queue_depth}.log mix ${DEV}_mix_table
		done
	done
done


nvme smart-log /dev/$DEV > ./"$DEV"/test_log/"$DEV"_smart_info_after.log
nvme error-log /dev/$DEV > ./"$DEV"/test_log/"$DEV"_error_log_after.log

mv "$DEV"_*.log ./"$DEV"/test_log
mv "$DEV"_*.csv ./"$DEV"/test_data

date
echo "test done"

运行如下:
Linux下Python脚本的编写解析fio测试log(四)

混合读写:
Linux下Python脚本的编写解析fio测试log(四)文章来源地址https://www.toymoban.com/news/detail-477336.html

到了这里,关于Linux下Python脚本的编写解析fio测试log(四)的文章就介绍完了。如果您还想了解更多内容,请在右上角搜索TOY模板网以前的文章或继续浏览下面的相关文章,希望大家以后多多支持TOY模板网!

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处: 如若内容造成侵权/违法违规/事实不符,请点击违法举报进行投诉反馈,一经查实,立即删除!

领支付宝红包 赞助服务器费用

相关文章

  • Linux性能测试工具 stress,stress-ng,sysbench,fio以及检测dstat

    通过man帮助,可以看到官方对dstat的定义为:多功能系统资源统计生成工具( versatile tool for generating system resource statistics)。在获取的信息上有点类似于top、free、iostat、vmstat等多个工具的合集,官方解释为vmstat、iostat、ifstat等工具的多功能替代品,且添加了许多额外的功能(

    2024年02月05日
    浏览(33)
  • jmeter编写测试脚本大全

    目录 一、背景 二、按照功能划分 2.1 加密处理、验签处理 2.2 jmeter 使用beanshell 编写脚本 2.3 jmeter脚本报错大全 2.4 jmeter打印log 2.5 jmeter压测java代码 2.6 jmeter读取数据保证每个线程拿到的数据唯一 2.7 jmeter设置并发量线程数为变量 2.8 beanshell 将数据写入到本地txt 2.9 beanshell 读

    2023年04月10日
    浏览(25)
  • JMeter 测试脚本编写技巧

     是一款开源软件,用于进行负载测试、性能测试及功能测试。测试人员可以使用 JMeter 编写测试脚本,模拟多种不同的负载情况,从而评估系统的性能和稳定性。以下是编写 JMeter 测试脚本的步骤。 第 1 步:创建测试计划 在JMeter中,测试计划是测试的最高级别,它包含了各

    2024年02月07日
    浏览(30)
  • 如何用 JMeter 编写性能测试脚本?

    Apache JMeter 应该是应用最广泛的性能测试工具。怎么用 JMeter 编写性能测试脚本? img HTTP 请求各项信息说明(以 JMeter 5.1 为例)。 如下图所示: Web服务器: 指定协议、HTTP 请求的主机地址和端口号,不需要加上“ http:// ”,JMeter 会自动加上,一般的 Web 服务端口号默认是 8

    2024年04月08日
    浏览(28)
  • 自动化测试脚本编写(超详细)

    🍅 视频学习: 文末有免费的配套视频可观看 🍅  关注公众号【互联网杂货铺】,回复 1  ,免费获取软件测试全套资料,资料在手,涨薪更快 什么是自动化测试? 自动化测试是验证和验证软件是否满足所有用户需求,并使用自动化工具按预期运行。它检查在产品开发阶段

    2024年04月26日
    浏览(28)
  • 【测试技术-性能测试-实战1】使用Loadrunner自带的系统学习脚本编写和脚本增强。

    Hp software-start hp web tours server Hp software-hp web tours application 打开Virtual User Generator 创建脚本,协议选择Web - HTTP/HTML,输入脚本名,选择好脚本存放路径,点击create即可创建好一个空白脚本。 设置录制选项,点击record按钮,设置完基本信息后,点击右下角的Recording Options,设置录

    2024年02月02日
    浏览(32)
  • 渗透测试-sqlmap绕过WAF脚本编写

    Sqlmap一款功能强大集成了多种数据库识别及注入方式,多用于识别和利用Web 应用程序注入漏洞的工具,它的优点在于集成了大量 payload,对检测与利用的自动化处理(数据库指纹、访问底层文件系统、执行命令)。 我们之前学习时间盲注的时候,可以针对时间盲注类型的漏洞

    2024年02月09日
    浏览(42)
  • 自动化用例编写思路 (使用pytest编写一个测试脚本)

    目录 一,明确测试对象 二,编写测试用例 构造请求数据 封装测试代码 断言设置 三,执行脚本获取测试结果 四,总结 经过之前的学习铺垫,我们尝试着利用pytest框架编写一条接口自动化测试用例,来厘清接口自动化用例编写的思路。 我们在百度搜索天气查询,会出现如下

    2024年02月16日
    浏览(45)
  • 【Postman】测试(Tests)脚本编写和断言详解

    测试确认您的 API 按预期工作,服务之间的集成运行可靠,并且新开​​发没有破坏任何现有功能。您可以使用 JavaScript 为 Postman API 请求编写测试脚本。当您的 API 项目出现问题时,您还可以使用测试代码来帮助调试过程。例如,您可以编写一个测试,通过发送包含不完整数

    2023年04月16日
    浏览(32)
  • Selenium如何用于编写自动化测试脚本?

    Selenium如何用于编写自动化测试脚本?它提供了许多测试工具和API,可以与浏览器交互,模拟用户操作,检查网页的各个方面。下面是一些步骤,可以帮助你编写Selenium自动化测试脚本。 1、安装Selenium库和浏览器驱动程序 首先,你需要安装Selenium库和浏览器驱动程序,例如C

    2024年02月09日
    浏览(29)

觉得文章有用就打赏一下文章作者

支付宝扫一扫打赏

博客赞助

微信扫一扫打赏

请作者喝杯咖啡吧~博客赞助

支付宝扫一扫领取红包,优惠每天领

二维码1

领取红包

二维码2

领红包