AntDB现已开源:
开源url:https://github.com/ADBSQL/AntDB
QQ交流群:496464280
背景: AntDB 是一款源自于 PG 内核的通用分布式事务性关系数据库。目前需要验证数据库的读写能力。第一个想到的是pgbench,postgresql提供的一个压测工具,不过内部使用的是libpq通讯协议,与生产使用的jdbc扩展协议有点差别;第二个想到的LR,想想LR部署安装的繁杂,关键是不免费,用破解版涉及商用可能被起诉的风险;第三个想到的是JMeter,小巧的纯java写的可以用来测试数据库的工具。
JMeter与pgbench相比:
AntDB是基于postgresql内核开发的,熟悉postgresql的都比较了解pgbench工具。pgbench是一种在PostgreSQL上运行基准测试的简单程序,用C语言开发的,内部走的是libpq,效率非常高。但是考虑到现场使用的是JAVA开发,调用jdbc走的扩展协议,所以本次不采用pgbench工具。
JMeter与LoadRunner相比:
JMeter 是一款开源的测试工具,虽然与LoadRunner相比,没有LR功能那么强大,结果分析也没有LR那么详细。但JMeter有其本身的优势:
1、开源,是一款开源的免费软件,免费!免费!免费!重要的事情说三遍…
2、小巧,相比LR几个G大小,它非常小巧,不需要安装,但需要JDK环境,因为它是使用java开发的工具;
3、可以运行在Windows以及linux环境中;
4、功能相对比较全面,jmeter设计之初只是一个简单的web性能测试工具,但经过不段的更新扩展,现在可以完成数据库、FTP、LDAP、WebService等方面的测试。因为它的开源性,当然你也可以根据自己的需求扩展它的功能。
言归正传,下面开始使用JMeter来测试AntDB:
安装JMeter:
1、http://jmeter.apache.org/下载最新版本的JMeter,上传linux服务器上,本次下载的是apache-jmeter-2.13版本;
2、解压JMeter:
tar -zxvf apache-jmeter-2.13.tgz
3、检查当前用户的JAVA版本,需要注意的是最新的apache-jmeter-3.3版本要求java 1.8并且不支持java1.9,下载的时候注意看官网版本要求
[zhugy@localhost4 bin]$ java -version
java version "1.7.0_111"
OpenJDK Runtime Environment (rhel-2.6.7.2.el6_8-x86_64 u111-b01)
OpenJDK 64-Bit Server VM (build 24.111-b01, mixed mode)
4、将JMeter的可执行目录添加到环境变量中去:apache-jmeter-2.13/bin
配置测试AntDB驱动和配置文件:
1、上传AntDB的jdbc驱动(基于postgresql的jdbc驱动作了一些调整)到JMeter的lib目录下:
[zhugy@localhost4 lib]$ pwd
/home/zhugy/jmeter/apache-jmeter-2.13/lib
[zhugy@localhost4 lib]$ ll| grep postgresql
-rw-r--r-- 1 zhugy zhugy 709082 Sep 5 16:58 postgresql-42.1.5.jre7-SNAPSHOT.jar
2、以insert为例,完整的jmeter的jmx配置如下:
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="2.8" jmeter="2.13 r1665067">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="测试计划" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Argume
nts" testname="用户定义的变量" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="测试adb单查询性能" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="L
oopController" testname="循环控制器" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">10000</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">0</stringProp>
<longProp name="ThreadGroup.start_time">1475998829000</longProp>
<longProp name="ThreadGroup.end_time">1475998829000</longProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration">180</stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
</ThreadGroup>
<hashTree>
<JDBCSampler guiclass="TestBeanGUI" testclass="JDBCSampler" testname="JDBC Request" enabled="true">
<stringProp name="dataSource">postgresql</stringProp>
<stringProp name="query">INSERT INTO customer(cust_id, credit_id, service_grade_id, credit_limit_id, party_id, cust
_name, common_region_id, cust_source_id, cust_order_id) VALUES(${cust_id}, 1112, 3, 10010, 10000, '测试用户', 200,
101, 10011);</stringProp>
<stringProp name="queryArguments"></stringProp>
<stringProp name="queryArgumentsTypes"></stringProp>
<stringProp name="queryTimeout"></stringProp>
<stringProp name="queryType">Callable Statement</stringProp>
<stringProp name="resultSetHandler">Store as String</stringProp>
<stringProp name="resultVariable"></stringProp>
<stringProp name="variableNames"></stringProp>
</JDBCSampler>
<hashTree/>
<JDBCDataSource guiclass="TestBeanGUI" testclass="JDBCDataSource" testname="JDBC Connection Configuration" enabled="t
rue">
<boolProp name="autocommit">true</boolProp>
<stringProp name="checkQuery"></stringProp>
<stringProp name="connectionAge">5000</stringProp>
<stringProp name="dataSource">postgresql</stringProp>
<stringProp name="dbUrl">jdbc:postgresql://10.1.226.201:6603/postgres?binaryTransfer=false&forcebinary=false</s
tringProp>
<stringProp name="driver">org.postgresql.Driver</stringProp>
<boolProp name="keepAlive">true</boolProp>
<stringProp name="password">123456</stringProp>
<stringProp name="poolMax">100</stringProp>
<stringProp name="timeout">10000</stringProp>
<stringProp name="transactionIsolation">DEFAULT</stringProp>
<stringProp name="trimInterval">60000</stringProp>
<stringProp name="username">zgy</stringProp>
</JDBCDataSource>
<hashTree/>
<RandomVariableConfig guiclass="TestBeanGUI" testclass="RandomVariableConfig" testname="Random Variable" enabled="tru
e">
<stringProp name="variableName">cust_id</stringProp>
<stringProp name="outputFormat"></stringProp>
<stringProp name="minimumValue">1</stringProp>
<stringProp name="maximumValue">20000000</stringProp>
<stringProp name="randomSeed"></stringProp>
<boolProp name="perThread">false</boolProp>
</RandomVariableConfig>
<hashTree/>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>
针对上述流程文件,分解为几步:
- 线程组设置:
ThreadGroup.num_threads 线程数,就是我们常说的并发数
LoopController.loops 循环次数
ThreadGroup.ramp_time 启动时间,类似于LR的准备时间
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="测试adb单查询性能" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="L
oopController" testname="循环控制器" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">10000</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">0</stringProp>
<longProp name="ThreadGroup.start_time">1475998829000</longProp>
<longProp name="ThreadGroup.end_time">1475998829000</longProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration">180</stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
</ThreadGroup>
- AntDB连接设置:
dbUrl: antdb的URL
driver: 驱动类型
username:数据库用户名
password: 数据库密码
checkQuery:连接检查,比方配置select 1,每次建立连接后会执行,测试中一般不设置
<JDBCDataSource guiclass="TestBeanGUI" testclass="JDBCDataSource" testname="JDBC Connection Configuration" enabled="t
rue">
<boolProp name="autocommit">true</boolProp>
<stringProp name="checkQuery"></stringProp>
<stringProp name="connectionAge">5000</stringProp>
<stringProp name="dataSource">postgresql</stringProp>
<stringProp name="dbUrl">jdbc:postgresql://10.1.226.201:6603/postgres?binaryTransfer=false&forcebinary=false</s
tringProp>
<stringProp name="driver">org.postgresql.Driver</stringProp>
<boolProp name="keepAlive">true</boolProp>
<stringProp name="password">123456</stringProp>
<stringProp name="poolMax">100</stringProp>
<stringProp name="timeout">10000</stringProp>
<stringProp name="transactionIsolation">DEFAULT</stringProp>
<stringProp name="trimInterval">60000</stringProp>
<stringProp name="username">zgy</stringProp>
</JDBCDataSource>
- 变量设置:
variableName: 变量名
minimumValue: 最小值
maximumValue: 最大值
有几种取值方式,感兴趣可以自己去试验,本次选取的是:Random(取随机数)
<RandomVariableConfig guiclass="TestBeanGUI" testclass="RandomVariableConfig" testname="Random Variable" enabled="tru
e">
<stringProp name="variableName">cust_id</stringProp>
<stringProp name="outputFormat"></stringProp>
<stringProp name="minimumValue">1</stringProp>
<stringProp name="maximumValue">20000000</stringProp>
<stringProp name="randomSeed"></stringProp>
<boolProp name="perThread">false</boolProp>
</RandomVariableConfig>
- jdbc requst:
dataSource: 取上面AntDB连接的name
query: 执行的sql,引用变量的话用${变量名}
queryType: 对应jdbc的方法调用
<JDBCSampler guiclass="TestBeanGUI" testclass="JDBCSampler" testname="JDBC Request" enabled="true">
<stringProp name="dataSource">postgresql</stringProp>
<stringProp name="query">INSERT INTO customer(cust_id, credit_id, service_grade_id, credit_limit_id, party_id, cust
_name, common_region_id, cust_source_id, cust_order_id) VALUES(${cust_id}, 1112, 3, 10010, 10000, '测试用户', 200,
101, 10011);</stringProp>
<stringProp name="queryArguments"></stringProp>
<stringProp name="queryArgumentsTypes"></stringProp>
<stringProp name="queryTimeout"></stringProp>
<stringProp name="queryType">Callable Statement</stringProp>
<stringProp name="resultSetHandler">Store as String</stringProp>
<stringProp name="resultVariable"></stringProp>
<stringProp name="variableNames"></stringProp>
</JDBCSampler>
到此,安装和配置部分完成。
测试AntDB:
执行很简单:
[zhugy@localhost4 antdb2.2]$ vim jmeter.sh
#!/bin/bash
jmx=$1
jmeter -n -t $jmx -l jmeterlog.jtl
[zhugy@localhost4 adb2.2]$ sh jmeter.sh insert.jmx
会在当前目录下生成两个文件:
jmeter.log 记录执行过程日志,包括性能结果记录
jmeterlog.jtl 记录每个请求的响应
[zhugy@localhost4 antdb2.2]$ ll
-rw-rw-r-- 1 zhugy zhugy 201100 Feb 7 16:36 jmeter.log
-rw-rw-r-- 1 zhugy zhugy 19662 Feb 7 16:36 jmeterlog.jtl
[zhugy@localhost4 antdb2.2]$ tail 20f jmeter.log
tail: cannot open `20f' for reading: No such file or directory
==> jmeter.log <==
2017/11/29 09:54:18 INFO - jmeter.engine.StandardJMeterEngine: All thread groups have been started
2017/11/29 09:54:18 INFO - jmeter.threads.JMeterThread: Thread started: 测试adb单查询性能 1-1
2017/11/29 09:54:18 INFO - jmeter.samplers.SampleResult: Note: Sample TimeStamps are START times
2017/11/29 09:54:18 INFO - jmeter.samplers.SampleResult: sampleresult.default.encoding is set to ISO-8859-1
2017/11/29 09:54:18 INFO - jmeter.samplers.SampleResult: sampleresult.useNanoTime=true
2017/11/29 09:54:18 INFO - jmeter.samplers.SampleResult: sampleresult.nanoThreadSleep=5000
2017/11/29 09:54:18 INFO - jmeter.threads.JMeterThread: Thread is done: 测试adb单查询性能 1-1
2017/11/29 09:54:18 INFO - jmeter.threads.JMeterThread: Thread finished: 测试adb单查询性能 1-1
2017/11/29 09:54:18 INFO - jmeter.engine.StandardJMeterEngine: Notifying test listeners of end of test
2017/11/29 09:54:18 INFO - jmeter.reporters.Summariser: summary = 1 in 0.3s = 3.2/s Avg: 199 Min: 199 Max: 199 Err: 0 (0.00%)
[zhugy@localhost4 antdb2.2]$ tail 20f jmeterlog.jtl
tail: cannot open `20f' for reading: No such file or directory
==> jmeterlog.jtl <==
1511857675459,208,JDBC Request,200,OK,测试adb单查询性能 1-1,text,true,64,1,1,29
1511857740293,194,JDBC Request,200,OK,测试adb单查询性能 1-1,text,true,64,1,1,30
1511919740601,194,JDBC Request,200,OK,测试adb单查询性能 1-1,text,true,64,1,1,29
1511920458410,199,JDBC Request,200,OK,测试adb单查询性能 1-1,text,true,64,1,1,29