转载出处:https://blog.csdn.net/l1028386804/article/details/80341251
一、项目背景
基于java开发的功能强大、配置灵活的数据库之间的同步工具,和数据产生器一样,均是前段时间因为项目需要编写的小工具,在实际应用场景中,我们经常需要定期将一个数据库的数据同步到另外一个数据库中,常见的一种做法是将源数据库的数据dump为sql文件,然后到目标数据库执行sql文件完成数据库的导入,但是这种方法至少存在以下问题:
- 需要手工操作,效率低
- 当涉及数据表较多时,容易遗漏、出错
- 如果要定期同步,操作人容易忘记
- 难以应付频繁变更数据表或者字段
针对以上存在的问题,将珍贵人力从这种重复、无意义的工作中解脱出来,特意开发这个小工具,其中主要配置主要在jobs.xml中完成。
二、项目结构
项目整体结构如下图:
三、项目功能
- MySQL——>MySQL
- SQLServer——>SQLServer
- MySQL——>SQLServer
- SQLServer——>MySQL
注:——>左边的代码源数据库,——>右边代表的是目标数据库,具体解释如下:
- 支持MySQL向MySQL同步数据
- 支持SQLServer向SQLServer同步数据
- 支持MySQL向SQLServer同步数据
- 支持SQLServer向MySQL同步数据
四、具体功能实现
1、创建数据库信息类DBInfo
这个类主要是存储一些数据库相关的信息,比如数据库驱动、数据库连接、用户名和密码等,具体见如下代码:
- package io.mykit.db.sync.provider.entity;
- /**
- * 数据库信息
- * @author liuyazhuang
- *
- */
- public class DBInfo {
- //数据库连接
- private String url;
- //数据库用户名
- private String username;
- //数据库密码
- private String password;
- //数据库类型(对应mysql还是sqlserver)
- private String dbtype;
- //数据库驱动
- private String driver;
- public String getUrl() {
- return url;
- }
- public void setUrl(String url) {
- this.url = url;
- }
- public String getUsername() {
- return username;
- }
- public void setUsername(String username) {
- this.username = username;
- }
- public String getPassword() {
- return password;
- }
- public void setPassword(String password) {
- this.password = password;
- }
- public String getDbtype() {
- return dbtype;
- }
- public void setDbtype(String dbtype) {
- this.dbtype = dbtype;
- }
- public String getDriver() {
- return driver;
- }
- public void setDriver(String driver) {
- this.driver = driver;
- }
- }
2、创建定时同步任务信息类JobInfo
这个类主要是存储一些与定时任务相关的基本信息,具体见如下代码:
- package io.mykit.db.sync.provider.entity;
- /**
- * 任务信息
- * @author liuyazhuang
- *
- */
- public class JobInfo {
- //任务名称
- private String name;
- //任务表达式
- private String cron;
- //源数据源sql
- private String srcSql;
- //目标数据表
- private String destTable;
- //目标表数据字段
- private String destTableFields;
- //目标表主键
- private String destTableKey;
- //目标表可更新的字段
- private String destTableUpdate;
- public String getName() {
- return name;
- }
- public void setName(String name) {
- this.name = name;
- }
- public String getCron() {
- return cron;
- }
- public void setCron(String cron) {
- this.cron = cron;
- }
- public String getSrcSql() {
- return srcSql;
- }
- public void setSrcSql(String srcSql) {
- this.srcSql = srcSql;
- }
- public String getDestTable() {
- return destTable;
- }
- public void setDestTable(String destTable) {
- this.destTable = destTable;
- }
- public String getDestTableFields() {
- return destTableFields;
- }
- public void setDestTableFields(String destTableFields) {
- this.destTableFields = destTableFields;
- }
- public String getDestTableKey() {
- return destTableKey;
- }
- public void setDestTableKey(String destTableKey) {
- this.destTableKey = destTableKey;
- }
- public String getDestTableUpdate() {
- return destTableUpdate;
- }
- public void setDestTableUpdate(String destTableUpdate) {
- this.destTableUpdate = destTableUpdate;
- }
- }
3、创建字符串工具类SpringUtils
这个类主要是为字符串的操作提供统一的工具支持,在这个小工具中,本类主要的作用就是判断给定的字符串是否为空,具体见如下代码:
- package io.mykit.db.sync.provider.utils;
- /**
- * 字符串工具类
- * @author liuyazhuang
- *
- */
- public class StringUtils {
- public static boolean isEmpty(String str){
- return str == null || "".equals(str.trim());
- }
- }
4、创建工具类Tool
此类的主要作用就是随机生成一个给定长度的字符串,具体见如下代码:
- package io.mykit.db.sync.provider.utils;
- /**
- * 工具类
- *
- * @author liuyazhuang
- *
- */
- public class Tool {
- public static String generateString(int length) {
- if (length < 1)
- length = 6;
- String str = "ABCDEFGHIJKLMNOPQRSTUVWXYZ";
- String genStr = "";
- for (int index = 0; index < length; index++) {
- genStr = genStr + str.charAt((int) ((Math.random() * 100) % 26));
- }
- return genStr;
- }
- }
5、定义常量类Constants
这个类中主要为此工程提供常量信息,标识数据库的类型,具体代码如下:
- package io.mykit.db.sync.provider.constants;
- /**
- * 常量
- * @author liuyazhuang
- *
- */
- public class Constants {
- /**
- * sqlserver数据库
- */
- public static final String TYPE_DB_SQLSERVER = "sqlserver";
- /**
- * MySQL数据库
- */
- public static final String TYPE_DB_MYSQL = "mysql";
- }
以上五个类是我们实现数据库数据同步的基础支持类,创建完以上四个类之后,我们就开始编写具体的同步业务了。
6、创建同步数据库的抽象接口DBSync
这个接口主要是定义了同步数据库的方法,具体代码如下:
- package io.mykit.db.sync.provider.sync;
- import java.sql.Connection;
- import java.sql.SQLException;
- import io.mykit.db.sync.provider.entity.JobInfo;
- /**
- * 同步数据库的抽象接口
- * @author liuyazhuang
- *
- */
- public interface DBSync {
- /**
- *
- * @param paramString:同步参数
- * @param paramConnection:数据库连接
- * @param paramJobInfo:同步任务
- * @return
- * @throws SQLException
- */
- String assembleSQL(String paramString, Connection paramConnection, JobInfo paramJobInfo) throws SQLException;
- /**
- *
- * @param sql:要执行的SQL语句
- * @param conn:数据库连接
- * @throws SQLException
- */
- void executeSQL(String sql, Connection conn) throws SQLException;
- }
7、创建数据库同步抽象类AbstractDBSync
这个类主要是抽象同步业务,目前主要提供的方法为:消除从job.xml文件中读取出的数据存在的空格,具体代码如下:
- package io.mykit.db.sync.provider.sync.impl;
- import io.mykit.db.sync.provider.sync.DBSync;
- /**
- * 执行数据库同步的抽象类
- * @author liuyazhuang
- *
- */
- public abstract class AbstractDBSync implements DBSync {
- /**
- * 去除String数组每个元素中的空格
- * @param arr
- * @return
- */
- protected String[] trimArrayItem(String[] src){
- if(src == null || src.length == 0) return src;
- String[] dest = new String[src.length];
- for(int i = 0; i < src.length; i++){
- dest[i] = src[i].trim();
- }
- return dest;
- }
- }
8、创建MySQL数据库同步类MySQLSync
此类主要实现的是MySQL数据库之前的同步操作,具体业务见如下代码:
- package io.mykit.db.sync.provider.sync.impl;
- import java.sql.Connection;
- import java.sql.PreparedStatement;
- import java.sql.ResultSet;
- import java.sql.SQLException;
- import io.mykit.db.sync.provider.entity.JobInfo;
- import io.mykit.db.sync.provider.sync.DBSync;
- import io.mykit.db.sync.provider.utils.Tool;
- /**
- * 实现MySQL同步数据库
- * @author liuyazhuang
- *
- */
- public class MySQLSync extends AbstractDBSync implements DBSync {
- @Override
- public String assembleSQL(String srcSql, Connection conn, JobInfo jobInfo) throws SQLException {
- String uniqueName = Tool.generateString(6) + "_" + jobInfo.getName();
- String[] fields = jobInfo.getDestTableFields().split(",");
- fields = this.trimArrayItem(fields);
- String[] updateFields = jobInfo.getDestTableUpdate().split(",");
- updateFields = this.trimArrayItem(updateFields);
- String destTable = jobInfo.getDestTable();
- String destTableKey = jobInfo.getDestTableKey();
- PreparedStatement pst = conn.prepareStatement(srcSql);
- ResultSet rs = pst.executeQuery();
- StringBuffer sql = new StringBuffer();
- sql.append("insert into ").append(jobInfo.getDestTable()).append(" (").append(jobInfo.getDestTableFields()).append(") values ");
- long count = 0;
- while (rs.next()) {
- sql.append("(");
- for (int index = 0; index < fields.length; index++) {
- sql.append("'").append(rs.getString(fields[index])).append(index == (fields.length - 1) ? "'" : "',");
- }
- sql.append("),");
- count++;
- }
- if (rs != null) {
- rs.close();
- }
- if (pst != null) {
- pst.close();
- }
- if (count > 0) {
- sql = sql.deleteCharAt(sql.length() - 1);
- if ((!jobInfo.getDestTableUpdate().equals("")) && (!jobInfo.getDestTableKey().equals(""))) {
- sql.append(" on duplicate key update ");
- for (int index = 0; index < updateFields.length; index++) {
- sql.append(updateFields[index]).append("= values(").append(updateFields[index]).append(index == (updateFields.length - 1) ? ")" : "),");
- }
- return new StringBuffer("alter table ").append(destTable).append(" add constraint ").append(uniqueName).append(" unique (").append(destTableKey).append(");").append(sql.toString())
- .append(";alter table ").append(destTable).append(" drop index ").append(uniqueName).toString();
- }
- return sql.toString();
- }
- return null;
- }
- @Override
- public void executeSQL(String sql, Connection conn) throws SQLException {
- PreparedStatement pst = conn.prepareStatement("");
- String[] sqlList = sql.split(";");
- for (int index = 0; index < sqlList.length; index++) {
- pst.addBatch(sqlList[index]);
- }
- pst.executeBatch();
- conn.commit();
- pst.close();
- }
- }
9、创建SQLServer数据库同步类SQLServerSync
这个类主要实现了SQLServer数据库之前的数据同步操作,具体业务见如下代码:
- package io.mykit.db.sync.provider.sync.impl;
- import java.sql.Connection;
- import java.sql.PreparedStatement;
- import java.sql.ResultSet;
- import java.sql.SQLException;
- import java.sql.Statement;
- import org.apache.log4j.Logger;
- import io.mykit.db.sync.provider.entity.JobInfo;
- import io.mykit.db.sync.provider.sync.DBSync;
- /**
- * SQLServer同步的数据库的实现
- * @author liuyazhuang
- *
- */
- public class SQLServerSync extends AbstractDBSync implements DBSync {
- private Logger logger = Logger.getLogger(SQLServerSync.class);
- @Override
- public String assembleSQL(String srcSql, Connection conn, JobInfo jobInfo) throws SQLException {
- String fieldStr = jobInfo.getDestTableFields();
- String[] fields = jobInfo.getDestTableFields().split(",");
- fields = this.trimArrayItem(fields);
- String[] updateFields = jobInfo.getDestTableUpdate().split(",");
- updateFields = this.trimArrayItem(updateFields);
- String destTableKey = jobInfo.getDestTableKey();
- String destTable = jobInfo.getDestTable();
- Statement stat = conn.createStatement();
- ResultSet rs = stat.executeQuery(srcSql);
- StringBuffer sql = new StringBuffer();
- long count = 0;
- while (rs.next()) {
- sql.append("if not exists (select ").append(destTableKey).append(" from ").append(destTable).append(" where ").append(destTableKey).append("='").append(rs.getString(destTableKey))
- .append("')").append("insert into ").append(destTable).append("(").append(fieldStr).append(") values(");
- for (int index = 0; index < fields.length; index++) {
- sql.append("'").append(rs.getString(fields[index])).append(index == (fields.length - 1) ? "'" : "',");
- }
- sql.append(") else update ").append(destTable).append(" set ");
- for (int index = 0; index < updateFields.length; index++) {
- sql.append(updateFields[index]).append("='").append(rs.getString(updateFields[index])).append(index == (updateFields.length - 1) ? "'" : "',");
- }
- sql.append(" where ").append(destTableKey).append("='").append(rs.getString(destTableKey)).append("';");
- count++;
- // this.logger.info("第" + count + "耗时: " + (new Date().getTime() - oneStart) + "ms");
- }
- this.logger.info("总共查询到 " + count + " 条记录");
- if (rs != null) {
- rs.close();
- }
- if (stat != null) {
- stat.close();
- }
- return count > 0 ? sql.toString() : null;
- }
- @Override
- public void executeSQL(String sql, Connection conn) throws SQLException {
- PreparedStatement pst = conn.prepareStatement(sql);
- pst.executeUpdate();
- conn.commit();
- pst.close();
- }
- }
10、创建同步对象的工厂类DBSyncFactory
这里,我们以工厂的形式来创建MySQLSync类或者SQLServerSync类,具体创建哪个类是根据传递的数据库类型决定的,具体见如下代码:
- package io.mykit.db.sync.provider.factory;
- import io.mykit.db.sync.provider.constants.Constants;
- import io.mykit.db.sync.provider.sync.DBSync;
- import io.mykit.db.sync.provider.sync.impl.MySQLSync;
- import io.mykit.db.sync.provider.sync.impl.SQLServerSync;
- import io.mykit.db.sync.provider.utils.StringUtils;
- /**
- * 创建同步对象的工厂类
- * @author liuyazhuang
- *
- */
- public class DBSyncFactory {
- /**
- * 根据数据库的类型创建不同的同步数据库数据的对象
- * @param type:数据库类型
- * @return:同步数据库数据的对象
- */
- public static DBSync create(String type){
- if(StringUtils.isEmpty(type)) return null;
- switch (type) {
- case Constants.TYPE_DB_MYSQL:
- return new MySQLSync();
- case Constants.TYPE_DB_SQLSERVER:
- return new SQLServerSync();
- default:
- return null;
- }
- }
- }
11、创建同步数据库任务的具体实现类JobTask
这个类实现org.quartz.Job接口,主要实现定时任务同步数据库,具体见如下代码:
- package io.mykit.db.sync.provider;
- import java.sql.Connection;
- import java.sql.DriverManager;
- import java.sql.SQLException;
- import java.util.Date;
- import org.apache.log4j.Logger;
- import org.quartz.Job;
- import org.quartz.JobDataMap;
- import org.quartz.JobExecutionContext;
- import org.quartz.JobExecutionException;
- import io.mykit.db.sync.provider.entity.DBInfo;
- import io.mykit.db.sync.provider.entity.JobInfo;
- import io.mykit.db.sync.provider.factory.DBSyncFactory;
- import io.mykit.db.sync.provider.sync.DBSync;
- /**
- * 同步数据库任务的具体实现
- * @author liuyazhuang
- *
- */
- public class JobTask implements Job {
- private Logger logger = Logger.getLogger(JobTask.class);
- /**
- * 执行同步数据库任务
- *
- */
- @Override
- public void execute(JobExecutionContext context) throws JobExecutionException {
- this.logger.info("开始任务调度: " + new Date());
- Connection inConn = null;
- Connection outConn = null;
- JobDataMap data = context.getJobDetail().getJobDataMap();
- DBInfo srcDb = (DBInfo) data.get("srcDb");
- DBInfo destDb = (DBInfo) data.get("destDb");
- JobInfo jobInfo = (JobInfo) data.get("jobInfo");
- String logTitle = (String) data.get("logTitle");
- try {
- inConn = createConnection(srcDb);
- outConn = createConnection(destDb);
- if (inConn == null) {
- this.logger.info("请检查源数据连接!");
- return;
- } else if (outConn == null) {
- this.logger.info("请检查目标数据连接!");
- return;
- }
- DBSync dbHelper = DBSyncFactory.create(destDb.getDbtype());
- long start = new Date().getTime();
- String sql = dbHelper.assembleSQL(jobInfo.getSrcSql(), inConn, jobInfo);
- this.logger.info("组装SQL耗时: " + (new Date().getTime() - start) + "ms");
- if (sql != null) {
- this.logger.debug(sql);
- long eStart = new Date().getTime();
- dbHelper.executeSQL(sql, outConn);
- this.logger.info("执行SQL语句耗时: " + (new Date().getTime() - eStart) + "ms");
- }
- } catch (SQLException e) {
- this.logger.error(logTitle + e.getMessage());
- this.logger.error(logTitle + " SQL执行出错,请检查是否存在语法错误");
- } finally {
- this.logger.error("关闭源数据库连接");
- destoryConnection(inConn);
- this.logger.error("关闭目标数据库连接");
- destoryConnection(outConn);
- }
- }
- /**
- * 创建数据库连接
- * @param db
- * @return
- */
- private Connection createConnection(DBInfo db) {
- try {
- Class.forName(db.getDriver());
- Connection conn = DriverManager.getConnection(db.getUrl(), db.getUsername(), db.getPassword());
- conn.setAutoCommit(false);
- return conn;
- } catch (Exception e) {
- this.logger.error(e.getMessage());
- }
- return null;
- }
- /**
- * 关闭并销毁数据库连接
- * @param conn
- */
- private void destoryConnection(Connection conn) {
- try {
- if (conn != null) {
- conn.close();
- conn = null;
- this.logger.error("数据库连接关闭");
- }
- } catch (SQLException e) {
- e.printStackTrace();
- }
- }
- }
12、创建同步数据库资源的整合类DBSyncBuilder
这个类的主要作用是整合本工程的所有资源,比如:读取相关的配置文件,通过工厂类DBSyncFactory实例化具体的同步对象,启动定时任务,同步数据库数据等。具体见如下代码:
- package io.mykit.db.sync.build;
- import static org.quartz.CronScheduleBuilder.cronSchedule;
- import static org.quartz.JobBuilder.newJob;
- import static org.quartz.TriggerBuilder.newTrigger;
- import java.lang.reflect.Field;
- import java.util.ArrayList;
- import java.util.Iterator;
- import java.util.List;
- import org.apache.log4j.Logger;
- import org.dom4j.Element;
- import org.dom4j.io.SAXReader;
- import org.quartz.CronTrigger;
- import org.quartz.JobDetail;
- import org.quartz.Scheduler;
- import org.quartz.SchedulerFactory;
- import org.quartz.impl.StdSchedulerFactory;
- import io.mykit.db.sync.provider.JobTask;
- import io.mykit.db.sync.provider.entity.DBInfo;
- import io.mykit.db.sync.provider.entity.JobInfo;
- /**
- * 同步数据库数据的Builder对象
- * @author liuyazhuang
- *
- */
- public class DBSyncBuilder {
- private DBInfo srcDb;
- private DBInfo destDb;
- private List<JobInfo> jobList;
- private String code;
- private static Logger logger = Logger.getLogger(DBSyncBuilder.class);
- private DBSyncBuilder(){
- }
- /**
- * 创建DBSyncBuilder对象
- * @return DBSyncBuilder对象
- */
- public static DBSyncBuilder builder(){
- return new DBSyncBuilder();
- }
- /**
- * 初始化数据库信息并解析jobs.xml填充数据
- * @return DBSyncBuilder对象
- */
- public DBSyncBuilder init() {
- srcDb = new DBInfo();
- destDb = new DBInfo();
- jobList = new ArrayList<JobInfo>();
- SAXReader reader = new SAXReader();
- try {
- // 读取xml的配置文件名,并获取其里面的节点
- Element root = reader.read("jobs.xml").getRootElement();
- Element src = root.element("source");
- Element dest = root.element("dest");
- Element jobs = root.element("jobs");
- // 遍历job即同步的表
- for (@SuppressWarnings("rawtypes")
- Iterator it = jobs.elementIterator("job"); it.hasNext();) {
- jobList.add((JobInfo) elementInObject((Element) it.next(), new JobInfo()));
- }
- //
- elementInObject(src, srcDb);
- elementInObject(dest, destDb);
- code = root.element("code").getTextTrim();
- } catch (Exception e) {
- e.printStackTrace();
- }
- return this;
- }
- /**
- * 解析e中的元素,将数据填充到o中
- * @param e 解析的XML Element对象
- * @param o 存放解析后的XML Element对象
- * @return 存放有解析后数据的Object
- * @throws IllegalArgumentException
- * @throws IllegalAccessException
- */
- public Object elementInObject(Element e, Object o) throws IllegalArgumentException, IllegalAccessException {
- Field[] fields = o.getClass().getDeclaredFields();
- for (int index = 0; index < fields.length; index++) {
- fields[index].setAccessible(true);
- fields[index].set(o, e.element(fields[index].getName()).getTextTrim());
- }
- return o;
- }
- /**
- * 启动定时任务,同步数据库的数据
- */
- public void start() {
- for (int index = 0; index < jobList.size(); index++) {
- JobInfo jobInfo = jobList.get(index);
- String logTitle = "[" + code + "]" + jobInfo.getName() + " ";
- try {
- SchedulerFactory sf = new StdSchedulerFactory();
- Scheduler sched = sf.getScheduler();
- JobDetail job = newJob(JobTask.class).withIdentity("job-" + jobInfo.getName(), code).build();
- job.getJobDataMap().put("srcDb", srcDb);
- job.getJobDataMap().put("destDb", destDb);
- job.getJobDataMap().put("jobInfo", jobInfo);
- job.getJobDataMap().put("logTitle", logTitle);
- logger.info(jobInfo.getCron());
- CronTrigger trigger = newTrigger().withIdentity("trigger-" + jobInfo.getName(), code).withSchedule(cronSchedule(jobInfo.getCron())).build();
- sched.scheduleJob(job, trigger);
- sched.start();
- } catch (Exception e) {
- logger.info(logTitle + e.getMessage());
- logger.info(logTitle + " run failed");
- continue;
- }
- }
- }
- }
13、创建工程的入口类Main
此类为启动工程的入口类,具体见如下代码:
- package io.mykit.db.sync;
- import io.mykit.db.sync.build.DBSyncBuilder;
- /**
- * 程序入口
- * @author liuyazhuang
- *
- */
- public class Main {
- public static void main(String[] args) {
- DBSyncBuilder.builder().init().start();
- }
- }
五、资源配置
写完具体的业务代码后,我们就要完善相关的配置文件信息了。
1、创建配置文件jobs.xml
这个文件是我们整个工程中最核心的配置文件,在这个文件中定义了同步的源数据库信息和目标数据库信息,同步任务等,同时定义了同步数据的数据表和数据字段等信息,具体参见如下配置
- <?xml version="1.0" encoding="UTF-8"?>
- <root>
- <code>4500000001</code>
- <!-- <source>
- <url>jdbc:oracle:thin:@192.168.1.179:1521:XE</url>
- <username>test</username>
- <password>test</password>
- <dbtype>oracle</dbtype>
- <driver>oracle.jdbc.driver.OracleDriver</driver>
- </source>
- <dest>
- <url>jdbc:sqlserver://192.168.1.191:1433;DatabaseName=test</url>
- <username>test</username>
- <password>test</password>
- <dbtype>sqlserver</dbtype>
- <driver>com.microsoft.sqlserver.jdbc.SQLServerDriver</driver>
- </dest> -->
- <source>
- <url>jdbc:mysql://192.168.209.121:3306/test</url>
- <username>root</username>
- <password>root</password>
- <dbtype>mysql</dbtype>
- <driver>com.mysql.jdbc.Driver</driver>
- </source>
- <dest>
- <url>jdbc:mysql://127.0.0.1:3306/test</url>
- <username>root</username>
- <password>root</password>
- <dbtype>mysql</dbtype>
- <driver>com.mysql.jdbc.Driver</driver>
- </dest>
- <jobs>
- <job>
- <name>1</name>
- <cron>0/10 * * * * ?</cron>
- <srcSql>select user_id, account,password from user</srcSql>
- <destTable>client_user</destTable>
- <destTableFields>user_id, account</destTableFields>
- <destTableKey>user_id</destTableKey>
- <destTableUpdate>account</destTableUpdate>
- </job>
- </jobs>
- </root>
2、创建log4j.properties
这个就不多说了,具体如下:
- log4j.rootCategory=INFO,A1,A2,A3
- log4j.appender.A1=org.apache.log4j.ConsoleAppender
- log4j.appender.A1.layout=org.apache.log4j.PatternLayout
- log4j.appender.A1.layout.ConversionPattern=%4p [%t] (%F:%L) - %m%n
- log4j.appender.A2=org.apache.log4j.RollingFileAppender
- log4j.appender.A2.File=./databaseSync.log
- log4j.appender.A2.MaxFileSize = 10MB
- log4j.appender.A2.MaxBackupIndex = 10
- log4j.appender.A2.layout=org.apache.log4j.PatternLayout
- log4j.appender.A2.layout.ConversionPattern=%d{yyyy-MM-dd hh:mm:ss}:%p %t %c - %m%n
3、创建pom.xml
这个文件定义了我们的项目结构和依赖,具体如下:
- <?xml version="1.0"?>
- <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
- <modelVersion>4.0.0</modelVersion>
- <groupId>io.mykit</groupId>
- <artifactId>mykit-db-sync-provider</artifactId>
- <version>1.0.0</version>
- <name>mykit-db-sync-provider</name>
- <licenses>
- <license>
- <name>Apache 2</name>
- <url>http://www.apache.org/licenses/LICENSE-2.0.txt</url>
- <distribution>repo</distribution>
- <comments>A business-friendly OSS license</comments>
- </license>
- </licenses>
- <properties>
- <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
- <skip_maven_deploy>false</skip_maven_deploy>
- <jdk.version>1.8</jdk.version>
- <spring.version>4.1.0.RELEASE</spring.version>
- </properties>
- <dependencies>
- <dependency>
- <groupId>org.slf4j</groupId>
- <artifactId>slf4j-log4j12</artifactId>
- <version>1.7.2</version>
- </dependency>
- <dependency>
- <groupId>commons-logging</groupId>
- <artifactId>commons-logging</artifactId>
- <version>1.1.1</version>
- </dependency>
- <dependency>
- <groupId>junit</groupId>
- <artifactId>junit</artifactId>
- <version>4.12</version>
- <scope>test</scope>
- </dependency>
- <dependency>
- <groupId>mysql</groupId>
- <artifactId>mysql-connector-java</artifactId>
- <version>5.1.22</version>
- </dependency>
- <dependency>
- <groupId>dom4j</groupId>
- <artifactId>dom4j</artifactId>
- <version>1.6.1</version>
- </dependency>
- <!-- <dependency>
- <groupId>com.microsoft.sqlserver</groupId>
- <artifactId>sqljdbc4</artifactId>
- <version>1.0</version>
- </dependency> -->
- <dependency>
- <groupId>org.quartz-scheduler</groupId>
- <artifactId>quartz</artifactId>
- <version>2.1.3</version>
- </dependency>
- <!-- <dependency>
- <groupId>com.oracle</groupId>
- <artifactId>ojdbc6</artifactId>
- <version>11.2.0.3</version>
- </dependency> -->
- </dependencies>
- <build>
- <plugins>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-jar-plugin</artifactId>
- <configuration>
- <archive>
- <manifest>
- <addClasspath>true</addClasspath>
- <classpathPrefix>lib/</classpathPrefix>
- <mainClass>io.mykit.db.sync.Main</mainClass>
- </manifest>
- </archive>
- </configuration>
- </plugin>
- <plugin>
- <groupId>org.apache.maven.plugins</groupId>
- <artifactId>maven-dependency-plugin</artifactId>
- <executions>
- <execution>
- <id>copy</id>
- <phase>package</phase>
- <goals>
- <goal>copy-dependencies</goal>
- </goals>
- <configuration>
- <outputDirectory>${project.build.directory}/lib</outputDirectory>
- </configuration>
- </execution>
- </executions>
- </plugin>
- <plugin>
- <groupId>org.mortbay.jetty</groupId>
- <artifactId>maven-jetty-plugin</artifactId>
- <version>6.1.10</version>
- </plugin>
- </plugins>
- <resources>
- <resource>
- <directory>src/main/java</directory>
- <includes>
- <include>**/*.properties</include>
- <include>**/*.xml</include>
- </includes>
- <filtering>true</filtering>
- </resource>
- </resources>
- </build>
- </project>
至此,我们就实现了基于java开发的功能强大、配置灵活的数据库之间的同步工具,大家可以根据具体需求修改job.xml中的相关配置信息即可实现数据库之前的同步。