13-jdbc批量插入

mybatis批量插入

public void insertBatch(List<User> userList) {
    List<List<User>> partition = ListUtil.partition(userList, 100);
    for (List<User> users : partition) {
        userDao.insertBatch(users);
    }
}

<insert id="insertBatch">
    INSERT INTO `demo`.`user` (`username`, `address`, `remark`, `age`, `create_time`)
        VALUES
        <foreach collection="users" index="" item="user" separator=",">
            (#{user.username,jdbcType=VARCHAR},
            #{user.address,jdbcType=VARCHAR},
            #{user.remark,jdbcType=VARCHAR},
            #{user.age,jdbcType=INTEGER},
            now())
        </foreach>
</insert>

BatchExecutor插入

mybatis提供了三种sql执行器，分别是SIMPLE(默认)，REUSE，BATCH：

• SIMPLE（SimpleExecutor），相当于JDBC的PreparedStatement.execute(sql)执行完毕即关闭即 PreparedStatement.close()
• REUSE（ReuseExecutor），相当于JDBC的PreparedStatement.execute(sql) 执行完不关闭，而是将PreparedStatement存入 Map<String, Statement>中缓存，其中key为执行的sql模板；
• BATCH（BatchExecutor），相当于JDBC语句的PreparedStatement.addBatch(sql)，即仅将执行SQL加入到批量计划但是不真正执行， 所以此时不会执行返回受影响的行数，而只有执行PreparedStatement.execteBatch()后才会真正执行sql

@Autowired
private SqlSessionFactory sqlSessionFactory;

@Override
public void insertBatchType(List<User> userList) {
    SqlSession sqlSession = sqlSessionFactory.openSession(ExecutorType.BATCH);
    UserDao mapper = sqlSession.getMapper(UserDao.class);
    try {
        for (User user : userList) {
            mapper.insert(user);
        }
        sqlSession.commit();
    } catch (Exception e) {
        System.out.println("批量导入数据异常，事务回滚");
        sqlSession.rollback();
    } finally {
        sqlSession.close();
    }
}

JDBC插入

当然也可以使用原生的JDBC的方式进行批量插入，使用statement.addBatch();的方式，也是很快的

@Resource(name = "dataSource")
private DataSource dataSource;

@Override
public void insertJdbc(List<User> userList) throws SQLException {
    Connection connection = null;
    PreparedStatement statement = null;
    try {
        connection = dataSource.getConnection();
        connection.setAutoCommit(false);
        String sql = "INSERT INTO `user` (`username`, `address`, `remark`, `age`, `create_time`) " +
                "VALUES (?,?,?,?,now()) ";
        statement = connection.prepareStatement(sql);
        for (User user : userList) {
            statement.setString(1, user.getUsername());
            statement.setString(2, user.getAddress());
            statement.setString(3, user.getRemark());
            statement.setInt(4, user.getAge());
            statement.addBatch();
        }
        statement.executeBatch();
        connection.commit();
    } catch (SQLException throwables) {
        throwables.printStackTrace();
    } finally {
        statement.close();
        connection.close();
    }
}

测试效率

@Test
public void test_for_user() throws SQLException {
    List<User> userList = new ArrayList<>();
    for (int i = 0; i < 600000; i++) {
        User user = new User();
        user.setUsername("张三" + i);
        user.setAddress("上海" + i);
        user.setRemark("备注" + i);
        user.setAge(i);
        userList.add(user);
    }

    StopWatch stopWatch = new StopWatch();

    stopWatch.start("mybatis批量插入");
    userService.insertBatch(userList);
    stopWatch.stop();
    System.out.println(stopWatch.getLastTaskName() + ":" + stopWatch.getLastTaskTimeMillis());

    stopWatch.start("BatchType插入");
    userService.insertBatchType(userList);
    stopWatch.stop();
    System.out.println(stopWatch.getLastTaskName() + ":" + stopWatch.getLastTaskTimeMillis());

    stopWatch.start("JDBC-BatchType插入");
    userService.insertJdbc(userList);
    stopWatch.stop();
    System.out.println(stopWatch.getLastTaskName() + ":" + stopWatch.getLastTaskTimeMillis());

}

执行效率从高到低依次是：JDBC-BatchType插入 > BatchType插入 > mybatis批量插入