Zabbix DB历史表由Innodb转换成Tokudb引擎总结

2022-07-26 17:48:34

线上Zabbix监控服务器DB采用的是MySQL一主一从。监控历史数据公司要求保留2年，历史数据相关的表做了表分区，过期数据自动删除。随着线上Zabbix监控设备数的不断增加，服务器磁盘空间增长的很厉害，3.7T的磁盘空间目前已经用了3.3T。尝试用pt-online工具压缩数据表，由于历史数据没有主键不能压缩。

最后考虑把Zabbix历史相关的数据表由Innodb引擎转换成TokuDB引擎从而来减少磁盘空间。由于采用TokuDB压缩历史数据会损耗一定的CPU资源，申请一台HP G10服务器，内存64G 8块600G硬盘做Raid5专门做Zabbix DB服务器。

1.Percona数据库安装并开启Tokudb存储引擎。

1）安装percona数据库，采用官方最新的RPM进行安装。

[root@ZBXTKDB01 software]# wget https://www.percona.com/downloads/Percona-Server-5.6/Percona-Server-5.6.42-84.2/binary/redhat/6/x86_64/Percona-Server-5.6.42-84.2-r6b2b987-el6-x86_64-bundle.tar
[root@ZBXTKDB01 software]# tar -xvf Percona-Server-5.6.42-84.2-r6b2b987-el6-x86_64-bundle.tar
[root@ZBXTKDB01 software]# rpm -ivh Percona-Server-server-56-5.6.42-rel84.2.el6.x86_64.rpm\ Percona-Server-client-56-5.6.42-rel84.2.el6.x86_64.rpm \Percona-Server-shared-56-5.6.42-rel84.2.el6.x86_64.rpm
[root@ZBXTKDB01 software]# service mysql start

2）安装LIBJEMALLOC库。TokuDB存储引擎需要libjemalloc library 3.3.0或者更新，RPM安装库文件默认在/usr/lib64/libjemalloc.so.1。

[root@ZBXTKDB01 software]# rpm -ivh jemalloc-3.6.0-1.el6.art.x86_64.rpm

3）安装Tokudb插件，RPM 安装的软件包Tokudb插件默认在 /usr/lib64/mysql/plugin/ha_tokudb.so。

[root@ZBXTKDB01 software]# rpm -ivh Percona-Server-tokudb-56-5.6.42-rel84.2.el6.x86_64.rpm

4）加载Tokudb引擎需要修改下面的内核参数并写到/etc/rc.local里面。

echo never > /sys/kernel/mm/transparent_hugepage/enabled
echo never > /sys/kernel/mm/transparent_hugepage/defrag

5）安装Tokudb引擎。

[root@ZBXTKDB01 ~]# ps_tokudb_admin --enable -uroot
Checking if Percona Server is running with jemalloc enabled...
INFO: Percona Server is running with jemalloc enabled.

Checking transparent huge pages status on the system...
INFO: Transparent huge pages are currently disabled on the system.

Checking if thp-setting=never option is already set in config file...
INFO: Option thp-setting=never is set in the config file.

Checking TokuDB engine plugin status...
INFO: TokuDB engine plugin is not installed.

Installing TokuDB engine...
INFO: Successfully installed TokuDB engine plugin.

6）可以通过手工的方式加载Tokudb存储引擎。

INSTALL PLUGIN tokudb SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_file_map SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_fractal_tree_info SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_fractal_tree_block_map SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_trx SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_locks SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_lock_waits SONAME ‘ha_tokudb.so‘;
INSTALL PLUGIN tokudb_background_job_status SONAME ‘ha_tokudb.so‘;

7）验证MySQL数据库的Tokudb引擎是否安装成功。通过show engines或者show plugins命令都可以查看。

mysql> show engines;
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| Engine             | Support | Comment                                                                    | Transactions | XA   | Savepoints |
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+------------+
| PERFORMANCE_SCHEMA | YES     | Performance Schema                                                         | NO           | NO   | NO         |
| MRG_MYISAM         | YES     | Collection of identical MyISAM tables                                      | NO           | NO   | NO         |
| CSV                | YES     | CSV storage engine                                                         | NO           | NO   | NO         |
| BLACKHOLE          | YES     | /dev/null storage engine (anything you write to it disappears)             | NO           | NO   | NO         |
| MEMORY             | YES     | Hash based, stored in memory, useful for temporary tables                  | NO           | NO   | NO         |
| TokuDB             | YES     | Percona TokuDB Storage Engine with Fractal Tree(tm) Technology             | YES          | YES  | YES        |
| MyISAM             | YES     | MyISAM storage engine                                                      | NO           | NO   | NO         |
| ARCHIVE            | YES     | Archive storage engine                                                     | NO           | NO   | NO         |
| FEDERATED          | NO      | Federated MySQL storage engine                                             | NULL         | NULL | NULL       |
| InnoDB             | DEFAULT | Percona-XtraDB, Supports transactions, row-level locking, and foreign keys | YES          | YES  | YES        |
+--------------------+---------+----------------------------------------------------------------------------+--------------+------+--------
```----+

8）由于原来Zabbix数据库版本5.5.20的安装的，可以通过下面的命令update数据库。

[root@ZBXTKDB01 ~]# mysql_upgrade -uroot -p
Enter password:
Looking for ‘mysql‘ as: mysql
Looking for ‘mysqlcheck‘ as: mysqlcheck
Running ‘mysqlcheck‘ with connection arguments: ‘--socket=/mysql/3307/mysql.sock‘ ‘--port=3307‘


9）MySQL my.conf配置文件的参数优化。主要优化了Innodb和Tokubdb参数。如果是源码安装的数据库，Tokudb扩展及LIBJEMALLOC库需要在my.conf配置文件里面指定Tokudb插件和LIBJEMALLOC库的路径地址。

[root@ZBXTKDB01 ~]# cat /etc/my.cnf
[client]
socket = /mysql/3307/mysql.sock
port = 3307

[mysql]
no-auto-rehash

[mysqld]
server-id = 6243
port = 3307
user = mysql
basedir = /usr
datadir = /mysql/3307/data
socket = /mysql/3307/mysql.sock
skip-name-resolve
default_storage_engine = innodb

binlog-do-db = zabbix
log-bin = /mysql/3307/binlog/mysql-bin
binlog_format = ROW
binlog_cache_size = 20M
expire_logs_days = 10

#slow_query_log = 1
slow_query_log_file = /mysql/3307/data/mysql-slow.log
long_query_time = 20
log_error = /mysql/3307/mysql_error.log
pid-file = /mysql/3307/mysql.pid

max_connections = 3000
back_log = 100
max_connect_errors = 1000000
max_allowed_packet = 64M
max_heap_table_size = 300M
sort_buffer_size = 2M
join_buffer_size = 2M
thread_cache_size = 400
query_cache_type = 1
query_cache_size = 128M
query_cache_limit = 2M
tmp_table_size = 1024M
#table_cache = 1000

key_buffer_size = 128M
read_buffer_size = 8M
read_rnd_buffer_size = 8M
bulk_insert_buffer_size = 10M
myisam_sort_buffer_size = 16M
myisam_max_sort_file_size = 4G
myisam_repair_threads = 1

innodb_file_per_table = 1
innodb_buffer_pool_size = 10G
innodb_use_sys_malloc = 1
innodb_data_file_path = ibdata01:400M:autoextend
innodb_log_buffer_size = 10M
innodb_log_file_size = 400M
innodb_log_files_in_group = 2
innodb_open_files = 2048
innodb_flush_log_at_trx_commit = 2
innodb_lock_wait_timeout = 60

tokudb_directio = 1
tokudb_row_format = tokudb_lzma
tokudb_cache_size=30G
optimizer_switch=index_condition_pushdown=off
tokudb_read_buf_size = 4K
tokudb_commit_sync = 0
tokudb_fsync_log_period = 60000
tokudb_fs_reserve_percent = 2
tokudb-data-dir = /mysql/3307/tokudb
tokudb-log-dir=

[mysqldump]
quick
max_allowed_packet = 16M

[myisamchk]
key_buffer_size = 512M
sort_buffer_size = 512M
read_buffer = 8M
write_buffer = 8M

[mysqlhotcopy]
interactive-timeout

[mysqld_safe]
thp-setting = never
open-files-limit = 8192


2.在安装Tokudb存储引擎的服务器上面开启Zabbix数据库新的从库。主从同步和MySQL二进制文件有关，和存储引擎无关，即Innodb存储引擎的MySQL表的数据可以同步到Tokudb存储引擎的数据表里面。

1）停止该服务器和原从库MySQL服务，将data目录拷贝到该服务器上面。

2）根据原来从库服务器数据库data目录里面/data/mysql/3307/data/master.info记录主从复制的信息，开启新服务器的主从复制。

#主库上面添加主从复制的账号
grant replication slave on . to ‘xxx‘@‘192.168.6.243‘identified by ‘xxxxxx‘
#新服务器上面开启主从同步
change master to
master_host=‘92.168.6.244‘,
master_port=3307,
master_user=‘xxx‘,
master_password=‘xxxxxx’,
master_log_file=‘msql-bin.xxxx’,
master_log_pos=xxxx;


3）验证新从库的数据和主库的数据是否正常同步，如果出现数据不一致可以通过下面的命令解决。

stop slave;
set global sql_slave_skip_counter=1;
start slave ;

set global slave_exec_mode=idempotent;
set global slave_exec_mode=STRICT;


4）Tokudb存储引擎的转换，将Zabbix数据库历史数据相关的表(history、history_uint、trends、tredns_uint 、history_str、history_log、history_text)全部转换成Tokudb存储引擎。

`mysql> alter table history_uint engine=TokuDB;`

由Innodb存储引擎转换成Tokudb存储引擎应该注意以下事项：

1）在执行alter table history_uint engine=TokuDB 命令之后，首先数据库库会执行MDL（Meta Data lock元数据锁），把Innodb表里面每条数据拿出来放存放到tokudb-data-dir目录里面的临时文件里。等所有的数据取出来存储到临时文件之后，在合并临时文件里面的数据到对应的Tokudb数据表里面。然后删除Innodb数据文件并释放磁盘空间。

[root@ZBXTKDB01 ~]# ls -l /mysql/3307/tokudb/ |more
total 214125632
-rw------- 1 mysql mysql 0 Dec 20 17:35 __tokudb_lock_dont_delete_me_data
-rw------- 1 mysql mysql 0 Dec 20 17:35 __tokudb_lock_dont_delete_me_temp
-rw------- 1 mysql mysql 1921350 Dec 24 15:07 tokuld102q5N
-rw------- 1 mysql mysql 1948116 Dec 24 13:16 tokuld106Z4m
-rw------- 1 mysql mysql 1936454 Dec 24 14:10 tokuld10AgeQ
-rw------- 1 mysql mysql 1936518 Dec 24 14:42 tokuld10DHEW
-rw------- 1 mysql mysql 1938374 Dec 24 14:42 tokuld10DJqt
-rw------- 1 mysql mysql 1943288 Dec 24 14:51 tokuld10FlFb
-rw------- 1 mysql mysql 1943969 Dec 24 12:42 tokuld10jIQP



2）由于在转换成TokuDB存储引擎的时候，会生成成千上万的临时文件，确保数据库分区空间可用性。建议1000G的Innodb转换成Tokudb的时候至少预留300G磁盘空间。

3）Innodb大表在转换成TokuDB存储引擎的时候，特别耗时。经测试history_uint表总共有230亿左右的数据，Fetched到临时表总共花费了10小时30分钟，Loading合成临时文件也需要26个小时左右。

建议把命令放在后台以脚本的方式去执行，如果出错还可以查看具体出错的日志。

`[root@ZBXTKDB01 shell]# nohup ChangeInnodbToTokudb.sh &;`

4）在后台执行alter table history_uint engine=TokuDB的时候，程序突然中断报下面的错误。需要调大innodb_buffer_pool_size 的数值。

`ERROR 1206 (HY000) at line 1: The total number of locks exceeds the lock table size`

Zabbix DB历史表由Innodb转换成Tokudb引擎总结

码农公寓

相关文章