实验环境：
vm1-ning:172.16.3.1/16 vm2-hong:172.16.3.2/16drbd操作步骤：
1、创建mysql的drbd资源磁盘(两节点都需要创建) [root@hong ~]# fdisk /dev/sda n p +5G w [root@hong ~]# kpartx -af /dev/sda [root@hong ~]# partx -a /dev/sda2、drbd设置（全局配置文件这里和web高可用中的一样没有改变，不再次说明。）
[root@hong ~]# cd /etc/drbd.d [root@hong drbd.d]# ls global_common.conf global_common.conf.bak global_common.conf.rpmsave web.res [root@hong drbd.d]# cp web.res mydata.res [root@hong drbd.d]# vim mydata.res 定义资源名称 [root@hong drbd.d]# cat mydata.res resource mydata { 定义资源名称也是唯一的，不能相同 on ning { device /dev/drbd1; 指定drdb设备名称为drbd1 disk /dev/sda4; 定义的分区磁盘 address 172.16.3.1:7790; 两个资源之间不能使用相同的端口号 meta-disk internal; } on hong { device /dev/drbd1; disk /dev/sda4; address 172.16.3.2:7790; meta-disk internal; } } [root@hong drbd.d]# scp mydata.res 172.16.3.1:/etc/drbd.d/ 双方节点上都需要配置这样复制过去即可 root@172.16.3.1's password: mydata.res100% 278 0.3KB/s 00:00
3、初始化drbd在两节点上都需要初始化，并启动drbd [root@hong drbd.d]# drbdadm create-md mydata Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. [root@ning ~]# drbdadm create-md mydata Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. [root@hong drbd.d]# service drbd start [root@ning ~]# service drbd start4、提升一个节点为主节点进行分区数据同步和格式化分区
[root@hong drbd.d]# drbd-overview （查看当前状态）
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- 1:mydata/0 Connected Secondary/Secondary Inconsistent/Inconsistent C r----- （分区还没有进行同步，数据不一致状态） [root@hong drbd.d]# drbdadm primary --force mydata （提升hong节点为主节点，进行数据同步，并进行格式化） [root@hong drbd.d]# drbd-overview 0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- 1:mydata/0 SyncSource Primary/Secondary UpToDate/Inconsistent C r---n- [>....................] sync'ed: 4.3% (4912/5128)M （正在同步块按照位在对齐） [root@hong drbd.d]# drbd-overview 0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- 1:mydata/0 Connected Primary/Secondary UpToDate/UpToDate C r----- （分区同步完成） [root@hong ~]# mke2fs -t ext4 /dev/drbd1 （对分区进行格式化） mke2fs 1.41.12 (17-May-2010) Filesystem label= OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe blocks 328656 inodes, 1313263 blocks 65663 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=1346371584 41 block groups 32768 blocks per group, 32768 fragments per group 8016 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736Writing inode tables: done
Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override. mysql配置： 1、创建mysql用户（注意两节点的mysql的ID号要一致） hong节点： [root@hong drbd.d]# groupadd -r -g 306 mysql [root@hong drbd.d]# useradd -r -g 306 -u 306 mysql [root@hong drbd.d]# id mysql uid=306(mysql) gid=306(mysql) groups=306(mysql) [root@hong ~]# mkdir /mydata 创建数据的存储目录ning节点：
[root@ning ~]# groupadd -r -g 306 mysql [root@ning ~]# useradd -r -g 306 -u 306 mysql [root@ning ~]# id mysql uid=306(mysql) gid=306(mysql) groups=306(mysql) [root@ning ~]# mkdir /mydata 创建数据的存储目录（磁盘的挂载点）2、安装mysql
（1）在主节点hong上挂载/dev/drbd1到数据的存储目录/mydata
[root@hong ~]# mount /dev/drbd1 /mydata/ （2）在挂载目录/mydata下创建数据存储目录为data，并设置属主组 [root@hong ~]# cd /mydata/ [root@hong mydata]# mkdir data [root@hong mydata]# chown -R mysql.mysql data [root@hong mydata]# ll total 20 drwxr-xr-x 2 mysql mysql 4096 Aug 16 17:21 data drwx------ 2 root root 16384 Aug 16 17:16 lost+found 验证下是否两节点都是有data目录的 [root@hong ~]# umount /mydata [root@hong ~]# drbdadm secondary mydata 在hong上降级drbd为从节点 [root@hong ~]# drbd-overview 0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- 1:mydata/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- [root@ning ~]# drbdadm primary mydata 在ning节点上提升为主节点 [root@ning ~]# drbd-overview 0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- 1:mydata/0 Connected Primary/Secondary UpToDate/UpToDate C r----- [root@ning ~]# mount /dev/drbd1 /mydata/ [root@ning ~]# cd /mydata/ [root@ning mydata]# ll total 20 drwxr-xr-x 2 mysql mysql 4096 Aug 16 17:21 data （查看是否data目录为mysql属主组） drwx------ 2 root root 16384 Aug 16 17:16 lost+found （3）安装mysql软件 包： mariadb-5.5.36-linux-x86_64.tar.gz ning节点上： [root@ning ~]# tar xf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local [root@ning ~]# cd /usr/local [root@ning local]# ln -sv mariadb-5.5.36-linux-x86_64/ mysql `mysql' -> `mariadb-5.5.36-linux-x86_64/' [root@ning local]# chown -R root.mysql mysql/* [root@ning local]# cd mysql [root@ning mysql]# scripts/mysql_install_db --datadir=/mydata/data/ --user=mysql[root@ning mysql]# mkdir /etc/mysql
[root@ning mysql]# cp support-files/my-large.cnf /etc/mysql/my.cnf [root@ning mysql]# vim /etc/mysql/my.cnf [mysqld] datadir = /mydata/data [root@ning mysql]# cp support-files/my-large.cnf /etc/mysql/my.cnf [root@ning mysql]# vim /etc/mysql/my.cnf [root@ning mysql]# vim /etc/mysql/my.cnf [root@ning mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld [root@ning mysql]# chmod +x /etc/rc.d/init.d/mysqld [root@ning mysql]# chkconfig --add mysqld [root@ning mysql]# chkconfig mysqld off [root@ning mysql]# service mysqld start Starting MySQL.... [ OK ] [root@ning mysql]# ls /mydata/data aria_log.00000001 ibdata1 ib_logfile1 mysql-bin.000001 ning.pid test aria_log_control ib_logfile0 mysql mysql-bin.index performance_schema [root@ning mysql]# /usr/local/mysql/bin/mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 2 Server version: 5.5.36-MariaDB-log MariaDB ServerCopyright (c) 2000, 2014, Oracle, Monty Program Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> create database mysdb;
Query OK, 1 row affected (0.00 sec)MariaDB [(none)]> \q
Bye 到这里mariadb数据库已经安装完成 [root@ning mysql]# service mysqld stop Shutting down MySQL. [ OK ] [root@ning mysql]# cd [root@ning ~]# umount /dev/drbd1 卸载挂载点 [root@ning ~]# drbdadm secondary mydata 降级为从节点 [root@ning ~]# drbd-overview 0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- 1:mydata/0 Connected Secondary/Secondary UpToDate/UpToDate C r----- [root@ning ~]# scp mariadb-5.5.36-linux-x86_64.tar.gz 172.16.3.2:/root 复制包到ning节点，进行安装 root@172.16.3.2's password: mariadb-5.5.36-linux-x86_64.tar.gz 100% 212MB 26.5MB/s 00:08 hong节点： [root@hong ~]# drbdadm primary mydata 提升hong节点为主节点 [root@hong ~]# mount /dev/drbd1 /mydata 挂载分区 [root@hong ~]# scp -r 172.16.3.1:/etc/mysql /etc/ 并复制ning节点上的mysqld配置文件，（因为配置文件是一样的） root@172.16.3.1's password: my.cnf 100% 4924 4.8KB/s 00:00 [root@hong ~]# tar xf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local [root@hong ~]# cd /usr/local [root@hong local]# ln -sv mariadb-5.5.36-linux-x86_64/ mysql `mysql' -> `mariadb-5.5.36-linux-x86_64/' [root@hong local]# cd mysql [root@hong mysql]# chown -R root.mysql ./* 这里说明下，不需要对mariadb做初始化了，因为数据已经有了，在ning节点上做的。在这里只需要提供服务启动脚本即可 [root@hong mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld [root@hong mysql]# chmod +x /etc/rc.d/init.d/mysqld [root@hong mysql]# chkconfig --add mysqld [root@hong mysql]# service mysqld start Starting MySQL.... [ OK ] [root@hong mysql]# /usr/local/mysql/bin/mysql Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 2 Server version: 5.5.36-MariaDB-log MariaDB ServerCopyright (c) 2000, 2014, Oracle, Monty Program Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases; 查看在ning节点上创建的数据库是存在的
+--------------------+ | Database | +--------------------+ | information_schema | | mysdb | 看这里呵呵 | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.05 sec) MariaDB [(none)]> grant all on *.* to root@'172.16.%.%' identified by 'ning'; 授权远程连接用户 Query OK, 0 rows affected (0.03 sec)MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec) 测试下是否能够连接上数据库：（随便找个节点上测试的（这节点上需要有mysql客户端）） [root@liang ~]# mysql -uroot -h 172.16.3.2 -p Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.5.36-MariaDB-log MariaDB ServerCopyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+ | Database | +--------------------+ | information_schema | | mysdb | | mysql | 这里可以看到创建的数据库 | performance_schema | | test | +--------------------+ 5 rows in set (0.00 sec)mysql> \q
Bye （4）测试完成后，在两节点上关闭mysqld和drbd开始配置corosync [root@hong mysql]# service mysqld stop Shutting down MySQL. [ OK ] [root@hong mysql]# umount /mydata [root@hong mysql]# drbdadm secondary mydata [root@hong mysql]# service drbd stop Stopping all DRBD resources: . [root@ning ~]# service drbd stop Stopping all DRBD resources: . 配置corosync集群设置 1、定义drbd [root@ning ~]# crm crm(live)# configure 定义drbd资源 crm(live)configure# primitive mydrbd ocf:linbit:drbd params drbd_resource=mydata op monitor role=Master interval=10s timeout=20s op monitor role=Slave interval=20s timeout=30s op start timeout=240s op stop timeout=100s 注意下定义时间间隔的时候主从不能一样。 crm(live)configure# verify 定义drbd克隆资源 crm(live)configure# master ms_mydrbd mydrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=True crm(live)configure# verify 定义挂载磁盘资源 crm(live)configure# primitive myfs ocf:heartbeat:Filesystem params device=/dev/drbd1 directory=/mydata fstype=ext4 op monitor interval=20s timeout=60s op start timeout=60s op stop timeout=60s crm(live)configure# verify 定义mysqlIP资源 crm(live)configure# primitive myip ocf:heartbeat:IPaddr params ip=172.16.3.101 op monitor interval=30s timeout=20s crm(live)configure# verify 定义mysql服务启动资源 crm(live)configure# primitive myserver lsb:mysqld op monitor interval=30s timeout=20s crm(live)configure# verify 定义规则 crm(live)configure# group myservice myip myfs myserver 定义组名称为myservice资源名称为myip\myfs\myserver crm(live)configure# colocation myfs_with_ms_mydrbd_master inf: myfs ms_mydrbd:Master 定义myfs挂载磁盘资源一定和ms_mydrbd克隆资源在一起 crm(live)configure# verify crm(live)configure# order myfs_after_ms_mydrbd_master mandatory: ms_mydrbd:promote myfs:start 定义启动顺序挂载磁盘my_fs一定在克隆资源（成为主节点）ms_mydrbd之后启动 crm(live)configure# colocation myserver_with_myfs inf: myserver myfs 定义mysql服务器启动一定要和挂载磁盘my_fs在一起 crm(live)configure# verify crm(live)configure# commit crm(live)# status (查看状态信息) Last updated: Fri Sep 19 13:16:37 2014 Last change: Fri Sep 19 12:26:20 2014 via cibadmin on hong Stack: classic openais (with plugin) Current DC: ning - partition with quorum Version: 1.1.10-14.el6-368c726 2 Nodes configured, 2 expected votes 5 Resources configuredOnline: [ hong ning ] （两个节点都在线）
Master/Slave Set: ms_mydrbd [mydrbd]
Masters: [ ning ] Slaves: [ hong ] Resource Group: myservice myip (ocf::heartbeat:IPaddr): Started ning myfs (ocf::heartbeat:Filesystem): Started ning myserver (lsb:mysqld): Started ning 测试corosync+mariadb+drbd 1、测试是否ning节点下线，资源将被切换到hong节点上去 [root@ning ~]# crm node standby 让ning节点下线 [root@ning ~]# crm status Last updated: Fri Sep 19 13:24:33 2014 Last change: Fri Sep 19 13:24:25 2014 via crm_attribute on ning Stack: classic openais (with plugin) Current DC: ning - partition with quorum Version: 1.1.10-14.el6-368c726 2 Nodes configured, 2 expected votes 5 Resources configuredNode ning: standby
Online: [ hong ]Master/Slave Set: ms_mydrbd [mydrbd]
Masters: [ hong ] drbd将自动切换到hong节点上 Stopped: [ ning ] Resource Group: myservice myip (ocf::heartbeat:IPaddr): Started hong myfs (ocf::heartbeat:Filesystem): Started hong myserver (lsb:mysqld): Started hong 在hong 节点查看下mysqlIP； [root@hong ~]# ip addr show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000 link/ether 00:0c:29:d7:f7:9c brd ff:ff:ff:ff:ff:ff inet 172.16.3.2/16 brd 172.16.255.255 scope global eth0 inet 172.16.3.101/16 brd 172.16.255.255 scope global secondary eth0 这里是有mysqlIP inet6 fe80::20c:29ff:fed7:f79c/64 scope link valid_lft forever preferred_lft forever 在liang节点上尝试连接下mariadb [root@liang ~]# mysql -uroot -h 172.16.3.101 -p 这里连接的mysqlIP的地址 Enter password: Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 4 Server version: 5.5.36-MariaDB-log MariaDB ServerCopyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective owners.Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+ | Database | +--------------------+ | information_schema | | mysdb |这里有我们在上面测试用到的mysdb | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.10 sec) 2、测试mysqld手动当掉，是否会自动启动？ # service mysqld stop # ss -tnl 观察3306端口将会看到30s后会自动上线