MySQL/MairaDB Master High Active (MHA) 安裝、設定、測試步驟

1/24/2022 08:00:00 上午

因為測試使用，所以只搞兩台測試機來測試，

實際正式環境是使用三台，master / slave / manager (MHA)

以下為本次測試環境

HostnameIPAddressLinux
Master10.211.55.6 / 10.211.55.10 (VIP)CentOS 7.8
Slave / Manager10.211.55.7CentOS 7.8


環境安裝:

yum install net-tools
yum install perl-Parallel-ForkManager
yum install perl-Log-Dispatch
yum install perl-Config-Tiny
yum install perl-DBD-MySQL

# 以下是測試過程中，還有另外安裝的項目
yum install perl-Module-Install
yum install perl-IPC-Run3
yum install *YAML*
yum install gcc
yum install centos-release-scl-rh
yum install rh-perl526-perl-Specio
rpm -ivy http://repo.openfusion.net/centos7-x86_64/perl-Test-Needs-0.002005-1.of.el7.noarch.rpm

SSH Keyless (每台互連均要設定)

ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.211.55.6

ssh-keygen
ssh-copy-id -i /root/.ssh/id_rsa.pub root@10.211.55.7

MySQL 加入 MHA 監控帳號 ( 從Master設定，再同步至Slave端 )

create user 'mha_manager'@'10.211.55.%' identified by 'mhapass';
grant reload, super on *.* to 'mha_manager'@'10.211.55.%';
grant select on mysql.* to 'mha_manager'@'10.211.55.%';
grant all on mysql.apply_diff_relay_logs to 'mha_manager'@'10.211.55.%';
grant all on mysql.apply_diff_relay_logs_test to 'mha_manager'@'10.211.55.%';

 

採用 Master High Active (MHA) 0.58 版本，從官網下載 source code 來安裝，安裝過程使用 root 身份進行安裝

Node 節點:

git clone https://github.com/yoshinorim/mha4mysql-node.git
cd mha4mysql-node/
perl Makefile.PL
make
mkae install

Manager 節點:

git clone https://github.com/yoshinorim/mha4mysql-manager.git
cd mha4mysql-manager
perl -MCPAN -e "install Config::Tiny"
perl -MCPAN -e "install Log::Dispatch"
perl -MCPAN -e "install Parallel::ForkManager"
perl Makefile.PL
make
make install

Manager Conf 設定 ( 可參考 mha4mysql-manager/samples/conf/masterha_default.cnf )

[server default]
user=mha_manager
password=mhapass
ssh_user=root
master_binlog_dir=/var/lib/mysql/,/binlog/3306/
remote_workdir=/data/manager
#secondary_check_script= masterha_secondary_check -s remote_host1 -s remote_host2
ping_interval=3
repl_user=repl
repl_password=repl
master_ip_failover_script="/opt/mha4mysql-manager-0.56/samples/scripts/master_ip_failover"
master_ip_online_change_script="/opt/mha4mysql-manager-0.56/samples/scripts/master_ip_online_change"
# shutdown_script= /script/masterha/power_manager
# report_script= /script/masterha/send_report

[server1]
hostname=10.211.55.7
candidate_master=1

[server2]
hostname=10.211.55.6
candidate_master=1

SSH 測試

masterha_check_ssh --conf=/opt/mhamanager/masterha_default.cnf

 

Thu May 28 15:54:56 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu May 28 15:54:56 2020 - [info] Reading application default configuration from /opt/mhamanager/masterha_default.cnf..
Thu May 28 15:54:56 2020 - [info] Reading server configuration from /opt/mhamanager/masterha_default.cnf..
Thu May 28 15:54:56 2020 - [info] Starting SSH connection tests..
Thu May 28 15:54:57 2020 - [debug] 
Thu May 28 15:54:56 2020 - [debug]  Connecting via SSH from root@10.211.55.6(10.211.55.6:22) to root@10.211.55.7(10.211.55.7:22)..
Thu May 28 15:54:57 2020 - [debug]   ok.
Thu May 28 15:54:58 2020 - [debug] 
Thu May 28 15:54:56 2020 - [debug]  Connecting via SSH from root@10.211.55.7(10.211.55.7:22) to root@10.211.55.6(10.211.55.6:22)..
Thu May 28 15:54:57 2020 - [debug]   ok.
Thu May 28 15:54:58 2020 - [info] All SSH connection tests passed successfully.

測試 Repl 功能是否正常

masterha_check_repl --conf=/opt/mhamanager/masterha_default.cnf

Thu May 28 16:10:59 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.
Thu May 28 16:10:59 2020 - [info] Reading application default configuration from /opt/mhamanager/masterha_default.cnf..
Thu May 28 16:10:59 2020 - [info] Reading server configuration from /opt/mhamanager/masterha_default.cnf..
Thu May 28 16:10:59 2020 - [info] MHA::MasterMonitor version 0.58.
Thu May 28 16:11:00 2020 - [error][/usr/local/share/perl5/MHA/ServerManager.pm, ln193] There is no alive slave. We can't do failover
Thu May 28 16:11:00 2020 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln427] Error happened on checking configurations.  at /usr/local/share/perl5/MHA/MasterMonitor.pm line 329.
Thu May 28 16:11:00 2020 - [error][/usr/local/share/perl5/MHA/MasterMonitor.pm, ln525] Error happened on monitoring servers.
Thu May 28 16:11:00 2020 - [info] Got exit code 1 (Not master dead).

MySQL Replication Health is NOT OK!

Repl 測試有錯誤，因為Slave防火牆沒關… 

關掉後重試，會發生不認得 super_read_only 這個參數，google 發現問題所在，這是 Mariadb 10.1以後才有的參數，只能降 MHA 的版本！

Thu May 28 16:52:52 2020 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='mha_manager' --slave_host=10.211.55.6 --slave_ip=10.211.55.6 --slave_port=3306 --workdir=/data/manager --target_version=10.5.3-MariaDB-log --manager_version=0.56 --relay_log_info=/binlog/3306/relay-log.info  --relay_dir=/data/3306/  --slave_pass=xxx

Thu May 28 16:52:52 2020 - [info]   Connecting to root@10.211.55.6(10.211.55.6:22).. 

  Checking slave recovery environment settings..

    Opening /binlog/3306/relay-log.info ... ok.

    Relay log found at /binlog/3306, up to relay-bin.000014

    Temporary relay log file is /binlog/3306/relay-bin.000014

    Checking if super_read_only is defined and turned on..DBD::mysql::st execute failed: Unknown system variable 'super_read_only' at /usr/local/share/perl5/MHA/SlaveUtil.pm line 245.

改裝 0.56 版本 ( manager、node ) 重新測試 ( 下載頁面 )

Thu May 28 17:12:14 2020 - [warning] Global configuration file /etc/masterha_default.cnf not found. Skipping.

Thu May 28 17:12:14 2020 - [info] Reading application default configuration from /root/masterha_default.cnf..

Thu May 28 17:12:14 2020 - [info] Reading server configuration from /root/masterha_default.cnf..

Thu May 28 17:12:14 2020 - [info] MHA::MasterMonitor version 0.56.

Thu May 28 17:12:15 2020 - [info] GTID failover mode = 0

Thu May 28 17:12:15 2020 - [info] Dead Servers:

Thu May 28 17:12:15 2020 - [info] Alive Servers:

Thu May 28 17:12:15 2020 - [info]   10.211.55.7(10.211.55.7:3306)

Thu May 28 17:12:15 2020 - [info]   10.211.55.6(10.211.55.6:3306)

Thu May 28 17:12:15 2020 - [info] Alive Slaves:

Thu May 28 17:12:15 2020 - [info]   10.211.55.6(10.211.55.6:3306)  Version=10.5.3-MariaDB-log (oldest major version between slaves) log-bin:enabled

Thu May 28 17:12:15 2020 - [info]     Replicating from 10.211.55.7(10.211.55.7:3306)

Thu May 28 17:12:15 2020 - [info]     Primary candidate for the new Master (candidate_master is set)

Thu May 28 17:12:15 2020 - [info] Current Alive Master: 10.211.55.7(10.211.55.7:3306)

Thu May 28 17:12:15 2020 - [info] Checking slave configurations..

Thu May 28 17:12:15 2020 - [info]  read_only=1 is not set on slave 10.211.55.6(10.211.55.6:3306).

Thu May 28 17:12:15 2020 - [warning]  relay_log_purge=0 is not set on slave 10.211.55.6(10.211.55.6:3306).

Thu May 28 17:12:15 2020 - [info] Checking replication filtering settings..

Thu May 28 17:12:15 2020 - [info]  binlog_do_db= , binlog_ignore_db= 

Thu May 28 17:12:15 2020 - [info]  Replication filtering check ok.

Thu May 28 17:12:15 2020 - [info] GTID (with auto-pos) is not supported

Thu May 28 17:12:15 2020 - [info] Starting SSH connection tests..

Thu May 28 17:12:17 2020 - [info] All SSH connection tests passed successfully.

Thu May 28 17:12:17 2020 - [info] Checking MHA Node version..

Thu May 28 17:12:17 2020 - [info]  Version check ok.

Thu May 28 17:12:17 2020 - [info] Checking SSH publickey authentication settings on the current master..

Thu May 28 17:12:18 2020 - [info] HealthCheck: SSH to 10.211.55.7 is reachable.

Thu May 28 17:12:18 2020 - [info] Master MHA Node version is 0.58.

Thu May 28 17:12:18 2020 - [info] Checking recovery script configurations on 10.211.55.7(10.211.55.7:3306)..

Thu May 28 17:12:18 2020 - [info]   Executing command: save_binary_logs --command=test --start_pos=4 --binlog_dir=/var/lib/mysql/,/var/log/mysql --output_file=/data/manager/save_binary_logs_test --manager_version=0.56 --start_file=mysql-bin.000007 

Thu May 28 17:12:18 2020 - [info]   Connecting to root@10.211.55.7(10.211.55.7:22).. 

  Creating /data/manager if not exists..    ok.

  Checking output directory is accessible or not..

   ok.

  Binlog found at /var/lib/mysql/, up to mysql-bin.000007

Thu May 28 17:12:19 2020 - [info] Binlog setting check done.

Thu May 28 17:12:19 2020 - [info] Checking SSH publickey authentication and checking recovery script configurations on all alive slave servers..

Thu May 28 17:12:19 2020 - [info]   Executing command : apply_diff_relay_logs --command=test --slave_user='mha_manager' --slave_host=10.211.55.6 --slave_ip=10.211.55.6 --slave_port=3306 --workdir=/data/manager --target_version=10.5.3-MariaDB-log --manager_version=0.56 --relay_log_info=/binlog/3306/relay-log.info  --relay_dir=/data/3306/  --slave_pass=xxx

Thu May 28 17:12:19 2020 - [info]   Connecting to root@10.211.55.6(10.211.55.6:22).. 

  Checking slave recovery environment settings..

    Opening /binlog/3306/relay-log.info ... ok.

    Relay log found at /binlog/3306, up to relay-bin.000016

    Temporary relay log file is /binlog/3306/relay-bin.000016

    Testing mysql connection and privileges.. done.

    Testing mysqlbinlog output.. done.

    Cleaning up test file(s).. done.

Thu May 28 17:12:19 2020 - [info] Slaves settings check done.

Thu May 28 17:12:19 2020 - [info] 

10.211.55.7(10.211.55.7:3306) (current master)

 +--10.211.55.6(10.211.55.6:3306)

Thu May 28 17:12:19 2020 - [info] Checking replication health on 10.211.55.6..

Thu May 28 17:12:19 2020 - [info]  ok.

Thu May 28 17:12:19 2020 - [warning] master_ip_failover_script is not defined.

Thu May 28 17:12:19 2020 - [warning] shutdown_script is not defined.

Thu May 28 17:12:19 2020 - [info] Got exit code 0 (Not master dead).

MySQL Replication Health is OK.

確認沒問題，啟動 MHA 的服務吧！

nohup masterha_manager --conf=/root/masterha_default.cnf > /tmp/mha.log 2>&1 &

 

測試：Online switchover，要先調整 sample 提供的 master_ip_online_change 腳本

#!/usr/bin/env perl

#  Copyright (C) 2011 DeNA Co.,Ltd.
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#  Foundation, Inc.,
#  51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA

## Note: This is a sample script and is not complete. Modify the script based on your environment.

use strict;
use warnings FATAL => 'all';

use Getopt::Long;
use MHA::DBHelper;
use MHA::NodeUtil;
use Time::HiRes qw( sleep gettimeofday tv_interval );
use Data::Dumper;

my $_tstart;
my $_running_interval = 0.1;
my (
  $command,              $orig_master_is_new_slave, $orig_master_host,
  $orig_master_ip,       $orig_master_port,         $orig_master_user,
  $orig_master_password, $orig_master_ssh_user,     $new_master_host,
  $new_master_ip,        $new_master_port,          $new_master_user,
  $new_master_password,  $new_master_ssh_user,
);

####新增區塊####
my $vip = '10.211.55.10/24';
my $key = "0";
my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down";
my $exit_code = 0;
####新增區塊####

GetOptions(
  'command=s'                => \$command,
  'orig_master_is_new_slave' => \$orig_master_is_new_slave,
  'orig_master_host=s'       => \$orig_master_host,
  'orig_master_ip=s'         => \$orig_master_ip,
  'orig_master_port=i'       => \$orig_master_port,
  'orig_master_user=s'       => \$orig_master_user,
  'orig_master_password=s'   => \$orig_master_password,
  'orig_master_ssh_user=s'   => \$orig_master_ssh_user,
  'new_master_host=s'        => \$new_master_host,
  'new_master_ip=s'          => \$new_master_ip,
  'new_master_port=i'        => \$new_master_port,
  'new_master_user=s'        => \$new_master_user,
  'new_master_password=s'    => \$new_master_password,
  'new_master_ssh_user=s'    => \$new_master_ssh_user,
);

exit &main();

sub current_time_us {
  my ( $sec, $microsec ) = gettimeofday();
  my $curdate = localtime($sec);
  return $curdate . " " . sprintf( "%06d", $microsec );
}

sub sleep_until {
  my $elapsed = tv_interval($_tstart);
  if ( $_running_interval > $elapsed ) {
    sleep( $_running_interval - $elapsed );
  }
}
sub get_threads_util {
  my $dbh                    = shift;
  my $my_connection_id       = shift;
  my $running_time_threshold = shift;
  my $type                   = shift;
  $running_time_threshold = 0 unless ($running_time_threshold);
  $type                   = 0 unless ($type);
  my @threads;

  my $sth = $dbh->prepare("SHOW PROCESSLIST");
  $sth->execute();

  while ( my $ref = $sth->fetchrow_hashref() ) {
    my $id         = $ref->{Id};
    my $user       = $ref->{User};
    my $host       = $ref->{Host};
    my $command    = $ref->{Command};
    my $state      = $ref->{State};
    my $query_time = $ref->{Time};
    my $info       = $ref->{Info};
    $info =~ s/^\s*(.*?)\s*$/$1/ if defined($info);
    next if ( $my_connection_id == $id );
    next if ( defined($query_time) && $query_time < $running_time_threshold );
    next if ( defined($command)    && $command eq "Binlog Dump" );
    next if ( defined($user)       && $user eq "system user" );
    next
      if ( defined($command)
      && $command eq "Sleep"
      && defined($query_time)
      && $query_time >= 1 );

    if ( $type >= 1 ) {
      next if ( defined($command) && $command eq "Sleep" );
      next if ( defined($command) && $command eq "Connect" );
    }

    if ( $type >= 2 ) {
      next if ( defined($info) && $info =~ m/^select/i );
      next if ( defined($info) && $info =~ m/^show/i );
    }

    push @threads, $ref;
  }
  return @threads;
}

sub main {
  if ( $command eq "stop" ) {
    ## Gracefully killing connections on the current master
    # 1. Set read_only= 1 on the new master
    # 2. DROP USER so that no app user can establish new connections
    # 3. Set read_only= 1 on the current master
    # 4. Kill current queries
    # * Any database access failure will result in script die.
    my $exit_code = 1;
    eval {
      ## Setting read_only=1 on the new master (to avoid accident)
      my $new_master_handler = new MHA::DBHelper();

      # args: hostname, port, user, password, raise_error(die_on_error)_or_not
      $new_master_handler->connect( $new_master_ip, $new_master_port,
        $new_master_user, $new_master_password, 1 );
      print current_time_us() . " Set read_only on the new master.. ";
      $new_master_handler->enable_read_only();
      if ( $new_master_handler->is_read_only() ) {
        print "ok.\n";
      }
      else {
        die "Failed!\n";
      }
      $new_master_handler->disconnect();

      # Connecting to the orig master, die if any database error happens
      my $orig_master_handler = new MHA::DBHelper();
      $orig_master_handler->connect( $orig_master_ip, $orig_master_port,
        $orig_master_user, $orig_master_password, 1 );

      ## Drop application user so that nobody can connect. Disabling per-session binlog beforehand
      $orig_master_handler->disable_log_bin_local();
      print current_time_us() . " Drpping app user on the orig master..\n";
      ##Mark
      #FIXME_xxx_drop_app_user($orig_master_handler);

      ## Waiting for N * 100 milliseconds so that current connections can exit
      my $time_until_read_only = 15;
      $_tstart = [gettimeofday];
      my @threads = get_threads_util( $orig_master_handler->{dbh},
        $orig_master_handler->{connection_id} );
      while ( $time_until_read_only > 0 && $#threads >= 0 ) {
        if ( $time_until_read_only % 5 == 0 ) {
          printf
"%s Waiting all running %d threads are disconnected.. (max %d milliseconds)\n",
            current_time_us(), $#threads + 1, $time_until_read_only * 100;
          if ( $#threads < 5 ) {
            print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
              foreach (@threads);
          }
        }
        sleep_until();
        $_tstart = [gettimeofday];
        $time_until_read_only--;
        @threads = get_threads_util( $orig_master_handler->{dbh},
          $orig_master_handler->{connection_id} );
      }

      ## Setting read_only=1 on the current master so that nobody(except SUPER) can write
      print current_time_us() . " Set read_only=1 on the orig master.. ";
      $orig_master_handler->enable_read_only();
      if ( $orig_master_handler->is_read_only() ) {
        print "ok.\n";
      }
      else {
        die "Failed!\n";
      }

      ## Waiting for M * 100 milliseconds so that current update queries can complete
      my $time_until_kill_threads = 5;
      @threads = get_threads_util( $orig_master_handler->{dbh},
        $orig_master_handler->{connection_id} );
      while ( $time_until_kill_threads > 0 && $#threads >= 0 ) {
        if ( $time_until_kill_threads % 5 == 0 ) {
          printf
"%s Waiting all running %d queries are disconnected.. (max %d milliseconds)\n",
            current_time_us(), $#threads + 1, $time_until_kill_threads * 100;
          if ( $#threads < 5 ) {
            print Data::Dumper->new( [$_] )->Indent(0)->Terse(1)->Dump . "\n"
              foreach (@threads);
          }
        }
        sleep_until();
        $_tstart = [gettimeofday];
        $time_until_kill_threads--;
        @threads = get_threads_util( $orig_master_handler->{dbh},
          $orig_master_handler->{connection_id} );
      }

      ## Terminating all threads
      print current_time_us() . " Killing all application threads..\n";
      $orig_master_handler->kill_threads(@threads) if ( $#threads >= 0 );
      print current_time_us() . " done.\n";
      $orig_master_handler->enable_log_bin_local();
      $orig_master_handler->disconnect();

      ## After finishing the script, MHA executes FLUSH TABLES WITH READ LOCK
      ####新增區塊####
      &stop_vip();
      ####新增區塊####
      $exit_code = 0;
    };
    if ($@) {
      warn "Got Error: $@\n";
      exit $exit_code;
    }
    exit $exit_code;
  }
  elsif ( $command eq "start" ) {
    ## Activating master ip on the new master
    # 1. Create app user with write privileges
    # 2. Moving backup script if needed
    # 3. Register new master's ip to the catalog database

# We don't return error even though activating updatable accounts/ip failed so that we don't interrupt slaves' recovery.
# If exit code is 0 or 10, MHA does not abort
    my $exit_code = 10;
    eval {
      my $new_master_handler = new MHA::DBHelper();

      # args: hostname, port, user, password, raise_error_or_not
      $new_master_handler->connect( $new_master_ip, $new_master_port,
        $new_master_user, $new_master_password, 1 );

      ## Set read_only=0 on the new master
      $new_master_handler->disable_log_bin_local();
      print current_time_us() . " Set read_only=0 on the new master.\n";
      $new_master_handler->disable_read_only();

      ## Creating an app user on the new master
      print current_time_us() . " Creating app user on the new master..\n";
      ##Mark
      #FIXME_xxx_create_app_user($new_master_handler);
      $new_master_handler->enable_log_bin_local();
      $new_master_handler->disconnect();

      ## Update master ip on the catalog database, etc
      ####新增區塊####
      &start_vip();
      ####新增區塊####
      $exit_code = 0;
    };
    if ($@) {
      warn "Got Error: $@\n";
      exit $exit_code;
    }
    exit $exit_code;
  }
  elsif ( $command eq "status" ) {
    ####新增區塊####
    print "Checking the Status of the script.. OK \n";
    `ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_start_vip \"`;
    ####新增區塊####
    # do nothing
    exit 0;
  }
  else {
    &usage();
    exit 1;
  }
}

####新增區塊####
# A simple system call that enable the VIP on the new master
sub start_vip() {
  `ssh $new_master_ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# # A simple system call that disable the VIP on the old_master
sub stop_vip() {
  `ssh $orig_master_ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
####新增區塊####

sub usage {
  print
"Usage: master_ip_online_change --command=start|stop|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
  die;
}

切換測試 ( masterha_manager 服務必需是停止的狀態，不然無法 online 切換 ) 

masterha_master_switch --master_state=alive --conf=/root/masterha_default.cnf --orig_master_is_new_slave

 

測試：Failover 切換

要先調整原本 sample 提供的 master_ip_failover 腳本，調整完要啟用 masterha_manager 服務，才能後續的測試

#!/usr/bin/env perl

#  Copyright (C) 2011 DeNA Co.,Ltd.
#
#  This program is free software; you can redistribute it and/or modify
#  it under the terms of the GNU General Public License as published by
#  the Free Software Foundation; either version 2 of the License, or
#  (at your option) any later version.
#
#  This program is distributed in the hope that it will be useful,
#  but WITHOUT ANY WARRANTY; without even the implied warranty of
#  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#  GNU General Public License for more details.
#
#  You should have received a copy of the GNU General Public License
#   along with this program; if not, write to the Free Software
#  Foundation, Inc.,
#  51 Franklin Street, Fifth Floor, Boston, MA  02110-1301  USA

## Note: This is a sample script and is not complete. Modify the script based on your environment.

use strict;
use warnings FATAL => 'all';

use Getopt::Long;
use MHA::DBHelper;

my (
  $command,        $ssh_user,         $orig_master_host,
  $orig_master_ip, $orig_master_port, $new_master_host,
  $new_master_ip,  $new_master_port,  $new_master_user,
  $new_master_password
);

####新增區塊 Start####
my $vip = '10.211.55.10/24';
my $key = "0";
my $ssh_start_vip = "/sbin/ifconfig eth0:$key $vip";
my $ssh_stop_vip = "/sbin/ifconfig eth0:$key down";
####新增區塊 End####

GetOptions(
  'command=s'             => \$command,
  'ssh_user=s'            => \$ssh_user,
  'orig_master_host=s'    => \$orig_master_host,
  'orig_master_ip=s'      => \$orig_master_ip,
  'orig_master_port=i'    => \$orig_master_port,
  'new_master_host=s'     => \$new_master_host,
  'new_master_ip=s'       => \$new_master_ip,
  'new_master_port=i'     => \$new_master_port,
  'new_master_user=s'     => \$new_master_user,
  'new_master_password=s' => \$new_master_password,
);

exit &main();

sub main {
  if ( $command eq "stop" || $command eq "stopssh" ) {

    # $orig_master_host, $orig_master_ip, $orig_master_port are passed.
    # If you manage master ip address at global catalog database,
    # invalidate orig_master_ip here.
    my $exit_code = 1;
    eval {
      ####新增區塊 Start####
      &stop_vip();
      ####新增區塊 End####
      # updating global catalog, etc
      $exit_code = 0;
    };
    if ($@) {
      warn "Got Error: $@\n";
      exit $exit_code;
    }
    exit $exit_code;
  }
  elsif ( $command eq "start" ) {

    # all arguments are passed.
    # If you manage master ip address at global catalog database,
    # activate new_master_ip here.
    # You can also grant write access (create user, set read_only=0, etc) here.
    my $exit_code = 10;
    eval {
      my $new_master_handler = new MHA::DBHelper();

      # args: hostname, port, user, password, raise_error_or_not
      $new_master_handler->connect( $new_master_ip, $new_master_port,
        $new_master_user, $new_master_password, 1 );

      ## Set read_only=0 on the new master
      $new_master_handler->disable_log_bin_local();
      print "Set read_only=0 on the new master.\n";
      $new_master_handler->disable_read_only();

      ## Creating an app user on the new master
      print "Creating app user on the new master..\n";
      FIXME_xxx_create_user( $new_master_handler->{dbh} );
      $new_master_handler->enable_log_bin_local();
      $new_master_handler->disconnect();

      ## Update master ip on the catalog database, etc
      ## 這裡要Mark起來
      ##FIXME_xxx;
      ####新增區塊 Start####
      &start_vip();
      ####新增區塊 End####
      $exit_code = 0;
    };
    if ($@) {
      warn $@;

      # If you want to continue failover, exit 10.
      exit $exit_code;
    }
    exit $exit_code;
  }
  elsif ( $command eq "status" ) {
    ####新增區塊 Start####
    print "Checking the status of the script: ssh -t $ssh_user\@$orig_master_host \"$ssh_start_vip\"\n";
    `ssh -t $ssh_user\@$orig_master_host \"$ssh_start_vip\"`;
    ####新增區塊 End####
    # do nothing
    exit 0;
  }
  else {
    &usage();
    exit 1;
  }
}

####新增區塊 Start####
# A simple system call that enable the VIP on the new master
sub start_vip() {
  `ssh $ssh_user\@$new_master_host \" $ssh_start_vip \"`;
}
# # A simple system call that disable the VIP on the old_master
sub stop_vip() {
  `ssh $ssh_user\@$orig_master_host \" $ssh_stop_vip \"`;
}
####新增區塊 End####

sub usage {
  print
"Usage: master_ip_failover --command=start|stop|stopssh|status --orig_master_host=host --orig_master_ip=ip --orig_master_port=port --new_master_host=host --new_master_ip=ip --new_master_port=port\n";
}

停止 Master 服務測試

# 使用 systemctl Stop Service
systemctl stop mariadb

# 或是||||||||||||||||||||||||||||

# 使用 mysqladmin 指定 Socket 停止
mysqladmin -u root -S /mysql/3306/socket/mysql.sock shutdown

在 failover 後，manager 會自動關閉 ( 不知是程式 bug 還是原作者就要這樣做 )，並留下檔名 /var/tmp/masterha_default.failover.complete 的空白記錄檔，若下次要再切 failover，這個檔案要記得刪除，不然會無法切換。

 

附註：

問題1: 

Thu May 28 22:45:12 2020 - [error][/usr/local/share/perl5/MHA/MasterFailover.pm, ln309] Last failover was done at 2020/05/28 22:04:37. Current time is too early to do failover again. If you want to do failover, manually remove /var/tmp/masterha_default.failover.complete and run this script again.

說明了已經有執行過一次 failover 處理，因為MHA每次執行過failover後，會在一個地方留下 xxxxxx.failover.complete ( xxxxxx是指您manager的設定檔名稱 )，這個檔案要刪除

 

問題2:

切換時，VIP沒有跟著轉過去，原因可能有幾個

1. 沒裝 net-tools，因為會使用 ifconfig 來綁定 IP

2. 沒有權限，內文是以 Linux root 帳號來執行切換，正式環境會用一般 mysql 使用者帳號，因此一般使用者帳號要有sudo權限，腳本也併加入 sudo

 

參考文章：

安装MHA报的错误

CentOS Local设置，消除告警：warning: Setting locale failed

精华总结：MHA构建中的各种报错与解决

MHA构建MySQL高可用平台最佳实践

MysqlMaster切换方案MHA的探索与测试结果




JohnsonJ

張貼者： JohnsonJ

張貼留言

0 留言