mysql异常退出定位及解决记录

现象

运行正常的程序，早上出现了异常。

查看日志报错，是读写数据时失败。这个时候需要检查一下数据库是否正常。

先在终端里用mysql客户端连接一下吧。

mysql -uuser -p

输入密码，果然连接失败。

再查看mysql服务的状态，发现已经挂了：

# systemctl status mysqld.service
● mysqld.service - MySQL Server
   Loaded: loaded (/usr/lib/systemd/system/mysqld.service; enabled; vendor preset: disabled)
   Active: failed (Result: start-limit) since Thu 2020-07-23 05:05:17 CST; 4h 5min ago
     Docs: man:mysqld(8)
           http://dev.mysql.com/doc/refman/en/using-systemd.html
  Process: 3912 ExecStart=/usr/sbin/mysqld --daemonize --pid-file=/var/run/mysqld/mysqld.pid $MYSQLD_OPTS (code=exited, status=1/FAILURE)
  Process: 3894 ExecStartPre=/usr/bin/mysqld_pre_systemd (code=exited, status=0/SUCCESS)
 Main PID: 5668 (code=exited, status=2)

Jul 23 05:05:17 localhost.localdomain systemd[1]: Failed to start MySQL Server.
Jul 23 05:05:17 localhost.localdomain systemd[1]: Unit mysqld.service entered failed state.
Jul 23 05:05:17 localhost.localdomain systemd[1]: mysqld.service failed.
Jul 23 05:05:17 localhost.localdomain systemd[1]: mysqld.service holdoff time over, scheduling restart.
Jul 23 05:05:17 localhost.localdomain systemd[1]: Stopped MySQL Server.
Jul 23 05:05:17 localhost.localdomain systemd[1]: start request repeated too quickly for mysqld.service
Jul 23 05:05:17 localhost.localdomain systemd[1]: Failed to start MySQL Server.
Jul 23 05:05:17 localhost.localdomain systemd[1]: Unit mysqld.service entered failed state.
Jul 23 05:05:17 localhost.localdomain systemd[1]: mysqld.service failed.

定位

要查看一下mysql为什么挂了。需要查看日志。

日志在哪里？通过配置文件查看：cat /etc/my.cnf

# cat /etc/my.cnf
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.7/en/server-configuration-defaults.html

[mysqld]
#
# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
# innodb_buffer_pool_size = 128M
#
# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin
#
# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
# join_buffer_size = 128M
# sort_buffer_size = 2M
# read_rnd_buffer_size = 2M
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock

# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0

log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

错误日志位于：/var/log/mysqld.log。

vi打开后，直接跳到最后，发现如下：

很明显，是磁盘满了。

解决

查看一下系统磁盘空间使用情况：

% df -h 
Filesystem               Size  Used Avail Use% Mounted on
/dev/mapper/centos-root   50G   50G   20K 100% /

果然没空间了。但是空间都被谁吃了？

# du -h / --max-depth=1
# 查看根目录下各个文件夹占用磁盘大小
# max-depth控制目录层级

找到占用空间大且可以删除的文件。一般可能是日志文件。

进入到目录，删除目录下所有log文件：

find . -type f -name *.log -exec rm {} ;

解释一下：查找当前目录下所有的类型为普通文档的且以log结尾的文件，并删除之。

注意{}和之间有空格。

ok了。再查看一下空间状态。

此时可以重启mysql了：systemctl start mysqld.service

问题解决。