阅读量:2
网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。
一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！
Adapter 0-VD 1: Cache Policy:WriteBack, ReadAheadNone, Direct
or
MegaCli -LDGetProp -Cache -LALL -aALL
Adapter 0-VD 0: Cache Policy:WriteBack, ReadAheadNone, Direct
Adapter 0-VD 1: Cache Policy:WriteBack, ReadAheadNone, Direct
or
MegaCli -LDGetProp -DskCache -LALL -aALL
Adapter 0-VD 0: Disk Write Cache : Disk’s Default
Adapter 0-VD 1: Disk Write Cache : Disk’s Default
**3、设置磁盘缓存策略** 缓存策略解释：
WT (Write through
WB (Write back)
NORA (No read ahead)
RA (Read ahead)
ADRA (Adaptive read ahead)
Cached
Direct
例子：
MegaCli -LDSetProp WT|WB|NORA|RA|ADRA -L0 -a0
or
MegaCli -LDSetProp -Cached|-Direct -L0 -a0
or
enable / disable disk cache
MegaCli -LDSetProp -EnDskCache|-DisDskCache -L0 -a0
**4、创建/删除 阵列** 4.1 创建一个 raid5 阵列，由物理盘 2,3,4 构成，该阵列的热备盘是物理盘 5
MegaCli -CfgLdAdd -r5 [1:2,1:3,1:4] WB Direct -Hsp[1:5] -a0
4.2 创建阵列，不指定热备
MegaCli -CfgLdAdd -r5 [1:2,1:3,1:4] WB Direct -a0
4.3 删除阵列
MegaCli -CfgLdDel -L1 -a0
4.4 在线添加磁盘
MegaCli -LDRecon -Start -r5 -Add -PhysDrv[1:4] -L1 -a0
意思是，重建逻辑磁盘组1，raid级别是5，添加物理磁盘号：1:4。重建完后，新添加的物理磁盘会自动处于重建(同步)状态，这个 时候 fdisk -l是看不到阵列的空间变大的，只有在系统重启后才能看见。
**5、查看阵列初始化信息** 5.1 阵列创建完后，会有一个初始化同步块的过程，可以看看其进度。 MegaCli -LDInit -ShowProg -LALL -aALL
或者以动态可视化文字界面显示 MegaCli -LDInit -ProgDsply -LALL -aALL
5.2 查看阵列后台初始化进度 MegaCli -LDBI -ShowProg -LALL -aALL
或者以动态可视化文字界面显示 MegaCli -LDBI -ProgDsply -LALL -aALL
6、创建全局热备 指定第 5 块盘作为全局热备 MegaCli -PDHSP -Set [-EnclAffinity] [-nonRevertible] -PhysDrv[1:5] -a0
也可以指定为某个阵列的专用热备 MegaCli -PDHSP -Set [-Dedicated [-Array1]] [-EnclAffinity] [-nonRevertible] -PhysDrv[1:5] -a0
7、删除全局热备 MegaCli -PDHSP -Rmv -PhysDrv[1:5] -a0
8、将某块物理盘下线/上线 MegaCli -PDOffline -PhysDrv [1:4] -a0
MegaCli -PDOnline -PhysDrv [1:4] -a0
9、查看物理磁盘重建进度 MegaCli -PDRbld -ShowProg -PhysDrv [1:5] -a0
**查看raid 电池状态** 查看当前raid缓存状态，raid缓存状态设置为wb的话要注意电池放电事宜，设置电池放电模式为自动学习模式
/opt/MegaRAID/MegaCli/MegaCli64 -ldgetprop -dskcache -lall -aall
raid 电池设置相关
查看电池状态信息(Display BBU Status Information)
MegaCli -AdpBbuCmd -GetBbuStatus -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuStatus -aALL
查看电池容量（Display BBU Capacity Information）
MegaCli -AdpBbuCmd -GetBbuCapacityInfo -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuCapacityInfo –aALL
查看电池设计参数(Display BBU Design Parameters)
MegaCli -AdpBbuCmd -GetBbuDesignInfo -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuDesignInfo –aALL
查看电池属性（Display Current BBU Properties）
MegaCli -AdpBbuCmd -GetBbuProperties -aN|-a0,1,2|-aALL
MegaCli -AdpBbuCmd -GetBbuProperties –aALL
设置电池为学习模式为循环模式（Start BBU Learning Cycle）
Description Starts the learning cycle on the BBU.
No parameter is needed for this option.
MegaCli -AdpBbuCmd -BbuLearn -aN|-a0,1,2|-aALL
设置磁盘的缓存模式和访问方式 （Change Virtual Disk Cache and Access Parameters）
Description Allows you to change the following virtual disk parameters:
-WT (Write through), WB (Write back): Selects write policy.
-NORA (No read ahead), RA (Read ahead), ADRA (Adaptive read ahead): Selects read policy.
-Cached, -Direct: Selects cache policy.
-RW, -RO, Blocked: Selects access policy.
-EnDskCache: Enables disk cache.
-DisDskCache: Disables disk cache.
MegaCli -LDSetProp { WT | WB|NORA |RA | ADRA|-Cached|Direct} |
{-RW|RO|Blocked} |
{-Name[string]} |
{-EnDskCache|DisDskCache} –Lx |
-L0,1,2|-Lall -aN|-a0,1,2|-aALL
MegaCli -LDSetProp WT -L0 -a0
显示磁盘缓存和访问方式（Display Virtual Disk Cache and Access Parameters）
MegaCli -LDGetProp -Cache | -Access | -Name | -DskCache -Lx|-L0,1,2|
-Lall -aN|-a0,1,2|-aALL
Displays the cache and access policies of the virtual disk(s):
-WT (Write through), WB (Write back): Selects write policy.
-NORA (No read ahead), RA (Read ahead), ADRA (Adaptive read ahead): Selects read policy.
-Cache, -Cached, Direct: Displays cache policy.
-Access, -RW, -RO, Blocked: Displays access policy.
-DskCache: Displays physical disk cache policy.
Megaraid 必知必会 使用LSI的megaraid可以对raid进行有效监控。别的厂商比如HP,IBM也有自己的raid API
MegaCli -ldinfo -lall -aall
查询raid级别，磁盘数量，容量，条带大小。
MegaCli -cfgdsply -aALL |grep Policy
查询控制器cache策略
MegaCli -LDSetProp WB -L0 -a0
设置write back功能
MegaCli -LDSetProp CachedBadBBU -L0 -a0
设置即使电池坏了还是保持WB功能
MegaCli -AdpBbuCmd -BbuLearn a0
手动充电
MegaCli -FwTermLog -Dsply -aALL
查询日志
显示适配器个数： MegaCli -adpCount
显示所有适配器信息： MegaCli -AdpAllInfo -aAll
Critical Disks : 0
Failed Disks : 0
显示所有逻辑磁盘组信息： MegaCli -LDInfo -LALL -aAll
显示所有的物理信息： MegaCli -PDList -aAll
Media Error Count: 0
Other Error Count: 0
查看充电状态： MegaCli -AdpBbuCmd -GetBbuStatus -aALL
Learn Cycle Requested : No
Fully Charged : Yes
显示BBU(后备电池)状态信息： MegaCli -AdpBbuCmd -GetBbuStatus -aALL
显示BBU容量信息： MegaCli -AdpBbuCmd -GetBbuCapacityInfo -aALL
显示BBU设计参数： MegaCli -AdpBbuCmd -GetBbuDesignInfo -aALL
显示当前BBU属性： MegaCli -AdpBbuCmd -GetBbuProperties -aALL
显示Raid卡型号，Raid设置，Disk相关信息： MegaCli -cfgdsply -aALL
查看Cache 策略设置： MegaCli -cfgdsply -aALL |grep -i Policy
Current Cache Policy: WriteBack, ReadAheadNone, Direct, Write Cache OK if Bad BBU
查看充电进度百分比： MegaCli -AdpBbuCmd -GetBbuStatus -aALL
各种设备和磁盘的不同状态：
Device |Normal|Damage|Rebuild|Normal
Virtual Drive |Optimal|Degraded|Degraded|Optimal
Physical Drive |Online|Failed –> Unconfigured|Rebuild|Online
通过脚本检测RAID 磁盘状态 Linux下脚本 #!/bin/bash
#check raid disk status
MEGACLI="/opt/MegaRAID/MegaCli/MegaCli64 "
$MEGACLI -pdlist -aALL | grep “Firmware state” | awk -F : ‘{print $2}’ | awk -F , ‘{print $1}’ >/tmp/fireware.log
$MEGACLI -pdlist -aALL | grep -E “Media Error|Other Error” | awk -F : ‘{print $2}’ >/tmp/disk.log
for i in cat < /tmp/disk.log
do
if [ $i -ne 0 ]
then
curl “http://xxxxxxB&state=ALARM&description=raid_disk_error”
fi
done
for i in cat < /tmp/fireware.log
do
if [ $i != Online ]
then
curl “http://xxxxxxstate=ALARM&description=raid_disk_offline”
fi
done
**CentOS安装megacli** yum install megacli -y
查看Raid卡信息
megacli -LDInfo -Lall -aALL
查看硬盘信息：
megacli -PDList -aALL
**查看Raid卡信息** [root@ robin]# megacli -LDInfo -Lall -aALL
OSSpecificInitialize: Failed to load libsysfs.so.2.0.2 Please ensure that libsfs is present in the system.
The dependent library libsysfs.so.2.0.1 not available. Please contact LSI for distribution of the package
Adapter 0 – Virtual Drive Information:
Virtual Drive: 0 (Target Id: 0)
Name :
RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0
Size : 930.5 GB
Mirror Data : 930.5 GB
State : Optimal <<<< 这是正常的状态。
Strip Size : 64 KB
Number Of Drives per span:2 <<<< RAID组包含2块磁盘（但实际是4块500G的SSD做Raid10）。
Span Depth : 2 <<<< 深度若是1，说明实际只是RAID 1。2表示可以是RAID 10
Default Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAdaptive, Direct, No Write Cache if Bad BBU
Default Access Policy: Read/Write
Current Access Policy: Read/Write
Disk Cache Policy : Disk’s Default
Ongoing Progresses:
Background Initialization: Completed 41%, Taken 190 min.
Encryption Type : None
Default Power Savings Policy: Controller Defined
Current Power Savings Policy: None
Can spin up in 1 minute: No
LD has drives that support T10 power conditions: No
LD’s IO profile supports MAX power savings with cached writes: No
Bad Blocks Exist: No
Is VD Cached: No
 **查看硬盘信息：** [root@localhost ~]# megacli -PDList -aALL
###有如下样式的几段，说明实际有几块硬盘（下面是一块500G的SSD的信息）
Enclosure Device ID: 32
Slot Number: 3
Drive’s postion: DiskGroup: 0, Span: 1, Arm: 1
Enclosure position: 0
Device Id: 3
WWN: 50025388A0061518
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SATA
Raw Size: 465.761 GB [0x3a386030 Sectors]
Non Coerced Size: 465.261 GB [0x3a286030 Sectors]
Coerced Size: 465.25 GB [0x3a280000 Sectors]
Firmware state: Online, Spun Up
Device Firmware Level: BB0Q
Shield Counter: 0
Successful diagnostics completion on : N/A
SAS Address(0): 0x4433221107000000
Connected Port Number: 3(path0)
Inquiry Data: S1DHNSADA04008B Samsung SSD 840 EVO 500GB EXT0BB0Q
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Solid State Device
Drive: Not Certified
Drive Temperature :32C (89.60 F)
PI Eligibility: No
Drive is formatted for PI information: No
PI: No PI
Drive’s write cache : Disabled
Drive’s NCQ setting : Disabled
Port-0 :
Port status: Active
Port’s Linkspeed: 6.0Gb/s
Drive has flagged a S.M.A.R.T alert : No
备注： **RAID型号对应表信息如下：** RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0" #代表Raid 1
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0" #代表Raid 0
RAID Level : Primary-5, Secondary-0, RAID Level Qualifier-3" #代表Raid 5
RAID Level : Primary-1, Secondary-3, RAID Level Qualifier-0" #代表Raid10
命令行模式举例如下: lsiutil –p1 –a 21,1,0,0,0 和上边菜单操作一样，只不过不用手动去选择菜单项了
lsiutil -p 1 -i –s 显示端口1的设置和连接的设备
lsiutil –s 显示所有端口上连接的设备
lsiutil -p 2 -l 1 强制HBA卡端口2连接速度为1Gb
### 案例 ： **1、MegaCli操作1—创建与删除raid0** 服务器DELL R710加硬盘，准备在线做raid。 用MegaCli做阵列，最重要是就是这三个系数： Adapter #0 Enclosure Device ID: 32 Slot Number: 0 分别代表了 -PhysDrv[E0:S0,E1:S1,…] -aN #物理磁盘的 enclosure号:slot号 -a是阵列卡号 这些参数可以通过下面的命令得到。 查看阵列卡信息 lspci -v
/opt/MegaRAID/MegaCli/MegaCli64 -AdpAllInfo -aALL
查看所有物理硬盘状态 /opt/MegaRAID/MegaCli/MegaCli64 -PDList -a0
/opt/MegaRAID/MegaCli/MegaCli64 help |grep -i pdinfo
MegaCli -pdInfo -PhysDrv[E0:S0,E1:S1,…] -aN|-a0,1,2|-aALL
MegaCli -LdPdInfo -aN|-a0,1,2|-aALL
查看逻辑盘详细信息 /opt/MegaRAID/MegaCli/MegaCli64 -LdPdInfo -aALL
查看单个盘的详细信息 /opt/MegaRAID/MegaCli/MegaCli64 -pdInfo -PhysDrv[32:3] -aALL
Adapter 0: Device at Enclosure - 32, Slot - 3 is not found.
像我新增加的硬盘就是 [32:2]   这里我的硬盘被标记为“Foreign”外来配置，要清除一下，不然会报下面这个错误： The specified physical disk does not have the appropriate attributes to complete
the requested command.
Exit Code: 0x26
扫描外来配置的个数： /opt/MegaRAID/MegaCli/MegaCli64 -cfgforeign -scan -a0
There are 1 foreign configuration(s) on controller 0.
Exit Code: 0x00
清除外来配置： /opt/MegaRAID/MegaCli/MegaCli64 -cfgforeign -clear -a0
Foreign configuration 0 is cleared on controller 0.
Exit Code: 0x00
再次扫描外来配置的个数： /opt/MegaRAID/MegaCli/MegaCli64 -cfgforeign -scan -a0
There is no foreign configuration on controller 0.
Exit Code: 0x00
创建Raid0 /opt/MegaRAID/MegaCli/MegaCli64 -CfgLdAdd -r0[32:2] WB Direct -a0
Adapter 0: Created VD 2
Adapter 0: Configured the Adapter!!
Exit Code: 0x00
 删除阵列： /opt/MegaRAID/MegaCli/MegaCli6 -cfgclr -a0 清除所有的raid组的配置
/opt/MegaRAID/MegaCli/MegaCli6 -cfglddel -L0 -a0 删除指定的raid组(Target Id: 0)的raid组，可以通过上面的“查看逻辑盘详细信息”得到。
查看硬盘的重建情况： /opt/MegaRAID/MegaCli/MegaCli64 -PDRbld -ProgDsply -PhysDrv [12:7] -aALL
Device(Encl-12 Slot-7) is not in rebuild process
**2、MegaCli使用方法** 查看磁盘使用情况 [root@ ~]# MegaCli -PDList -Aall
Enclosure Device ID: 32
Slot Number: 0
Enclosure position: 0
Device Id: 0
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Online, Spun Up 在线
Enclosure Device ID: 32
Slot Number: 1
Enclosure position: 0
Device Id: 1
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Online, Spun Up
SAS Address(0): 0x5000c500586e6559
Enclosure Device ID: 32
Slot Number: 2
Enclosure position: 0
Device Id: 2
Sequence Number: 3
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 1.819 TB [0xe8e088b0 Sectors]
Non Coerced Size: 1.818 TB [0xe8d088b0 Sectors]
Coerced Size: 1.818 TB [0xe8d00000 Sectors]
Firmware state: Unconfigured(good), Spun Up 安装了磁盘，但是没有使用
Enclosure Device ID: 32
Slot Number: 3
Enclosure position: 0
Device Id: 3
Sequence Number: 3
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 1.819 TB [0xe8e088b0 Sectors]
Non Coerced Size: 1.818 TB [0xe8d088b0 Sectors]
Coerced Size: 1.818 TB [0xe8d00000 Sectors]
Firmware state: Unconfigured(good), Spun Up 安装了磁盘，但是没有使用
2块2T磁盘使用MegaCli做raid0 [root@ZAMPDMP-08 ~]# MegaCli -CfgLdAdd -r0 [32:2,32:3] WB Direct -a0
Adapter 0: Created VD 1
Adapter 0: Configured the Adapter!!
Exit Code: 0x00
单块4T磁盘做raid0 对7号槽的单块4T硬盘做raid0 MegaCli -CfgLdAdd -r0 [32:6] WB Direct -a0
单块盘要么调整raid 为jbod模式 要么就做成raid0 才能作为单块盘使用 无实际意义 只是raid卡的限制 查看raid [root@ZAMPDMP-08 ~]# MegaCli -LdPdInfo -aALL Adapter #0 Number of Virtual Disks: 2 Virtual Drive: 0 (Target Id: 0) 第一个raid，包括slot 0,1，对应插槽0和1，级别raid0 Name :
RAID Level : Primary-1, Secondary-0, RAID Level Qualifier-0
Size : 931.0 GB
State : Optimal
Strip Size : 64 KB
Number Of Drives : 2
Span Depth : 1
Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Access Policy : Read/Write
Disk Cache Policy : Disk’s Default
Encryption Type : None
Default Power Savings Policy: Controller Defined
Current Power Savings Policy: None
Can spin up in 1 minute: Yes
LD has drives that support T10 power conditions: Yes
LD’s IO profile supports MAX power savings with cached writes: No
Bad Blocks Exist: No
Number of Spans: 1
Span: 0 - Number of PDs: 2
PD: 0 Information
Enclosure Device ID: 32
Slot Number: 0
Enclosure position: 0
Device Id: 0
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Online, Spun Up
SAS Address(0): 0x5000c500586e2169
SAS Address(1): 0x0
Connected Port Number: 0(path0)
Inquiry Data: SEAGATE ST1000NM0023 GS0DZ1W1SRX0
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :37C (98.60 F)
PD: 1 Information
Enclosure Device ID: 32
Slot Number: 1
Enclosure position: 0
Device Id: 1
Sequence Number: 2
Media Error Count: 0
Other Error Count: 0
Predictive Failure Count: 0
Last Predictive Failure Event Seq Number: 0
PD Type: SAS
Raw Size: 931.512 GB [0x74706db0 Sectors]
Non Coerced Size: 931.012 GB [0x74606db0 Sectors]
Coerced Size: 931.0 GB [0x74600000 Sectors]
Firmware state: Online, Spun Up
SAS Address(0): 0x5000c500586e6559
SAS Address(1): 0x0
Connected Port Number: 1(path0)
Inquiry Data: SEAGATE ST1000NM0023 GS0DZ1W1SQS1
FDE Capable: Not Capable
FDE Enable: Disable
Secured: Unsecured
Locked: Unlocked
Needs EKM Attention: No
Foreign State: None
Device Speed: 6.0Gb/s
Link Speed: 6.0Gb/s
Media Type: Hard Disk Device
Drive Temperature :38C (100.40 F)
Virtual Drive: 1 (Target Id: 1) 第二个raid，包括slot 2,3，对应插槽2和4、3，级别raid0 Name :
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0
Size : 3.637 TB
State : Optimal
Strip Size : 64 KB
Number Of Drives : 2
Span Depth : 1
Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Access Policy : Read/Write
网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。
一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！
Media Type: Hard Disk Device
Drive Temperature :38C (100.40 F)
Virtual Drive: 1 (Target Id: 1) 第二个raid，包括slot 2,3，对应插槽2和4、3，级别raid0 Name :
RAID Level : Primary-0, Secondary-0, RAID Level Qualifier-0
Size : 3.637 TB
State : Optimal
Strip Size : 64 KB
Number Of Drives : 2
Span Depth : 1
Default Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteThrough, ReadAheadNone, Direct, No Write Cache if Bad BBU
Access Policy : Read/Write
[外链图片转存中…(img-ImZL1Sxe-1714989520992)]
[外链图片转存中…(img-3OVNnMYK-1714989520992)]
网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。
一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！
