1、1-11 LVM 实战1.1 现场环境1. 久远的案例回顾当时操作系统是 RHEL4U8，存储为 DELL MD3000，数据库 oracle10g，数据库文件存放在存储上。并通过 ROSEHA 做集群。Linux 服务器通过 iscsi 单链路的方式挂载存储。原来 DELL MD3000 上有六个旧盘做了 raid5，由于没有删掉 raid5 就拔了几块旧盘，并在那几个位置上插了新盘，此时 raid5 开始恢复数据，造成 raid 模块记下了那几块新盘。导致最终 15块新盘全插上后，那几块盘都是一直亮黄灯。2. DELL 400 热线求助最终想到的办法是初始化 MD3000，在插完硬盘到 m
2、d3000 上后，执行如下操作：在存储管理软件安装的电脑上，确认了管理软件的安装目录后，在 CMD 命令行中输入命令C:Program FilesDellMD Storage Managerclientsmcli.exe 192.168.128.101 -c “clear storagearray configuration;“Performing syntax check.Syntax check complete.Executing script.Script execution complete.运行完后，黄灯都变成了绿灯，阵列框上的红灯也变成了紫灯，阵列正常。 命令说明：smcli.e
3、xe 192.168.128.101 -p 123456 -c “clear storagearray configuration;“ smcli.exe 是工具运行文件IP 地址要输管理软件连接 MD3000 时，带外管理的 IP 地址-p 密码，默认为没有密码。如果没有密码则不要此参数-c 所需要执行的操作 “clear storagearray configuration;“ 清除存储阵列配置如果 MD3000 认不出硬盘，也可以通过上述方式操作。1.2 查看两主机名及磁盘挂载情况1. 查看主机名Hostname2. 查看磁盘使用情况df h1-23. 查看磁盘挂载信息。检查/etc/f
4、stab4. 查看分区表信息。rootpacsserver1 # fdisk -lDisk /dev/sda: 299.4 GB, 299439751168 bytes255 heads, 63 sectors/track, 36404 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDevice Boot Start End Blocks Id System/dev/sda1 * 1 13 104391 83 Linux/dev/sda2 14 35360 283924777+ 83 Linux/dev/sda3 35361
5、36404 8385930 82 Linux swapDisk /dev/sdb: 2186.1 GB, 2186136256512 bytes255 heads, 63 sectors/track, 265782 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDisk /dev/sdb doesnt contain a valid partition tableDisk /dev/sdc: 2186.1 GB, 2186136256512 bytes255 heads, 63 sectors/track, 265782 cy
6、lindersUnits = cylinders of 16065 * 512 = 8225280 bytesDisk /dev/sdc doesnt contain a valid partition tableDisk /dev/sdd: 2186.1 GB, 2186136256512 bytes255 heads, 63 sectors/track, 265782 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDisk /dev/sdd doesnt contain a valid partition tableDis
7、k /dev/sde: 2186.1 GB, 2186136256512 bytes255 heads, 63 sectors/track, 265782 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesDisk /dev/sde doesnt contain a valid partition tableDisk /dev/sdf: 2186.1 GB, 2186136256512 bytes255 heads, 63 sectors/track, 265782 cylinders1-3Units = cylinders of
8、 16065 * 512 = 8225280 bytesDisk /dev/sdf doesnt contain a valid partition tableDisk /dev/sdg: 2040.1 GB, 2040109465600 bytes255 heads, 63 sectors/track, 248029 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytes1.3 创建 PVrootpacsserver1 # pvcreate /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /
9、dev/sdgPhysical volume “/dev/sdb“ successfully createdPhysical volume “/dev/sdc“ successfully createdPhysical volume “/dev/sdd“ successfully createdPhysical volume “/dev/sde“ successfully createdPhysical volume “/dev/sdf“ successfully createdPhysical volume “/dev/sdg“ successfully created1.4 创建 VG1.
10、 创建 VGvgcreate HHPACS /dev/sdb /dev/sdc /dev/sdd /dev/sde /dev/sdf /dev/sdgVolume group “HHPACS“ successfully created2. 查看 VGrootpacsserver1 # vgdisplay- Volume group -VG Name HHPACSSystem IDFormat lvm2Metadata Areas 6Metadata Sequence No 1VG Access read/writeVG Status resizableMAX LV 0Cur LV 0Open
11、LV 0Max PV 0Cur PV 6Act PV 6VG Size 11.80 TB1-4PE Size 4.00 MBTotal PE 3092474Alloc PE / Size 0 / 0Free PE / Size 3092474 / 11.80 TBVG UUID L3ey33-Vj1D-lcJr-RFKK-kFZF-EgMo-mPmCN21.5 创建 LV1. 创建 LVvgdisplay HHPACS | grep “Total PE“lvcreate -l 3092474 HHPACS -n pacsstoreLogical volume “pacsstore“ creat
12、ed2. 查看 LVlvdisplayrootpacsserver1 # lvdisplay- Logical volume -LV Name /dev/HHPACS/pacsstoreVG Name HHPACSLV UUID z5fZIQ-YLXv-pkju-EEPE-Eqtl-itkA-hru7cGLV Write Access read/writeLV Status available# open 0LV Size 11.80 TBCurrent LE 3092474Segments 6Allocation inheritRead ahead sectors 0Block device
13、 253:01.6 创建文件系统mke2fs -j /dev/HHPACS/pacsstore(PACS LV 的路径）注意：大于 8TB 的可以通过以上命令创建，或者通过 parted 方式创建。rootpacsserver1 # mke2fs -j /dev/HHPACS/pacsstoremke2fs 1.35 (28-Feb-2004)mke2fs: Filesystem too large. No more than 2*31-1 blocks(8TB using a blocksize of 4k) are currently supported.rootpacsserver1 #
14、 mkfs.ext3 /dev/HHPACS/pacsstore1-5mke2fs 1.35 (28-Feb-2004)mkfs.ext3: Filesystem too large. No more than 2*31-1 blocks(8TB using a blocksize of 4k) are currently supported.rootpacsserver1 # mkfs.ext3 -b 8192 /dev/HHPACS/pacsstoreWarning: blocksize 8192 not usable on most systems.mke2fs 1.35 (28-Feb
15、-2004)mkfs.ext3: 8192-byte blocks too big for system (max 4096)Proceed anyway? (y,n) yWarning: 8192-byte blocks too big for system (max 4096), forced to continueFilesystem label=OS type: LinuxBlock size=8192 (log=3)Fragment size=8192 (log=3)1581999936 inodes, 1583346688 blocks36217661 blocks (2.29%)
16、 reserved for the super userFirst data block=0Maximum filesystem blocks=158944665624163 block groups65528 blocks per group, 65528 fragments per group65472 inodes per groupSuperblock backups stored on blocks:65528, 196584, 327640, 458696, 589752, 1638200, 1769256, 3210872,5307768, 8191000, 15923304,
17、22476104, 40955000, 47769912, 143309736,157332728, 204775000, 429929208, 1023875000, 1101329096, 1289787624Writing inode tables: doneCreating journal (8192 blocks): doneWriting superblocks and filesystem accounting information: doneThis filesystem will be automatically checked every 28 mounts or180
18、days, whichever comes first. Use tune2fs -c or -i to override.1.7 删除 LVlvremove /dev/HHPACS/pacsstoreDo you really want to remove active logical volume “pacsstore“? y/n: yLogical volume “pacsstore“ successfully removed1-611T 的文件系统比较大，如果一块盘出现故障，恢复数据比较慢。所以建议做多个逻辑卷，之后再工作软链接的方式协调。1.8 创建多个 LVrootpacsserv
19、er1 # lvcreate -i4 -L 7.96T HHPACS -n pacsstoreUsing default stripesize 64.00 KBRounding up size to full physical extent 7.96 TBRounding size (2086667 extents) up to stripe boundary size (2086668 extents)Insufficient suitable allocatable extents for logical volume pacsstore: 1808 more requiredrootpa
20、csserver1 # lvcreate -i4 -L 7.95T HHPACS -n pacsstoreUsing default stripesize 64.00 KBRounding up size to full physical extent 7.95 TBRounding size (2084045 extents) up to stripe boundary size (2084048 extents)Logical volume “pacsstore“ createdrootpacsserver1 # lvcreate -L 3.84T HHPACS -n pacsstore1
21、Rounding up size to full physical extent 3.84 TBLogical volume “pacsstore1“ createdrootpacsserver1 # lvdisplayrootpacsserver1 # lvdisplay- Logical volume -LV Name /dev/HHPACS/pacsstoreVG Name HHPACSLV UUID dlj5RG-m5hM-Rzul-7Cei-ZFkp-QDdD-XEyhJ3LV Write Access read/writeLV Status available# open 0LV
22、Size 7.95 TBCurrent LE 2084048Segments 1Allocation inheritRead ahead sectors 0Block device 253:0- Logical volume -LV Name /dev/HHPACS/pacsstore1VG Name HHPACSLV UUID QV4SxH-u08k-oCBp-SlNi-yDAG-zQVr-HJGtclLV Write Access read/write1-7LV Status available# open 0LV Size 3.84 TBCurrent LE 1006633Segment
23、s 2Allocation inheritRead ahead sectors 0Block device 253:1rootpacsserver1 # vgdisplay- Volume group -VG Name HHPACSSystem IDFormat lvm2Metadata Areas 6Metadata Sequence No 5VG Access read/writeVG Status resizableMAX LV 0Cur LV 2Open LV 0Max PV 0Cur PV 6Act PV 6VG Size 11.80 TBPE Size 4.00 MBTotal P
24、E 3092474Alloc PE / Size 3090681 / 11.79 TBFree PE / Size 1793 / 7.00 GBVG UUID L3ey33-Vj1D-lcJr-RFKK-kFZF-EgMo-mPmCN21.9 挂载 LVmount /dev/HHPACS/pacsstore /data如果想让他开机自动挂载，则可以写到/etc/fstab 或者写到 /etc/rc.local 中。注意：如果数据库文件也放到 LV 上，及/dev/HHPACS/pacsstore 中，则需要添加到/etc/fstab 中，因为 Linux 启动先读 /etc/fstab，再启动
25、数据库，最后才读/etc/rc.local。1-81.10 卸载 LV有的时候进程或者用户在使用 LV，我们用 umount /dev/HHPACS/pacsstore，无法卸载。我们可以通过：1. fuser /dev/HHPACS/pacsstore ，查看哪个进程在使用。结束该进程。最暴力的方式 kill -9 PID2. umount l /dev/HHPACS/pacsstore命令 fuser 来完成1：# fuser cu mount_point 该命令结果显示文件系统的名称和拥有该进程的用户名2：# fuser ck mount_point 该命令杀掉所有访问该文件系统的进程3：# fuser c mount_point 该命令确认没有进程访问该文件系统4：# umount mount_point 1.11 故障解决1. 激活卷组当发生意外断电或者切换交换机时，可能发生 lv 找不到的现象。vgsan 和 vgchange ay 。其中 a 表示 available，有 y 和 n 可选，n 表示不激活卷组。2. 有问题找男人（ man）man mountman lvcreate
