Thursday, December 30, 2010

How to recover and start a VXVM volume where the volume is DISABLED ACTIVE and has a plex that is DISABLED RECOVER

Problem

How to recover and start a Veritas Volume Manager logical volume where the volume is DISABLED ACTIVE and has a plex that is DISABLED RECOVER

Solution

When a system encounters a problem with a volume or a plex, or if Veritas Volume Manager (VxVM) has any reason to believe that the data is not synchronized, VxVM changes the kernel state, KSTATE and state, STATE, of the volume and its plexes accordingly. The plex state can be stale, empty, nodevice, etc. A particular plex state does not necessarily mean that the data is good or bad. The plex state is representative of VxVM's perception of the data in a plex.

The output from the vxprint utility using the switches "-h" and "-t" (for more information about these switches and all applicable switches, see the man page for vxprint) displays information from records in VxVM disk group configurations, including the KSTATE and STATE of a volume and plex as indicated in columns 4 and 5 respectively in the table below. When viewing the configuration records of a VxVM disk group using the vxprint utility and the KSTATE and STATE fields display DISABLED ACTIVE for the volume and DISABLED RECOVER  for the plex, recovery steps need to be followed to bring the volume back to an ENABLED ACTIVE state so it can be mounted and make the file system accessible again.

From the below output, it can be seen that the KSTATE and STATE for the volume test is DISABLED ACTIVE and its plex test-01 is DISABLED RECOVER.

# vxprint -ht -g testdg
 
DG NAME NCONFIG NLOG MINORS GROUP-ID    
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE  
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
V NAME RVG KSTATE STATE LENGTH USETYPE PREFPLEX RDPOL
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
               

                 
dg testdg default default 84000 970356463.1203.alu      
                 
dm testdg01 c1t4d0s2 sliced 2179 8920560 -    
dm testdg02 c1t6d0s2 sliced 2179 8920560 -    
                 
v test - DISABLED ACTIVE 17840128 fsgen - SELECT
pl test-01 test DISABLED RECOVER 17841120 CONCAT - RW
sd testdg01-01 test-01 testdg01 0 8920560 0 c1t4d0 ENA
sd testdg02-01 test-01 testdg02 0 8920560 8920560 c1t6d0 ENA
                 


Follow these steps to change KSTATE and STATE of a plex that is DISABLED RECOVER to ENABLED ACTIVE so the volume can be recovered / started and the file system mounted:

1. Change the plex test-01 to the DISABLED STALE state:
# vxmend -g  diskgroup fix stale <plex_name>

For example:
# vxmend -g testdg fix stale test-01

This output shows the plex test-01 as DISABLED STALE:
# vxprint -ht -g testdg
 
DG NAME NCONFIG NLOG MINORS GROUP-ID      
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE    
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL  
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK  
V NAME RVG KSTATE STATE LENGTH USETYPE PREFPLEX RDPOL
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
                 
dg testdg default default 84000 970356463.1203.alu      
                 
dm testdg01 c1t4d0s2 sliced 2179 8920560 -    
dm testdg02 c1t6d0s2 sliced 2179 8920560 -    
                 
v test - DISABLED ACTIVE 17840128 fsgen - SELECT
pl test-01 test DISABLED STALE 17841120 CONCAT - RW
sd testdg01-01 test-01 testdg01 0 8920560 0 c1t4d0 ENA
sd testdg02-01 test-01 testdg02 0 8920560 8920560 c1t6d0 ENA


2. Change the plex test-01 to the DISABLED CLEAN state:
# vxmend -g diskgroup fix clean <plex_name>

For example:
# vxmend -g testdg fix clean test-01

This output shows the plex test-01 as DISABLED CLEAN:
# vxprint -ht -g testdg
 
DG NAME NCONFIG NLOG MINORS GROUP-ID      
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE    
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL  
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK  
V NAME RVG KSTATE STATE LENGTH USETYPE PREFPLEX RDPOL
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
                 
dg testdg default default 84000 970356463.1203.alu      
                 
dm testdg01 c1t4d0s2 sliced 2179 8920560 -    
dm testdg02 c1t6d0s2 sliced 2179 8920560 -    
                 
v test - DISABLED ACTIVE 17840128 fsgen - SELECT
pl test-01 test DISABLED CLEAN 17841120 CONCAT - RW
sd testdg01-01 test-01 testdg01 0 8920560 0 c1t4d0 ENA
sd testdg02-01 test-01 testdg02 0 8920560 8920560 c1t6d0 ENA


3. Start the volume test:
# vxvol -g diskgroup start  <volume>

For example:
# vxvol -g diskgroup start test

This output shows that the volume test and its plex test-01 are both ENABLED ACTIVE:
# vxprint -ht -g testdg
 
DG NAME NCONFIG NLOG MINORS GROUP-ID      
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE    
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL  
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK  
V NAME RVG KSTATE STATE LENGTH USETYPE PREFPLEX RDPOL
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
                 
dg testdg default default 84000 970356463.1203.alu      
                 
dm testdg01 c1t4d0s2 sliced 2179 8920560 -    
dm testdg02 c1t6d0s2 sliced 2179 8920560 -    
                 
v test - ENABLED ACTIVE 17840128 fsgen - SELECT
pl test-01 test ENABLED ACTIVE 17841120 CONCAT - RW
sd testdg01-01 test-01 testdg01 0 8920560 0 c1t4d0 ENA
sd testdg02-01 test-01 testdg02 0 8920560 8920560 c1t6d0 ENA


4. Mount the volume to its associated mount point (refer to the /etc/vfstab file if the mount point location is not known) if the file system is a Veritas File System (VxFS) file system:
# mount -F vxfs /dev/vx/dsk/diskgroup/volume /mount point

For example:
# mount -F vxfs /dev/vx/dsk/testdg/test /testvol

Note: An error may be generated stating that the file system needs to be checked for consistency. If this occurs, run the VxFS specific fsck utility (/usr/lib/fs/vxfs/fsck) where the default is to replay the intent log, instead of performing a full structural file system check which is usually sufficient to set the file system to CLEAN and allow the volume to be mounted

No comments:

Post a Comment