Monday, December 04, 2006
Fun with fsdb
If you've been around long enough, you'll eventually see something weird like the following:
What's happened is that the filesystem mounted at /fs/proj is mounted on a directory with insufficient permissions. But how can you see the current permissions are? If you do an 'ls -li', you'll note that the inode number of the underlying directory is listed:
But how can you look at the information contained in that underlying inode? If you try to stat the file, you'll end up with information about the root inode of the filesystem mounted there. If you could make a hard link to the inode, you could access it via that link, but you can't make a hard link to a directory. You can use fsdb, however, to look directly at the filesystem in question:
Inside fsdb, you can move around the filesystem and get some information aboutwhat's there. But because fsdb is going directly to the disk and not going through the OS, it has no information about what's mounted there (and resolving a path like /fs/proj won't ever cross filesystem boundaries.) Below, we set the current inode and then get information about it:
And we can see that the permissions on the file are very restrictive. We could unmount the filesystem, change the permissions, and remount, or we could keep playing with fsdb. (Note that this likely isn't the kind of thing you'd want to do on a production server if you like your job. Note also that you need the 'w' option to be able to write to the device.):
juser@server:/fs> ls -l total 4 drwxr-xr-x 3 root root 512 Dec 4 10:17 proj/ drwxr-xr-x 4 root root 512 Nov 29 12:34 repl/ juser@server:/fs> cd proj juserserver:/fs/proj> cd .. ..: Permission denied. juser@server:/fs/proj>
What's happened is that the filesystem mounted at /fs/proj is mounted on a directory with insufficient permissions. But how can you see the current permissions are? If you do an 'ls -li', you'll note that the inode number of the underlying directory is listed:
juser@server> ls -li total 4 6 drwxr-xr-x 3 root root 512 Dec 4 10:17 proj/ 7 drwxr-xr-x 4 root root 512 Nov 29 12:34 repl/ juser@server>
But how can you look at the information contained in that underlying inode? If you try to stat the file, you'll end up with information about the root inode of the filesystem mounted there. If you could make a hard link to the inode, you could access it via that link, but you can't make a hard link to a directory. You can use fsdb, however, to look directly at the filesystem in question:
juser@server> sudo fsdb /dev/rdsk/c0t0d0s0 fsdb of /dev/rdsk/c0t0d0s0 (Read only) -- last mounted on / fs_clean is currently set to FSLOG fs_state consistent (fs_clean CAN be trusted) /dev/rdsk/c0t0d0s0 > :cd /fs /dev/rdsk/c0t0d0s0 > :ls -l /fs: i#: 5 ./ i#: 2 ../ i#: 33371 .rsync/ i#: 16207 .rsync_root i#: 6 proj/ i#: 7 repl/ /dev/rdsk/c0t0d0s0 >
Inside fsdb, you can move around the filesystem and get some information aboutwhat's there. But because fsdb is going directly to the disk and not going through the OS, it has no information about what's mounted there (and resolving a path like /fs/proj won't ever cross filesystem boundaries.) Below, we set the current inode and then get information about it:
/dev/rdsk/c0t0d0s0 > 6:inode /dev/rdsk/c0t0d0s0 > ?i i#: 6 md: d---rwx------ uid: 0 gid: 0 ln: 3 bs: 2 sz : c_flags : 0 200 db#0: 2fd accessed: Mon Dec 4 11:32:30 2006 modified: Mon Dec 4 10:13:13 2006 created : Mon Dec 4 11:32:59 2006 /dev/rdsk/c0t0d0s0 >
And we can see that the permissions on the file are very restrictive. We could unmount the filesystem, change the permissions, and remount, or we could keep playing with fsdb. (Note that this likely isn't the kind of thing you'd want to do on a production server if you like your job. Note also that you need the 'w' option to be able to write to the device.):
juser@server> sudo fsdb -o w /dev/rdsk/c0t0d0s0 fsdb of /dev/rdsk/c0t0d0s0 (Opened for write) -- last mounted on / fs_clean is currently set to FSLOG fs_state consistent (fs_clean CAN be trusted) /dev/rdsk/c0t0d0s0 > 6:inode /dev/rdsk/c0t0d0s0 > ?i i#: 6 md: d---rwx------ uid: 0 gid: 0 ln: 3 bs: 2 sz : c_flags : 0 200 db#0: 2fd accessed: Mon Dec 4 11:32:30 2006 modified: Mon Dec 4 10:13:13 2006 created : Mon Dec 4 11:32:59 2006 /dev/rdsk/c0t0d0s0 > :md=+055 i#: 6 md: d---rwxr-xr-x uid: 0 gid: 0 ln: 3 bs: 2 sz : c_flags : 0 200 db#0: 2fd accessed: Mon Dec 4 11:32:30 2006 modified: Mon Dec 4 10:13:13 2006 created : Mon Dec 4 11:32:59 2006 /dev/rdsk/c0t0d0s0 >