Global File System

When creating a clustered GFS filesystem, you can use either of the following formats:

gfs_mkfs -p LockProtoName -t LockTableName -j NumberJournals BlockDevice

mkfs -t gfs -p LockProtoName -t LockTableName -j NumberJournals BlockDevice

When creating a local filesystem, you can use either of the following formats:

gfs_mkfs -p LockProtoName -j NumberJournals BlockDevice

mkfs -t gfs -p LockProtoName -j NumberJournals BlockDevice

In these examples, lock_dlm is the locking protocol that the file system uses, since this is a clustered file system. The cluster name is alpha, and the file system name is mydata1. The file system contains eight journals and is created on /dev/vg01/lvol0.

[root@ask-07 ~]#  gfs_mkfs -p lock_dlm -t alpha:mydata1 -j 8 /dev/vg01/lvol0
This will destroy any data on /dev/vg01/lvol0.

Are you sure you want to proceed? [y/n] y

Device:                    /dev/vg01/lvol0
Blocksize:                 4096
Filesystem Size:           136380192
Journals:                  8
Resource Groups:           2082
Locking Protocol:          lock_dlm
Lock Table:                alpha:mydata1

Syncing...
All Done

[root@ask-07 ~]#  mkfs -t gfs -p lock_dlm -t alpha:mydata1 -j 8 /dev/vg01/lvol0
This will destroy any data on /dev/vg01/lvol0.

Are you sure you want to proceed? [y/n] y

Device:                    /dev/vg01/lvol0
Blocksize:                 4096
Filesystem Size:           136380192
Journals:                  8
Resource Groups:           2082
Locking Protocol:          lock_dlm
Lock Table:                alpha:mydata1

Syncing...
All Done

In these examples, a second lock_dlm file system is made, which can be used in cluster alpha. The file system name is mydata2. The file system contains eight journals and is created on /dev/vg01/lvol1.

gfs_mkfs -p lock_dlm -t alpha:mydata2 -j 8 /dev/vg01/lvol1

mkfs -t gfs -p lock_dlm -t alpha:mydata2 -j 8 /dev/vg01/lvol1

Table 3.1, “Command Options: gfs_mkfs” describes the gfs_mkfs command options.

Mounting Without ACL Manipulation

mount BlockDevice MountPoint

Mounting With ACL Manipulation

mount -o acl BlockDevice MountPoint

In this example, the GFS file system on /dev/vg01/lvol0 is mounted on the /mydata1 directory.

mount /dev/vg01/lvol0 /mydata1

mount BlockDevice MountPoint -o option

The -o option argument consists of GFS-specific options (refer to Table 3.2, “GFS-Specific Mount Options”) or acceptable standard Linux mount -o options, or a combination of both. Multiple option parameters are separated by a comma and no spaces.

Table 3.2, “GFS-Specific Mount Options” describes the available GFS-specific -o option values that can be passed to GFS at mount time.

umount MountPoint

Display Tunable Parameters

gfs_tool gettune MountPoint

In this example, all GFS tunable parameters for the file system on the mount point /mnt/gfs are displayed.

[root@tng3-1]# gfs_tool gettune /mnt/gfs
ilimit1 = 100
ilimit1_tries = 3
ilimit1_min = 1
ilimit2 = 500
ilimit2_tries = 10
ilimit2_min = 3
demote_secs = 300
incore_log_blocks = 1024
jindex_refresh_secs = 60
depend_secs = 60
scand_secs = 5
recoverd_secs = 60
logd_secs = 1
quotad_secs = 5
inoded_secs = 15
glock_purge = 0
quota_simul_sync = 64
quota_warn_period = 10
atime_quantum = 3600
quota_quantum = 60
quota_scale = 1.0000   (1, 1)
quota_enforce = 1
quota_account = 1
new_files_jdata = 0
new_files_directio = 0
max_atomic_write = 4194304
max_readahead = 262144
lockdump_size = 131072
stall_secs = 600
complain_secs = 10
reclaim_limit = 5000
entries_per_readdir = 32
prefetch_secs = 10
statfs_slots = 64
max_mhc = 10000
greedy_default = 100
greedy_quantum = 25
greedy_max = 250
rgrp_try_threshold = 100
statfs_fast = 0

gfs_grow MountPoint

Before running the gfs_grow command:

The gfs_grow command provides a -T (test) option that allows you to see the results of executing the command without actually expanding the file system. Using this command with the -v provides additional information.

After running the gfs_grow command, you can run a df MountPoint command on the file system to check that the new space is now available in the file system.

In this example, the underlying logical volume for the file system file system on the /mnt/gfs directory is extended, and then the file system is expanded.

[root@tng3-1 ~]# lvextend -L35G /dev/gfsvg/gfslv
  Extending logical volume gfslv to 35.00 GB
  Logical volume gfslv successfully resized
[root@tng3-1 ~]# gfs_grow /mnt/gfs
FS: Mount Point: /mnt/gfs
FS: Device: /dev/mapper/gfsvg-gfslv
FS: Options: rw,hostdata=jid=0:id=196609:first=1
FS: Size: 5341168
DEV: Size: 9175040
Preparing to write new FS information...
Done.

gfs_grow [Options] {MountPoint | Device} [MountPoint | Device]

Table 3.3, “GFS-specific Options Available While Expanding A File System” describes the GFS-specific options that can be used while expanding a GFS file system.

gfs_jadd -j Number MountPoint

Before running the gfs_jadd command:

You can find out how many journals are currently used by the filesystem with the gfs_tool df MountPoint command. In the following example, the file system mounted at /mnt/gfs uses 8 journals.

[root@tng3-1 gfs]# gfs_tool df /mnt/gfs
/mnt/gfs:
  SB lock proto = "lock_dlm"
  SB lock table = "tng3-cluster:mydata1"
  SB ondisk format = 1309
  SB multihost format = 1401
  Block size = 4096
  Journals = 8
  Resource Groups = 76
  Mounted lock proto = "lock_dlm"
  Mounted lock table = "tng3-cluster:mydata1"
  Mounted host data = "jid=0:id=196609:first=1"
  Journal number = 0
  Lock module flags = 0
  Local flocks = FALSE
  Local caching = FALSE
  Oopses OK = FALSE

  Type           Total          Used           Free           use%
  ------------------------------------------------------------------------
  inodes         33             33             0              100%
  metadata       38             2              36             5%
  data           4980077        178            4979899        0%

After running the gfs_jadd command, you can run the gfs_tool df MountPoint command again to check that the new journals have been added to the file system.

In this example, one journal is added to the file system that is mounted at the /mnt/gfs directory. The underlying logical volume for this filesystem is extended before the journal can be added.

[root@tng3-1 ~]# lvextend -L35G /dev/gfsvg/gfslv
  Extending logical volume gfslv to 35.00 GB
  Logical volume gfslv successfully resized
[root@tng3-1 ~]# gfs_jadd -j1 /mnt/gfs
FS: Mount Point: /mnt/gfs
FS: Device: /dev/mapper/gfsvg-gfslv
FS: Options: rw,hostdata=jid=0:id=196609:first=1
FS: Size: 5242877
DEV: Size: 9175040
Preparing to write new FS information...
Done.

In this example, two journals are added to the file system on the /mnt/gfs directory.

[root@tng3-1 ~]# fs_jadd -j2 /mnt/gfs
FS: Mount Point: /mnt/gfs
FS: Device: /dev/mapper/gfsvg-gfslv
FS: Options: rw,hostdata=jid=0:id=196609:first=1
FS: Size: 5275632
DEV: Size: 9175040
Preparing to write new FS information...
Done.

gfs_jadd [Options] {MountPoint | Device} [MountPoint | Device]

Table 3.4, “GFS-specific Options Available When Adding Journals” describes the GFS-specific options that can be used when adding journals to a GFS file system.

Setting and Clearing the inherit_jdata Flag

gfs_tool setflag inherit_jdata Directory
gfs_tool clearflag inherit_jdata Directory

Setting and Clearing the jdata Flag

gfs_tool setflag jdata File
gfs_tool clearflag jdata File

This example shows setting the inherit_jdata flag on a directory. All files created in the directory or any of its subdirectories will have the jdata flag assigned automatically. Any data written to the files will be journaled. This example also shows the gfs_tool stat command you can use to verify what flags are set for a directory; the output has been elided to show only the relevant information.

[root@tng3-1]# gfs_tool setflag inherit_jdata /mnt/gfs/data
[root@tng3-1]# gfs_tool stat /mnt/gfs/data
...
Flags:
  inherit_jdata

This example shows setting the jdata flag on a file. The file must have a size of zero when you set this flag. Any data written to the file will be journaled. This example also shows the gfs_tool stat command you can use to verify what flags are set for a file; the output has been elided to show only the relevant information.

[root@tng3-1]# gfs_tool setflag jdata /mnt/gfs/datafile
[root@tng3-1]# gfs_tool stat /mnt/gfs/datafile
...
Flags:
  jdata

Start Suspension

gfs_tool freeze MountPoint

End Suspension

gfs_tool unfreeze MountPoint

This example suspends writes to file system /gfs.

gfs_tool freeze /gfs

This example ends suspension of writes to file system /gfs.

gfs_tool unfreeze /gfs

gfs_fsck -y BlockDevice

In this example, the GFS file system residing on block device /dev/gfsvg/gfslv is repaired. All queries to repair are automatically answered with yes. Because this example uses the -v (verbose) option, the sample output is extensive and repetitive lines have been elided.

[root@tng3-1]# gfs_fsck -v -y /dev/gfsvg/gfslv
Initializing fsck
Initializing lists...
Initializing special inodes...
Validating Resource Group index.
Level 1 check.
92 resource groups found.
(passed)
Setting block ranges...
Creating a block list of size 9175040...
Clearing journals (this may take a while)Clearing journal 0
Clearing journal 1
Clearing journal 2
...
Clearing journal 10

Journals cleared.
Starting pass1
Checking metadata in Resource Group 0
Checking metadata in Resource Group 1
...
Checking metadata in Resource Group 91
Pass1 complete      
Starting pass1b
Looking for duplicate blocks...
No duplicate blocks found
Pass1b complete      
Starting pass1c
Looking for inodes containing ea blocks...
Pass1c complete      
Starting pass2
Checking directory inodes.
Pass2 complete      
Starting pass3
Marking root inode connected
Checking directory linkage.
Pass3 complete      
Starting pass4
Checking inode reference counts.
Pass4 complete      
Starting pass5
...
Updating Resource Group 92
Pass5 complete      
Writing changes to disk
Syncing the device.
Freeing buffers.

For a Normal Symbolic Link

ln -s TargetLinkName

For a Variable Symbolic Link

ln -s VariableLinkName

In this example, there are three nodes with hostnames n01, n02 and n03. Applications on each node uses directory /gfs/log/, but the administrator wants these directories to be separate for each node. To do this, no actual log directory is created; instead, an @hostname CDPN link is created with the name log. Individual directories /gfs/n01/, /gfs/n02/, and /gfs/n03/ are created that will be the actual directories used when each node references /gfs/log/.

n01# cd /gfs
n01# mkdir n01 n02 n03
n01# ln -s @hostname log

n01# ls -l /gfs 
lrwxrwxrwx 1 root root 9 Apr 25 14:04 log -> @hostname/ 
drwxr-xr-x 2 root root 3864 Apr 25 14:05 n01/ 
drwxr-xr-x 2 root root 3864 Apr 25 14:06 n02/ 
drwxr-xr-x 2 root root 3864 Apr 25 14:06 n03/ 

n01# touch /gfs/log/fileA 
n02# touch /gfs/log/fileB
n03# touch /gfs/log/fileC

n01# ls /gfs/log/ 
fileA 
n02# ls /gfs/log/ 
fileB 
n03# ls /gfs/log/
fileC