Linux File Systems: Ext2 vs Ext3 vs Ext4 vs XFS

ext2, ext3, ext4 and xfs are all filesystems created for Linux. This article explains the following:

• High level difference between these filesystems.
• How to create these filesystems.
• How to convert from one filesystem type to another.

Ext2

• Ext2 stands for second extended file system.
• It was introduced in 1993. Developed by Rémy Card.
• This was developed to overcome the limitation of the original ext file system.
• Ext2 does not have journaling feature.
• On flash drives, usb drives, ext2 is recommended, as it doesn’t need to do the over head of journaling.
• Maximum individual file size can be from 16 GB to 2 TB
• Overall ext2 file system size can be from 2 TB to 32 TB

Ext3

• Ext3 stands for third extended file system.
• It was introduced in 2001. Developed by Stephen Tweedie.
• Starting from Linux Kernel 2.4.15 ext3 was available.
• The main benefit of ext3 is that it allows journaling.
• Journaling has a dedicated area in the file system, where all the changes are tracked. When the system crashes, the possibility of file system corruption is less because of journaling.
• Maximum individual file size can be from 16 GB to 2 TB
• Overall ext3 file system size can be from 2 TB to 32 TB
• There are three types of journaling available in ext3 file system.
  • Journal – Metadata and content are saved in the journal.
  • Ordered – Only metadata is saved in the journal. Metadata are journaled only after writing the content to disk. This is the default.
  • Writeback – Only metadata is saved in the journal. Metadata might be journaled either before or after the content is written to the disk.
• You can convert a ext2 file system to ext3 file system directly (without backup/restore).

Ext4

• Ext4 stands for fourth extended file system.
• It was introduced in 2008.
• Starting from Linux Kernel 2.6.19 ext4 was available.
• Supports huge individual file size and overall file system size.
• Maximum individual file size can be from 16 GB to 16 TB
• Overall maximum ext4 file system size is 1 EB (exabyte). 1 EB = 1024 PB (petabyte). 1 PB = 1024 TB (terabyte).
• Directory can contain a maximum of 64,000 subdirectories (as opposed to 32,000 in ext3)
• You can also mount an existing ext3 fs as ext4 fs (without having to upgrade it).
• Several other new features are introduced in ext4: multiblock allocation, delayed allocation, journal checksum. fast fsck, etc. All you need to know is that these new features have improved the performance and reliability of the filesystem when compared to ext3.
• In ext4, you also have the option of turning the journaling feature “off”.

XFS File System

• The XFS file system is an extension of the extent file system. 
• The XFS is a high-performance 64-bit journaling file system. 
• The support of the XFS was merged into Linux kernel in around 2002 and In 2009 Red Hat Enterprise Linux version 5.4 usage of the XFS file system. 
• XFS supports maximum file system size of 8 exbibytes for the 64-bit file system. 
• There is some comparison of XFS file system is XFS file system can’t be shrunk and poor performance with deletions of the large numbers of files. 
• Now, the RHEL 7.0 uses XFS as the default filesystem.

Creating an ext2, or ext3, or ext4 filesystem

Once you’ve partitioned your hard disk using fdisk command, use mke2fs to create either ext2, ext3, or ext4 file system.


Create an ext2 file system:

mke2fs /dev/sda1

Create an ext3 file system:

mkfs.ext3 /dev/sda1

(or)

mke2fs –j /dev/sda1

Create an ext4 file system:

mkfs.ext4 /dev/sda1

 (or)

 mke2fs -t ext4 /dev/sda1

Converting ext2 to ext3

For example, if you are upgrading /dev/sda2 that is mounted as /home, from ext2 to ext3, do the following.

umount /dev/sda2

tune2fs -j /dev/sda2

mount /dev/sda2 /home

Note: You really don’t need to umount and mount it, as ext2 to ext3 conversion can happen on a live file system. But, I feel better doing the conversion offline.

Converting ext3 to ext4

If you are upgrading /dev/sda2 that is mounted as /home, from ext3 to ext4, do the following.

umount /dev/sda2

tune2fs -O extents,uninit_bg,dir_index /dev/sda2

e2fsck -pf /dev/sda2

mount /dev/sda2 /home

Again, try all of the above commands only on a test system, where you can afford to lose all your data.

____________________________________________

What is a Journaling Filesystem?

A journaling filesystem keeps a journal or log of the changes that are being made to the filesystem during disk writing that can be used to rapidly reconstruct corruptions that may occur due to events such a system crash or power outage. The level of journaling performed by the file system can be configured to provide a number of levels of logging depending on your needs and performance requirements.

What are the Advantages of a Journaling Filesystem?

There are a number of advantages to using a journaling files system.

Both the size and volume of data stored on disk drives has grown exponentially over the years. The probelm with a non-journaled file system is that following a crash the fsck (filesystem consistency check) utility has to be run. fsck will scan the entire filesystem validating all entries and making sure that blocks are allocated and referenced correctly. If it finds a corrupt entry it will attempt to fix the problem. The issues here are two-fold. Firstly, the fsck utility will not always be able to repair damage and you will end up with data in the lost+found directory. This is data that was being used by an application but the system no longer knows where they were reference from. The other problem is the issue of time. It can take a very long time to complete the fsck process on a large file system leading to unacceptable down time.

A journaled file system records information in a log area on a disk (the journal and log do not need to be on the same device) during each write. This is a essentially an "intent to commit" data to the filesystem. The amount of information logged is configurable and ranges from not logging anything, to logging what is known as the "metadata" (i.e ownership, date stamp information etc), to logging the "metadata" and the data blocks that are to be written to the file. Once the log is updated the system then writes the actual data to the appropriate areas of the filesystem and marks an entry in the log to say the data is committed.

After a crash the filesystem can very quickly be brought back on-line using the journal log reducing what could take minutes using fsck to seconds with the added advantage that there is considerably less chance of data loss or corruption.


What is a Journal Checkpoint?

When a file is accessed on the filesystem, the last snapshot of that file is read from the disk into memory. The journal log is then consulted to see if any uncommitted changes have been made to the file since the data was last written to the file (essentially looking for an "intention to commit" in the log entry as described above). At particular points the filesystem will update file data on the disk from the uncommited log entries and trim those entries from the log. Committing operations from the log and synchronizing the log and its associated filesystem is called a checkpoint.

What are the disadvantages of a Journaled Filesystem?

Nothing in life is is free and ext3 and journaled filesystems are no exception to the rule. The biggest draw back of journaling is in the area of performance simply because more disk writes are required to store information in the log. In practice, however, unless you are running system where disk performance is absolutely critical the performance difference will be negligable.

____________________________________________________

Common Commands for ext3 and ext4 Compared to XFS

Task	ext3/4	XFS
Create a file system	mkfs.ext4 or mkfs.ext3	mkfs.xfs
File system check	e2fsck	xfs_repair
Resizing a file system	resize2fs	xfs_growfs
Save an image of a file system	e2image	xfs_metadump and xfs_mdrestore
Label or tune a file system	tune2fs	xfs_admin
Backup a file system	dump and restore	xfsdump and xfsrestore

Linux and Unix dd command

About dd

The dd command copies a file, converting the format of the data in the process, according to the operands specified.

Syntax of ‘dd’ command.

dd if=<source file name> of=<target file name> [Options]

dd [OPERAND]...
dd OPTION

Operands
bs=BYTES  : read and write BYTES bytes at a time (also see ibs=,obs=)

cbs=BYTES  : convert BYTES bytes at a time

conv=CONVS  : convert the file as per the comma separated symbol list. Each symbol may be one of                             the following, and represents a specific type of conversion:

                          ascii
                          from EBCDIC to ASCII

                          ebcdic
                         from ASCII to EBCDIC

                         ibm
                        from ASCII to alternate EBCDIC

                        block
                        pad newline-terminated records with spaces to cbs-size

                       unblock
                       replace trailing spaces in cbs-size records with newline

                      lcase
                      change upper case to lower case

                      nocreat
                     do not create the output file

                     excl
                    fail if the output file already exists

                   notrunc
                   do not truncate the output file

                  ucase
                  change lower case to upper case

                 swab
                 swap every pair of input bytes

                noerror
                continue after read errors

                sync
                pad every input block with NULs to ibs-size; when used with block or unblock, 

                pad with spaces rather than NULs

               fdatasync
              physically write output file data before finishing

              fsync
              likewise, but also write metadata.


count=BLOCKS  :  copy only BLOCKS input blocks

ibs=BYTES : read BYTES bytes at a time (default: 512)

if=FILE   : read from FILE instead of stdin

iflag=FLAGS  : read as per the comma separated symbol list. Each symbol may be 

                           one of the following:

                          append
                          append mode (makes sense only for output; conv=notrunc suggested)

                         direct
                         use direct I/O for data

                         directory
                         fail unless a directory

                        dsync
                        use synchronized I/O for data

                        sync
                       likewise, but also for metadata

                       fullblock
                       accumulate full blocks of input (iflag only)

                       nonblock
                      use non-blocking I/O

                       noatime
                      do not update access time

                      noctty
                     do not assign controlling terminal from file

                     nofollow
                     do not follow symlinks.


obs=BYTES  :  write BYTES bytes at a time (default: 512)
of=FILE  : write to FILE instead of stdout
oflag=FLAGS :  write as per the comma separated symbol list
seek=BLOCKS :  skip BLOCKS obs-sized blocks at start of output
skip=BLOCKS  : skip BLOCKS ibs-sized blocks at start of input
status=noxfer  : suppress transfer statistics

Options
--help       :   Display help and exit.
--version  :    Display version information and exit.

Numerical Suffixes

BLOCKS and BYTES may be followed by the following multiplicative suffixes:

c=1
w=2
b=512
kB=1000
K=1024
MB=1000*1000
M=1024*1024
xM=M
GB=1000*1000*1000
G=1024*1024*1024

and so on for T (terabytes), P (petabytes), E (exabytes), Z (zettabytes), and Y (yottabytes).

This tool can be used for:
• Backing up and restoring an entire hard drive or a partition.
• Copy regions of raw device files like backing up MBR (master boot record).
• Converting data formats like ASCII to EBCDIC.
• Converting lowercase to uppercase and vice versa.
• Creating files with fixed size.

dd examples

Caution: Use dd cautiously — improper usage or entering the wrong values could inadvertently wipe, destroy, or overwrite the data on your hard drive.

dd if=/dev/sr0 of=/home/hope/exampleCD.iso bs=2048 conv=noerror,sync


Create a ISO disc image from the CD in the computer.

dd if=/dev/sda of=~/disk1.img

Create an img file of the /dev/sda hard drive. To restore that image type: dd if=disk1.img of=/dev/sda

dd if=/dev/sda of=/dev/sdb

Copy the contents from the if= drive /dev/sda to the of= drive /dev/sdb.
Related commands

cp — Copy files and directories.
fdisk — A disk partioning utility. 

---------------------------------------------------------------------------------------

Example 1. Backup Entire Harddisk

To backup an entire copy of a hard disk to another hard disk connected to the same system, execute the dd command as shown below. In this dd command example, the UNIX device name of the source hard disk is /dev/hda, and device name of the target hard disk is /dev/hdb.

# dd if=/dev/sda of=/dev/sdb

• “if” represents inputfile, and “of” represents output file. So the exact copy of /dev/sda will be available in /dev/sdb.
• If there are any errors, the above command will fail. If you give the parameter “conv=noerror” then it will continue to copy if there are read errors.
• Input file and output file should be mentioned very carefully, if you mention source device in the target and vice versa, you might loss all your data.

In the copy of hard drive to hard drive using dd command given below, sync option allows you to copy everything using synchronized I/O.

# dd if=/dev/sda of=/dev/sdb conv=noerror,sync

---------------------------------------------------------------------------------------

Example 2. Create an Image of a Hard Disk

Instead of taking a backup of the hard disk, you can create an image file of the hard disk and save it in other storage devices.There are many advantages to backing up your data to a disk image, one being the ease of use. This method is typically faster than other types of backups, enabling you to quickly restore data following an unexpected catastrophe.

# dd if=/dev/hda of=~/hdadisk.img

The above creates the image of a harddisk /dev/hda

---------------------------------------------------------------------------------------

Example 3. Restore using Hard Disk Image

To restore a hard disk with the image file of an another hard disk, use the following dd command example.

# dd if=hdadisk.img of=/dev/hdb

The image file hdadisk.img file, is the image of a /dev/hda, so the above command will restore the image of /dev/hda to /dev/hdb.

---------------------------------------------------------------------------------------

Example 4. Creating a Floppy Image

Using dd command, you can create a copy of the floppy image very quickly. In input file, give the floppy device location, and in the output file, give the name of your floppy image file as shown below.

# dd if=/dev/fd0 of=myfloppy.img

---------------------------------------------------------------------------------------

Example 5. Backup a Partition

You can use the device name of a partition in the input file, and in the output either you can specify your target path or image file as shown in the dd command example below.

# dd if=/dev/hda1 of=~/partition1.img

---------------------------------------------------------------------------------------

Example 6. CDROM Backup

dd command allows you to create an iso file from a source file. So we can insert the CD and enter dd command to create an iso file of a CD content.

# dd if=/dev/cdrom of=tgsservice.iso bs=2048

dd command reads one block of input and process it and writes it into an output file. You can specify the block size for input and output file. In the above dd command example, the parameter “bs” specifies the block size for the both the input and output file. So dd uses 2048bytes as a block size in the above command.

Note: If CD is auto mounted, before creating an iso image using dd command, its always good if you unmount the CD device to avoid any unnecessary access to the CD ROM.

how to set the ssh timeout in linux

vim /etc/ssh/sshd_config


ClientAliveInterval 600    ---------------------change the value
ClientAliveCountMax 3

That will give you an timeout of 30 minutes (600 sec x 3)

after that restast sshd service and try