Extended 2 Filesystem Format, and Similar/Successor Filesystem Formats

Extended File System

This, naturally, predated Ext2. Since Ext2 became the popular native filesystem for Linux machines early on, it will probably be unlikely that Ext will be seen in regular usage today. However, some details for those interested in historical usage are mentioned by Design and Implementation fo the Second Extended Filesystem.

Ext2: Second Extended File System

Ext2 may be more compatible with older versions of operating systems using the Linux kernel, as well as other operating systems like OpenBSD. It's probably the most compatible method, although lacks some of the nice data integrity features of newer versions.

The e2fsprogs website has some resources by the author of widely used software that handles this filesystem format. One example is the Ext2fs Home Page on the e2fsprogs website which mentions some software that is about this file system. Design and Implementation fo the Second Extended Filesystem discusses the history of this filesystem format.

Ext3
[#ext3lik2]: Much compatibility with Ext2

Wikipedia's article on Ext3 documents the following as features of Ext3 which cause the file system to not be a valid Ext2 file system: Journaling, online file system growth, and the “Htree indexing for larger directories”. The page goes on to say, “Without these, any ext3 file system is also a valid ext2 file system.” In many cases, upgrading from Ext2 and Ext3, as well as downgrading from Ext3 to Ext2, are apparently fairly straightforward operations. While Wikipedia's article on Ext3: section on size limits says the maximum filesize for a file system with a 1KB block size is 2TB, Wikipedia's article on Ext2: section called “File system limits” says the maximum filesize for a file system with a 1KB block size is “4* TB”.

An Ext3 file system (partition) which has been properly dismounted is often a valid Ext2 file system. However, an Ext3 file system that has been mounted, and hasn't yet been dismounted or had a file system check performed, may often have a journal that needs to be “replayed” for the data to be considered valid. Mounting such a file system as an Ext2 file system, and doing so in read/write mode, may result in the journal being wiped up, causing the loss of data that the journal was keeping track of. Mounting in read-only mode isn't so likely to cause such data loss, but some data might not be properly read until the journal is replayed. (Running the tune2fs -l command on a drive that needs such clean-up may show one of the Filesystem features as being needs_recovery.)

Ext3 Journaling

An Ext3 file system supports some new features over Ext2, the most important of which may be an “inode” (section of data) which is called a “journal”. This journal is designed to reduce data corruption when a system doesn't unmount the file system in a “clean”, preferred fashion before the file system stops being mounted. The clearest likely cause of a file system stopping in such a manner is a system is reset or powered off without proper warning.

Using a journal requires support by the kernel. OpenBSD FAQ 8: section on Journaling (OpenBSD FAQ 8.21) notes that OpenBSD does not support this. The journal is likely usually invisible. However, it may be visible when tune2fs creates a journal on an Ext3 file system (converting the file system to Ext3 if the filesystem is an Ext2 filesystem at the start of the process). In such a case, the journal is in an “immutable” file called “.journal”. The e2fsck disk-checking command will convert such a visible file to an invisible file.

Wikipedia's article on Ext3: section on Compression notes that e3compr is not compatible with journals.

Ext3 Journaling design paper (PDF file) from the e2fsprogs web site may be of interest to those seeking such details.

[#ext3vsnw]: Comparison with newer file systems

Wikipedia's page on Ext3: “Advantages” section says that even though some “filesystem features such as dynamic inode allocation and extents could be considered a disadvantage, in terms of recoverability this gives ext3 a significant advantage over file systems with those features. The file system metadata is all in fixed, well-known locations, and there is some redundancy inherent in the data structures that may allow ext2 and ext3 to be recoverable in the face of significant data corruption, where tree-based file systems may not be recoverable.”

The journal may have some downside(s) relating to the section of the disk with the journal being constantly used, more than others sections of the disk. Wikipedia's page on Ext2 notes, “Recent kernels, however, support a journal-less mode of ext4, which would offer the same benefit along with a number of ext4-specific benefits.”

Although it doesn't support snapshot features, see Next3.

[#next3fs]: Next3
Next3 is basically Ext3 with added support for snapshots.
[#ext4fs]: Ext4

Ext4 has a number of improvements, including using “extents” instead of “the traditional block mapping scheme used by” ext3 (quoting Wikipedia's page for Ext4). Using extents may improve speed, while breaking compatibility with ext3 when the ext4 drive is using extents. Using extents may also reduce data recovery, as commented on a bit in the ext3 section.

Wikipedia's page on Btrfs identifies Theodore T'so as “the principal developer of the ext3 and ext4 file systems”, and Wikipedia's page of Theodore T'so identifies him as the primary developer and maintainer of e2fsprogs. Arstechnica summary of Linux kernel panel noted that Ts'o “believes that the way forward is Oracle's open source Btrfs filesystem”. He has been quoted described Oracle's open source Btrfs as being

[#e2fspmix]: DATA LOSS Warning: Be wary of mixing packages.
The dangerous problem

There is a command that comes with the e2fsprogs package, e2fsck, which may be better to use than fsck -t ext2fs if the filesystem was created using the e2fsprogs package's mke2fs command. Differences in filesystem structure could cause filesystem corruption to be detected by “fsck -t ext2fs” on a filesystem volume, if the volume was created with the e2fsprogs package's mke2fs command. If fsck -t ext2fs is allowed to proceed to attempt to fix this perceived corruption, files and/or directories may be removed and/or be inaccessible. Do not proceed with repairing such data (before getting a conveniently restorable backup of all data on the volume)!

(This has been seen with OpenBSD's fsck_ext2fs command.)

Specifically, at the time of this writing, the fsck_ext2fs program within the latest version of OpenBSD (OpenBSD/amd64 4.9), seems to have some problems. Namely, at least with drives created using the e2fsprogs software package instead of being created with the newfs_ext2 command that is built into OpenBSD, this fsck_ext2fs detects and reports filesystem errors. These filesystem errors are detected even though the filesystem seems to be fine before fsck_ext2fs attempts to make some changes. The program may make changes to try to fix these detected “errors”, and these changes may be destructive, causing loss to data that could previously have been accessed.

Note that simply deciding to not manually run fsck_ext2fs is not necessarily sufficiently safe. The operating system's default behavior may involve running fsck (which may run fsck_ext2fs) automatically when the drive is mounted, which may occur rather automatically when the system is being booted. (This is warned about in the section about making an ext2fs filesystem volume.) (A possible approach to try to work around that automated behavior has been explored a bit by a section about controlling when mounted drives boot. However, be warned that the attempted workaround might have, eventually, been found to not work as intended. So, plan to do sufficient research/testing/development before trying that approach.)

The safe thing to do may be to use OpenBSD's newfs_ext2 program. Note that this is not considered to be a bug with the e2fsprogs package's mke2fs command. The e2fsprogs was created and maintained by the widely-recognized creator/maintainer of the Ext2 filesysitem. The cause of the problem is simply that OpenBSD's fsck_ext2fs is not recognizing the output of the widely accepted mke2fs command from e2fsprogs. The reason for recommending using newfs_ext2 is just that this approach seems to be less likely to cause data corruption problems.

Logged experiences

Here are some early findings. To experience this: create a new Ext2 partition. Perhaps fsck_ext2fs will already have problems. Otherwise, mount the partition and make a subdirectory (all as a superuser) and then chown so the subdirectory is owned by a non-root user. Unmount. If there are still no errors, mount and unmount. It seems that these actions should be sufficient to result in a filesystem that fsck_ext2fs will not like.

Workaround

The solution may be to download the e2fsprogs package, and then run:

ls -l /sbin/fsck_ext2fs
If the file is a normal file

If that is a normal file (so the first character of the output is a hyphen), run the following (one command line at a time):

sudo mv -i /sbin/fsck_ext2fs /sbin/fsck_ext2fs_orig

Then, if that doesn't prompt about overwriting a file, and if that succeeded, that file is likely backed up okay now.

Otherwise...

If this does prompt about overwriting a file, the wisest course of action may be to say n (meaning “no”, do not proceed”), and then manually investigate just what the backup file /sbin/fsck_ext2fs_orig is, and perhaps also investigate just what the file /sbin/fsck_ext2fs is. Hopefully the answers for this investigation are readily available in documentation that stated how the /sbin/fsck_ext2fs_orig file was created (e.g. perhaps created by following these instructions earlier).

If that file was just moved, and so it no longer exists in its original location, then do the following in order to make the original location succeed in running an executable.

sudo ln -s /usr/local/sbin/e2fsck /sbin/fsck_ext2fs

If this sort of workaround is performed, remember to undo this sort of change before any sort of operating system upgrade (e.g. before upgrading to -stable), in case the old executable gets updated at all. Also, remember to re-apply this change after applying any sort of operating system upgrade, unless further testing is going to be getting performed. (Having some sort of centralized documentation per machine may help such things be remembered, if such documentation is readily checked before making such changes. So, update such documentation with these reminders so they can help later.)

More logs

Some details/output of the problem:

An Old forum post about fsck shows an example:

# fsck -t ext2fs /dev/wd1i
** /dev/rwd1i (NO WRITE)
** Last Mounted on
** Phase 1 - Check Blocks and Sizes
** Last Mounted on
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
** Phase 3 - Check Connectivity
** Phase 4 - Check Reference Counts
LINK COUNT FILE I=4415266  OWNER=rsync MODE=100700
SIZE=7585 MTIME=Apr 15 17:28 2005  COUNT 2 SHOULD BE 1
ADJUST? no

** Phase 5 - Check Cyl groups
SUMMARY INFORMATIONS WRONG FOR CG #22
SALVAGE? no

BLK(S) MISSING IN BIT MAPS #22
SALVAGE? no

SUMMARY INFORMATIONS WRONG FOR CG #52
SALVAGE? no

BLK(S) MISSING IN BIT MAPS #52
SALVAGE? no

SUMMARY INFORMATIONS WRONG FOR CG #270
SALVAGE? no

BLK(S) MISSING IN BIT MAPS #270
SALVAGE? no

SUPERBLK SUMMARY INFORMATION BAD
SALVAGE? no

77524 files, 7471772 used, 2146775 free


One might think that the above output looks a bit altered: There is no date for the last mount. Also, the output of “(NO WRITE)” suggests there may have been a -n parameter given to fsck. However, subsequent checks suggest maybe the missing date and the “(NO WRITE)” text may actually appear, so those might actually not be reliable indications that the output got altered. The third and fourth lines of output look to be repeated.

However, output similar to this has been seen by another person (on another system). There may be many more (dozens? hundreds?) of such error messages (referencing incrementing bit map numbers).

Or, there may be fewer errors, like the following:

$ sudo fsck -t ext2fs /dev/wd1i
** /dev/rsd0j (NO WRITE)
** File system is already clean
** Last Mounted on
** Phase 1 - Check Blocks and Sizes
** Phase 2 - Check Pathnames
ROOT INODE UNALLOCATED
ALLOCATE? no


$

Here is some example output of a drive that showed this behavior (on both of the Linux partitions using a type identifier value of 83):

fdisk: 1> p
Disk: sd0       geometry: 121601/255/63 [1953525168 Sectors]
Offset: 0       Signature: 0xAA55
            Starting         Ending         LBA Info:
 #: id      C   H   S -      C   H   S [       start:        size ]
-------------------------------------------------------------------------------
*0: A6  16710   0   1 -  33419  85  16 [   268446150:   268435456 ] OpenBSD
 1: 82  34919   0   1 -  38052   0   3 [   560973735:    50331648 ] Linux swap
 2: 83  38053   0   1 -  54762  85  16 [   611321445:   268435456 ] Linux files*
 3: 83  54764   0   1 - 121600 254  63 [   879783660:  1073736405 ] Linux files*
fdisk: 1>

Here is how the actual drive was detected by OpenBSD's startup sequence:

sd0 at scsibus0 targ 0 lun 0: <ATA, WDC WD1002FAEX-0, 05.0> SCSI3 0/direct fixed
sd0: 953869MB, 512 bytes/sec, 1953525168 sec total
[#extfscpt]: Ext family's compatibility

The different major versions of the Ext filesystems have been largely compatible, when feasible, with previous versions. When specific new features are not being used, even old filesystem software (that has code specific to filesystem volume handling, like code that is run by mount), might be able to work with volumes using a newer version. Kudos to those decisions that have enabled such excellent compatibility. For specific details, see section about Ext3 being compatible with Ext2.

See warning about data loss from mixing software suites.

In Linux, the Ext2 filesystem may be named ext2, while in BSD this same filesystem type may be named ext2fs. (Such names may be used when specifying the type of operating system to software used for creating or testing a filesystem volume.) Such a different doesn't exist with Ext3 in OpenBSD (and perhaps others), namely because OpenBSD does not support Ext3. (Such a difference might exist in other operating systems.)

[#mkane2fs]: Creating an ext2fs filesystem volume

First, choose the desired set of software, and be sure to stick with that set of software. Some details about the reason for this are in the warning about data loss from mixing software suites. If an operating system comes with software tools designed to support Ext2, that software might be somehow custom-tailored to work best (most compatibly) with that operating system. In that case, using the pre-bundled internal tools might be the safer bet (rather than using software from the e2fsprogs collection of software.) So, if commands, other than those from e2fsprogs, are available, and if these other commands can do what is wanted, see about using what is available (instead of installing e2fsprogs). (This might be especially true when using a kernel other than Linux.) This generalized advice (about whether it is better to use software bundled with the operating system or to use e2fsprogs) may not apply in some cases where the operating system actually does utilize mke2fs as the standard (possibly pre-bundled) tools for handling Ext2 filesystem volumes. (In this case, there is absolutely no difference between using mke2fs and what comes with the operating system.) In other operating systems, some of the options for mke2fs may match options using the operating system's native software tools for this task, but there may also be some variations. The safest approach is to review documentation (and, as needed, other websites as well) enough to be familiar with the ins and outs of the program, before using the software.

Be aware of the ongoing potential issue that threatens data loss when the filesystems may be tested. Such testing of a filesystem volume may happen rather automatically whenever the filesystem volume is mounted, including when the system is restarted. Note that this isn't meant to suggest that mke2fs is terrible in general, and one should not feel a need for alarm if the the e2fsprogs collection of software is the only thing providing utilities for the software. The statement is simply that mixing different software has been known to lead to problematic results.

For generalized details about performing this task using software that comes with the operating system, see the section about making a filesystem volume.

Additional details about creating a filesystem volume may also be in the commentary in the section about Ext filesystem compatibility.

Here are some additional details which may be specific to Ext2.

Using mke2fs from e2fsprogs

At the risk of being overly redundant (a risk being taken now because losing lots of data can certainly be very unpleasant!), note the danger of substantial data loss posed by mixing e2fsprogs with other software packages. This can happen rather automatically when the filesystem volume is mounted, including when the computer is rebooted.

See the /etc/mke2fs.conf file. The file from OpenBSD's e2fsprogs package matches Debian's (functionally, though white space may be different). Is Ubuntu's or CentOS's different, and perhaps even named something different, like mkfs.ext2.conf or something like that?

Some reference to understand this file may be found in (3rd party?) man page for mke2fs and/or (3rd party?) man page for /etc/mke2fs.conf. The [defaults] section may impact the defaults used for various parameters. The [fs_types] section may be the largest section of the default /etc/mke2fs.conf file, and can be used by the -T command line parameter to the mke2fs command.

The mke2fs command may make an mke2fs volume.

Some settings may not be easily changed after the file system is created. (Since data can be altered, in theory things could be changed, but there may not be any common implementation to change the settings). Some of those which may be impactful may include:

block size

(Unverified: Linux Questions post indicates “currently block size can't be bigger than page size. For most architectures page size is 4K”.) Wikipedia's page on Ext3: Citation note 10 seems to suggest this, saying, “In Linux, 8 KiB block size is only available on architectures which allow 8 KiB pages, such as Alpha.”

This can be affected with switches like -b or -T small. To determine what mke2fs would set the block size to, it might be safe to use mke2fs with the -n switch (so that mke2fs doesn't actually make changes). (Like any such super-dangerous command, it may be wise to check the documentation for mke2fs to make sure that the command line switch hasn't changed functionality with a new version.)

inode count

This can be a bit tricky to determine, but also can be quite important because a high inode count uses up space, while too low of an inode count can make a disk be unable to be written to. An example is described by Notatypewriter's Blog on choosing an inode value. If there is an existing file system that can be viewed, df -i will show inode usage. (The usage on some file systems, such as FAT32, may show as 100% used. If such a thing is seen, just ignore such unhelpful information.) If the %iused (which represents the percentage of inodes used) is higher than the Capacity (% of used space), then using a system with more inodes will likely end up being a better approach.

If in doubt, here are some general guidelines (which may or may not work all that well): For a general use system with lots of small files (like news articles, individual E-Mails saved, pictures, text files including HTML files, etc.), and perhaps small archive files (around a third of a MB or perhaps 5-20MB), the safe choices appear to be default, or to use -T news. Using -T news which may use up something like 6% of space, but will be less likely to hit an inode limit. Systems which are specifically expected to use few and very large files, such as databases, may have reason to explore using -T largefile or even -T largefile4 on some file systems. (Note: The -T largefile option refers to text in /etc/mke2fs.conf affects the inode_ratio and block size (similar to -i and -b options for mk2e2fs), and have nothing to do with -O large_file command line option. The -O large_file command line option should be set on any file system that will have at least one file that is over 2GB in size. Similarly, the ext4 feature of huge_file for supporting files over 2TB in size is unrelated.)

(Perhaps see also: -O large_file,resize_inode and -E resize=number options?)

http://en.wikipedia.org/wiki/Ext3#cite_note-0 states, “The maximum number of inodes (and hence the maximum number of files and directories) is set when the file system is created. If V is the volume size in bytes, then the default number of inodes is given by” V divided by 2 raised to the 13th power, which would be V divided by 8192, “(or the number of blocks, whichever is less), and the minimum by” V divided by 2 raised to the 23rd power, which would be V divided by 8,388,608 (8192x1024). “The default was deemed sufficient for most applications. The max number of subdirectories in one directory is fixed to 32000.”

Some other options to address. (These may or may not be easy to change: further research may be needed to determine how true that is or isn't.)

stride
According to information from Zimba's Performance Tuning Guidelines for Large Deployments: section about file systems, this should be set to the RAID stripe size divided by the block size. So if the RAID stripes are 128k and the block size is 4096 bytes (4k), then stride would be set to 32. Combined with the man page that says -E replaces the older -R switch, the parameter would be -E stride=32.

If there is a desire to use the features that are used by an already-existing filesystem volume (perhaps since the new filesystem volume will be used in the same data area that the old filesystem volume currently resides at), one may want to use tune2fs -l to show “Filesystem features:”.

Once the details for command line options are determined, the command to run will be mke2fs.

The following was used to create an ext2 filesystem in OpenBSD (before learning about newfs_ext2fs, which may be better to use on that platform):

date
time mke2fs -T news -O large_file -l /tmp/file_identifying_known_bad_blocks -v -n /dev/someHDD
echo $?

Then if all of the information provided looks accessible, remove the -n parameter.

Example output (from e2fsprogs in OpenBSD):

$ sudo time mke2fs -T news -O large_file -v /dev/sd0j
mke2fs 1.41.4 (27-Jan-2009)
fs_types for mke2fs.conf resolution: 'ext2', 'news'
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
33554432 inodes, 33554432 blocks
1677721 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
1024 block groups
32768 blocks per group, 32768 fragments per group
32768 inodes per group
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 36 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
    36174.25 real         0.27 user        66.07 sys
$ echo $?
0

For comparison,

In an earlier attempt, the following command line was used:

date
time mke2fs -T news -O large_file,resize_inode,dir_index,filetype,sparse_super -j -l /tmp/file_identifying_known_bad_blocks -v -n /dev/someHDD
echo $?

(The differences here were specifying a journal, and including some options that were found on an Ext3 system created by an Ubuntu disc.)

(And, since things looked good, the -n parameter was removed.)

The results were as follows:

The output of the mke2fs differed in the following ways: The fs_types mentioned ext3 instead of ext2, there was a “Creating journal (32768 blocks): done” line after the “Writing inode tables: done” line, and the filesystem said it would be checked after 39 mounts, not 36 mounts.

More importantly, the syntax led to a filesystem volume that had errors in OpenBSD after using chown and then umount and then mount and then umount. Then fsck would cause some data loss (if the loss hadn't already occurred). This might have been due to a bad physical disk, but badblocks did not find anything. (Although that might not be conclusive, it is being treated as a possibility.) (Update: The problems experienced were probably due to using fsck rather than the e2fsck command. This has since been noted by the danger of substantial data loss posed by mixing e2fsprogs with other software designed to handle Ext2 filesystems.

Making an Ext2 file system in OpenBSD

Perhaps the better options are to use newsfs_ext2fs or “ newsfs -t ext2fs ”. Another option may be to use e2fsprogs's mke2fs, which is described more in the section about creating a file system in Linux. Note that these programs do not use the precise same syntax.

The OpenBSD man page for newsfs_ext2fs indicates that -D 256 may assist with “compatibility with ext4.”

The disklabel needs to show the file system type is Linux Ext2, or else the -I parameter is needed (unless -F is used). To check the disklabel, use something like “ sudo disklabel /dev/sd0c ” (except using the proper device name: see making a filesystem: identifying a destination name for details about hard drive device names) and look in the fstype column.

The following is meant for OpenBSD (which has a sudo command available) , and this documentation example is assuming a proper BSD disklabel is in place. (However, newsfs_ext2fs may use an fstat system call (not to be confused with the fstat command) instead of relying on the disk label.)

sudo newfs -t ext2fs -o -D 256 -O 1 -V 4 -v customVolumeName -N NameOfDevice

For the name of device, either specify the raw device (e.g. /dev/rsd0a or the device's “relative path” (e.g. “sd0a”). Specifying the name of the relative path, in the /dev/ directory (e.g. /dev/sd0a) does NOT work.

The -v customVolumeName may be up to 16 characters, and can be modified later by adjusting/tuning the filesystem, using “tune2fs -L ”. However, the tune2fs command may not exist until after installing software, specifically the e2fsprogs software package. If that software is not installed (and if there are no plans to install that software), it may be most convenient to set the filesystem volume label here, during the process of creating the filesystem volume.

That should have gone very quickly since the filesystem volume wasn't actually created. If the output looks desirable, then run the command again with similar parameters, but eliminate the -N parameter. Prepend the word “time”, make an additional safety check to ensure that the correct device object is specified (so that the filesystem volume isn't created in the wrong location), and re-run the command (with that -N parameter now removed). The result may look something like the following:

# sudo time newfs_ext2fs -D 256 -O 1 -V 4 -v customVolumeName sd0j
/dev/rsd0j: 131072.0MB (268435456 sectors) block size 4096, fragment size 4096
        using 1024 block groups of 128.0MB, 32768 blks, 16384 inodes.
super-block backups (for fsck_ext2fs -b #) at:
   32768,    98304,   163840,   229376,   294912,   819200,   884736,  1605632,
 2654208,  4096000,  7962624, 11239424, 20480000, 23887872,
    3m25.92s real     0m2.60s user     0m18.94s system
# echo $?
0
#

(Actually, sudo wasn't needed in the above example, as can be determined by the prompt being a # instead of a $.)

Here is another example. (A newer version of OpenBSD was used, and the results of the time command look a bit different.)

$ sudo time newfs_ext2fs -D 256 -O 1 -V 4 -v chosenVolumeName sd0k
/dev/rsd0k: 524285.3MB (1073736405 sectors) block size 4096, fragment size 4096
        using 4096 block groups of 128.0MB, 32768 blks, 16384 inodes.
super-block backups (for fsck_ext2fs -b #) at:
    32768,     98304,    163840,    229376,    294912,    819200,    884736,
  1605632,   2654208,   4096000,   7962624,  11239424,  20480000,  23887872,
 71663616,  78675968, 102400000,
      859.24 real         9.41 user        73.69 sys
$ echo $?
0
$

Now that the filesystem is created, install software called e2fsprogs and then adjust/tune the filesystem volume.

Also, that package may provide a mklost+found command. Using that command may be good to use that command to make the lost+found/ directory. (For few further details, see man page for mklost+found.)

Creating an Ext3 filesystem volume

Much of this is similar to creating an Ext2 filesystem volume. (The information is not redundantly placed in this section.)

Perhaps see also: Ext4 Wiki @ Kernel.org: Considerations when creating ext3 filesystems

[#e2fsck]: Testing/Repairing Ext filesystem volumes

Preferably before potentially allowing a test to run (which may occur automatically when mounting, e.g. during a system reboot), be familiar with the danger of substantial data loss posed by mixing e2fsprogs with other software designed to handle Ext2 filesystems. This can happen rather automatically when the filesystem volume is mounted, including when the computer is rebooted.

Testing Ext may be supported by internal tools described by the section on testing/repairing filesystems.

It may make sense, before repairing, to use commands like e2image.

The e2fsprogs package may come with a program called e2fsck. For the most part, the basic purpose of the e2fsck is similar to using the fsck command, which is meant to test/repair the filesystem volume. However, using e2fsck -c will run badblocks, which is meant to perform hardware testing on a disk drive. Similarly, e2fsck -cc will run badblocks -n. (It does not appear that e2fsck has any parameters for requesting badblocks to destroy the filesystem volume's data by using a destructive write test.) So, if those parameters are used, the effect is different than performing a filesystem check like what fsck does.

The e2fsck command from e2progs does seem to support more parameters, such as using “ -l filename ” to use a file that was previously created by “ badblocks -o filename ”. (As such a file may be disk-specific, it makes sense to name these files after something like the disk device, so that other files can be used for other disks.) This may be an advantage to using the e2fsck software manually, rather than relying on some of the built-in support for performing some automatic checking of filesystem volumes.

Mounting an Ext filesystem

Note that although Ext2 drivers may often be able to mount an Ext3 filesystem, doing so may not be good if the Ext3 filesystem was dismounted uncleanly. Specifically, mounting it with read/write may cause data loss that could be prevented simply by mounting the drive with proper Ext3 Journal support (and then dismounting the drive cleanly).

For general details on mounting, see mount points.

FSDEXT2 supported read-only in Win95. This did not work on some machines, and did not support the newer style of extended partitions used by Win98. The author references support for Win98 being added, but the referenced web page is down: Archive of page for FDSEXT2 0.163 shows some info.

[#extfstdr]: Data recovery with Ext filesystems

This section contains some of the details specific to the Ext family of filesystem formats. For more information about recovering data, including warnings and other details that may help recovery be more likely to succeed, also be familiar with data recovery basics. There are some common warnings/approaches to take which should be followed (but which, to minimize redundancy, are not duplicated here.)

[#e2fsrcov]: Data recovery with Ext2 filesystem types

See: Official Guide to using TestDisk to undelete from ext2.

Using debugfs

Another method may be to use debugfs to recover an inode. This program is part of the e2fsprogs software package, and many systems that use the Linux kernel may have this software package installed by default. A “lsdel” (or “list_deleted_inodes” option may help to locate an iNode to remove. Then: “undelete inodeNumber outputFilename” (e.g.: “undelete 25728 frominod”). Other commands that might be helpful are “lr” (or “list_requests”), “help”, or “?”.

[#e3fsrcov]: Data recovery with Ext3 filesystems
According to Carlo Wood's article on Undeleting from Ext3, recovering from the main part of the Ext3 file system volume might not be doable, but recovering a recently deleted file may be possible by using information from the journal. Although this has been known to be done successfully, it may be time consuming (taking up a person's time) and should not be relied upon. The drive also should not be used (especially in a read-write state) while this time consuming work is done. The related software that was written to help with this process is called ext3grep.

If the above techniques are not available, then there is probably not going to be an easy way to recover the data. If the data is possible, recovery may take skills similar to being a software developer of filesystem drivers. This is not a task for the faint at heart. However, for anyone wanting to explore such a process, know that the e2fsprogs software package contains some software that might be helpful to get some information: dumpe2fs and/or debugfs. (Note that the process of using this information might not be anywhere close to easy.)

[#e2frag]: Filesystem volume fragmentation

See also: optimizing volumes.

First, a quick note for users of Microsoft Windows, MS-DOS, and probably other similar operating systems: Note that disk fragmentation may not be as big of an issue with Unix and similar filesystems. Many professions may choose not to try to defragment the disk at all. The reason this ends up being acceptable has to do with the filesystem driver implementations placing file data on the drive in a way that is less likely to cause fragmentation in the first place.

Ext2Resize FAQ has some details on this: “Stephen Tweedie's defrag tool only works on 1kB block filesystems, so it is generally not useful anymore. Andrew Morton has written a patch to provide the hooks for a defragger which can run while the filesystem is mounted, but it needs a user-space tool.” (This quoted information might be outdated, but this guide currently does not have more updated information.)