FAT, HPFS, and NTFS: A Comprehensive Guide to Windows File Systems

Table of Contents

This article delves into the distinctions among the File Allocation Table (FAT), High Performance File System (HPFS), and NT File System (NTFS) as they were implemented under Windows NT. Understanding these differences, along with their respective advantages and disadvantages, is crucial for managing storage effectively on Windows NT systems. Each file system was designed with different goals and constraints in mind, reflecting the evolution of storage technology and operating system requirements during that era.

FAT Overview¶

FAT stands as the most straightforward file system supported by Windows NT. Its fundamental structure revolves around the File Allocation Table (FAT), a critical data structure situated at the beginning of the volume. For resilience against data loss, two copies of the FAT are typically maintained. Both the FAT tables and the root directory are placed in fixed locations, a design choice that facilitates the reliable locating of system boot files during the startup process.

Disk space under the FAT file system is organized into clusters, the size of which is dependent on the overall volume size. When a file is created, an entry is added to the directory. This entry records the starting cluster number where the file’s data begins. Within the FAT table itself, each cluster entry acts as a pointer, either indicating that it holds the final piece of data for a file or directing the system to the next cluster in the chain.

Keeping the FAT table current is vital but can be time-consuming. Frequent updates are necessary to accurately reflect file locations and space usage. However, updating the FAT involves repositioning the disk’s read heads to the logical track zero each time, contributing to slower performance, especially with numerous small file operations or on fragmented disks. A failure to regularly update the FAT can lead to inconsistencies and potential data loss or corruption.

The directory structure in FAT is relatively simple, lacking inherent organization beyond a linear list. Files are typically placed in the first available directory entry. Furthermore, FAT supports a limited set of file attributes: read-only, hidden, system, and archive. These attributes provide basic control over file visibility and backup status but lack sophisticated permissions or metadata capabilities.

FAT Naming Convention¶

FAT adheres to the traditional 8.3 file naming standard. This format allows for a filename of up to eight characters, followed by a period (.), and then an extension of up to three characters. Only characters from the ASCII set are permitted.

A filename must begin with either a letter or a number. It can include most characters, but specifically excludes the following symbols: . " / \ [ ] : ; | = ,. Using any of these characters can result in unexpected behavior or errors. Additionally, filenames cannot contain spaces under the strict 8.3 convention.

Certain names are reserved for system devices and cannot be used for files or directories. These include: CON, AUX, COM1, COM2, COM3, COM4, LPT1, LPT2, LPT3, PRN, NUL. All characters used in filenames are automatically converted to uppercase by the FAT file system.

Advantages of FAT¶

While Windows NT itself does not support undelete utilities that directly access hardware (which is how traditional MS-DOS undelete programs work), a key advantage of using a FAT partition is the potential for data recovery. If a file is deleted from a FAT partition and the system is restarted under MS-DOS, it might be possible to recover the file using an MS-DOS undelete utility, provided the space hasn’t been overwritten.

FAT is generally best suited for drives or partitions that are relatively small, typically under approximately 200 MB. This is because FAT has very low overhead compared to more complex file systems. The space required for the FAT table and root directory is minimal on smaller volumes, leaving more space available for user data and keeping the table small enough for reasonably fast access.

Its simplicity and low overhead make FAT a viable choice for boot partitions on systems where compatibility with MS-DOS or older operating systems might be necessary. The widespread compatibility of FAT also means it can be easily accessed by virtually any operating system or boot disk. For very small volumes, the overhead of more advanced file systems would consume a disproportionate amount of space.

Disadvantages of FAT¶

The FAT file system becomes less efficient as the size of the volume increases. For drives or partitions exceeding approximately 200 MB, performance degrades significantly. This performance hit is due in part to the increasing size of the FAT table, which must be scanned and updated more frequently on larger, potentially more fragmented volumes.

A major limitation of FAT under Windows NT is the complete lack of file-level security permissions. Unlike NTFS, FAT partitions cannot be secured using the standard Windows NT access control lists (ACLs). This makes FAT unsuitable for volumes containing sensitive data or for multi-user environments where controlling access to specific files and folders is necessary.

FAT partitions also have significant size limitations. Under Windows NT, a single FAT partition is limited to a maximum size of 4 Gigabytes (GB). When accessed via MS-DOS, this limit is often even smaller, typically 2 GB. These limitations become restrictive when dealing with large hard drives common even in the Windows NT era.

Furthermore, as volumes grow and files are created and deleted, FAT partitions become increasingly fragmented. Fragmentation requires the disk heads to move excessively to read a single file, drastically reducing read/write speeds. Defragmentation utilities are needed to maintain performance, adding administrative overhead.

HPFS Overview¶

The High Performance File System (HPFS) was introduced with OS/2 version 1.2, primarily to address the limitations of FAT concerning larger hard drives and the growing need for enhanced file management features. HPFS aimed to improve upon FAT’s directory organization, extend filename capabilities, and introduce rudimentary security features (though Windows NT’s implementation had limitations regarding security). It was a significant step towards more modern file system designs.

HPFS retains a hierarchical directory structure similar to FAT but introduces automatic sorting of directory entries based on filenames, which improves lookup speed. Filenames were dramatically extended, supporting up to 254 characters, including double-byte characters, offering much greater flexibility than the 8.3 FAT limit. HPFS also allowed files to have extended attributes, enabling the storage of additional metadata beyond the basic FAT attributes.

A key architectural change was the move from cluster-based allocation (as in FAT) to sector-based allocation. HPFS allocates space in physical sectors, which are typically 512 bytes. This smaller allocation unit significantly reduces internal fragmentation (slack space within the last allocation unit of a file) compared to FAT, especially on larger volumes where FAT clusters could become very large (e.g., 32KB or 64KB).

Directory entries under HPFS store more detailed information than FAT entries. Beyond basic attributes, they include creation, modification, and access timestamps. Instead of pointing directly to the first data cluster, HPFS directory entries point to an FNODE (File Node). The FNODE is a structure that can either contain the file’s data directly (for small files) or hold pointers to other structures that ultimately locate the file’s data sectors. This provides a more flexible way to manage file allocation.

HPFS employs strategies to improve sequential file access speed. It attempts to allocate as much of a file’s data as possible in contiguous sectors on the disk. This minimizes the disk head movement required to read or write large files sequentially. This design was beneficial for applications that frequently processed large files in order.

The file system organizes a drive into a series of 8-MB bands. Ideally, a file is contained within one of these bands. Located between these bands are 2KB allocation bitmaps. These bitmaps track the allocation status (free or used) of sectors within their corresponding band. This banding structure aims to improve performance by reducing the travel distance for the disk head when searching for free space or file data; instead of returning to logical track zero, it only needs to go to the nearest band’s allocation bitmap.

Special Data Objects in HPFS¶

HPFS utilizes a couple of unique, crucial structures located at fixed logical sectors, which are vital for the file system’s operation.

Super Block: This object resides in logical sector 16. It contains essential information about the HPFS volume, most notably a pointer to the FNODE of the root directory. The Super Block is a single point of failure; if it becomes lost or corrupted, for example, due to a bad sector at this specific location, the entire contents of the HPFS partition become inaccessible, even if the rest of the disk is physically sound. Recovering data from such a situation is extremely difficult, potentially requiring specialized tools or manual reconstruction efforts.

Spare Block: Located in logical sector 17, the Spare Block holds a table used for “hot fixes” and the Spare Directory Block. Hot fixing is a fault-tolerance mechanism supported by HPFS (and later NTFS). When the file system attempts to write data to a sector and detects a write error (indicating a bad sector), it immediately redirects the write operation to a different, known-good sector from a pool of spare sectors. The original bad sector is then marked as unusable, transparently to the application performing the write. This automated handling of bad sectors prevents error messages like the “Abort, Retry, or Fail?” prompts common in older FAT systems when encountering a read or write error on a bad sector.

Advantages of HPFS¶

HPFS offered improved performance and features compared to FAT, making it a better choice for the larger hard drives becoming prevalent in the early-to-mid 1990s. It was best suited for volumes in the intermediate size range, specifically recommended for drives between approximately 200 MB and 400 MB. Within this range, its performance benefits over FAT were most apparent, and its overhead was not yet prohibitive.

The extended naming capabilities and better directory organization provided significant usability improvements over FAT. Users were no longer constrained by the restrictive 8.3 format, allowing for more descriptive filenames. The sorted directories also sped up file lookups within large folders compared to FAT’s linear search. The sector-based allocation helped conserve disk space compared to the larger clusters of FAT on bigger volumes.

Disadvantages of HPFS¶

Despite its improvements over FAT, HPFS also had notable disadvantages, particularly in the context of Windows NT. HPFS has a higher overhead than FAT, making it less efficient for very small volumes (under approximately 200 MB). The structures required by HPFS, such as the Super Block, Spare Block, FNODES, and allocation bitmaps, consume a larger portion of the disk space on small partitions compared to FAT’s relatively simple structure.

Performance with HPFS could also degrade on very large volumes, particularly those exceeding approximately 400 MB. While better than FAT on mid-range drives, HPFS was not designed to scale effectively to the multi-gigabyte drives that would become standard. A significant drawback for Windows NT users was that HPFS partitions could not have security permissions set on files or directories using the Windows NT security model. This limited its suitability for secure server or workstation environments.

Perhaps the most critical disadvantage for long-term use under Windows NT was its limited support lifespan. HPFS was only fully supported by Windows NT versions 3.1, 3.5, and 3.51. Importantly, Windows NT 4.0 and later versions cannot access HPFS partitions natively. This lack of forward compatibility meant that users upgrading from older NT versions or OS/2 would need to convert their HPFS volumes to FAT or NTFS to access them in Windows NT 4.0 and beyond.

NTFS Overview¶

The NT File System (NTFS) was designed specifically for Windows NT, aiming to be a robust, reliable, and feature-rich file system suitable for high-end workstations and servers. From a user perspective, NTFS continues the hierarchical directory model seen in FAT and HPFS, with directories also being sorted like in HPFS. However, the underlying architecture of NTFS differs significantly, removing the dependence on fixed, “special” locations like FAT tables or HPFS Super Blocks. NTFS treats all components of the file system, including its own metadata, as files themselves.

NTFS was built with several key objectives:

1. Reliability: To withstand system failures and disk errors, minimizing data loss and the need for extensive disk repair utilities.
2. A Platform for Added Functionality: To provide a flexible base for future features like security, compression, encryption, and advanced metadata.
3. Support POSIX Requirements: To meet the standards necessary for compatibility with POSIX-compliant applications and environments.
4. Removal of the Limitations: To overcome the size constraints, performance bottlenecks, and lack of features inherent in FAT and HPFS.

Reliability¶

Reliability was a paramount design goal for NTFS. This was achieved through several mechanisms: recoverability, elimination of fatal single-sector failures associated with critical structures, and support for hot fixing.

NTFS is a recoverable file system because it uses a transaction logging mechanism. Changes to the file system’s metadata (like creating, deleting, or renaming files, or modifying directory structures) are recorded in a log file before they are applied to the main file system structures. If the system crashes before a transaction is completed, the log can be used upon restart to either finish the incomplete transaction or roll it back to the last consistent state. This is fundamentally different from CHKDSK on FAT or HPFS, which primarily check the consistency of pointers after a potential corruption has occurred. NTFS’s logging allows for much faster recovery upon system restart.

Unlike FAT or HPFS, which rely on critical structures (FAT table, Super Block) located at fixed points that could cause total volume failure if corrupted, NTFS avoids this vulnerability. All metadata, including the Master File Table (MFT), which is central to NTFS, is treated as files. The MFT itself is mirrored; depending on the volume size, multiple copies of critical MFT sectors are stored in different locations on the disk. This redundancy prevents a single bad sector from rendering the entire volume inaccessible.

Similar to HPFS, NTFS incorporates hot fixing. When a write error occurs due to a bad sector, NTFS automatically remaps the write to a different, good sector and marks the original sector as bad. This process is invisible to the user and application, ensuring data integrity and preventing disruptive error messages.

Added Functionality¶

NTFS was designed as a platform, making it highly extensible. It fully integrates with and supports the comprehensive Windows NT security model. This allows administrators to set granular access control permissions (read, write, execute, delete, change permissions, take ownership) on individual files and directories, providing a secure environment for multi-user and network scenarios, a capability entirely missing in FAT and HPFS under NT.

NTFS supports multiple data streams per file. While a typical file has one main data stream, NTFS allows for alternative data streams (ADS) to be associated with a file, storing additional information that isn’t part of the primary file content. This feature has various uses, including storing metadata or application-specific information. NTFS also allows users and applications to define and store their own custom attributes associated with files, further enhancing its flexibility for specialized applications.

POSIX Support¶

NTFS was designed to meet key requirements of the POSIX.1 standard, which was important for compliance in certain technical and government environments. This makes NTFS the most POSIX-compliant file system supported by Windows NT.

The POSIX.1 features supported by NTFS include:

• Case-Sensitive Naming: Unlike FAT and HPFS under Windows NT, which treat filenames case-insensitively (e.g., “FILE.TXT” is the same as “file.txt”), NTFS stores filenames preserving the case used during creation (e.g., “README.TXT”, “Readme.txt”, and “readme.txt” can be displayed differently). Although NT’s native Win32 subsystem generally accesses files case-insensitively by default for backward compatibility, the underlying NTFS structure supports and preserves case, and POSIX-compliant applications running on NT can utilize the case-sensitive lookup behavior.
• Additional Time Stamp: Beyond the creation and modification times supported by FAT and HPFS, NTFS adds a third timestamp: the time the file was last accessed. This is a requirement of POSIX.1 and useful for various system and application purposes, such as tracking file usage.
• Hard Links: NTFS supports hard links, which allow multiple directory entries (filenames) to point to the same underlying file data on the volume. These links can even exist in different directories. Hard links differ from shortcuts (which are just pointers to a path) or symbolic links (which weren’t a native feature until later Windows versions) because they directly reference the file’s data structure. If one hard link is deleted, the data remains accessible as long as other hard links to it exist.

Remove Limitations¶

NTFS dramatically expanded the limits on file and volume sizes. Theoretically, NTFS volumes and files can be up to 2^64 bytes, which is an enormous size (16 exabytes), far exceeding the practical storage capacities of the time and effectively removing size as a limitation imposed by the file system itself. This scalability ensures NTFS can handle storage needs for the foreseeable future.

To provide flexibility and portability across different hardware platforms, NTFS returned to the concept of allocation units larger than a single sector, similar to FAT’s clusters. However, unlike FAT or HPFS’s fixed sector size, NTFS allows the cluster size (the basic unit of allocation) to be defined as multiples of the underlying hardware’s physical sector size. This allows administrators to choose an allocation size appropriate for the volume size and typical file usage, optimizing performance and reducing slack space more effectively than HPFS’s fixed sector size.

All filenames in NTFS are based on Unicode characters, providing support for a wide range of international character sets. While NTFS supports long filenames (up to 255 characters), it also automatically generates an 8.3 compatible alias for each file for backward compatibility with older applications and systems. This ensures that files created on NTFS can still be accessed, albeit with a truncated name, by applications that only understand the 8.3 format. The 255-character limit includes the full path length in some contexts, though the practical limit for a single filename part is high.

Advantages of NTFS¶

NTFS is the recommended file system for most volumes under Windows NT, particularly those of approximately 400 MB or larger. Its performance does not degrade significantly as volume size increases, unlike FAT and HPFS. The sophisticated internal structure, better allocation strategies, and features like transaction logging and hot fixing contribute to sustained performance on large disks.

The robust recoverability features of NTFS mean that serious file system corruption is rare, and recovery after a system crash is typically very fast. The need to run lengthy disk repair utilities like CHKDSK is drastically reduced compared to FAT and HPFS. This significantly increases system uptime and reliability, which is crucial for servers.

The integrated security model is a major advantage, allowing fine-grained control over access to data. This is essential for implementing secure multi-user environments and protecting sensitive information from unauthorized access. Support for large volumes and files removes practical storage constraints encountered with FAT and HPFS.

Disadvantages of NTFS¶

NTFS has a higher overhead than FAT due to its advanced features and structures (like the MFT, log file, etc.). This overhead consumes a noticeable amount of disk space, making it less efficient for very small volumes, typically below 400 MB. On a 100 MB partition, for example, the space consumed by NTFS system files could be 4 MB or more, a larger percentage than FAT’s overhead on the same size volume.

At the time Windows NT was prevalent (NT 4.0 era), NTFS did not include built-in file-level encryption features (like the Encrypting File System - EFS, which was introduced later in Windows 2000). While NTFS security prevents unauthorized access when the NT operating system is running, a malicious user could potentially boot the computer using another operating system (like MS-DOS or a Linux live CD) or specialized disk tools and bypass the NTFS permissions to view or copy data directly from the disk sectors, assuming the data wasn’t encrypted by other means. This highlights that NTFS security relies on the integrity of the NT operating system environment.

NTFS cannot be used to format floppy disks. The minimum overhead required for NTFS structures is larger than the capacity of a standard floppy disk. Therefore, Windows NT formats all floppy disks using the FAT file system to ensure compatibility and practicality.

NTFS Naming Conventions¶

NTFS filenames are much more flexible than those under FAT or HPFS. File and directory names can be up to 255 characters long, including any extensions. While NTFS preserves the case of the characters used when a file is created, it is generally case-insensitive when performing lookups by default in Windows NT’s Win32 subsystem for compatibility reasons. This means “MyFile.txt”, “myfile.txt”, and “MYFILE.TXT” can coexist if created with different casing (POSIX subsystem allows this), but a standard Win32 OpenFile operation for “myfile.txt” will typically find “MyFile.txt”.

Filenames can contain almost any character, including spaces and periods, except for a few reserved symbols that have special meaning: ? " / \ < > * | :. Although the maximum theoretical length for a filename component is 255 characters, practical limits, such as the maximum path length limit in Windows, can sometimes make it difficult to work with filenames approaching this limit in certain applications or command-line tools.

Comparing File Systems in Windows NT¶

To summarize the key characteristics and differences between FAT, HPFS, and NTFS under Windows NT:

Feature	FAT	HPFS	NTFS
Best Volume Size	< 200 MB	200-400 MB	> 400 MB
Performance (Large Vol)	Poor (degrades)	Fair (degrades somewhat)	Excellent (consistent)
Security (NT Perms)	No	No	Yes (Full NT ACL support)
Naming Convention	8.3 (Uppercase, ASCII)	Up to 254 chars (Case-preserving/sensitive)	Up to 255 chars (Case-preserving/sensitive)
Allocation Unit	Clusters (size varies)	Sectors (512 bytes)	Clusters (size varies, configurable)
Max File/Volume Size	2GB/4GB (MS-DOS/NT)	Approx 4GB	2^64 bytes (effectively unlimited)
Recoverability	Poor (requires external tools)	Limited (Super Block vulnerability)	Excellent (Transaction Logging)
Hot Fixing	No	Yes	Yes
Fixed Critical Locations	FAT Table, Root Directory	Super Block (Sector 16), Spare Block	No (MFT is file, mirrored sections)
POSIX Support	Limited	Limited	Good (Case-sensitive, timestamps, hard links)
Supported by NT 4.0+	Yes	No	Yes
Overhead on Small Vol	Low	Moderate	High
Floppy Support	Yes	No	No

This table highlights that while FAT offered basic compatibility, and HPFS provided some improvements for mid-range drives, NTFS was the file system truly designed to leverage the power, reliability, and security features of the Windows NT operating system architecture, making it the default and recommended choice for most modern volumes on this platform.

For further detailed information on these file systems within the Windows NT environment, consulting the Windows NT Resource Kit is highly recommended. These resources provide in-depth technical explanations and guidance for planning and managing disk storage.

What are your experiences working with FAT, HPFS, or NTFS volumes under Windows NT? Do you remember encountering specific advantages or disadvantages in practice? Share your thoughts and memories in the comments below!