BT

Introducing Windows New File System: ReFS

by Jonathan Allen on Jan 17, 2012 |

Nearly 20 years ago Microsoft introduced the New Technology File System as part of Windows NT. Since then NTFS has accumulated numerous features that, while promising at the time, turned out to be bad ideas. Meanwhile the state of the art has continued to advance in ways that are incompatible with the NTFS architecture. After a couple of failed starts, Microsoft is now promising a new file system.

Known as ReFS or Resilient File System, this new file system is designed to improve reliability, especially in the event of power loss or media degradation (i.e. the disk simply wearing out). Some of this reliability comes from fundamentally changing where and how file metadata is stored and updated. Rather than using in-place updates paired with a journal, metadata is updated using allocate-on-write semantics. If the power does go out during a write the file system simply continues using the old metadata entry rather than the partially written new one.

ReFS is compatible with the disk-spanning technology Storage Spaces. By itself Storage Spaces has several ways to improve performance and reliability, especially when dealing with catastrophic disk failure. Unfortunately there are subtler forms of data corruption that can occur. Using a checksum system, ReFS can detect when these errors occur. When that happens it then polls all of the Storage Spaces drives to determine which has a good copy of the file.

In order to proactively ensure drive and file integrity, ReFS will occasionally spot check files using a background process. This process compares the metadata and content of each copy of a file and, like in the example above, uses the good copies to correct the others. This is particularly important for rarely accessed files where corruption in one copy may not otherwise be caught before other copies happen to be corrupted as well.

Another reliability mechanism is what they are calling “integrity streams”. Just like metadata, integrity streams will use allocate-on-write semantics to reduce the chance that a failure while writing to disk will result in corruption of the only good copy of the file’s content. Integrity streams are not appropriate for all types of files; applications that require control over the physical file structure (e.g. databases) should disable this feature.

In order to improve compatibility ReFS has been built to run under the NTFS APIs with only the lowest levels swapped out. That said, there are numerous NTFS features that will not be supported in ReFS including:

  • Named streams
  • Object IDs
  • Short names
  • Compression
  • File level encryption (EFS)
  • User data transactions
  • Sparse Files
  • Hard-links
  • Extended attributes
  • Quotas

Other features such as “BitLocker encryption, Access Control Lists for security, USN journal, change notifications, symbolic links, junction points, mount points, reparse points, volume snapshots, file IDs, and oplocks” will still be supported.

The initial release of ReFS will be targeting file servers. This means it will only be available on Windows 8 Server and will not support bootable drives. Over time ReFS will be offered to client versions of Windows and eventually may be used as a boot drive. Given how dangerous flaws are when it comes to file systems, Microsoft felt that this go-slow approach would be more prudent.

One cannot perform an in-place conversion from NTFS to ReFS, a new drive must be created and the data copied across.

Hello stranger!

You need to Register an InfoQ account or to post comments. But there's so much more behind being registered.

Get the most out of the InfoQ experience.

Tell us what you think

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread
Community comments

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Allowed html: a,b,br,blockquote,i,li,pre,u,ul,p

Email me replies to any of my messages in this thread

Discuss

Educational Content

General Feedback
Bugs
Advertising
Editorial
InfoQ.com and all content copyright © 2006-2013 C4Media Inc. InfoQ.com hosted at Contegix, the best ISP we've ever worked with.
Privacy policy
BT