This is an overview review of Microsoft ReFS (Resilient File System) along with some resource links. ReFS is part of some Windows operating system platforms including Server 2016.

Some context here is that review can mean an in-depth deep dive product or technology review, while another meaning is to simply to refresh what you may already know about ReFS. For this post, the focus is on the latter, that is a bit of overview that also functions as a refresh review of what you may already know. However click here to see how ReFS and NTFS compare.

Click here to read more about Windows Server 2016, Storage Spaces Direct (S2D), Storage Replica (SR) and other related topics (or click on the image below).

Windows Server 2016 Welcome Screen - Source Server StorageIOlab.com

What Is Microsoft ReFS and Why It Matters

Microsoft ReFS (Resilient File System) is part of Windows Servers 2012, 2012 R2, 2016 as well as Windows 8.1 and 10 platforms as an alternative to NTFS file system. ReFS is designed not only for resiliency, also for scaling volumes beyond 256 TBytes (NTFS) to 4.7 Zettabytes (ZB). Note files size for both NTFS and ReFS is 18 Exabytes (EB). Click here to view various ReFS and NTFS data services, feature functionality along with limits. Part of being resilient means that ReFS is able to provide more data integrity protection to guard against logical data corruption.

Note while ReFS is the future for Windows-based platforms, NTFS is not going away anytime soon, after all, FAT (File Allocation Table) volumes are still supported after how many decades of being around? ReFS has been around for several years having existed in earlier WIndows operating systems as an option, however with Server 2016, its status is promoted to a more prominent role with more features, data services and functionality.

ReFS data services, features and functionality include:

• Resiliency - Automatic detection and online repair of data corruption issues
• Online repair - Isolate faults to localized area of data corruption for repair enabling volumes to stay online
• Storage Spaces integration - Leverage mirror or parity spaces for automatic detect and repair via alternate data copies. Note that with Windows Server 2016 Microsoft also has introduced Storage Spaces Direct (S2D).
• Data salvage - Should a volume become corrupt with no alternate copy (you should still have a backup), ReFS removes corrupt data from name space on a live volume. This capability enables surviving data to stay accessible while isolating the fault to the corrupted or damaged data.
• Integrity streams - Checksums for metadata and optionally file data that enable ReFS to detect corruptions in a reliable way.
• Proactive error correction - Besides validating data before reads and writes, ReFS also has a background scrubber data integrity check of volumes. This capability enables ReFS to proactively detect latent or silent data corruption so that corrective repair of damaged data can occur.
• Real-time tiering - When combined with S2D maximizes performance and space capacity across performance and capacity tiers using NVMe, flash SSD and HDDs devices. Writes occur to the performance tier, with large chunks de-staged to capacity tier. Read acceleration enabled via cache. Can support all flash (e.g. performance NVMe and capacity TLC or other flash SSD) as well as hybrid mix of HDD and SSD configurations.
• Block cloning for dynamic workloads including server virtualization such as accelerating checkpoint merge operations.
• Sparse VDL (Valid Data Length) improves virtual machine (VM) operations reducing time needed to create fixed size VHDs from 10s of minutes to seconds.
• Variable storage allocation cluster size of 4KB (for most environments) and 64KB (for environments with larger sequential file processing needs).

ReFS Deployment Options

Microsoft ReFS deployment options include:

• Basic disk (HDD, and SSD) - Leverage applications or other resiliency and protection solutions.
• SAS drive enclosures with storage spaces provides more data protection including availability as well as integrity and accessibility. Leverages classic storage spaces mirroring and parity protection for increased resiliency and availability.
• Storage Spaces Direct (S2D) - Increased scalability, real-time tiering and cache server storage I/O performance (effectiveness) and capacity (efficiency) optimization. For increased resiliency adds block clone and sparse Valid Data Length (VDL) to boost VHDX file performance operations (create, merge, expand). For resiliency, built-in checksums, online repair as well as leverage alternate data copies combined with S2D to detect as well as correct both metadata as well as primary data corruption issues. Optimized for large-scale and virtualized application workloads.

Where To Learn More

For those of you not as familiar with Microsoft Windows Server and related topics, or that simply need a refresh, here are several handy links as well as resources.

• Benchmarking Microsoft Hyper-V server, VMware ESXi and Xen Hypervisors (Via cisjournal PDF)
• BrightTalk Webinar - Software-Defined Data Centers (SDDC) are in your Future (if not already here)
• BrightTalk Weibniar - Software-Defined Data Infrastructures Enabling Software-Defined Data Centers
• Choosing drives and resiliency types in Storage Spaces Direct to meet performance and capacity requirements (Via TechNet)
• Data Protection for Modern Microsoft Environments (Redmond Magazine Webinar)
• Deep Dive: Volumes in Storage Spaces Direct (Via TechNet Blogs)
• Discover Storage Spaces Direct, the ultimate software-defined storage for Hyper-V (YouTube Video)
• DUPLICATE_EXTENTS_DATA structure (Via MSDN)
• Block cloning on ReFS (Via TechNet)
• DISKSPD now on GitHub, and the mysterious VMFLEET released (Via TechNet)
• Erasure Coding in Windows azure storage (Via Microsoft)
• Fault domain awareness in Windows Server 2016 (Via TechNet)
• Fault tolerance and storage efficiency in Storage Spaces Direct (Via TechNet)
• FSCTL_DUPLICATE_EXTENTS_TO_FILE control code (Via MSDN)
• Gaining Server Storage I/O Insight into Microsoft Windows Server 2016 (StorageIOblog)
• General information about SSD at www.thessdplace.com and NVMe at www.thenvmeplace.com
• Get the Windows Server 2016 evaluation bits here
• Happy 20th Birthday Windows Server, ready for Server 2016?
• How to run nested Hyper-V and Windows Server 2016 (Via Altaro and via MSDN)
• How to run Nested Windows Server and Hyper-V on VMware vSphere ESXi (Via Nokitel)
• Hyper-converged solution using Storage Spaces Direct in Windows Server 2016 (Via TechNet)
• Hyper-V large-scale VM performance for in-memory transaction processing (Via Technet)
• Introducing Windows Server 2016 (Free ebook from Microsoft Press)
• Large scale VM performance with Hyper-V and in-memory transaction processing (Via Technet)
• Microsoft Resilient File System (ReFS) overview (Via TechNet)
• Microsoft S2D Software Storage Bus (Via TechNet)
• Microsoft Storage Replica (SR) (Via TechNet)
• Microsoft Windows S2D Software Defined Storage (Via TechNet)
• NVMe, SSD and HDD storage configurations in Storage Spaces Direct TP5 (Via TechNet)
• Microsoft Azure Stack overview and related material via Microsoft
• ReFS integrity streams (Via TechNet)
• ReFS and NTFS feature, data services and functionality comparisons (Via TechNet)
• ReFS and NTFS limits (speeds and feeds via TechNet)
• Resilient File System aka ReFS (Via TechNet)
• Server 2016 Impact on VDI User Experience (Via LoginVSI)
• Server and Storage I/O Benchmark Tools: Microsoft Diskspd (Part I
• Setting up S2D with a 4 node configuration (Via StarWind blog)
• Setting up testing Windows Server 2016 and S2D using virtual machines (Via MSDN blogs)
• SQL Server workload (benchmark) Order Processing Benchmark using In-Memory OLTP (Via Github)
• Storage IOPS update with Storage Spaces Direct (Via TechNet)
• Storage throughput with Storage Spaces Direct (S2D TP5 (Via TechNet)
• Storage Spaces Direct hardware requirements (Via TechNet)
• Storage Spaces Direct in Windows Server 2016 (Via TechNet with Video)
• Storage Spaces Direct – Lab Environment Setup (Via Argon Systems)
• Understanding Software Defined Storage with S2D in Windows Server 2016 (Via TechNet)
• Understanding the cache in Storage Spaces Direct (Via TechNet)
• Various Windows Server and S2D lab scripts (Via Github)
• Volume resiliency and efficiency in Storage Spaces Direct (Via TechNet Blogs)
• What's New in Windows Server 2016 (Via TechNet)
• Windows Server 2016 Getting Started (Via TechNet)
• Windows Server 2016 and Active Directory (Redmond Magazine Webinar)
• Server StorageIO resources including added links, tools, reports, events and more

What This All Means

Now is as good of time as any to refresh (or enhance) your knowledge of ReFS and its current capabilities particular if you are involved with Microsoft environments. On the other hand, if you are not involved with Microsoft, take a few moments to update your insight and awareness of ReFS, storage spaces, S2D and other related capabilities including Windows Servers converged (desegregated) and hyper-converged (aggregated) options to avoid working off of or with stale data.

Ok, nuff said, for now...

Cheers
Gs