THIN PROVISIONING AND OFFLOADED DATA TRANSFER

A Joint Effort of the Storage Industry

A

The process to deliver a Windows application based on the industry standard

• Start the project blueprint with a group of key members

• Present the project proposal at the standard committee

• Propose the standard spec and requirements with a working group

• Implement the approved standard proposals

• Develop the test tools according to the standard spec.

• Complete and deliver the finalized solutions with qualified products

KEY CHANGES COMING… (1/2)

Windows & Thin Provisioning

Thinly-Provisioned LUNs will be a first-class citizen

– Identification

• Providing mechanisms for identifying thinly provisioned LUNs throughout the OS

• Ability to query the ‘mapped/unmapped’ state of LUN extents

– Notification

• Exposing events to indicate when LUNs cross thresholds boundaries

• Events will be consumable by management applications

KEY CHANGES COMING… (1/2)

Windows & Thin Provisioning

Thinly-Provisioned LUNs will be a first-class citizen

Optimization

Providing end-to-end transparency of application and file system allocations

All the way from the application layer through to hardware

Unmap requests provided both real-time and scheduled basis

Regular slab consolidation to ensure smallest storage footprints

Compatibility

Windows Logo requirement & tests for arrays that support thin provisioning

END TO END PLUMBING OF UNMAP(TRIM) / THIN PROVISIONING

The Storage e2e

Application(Like SQL, VHD’s etc.)

NTFS

Storport

Storage

Optimizer

Storage Hardware

File Level TRIM API

Delete Notification (TRIM)

UnMAP SCSI Command

Identification

• Windows will provide mechanisms to discover thinly-provisioned LUNs

• Management applications will be able to discover this attribute via an associated IOCTLs

• Provides the indication that other Windows features can take advantage of the thinly-

provisioned LUN.

• Ability to query the ‘mapped/unmapped’ state of LUN extents

• Potentially useful for management applications

Notifications• Windows will provide two types of Thin Provisioning notifications:

• Threshold Notification – Used when usage threshold is crossed

• Logged to Event Log for consumption by admin or storage management application

• Resource Exhaustion Notification – Used to indicate when capacity is exhausted

• Temporary Resource Exhaustion – Expected to be a temporary state while additional

resources are coming online

• OS will retry for a set period in this condition

• Permanent Resource Exhaustion – Critical Event

• Write will fail and OS will take LUN offline

OptimizationProviding end-to-end transparency of application and file system allocations

All the way from the application layer through to hardware

Unmap notifications provided both real-time and scheduled basis

Scheduled ‘re-trims’ for storage arrays that don’t prefer real-time unmap

Scheduled slab consolidation to ensure smallest storage footprints

Move data from sparsely populated slabs to densely populated slabs to further

improve capacity utilization

Mapped/unmapped state is a key input for this optimization

Compatibility / Logo Requirement

Logo Requirements for Storage Array supports Thin Provisioning

• Thinly provisioned LUN identification – Supported VPD Page and LBP VPD Page

• UNMAP/Trim Support – UNMAP Logo test implementation

• Get LBA Mapping State Information - Get LBA Status

• Report LBA Mapping Resource Status – LBP Log Page

• Handle threshold and resource exhaustion conditions – Sense Key and Additional Sense Code

Windows Offloaded Data Transfer enables host server to interact with storage

device to perform data movement through the high-speed storage network.

Windows Offloaded Data Transfer

Server1

or

Hyper-V

VM1

Physical Disk, VHD or SMB Shared Disk

Server2

or

Hyper-V

VM2Client-Server

Network

Storage

Network

Data Movement

Offload

Read

Return

Token

Offload

Write

Return

Result

Physical Disk, VHD or SMB Shared Disk

Copy Offload

Application

Token

Windows Offloaded Data Transfer

Offloaded Data Transfer (ODX)

Logo Requirements

• ODX Capability and Identification• Supported VPD Pages VPD page

• Third Party Copy VPD Page (Page Code 8Fh) +

Block Device ROD Token Limits descriptor

• ODX command support• Populate Token

• Write Using Token

• Received ROD Token Information

• Token Management during ODX operation

• Zero Well Known Token Support

Thanks to the following Key Members

• Dell - Lazarus Vekiarides, Daniel Soloman

• EMC - David Black,

• HP-3Par - Mothias Popp, Curtis Ballard

• NetApp - Fred Knight, John Meneghini, Ranga Sankar

• IBM – Kevin Butt, David Hartman

• Fujitsu – Kun Katsumata

• Microsoft – Jaivir Aithal, Tristan Griffith, David Chang