data storage organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf ·...
TRANSCRIPT
![Page 1: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/1.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Data Storage Organization
➢ It is clear we want to store data, in fact lots of it.
➢ We are given a disk which can hold several terabytes ofdata.
➢ How do we organize/structure this storage?
![Page 2: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/2.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Filing Cabinet
Ikea ERIK
![Page 3: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/3.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Directory Organization
➢ Filing cabinet.
➢ Drawers ↔ Disks / volumes.
- Folders ↔ Directories.
• Sheets of paper ↔ Files
![Page 4: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/4.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
![Page 5: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/5.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Link types
➢ Symbolic link
- New directory entry type.
- Contains file name (symbolic) of link target.
- Following this pointer: “resolving the link”.
➢ Hard link
- Duplicated directory entry.
- Directory entries point at “inodes”.
- In the “inode” a reference count is kept.
![Page 6: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/6.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
File System Mounting
![Page 7: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/7.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
File System Mounting
![Page 8: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/8.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Chapter 11: File system implementation
![Page 9: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/9.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Why Virtual File Systems?
➢ At different mount points, different types of filesystems may be mounted:
![Page 10: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/10.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Disk Block Allocation
![Page 11: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/11.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Chapter 12: Mass storage systems
![Page 12: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/12.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Modern hard drives
➢ Available up to ~10 TB.
![Page 13: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/13.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Solid State Disks
➢ These days available in different form factors, differentbuses (SATA, SAS, NVMe PCI).
![Page 14: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/14.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
SSD architecture➢ SSDs are built from collections of flash memories.
- The chips are organized into separate channels.- Communication on a channel can be interleaved, when one chip is busy,
we can continue communicating with another chip on that channel.➢ A microcontroller manages the flash memories and
communication with the host.
Source: K. Eshghi and R. Micheloni, SSD Architecture andPCI Express Interface, Springer 2013.
![Page 15: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/15.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
SSD architecture (2)
What is there to manage?
➢ Translation of host drive geometry (in particular for SATASSD) to SSD architecture.
- In fact, although the host thinks it is writing to contiguous blocks, thismay not be the case as blocks are dynamically remapped by the SSDmicrocontroller.
➢ Wear leveling
- Flash memory has a limited number of rewrite cycles before it breaks;so distribute the load uniformly.
➢ Bad block management
- Maintain a list of bad blocks; already initialized during testing at thefactory.
![Page 16: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/16.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
SSD architecture (3)
➢ The memories are grouped into pages.
➢ Pages are again combined into blocks.
➢ A collection of blocks forms the entire memory array.
Source: K. Eshghi and R. Micheloni, SSD Architecture andPCI Express Interface, Springer 2013.
![Page 17: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/17.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
SSD architecture (4)
➢ Read and write operations can be done at a page level.
- (So not at the level of individual memories!).
➢ Now here's an interesting catch:
- Before you can write a page, it MUST be empty.
- If it's not empty, it must first be erased.
- But: you can only erase entire blocks (!). (Hardware limitation)
➢ So, page overwrite implies:
- Read the entire block into a buffer.
- Erase the block.
- Write all pages back.
![Page 18: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/18.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
TRIM
➢ For each overwrite, many more write operations have tobe done (write amplification).
➢ This is not a very good idea, considering the limitednumber of cycles.
➢ What happens if a file is deleted?
- The OS only updates the file system data structures.
- It can rewrite the blocks when a new file is allocated there, not aproblem for magnetic disks.
- What does this mean for SSD?
![Page 19: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/19.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
TRIM (2)
➢ Say we have a block of which all pages, except one,contain data of deleted files.
➢ If we overwrite a page on this block, all pages arerewritten, also these blocks that contain deleted data.
➢ To alleviate this, the TRIM command was added to theATA command set (UNMAP in SCSI).
- OS support is needed!
- On file delete, the OS tells the SSD what blocks (pages on theSSD) were deleted. The SSD can then decide to no longerpreserve this data in future overwrites of neighboring pages.
![Page 20: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/20.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Tape silos
![Page 21: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/21.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
NFS vs. iSCSI
➢ NFS, CIFS, AFP
→ File-based systems. Remote host is accessed using requestsfor particular files. No knowledge about underlying file system.
➢ iSCSI
→ Block-based systems. Remote host is accessed using requestsfor particular disk blocks. Client determines file system.
![Page 22: Data Storage Organizationliacs.leidenuniv.nl/~rietveldkfd/courses/os2017/ch10-11-12-extra.pdf · Universiteit Leiden. Bij ons leer je de wereld kennen TRIM (2) Say we have a block](https://reader034.vdocuments.site/reader034/viewer/2022050109/5f473e7de591574be401fc32/html5/thumbnails/22.jpg)
Universiteit Leiden. Bij ons leer je de wereld kennen
Parity
➢ Odd parity
- 0 1 1 0 → 0 1 1 0 1
- Added parity bit causes number to the odd.
- We can close any bit and still recover, why?
➢ Even parity
- 0 1 1 0 → 0 1 1 0 0