clr: garbage collection inside out maoni stephens fun421 software design engineer microsoft...

CLR: Garbage Collection Inside CLR: Garbage Collection Inside OutOut

Maoni StephensMaoni StephensFUN421FUN421Software Design EngineerSoftware Design EngineerMicrosoft CorporationMicrosoft Corporation

AgendaAgenda

BasicsBasics

GenerationsGenerations

SegmentsSegments

AllocationAllocation

CollectionCollection

Large Object HeapLarge Object Heap

Different Flavors of GCDifferent Flavors of GC

PinningPinning

Tools and ResourcesTools and Resources

Generation 1Generation 1 Generation 0Generation 0

New Heap Begins with New GenerationNew Heap Begins with New GenerationAccessible References Keep Objects Alive Accessible References Keep Objects Alive Compacts Referenced ObjectsCompacts Referenced ObjectsObjects Promoted to Older Generation Objects Promoted to Older Generation New Allocations Rebuild New Generation New Allocations Rebuild New Generation

BasicsBasics

AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning


Generational GCGenerational GC

Three generationsThree generations

Most objects die in gen0Most objects die in gen0

Gen1 holds in the in flight dataGen1 holds in the in flight data

Gen2 holds the long lived dataGen2 holds the long lived data

Large object heapLarge object heap

Gen0 and Gen1 – ephemeral Gen0 and Gen1 – ephemeral generationsgenerations

AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning


GC Heap SegmentsGC Heap Segments

Unit of VirtualAllocUnit of VirtualAlloc

When CLR is loaded, initial segments When CLR is loaded, initial segments are allocatedare allocated

Additional segments reserved as Additional segments reserved as neededneeded

Committed/Decommitted as needed Committed/Decommitted as needed in the segmentin the segment

Deleted when not in useDeleted when not in use

VM hoarding feature in next CLR 1.1 VM hoarding feature in next CLR 1.1 SP and CLR 2.0 – SP and CLR 2.0 – STARTUP_HOARD_GC_VM startup flagSTARTUP_HOARD_GC_VM startup flag

Generations And Generations And SegmentsSegments

Start from the ephemeral segmentStart from the ephemeral segment

Ephemeral segment becomes a gen2 Ephemeral segment becomes a gen2 segmentsegment

Newly reserved segment becomes Newly reserved segment becomes ephemeralephemeral

Many gen2 segments, only one Many gen2 segments, only one ephemeralephemeral

Before GC #1Before GC #1

Gen1Gen1 Gen0Gen0


Gen2Gen2

Gen2Gen2

Gen2Gen2 Gen1Gen1 Gen0Gen0

Gen0Gen0




Before GC Before GC #100#100

Gen2Gen2


AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning



Cheap lock on UP; lock free on MPCheap lock on UP; lock free on MP

Moving a pointer forwardMoving a pointer forward

Clearing the memory for new objectsClearing the memory for new objects

Register for finalization if applicableRegister for finalization if applicable

Objects allocated together are Objects allocated together are close togetherclose together

AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning


CollectionCollectionWhenWhen

AllocationAllocation. Not enough space in . Not enough space in gen0.gen0.

Induced GCInduced GC. By calling . By calling System.GC.Collect().System.GC.Collect().

System pressureSystem pressure. .

CollectionCollectionHowHow

Suspend managed threadsSuspend managed threads

GCGC

Resume managed threadsResume managed threads

Two phases of GCTwo phases of GCMarkMark

CompactCompact

RootsRoots

StackStack

Handle tableHandle table

StaticsStatics

Older generation(s)Older generation(s)

Finalizer queueFinalizer queue

AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning



For objects that are 85,000 bytes or For objects that are 85,000 bytes or largerlarger

Compacting large objects costs a lotCompacting large objects costs a lotSo we only sweep (freelist)So we only sweep (freelist)

Many LOH segmentsMany LOH segments

Segments are handled similarly to Segments are handled similarly to small object heap segmentssmall object heap segments

Collection happens during gen2 GCsCollection happens during gen2 GCs

Collection – CostCollection – Cost

GC takes time – “% time in GC” GC takes time – “% time in GC” countercounter

If objects die in gen0 (survival rate is If objects die in gen0 (survival rate is 0) it’s the ideal situation0) it’s the ideal situation

The longer the object lives before The longer the object lives before being dead, the worse (with being dead, the worse (with exceptions)exceptions)

Gen0 and gen1 GCs should both be Gen0 and gen1 GCs should both be relatively cheap; gen2 GCs could cost relatively cheap; gen2 GCs could cost a lota lot

LOH – different cost modelLOH – different cost modelTemporary large objects could be badTemporary large objects could be bad

Should reuse if possibleShould reuse if possible

AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning


Concurrent GCConcurrent GC

Why do we have concurrent GCWhy do we have concurrent GCFor interactive applicationsFor interactive applications

How it’s doneHow it’s doneTrade some CPU and memory for shorter Trade some CPU and memory for shorter pause timepause time

Only for gen2Only for gen2

Done on a concurrent GC threadDone on a concurrent GC thread

How do you get concurrent GCHow do you get concurrent GCOn by defaultOn by default

Can be turned off via hosting or configCan be turned off via hosting or config

Server GCServer GC

Why do we have server GC – for server Why do we have server GC – for server apps thatapps that

Have a fairly consistent number of requestsHave a fairly consistent number of requests

Require high scalibility and high throughputRequire high scalibility and high throughput

How it’s doneHow it’s doneOne thread for each CPU, running at highest One thread for each CPU, running at highest prioritypriority

One ephemeral segment per CPUOne ephemeral segment per CPU

How do you get server GCHow do you get server GCOnly on via config or hostingOnly on via config or hosting

Hosts like ASP.NET and SQLCLR use server GCHosts like ASP.NET and SQLCLR use server GC

In CLR 1.1 it’s in mscorsvr.dll; in CLR 2.0 it’s in In CLR 1.1 it’s in mscorsvr.dll; in CLR 2.0 it’s in mscorwks.dllmscorwks.dll

WKS GCWKS GC SVR GCSVR GC

Where it runsWhere it runs On the user On the user thread that thread that triggered GCtriggered GC

On the GC On the GC threads running threads running at highest at highest prioritypriority

On a multi proc On a multi proc machinemachine

One small object One small object heap + One LOHheap + One LOH

N small object N small object heaps + N LOHsheaps + N LOHs

On a uni proc On a uni proc machinemachine

WKS GC + WKS GC + concurrent GC (if concurrent GC (if not turned off)not turned off)

WKS GC + WKS GC + concurrent GC concurrent GC OFFOFF

AgendaAgenda

BasicsBasics


SegmentsSegments





PinningPinning


PinningPinning

Why do we need pinning Why do we need pinning Interop with unmanaged codeInterop with unmanaged code

How objects get pinnedHow objects get pinnedUse of GCHandle of type Use of GCHandle of type GCHandleType.PinnedGCHandleType.Pinned

Allowed by language syntax, eg. fixed in Allowed by language syntax, eg. fixed in c#c#

Args get pinned by the interop frameArgs get pinned by the interop frame

Fragmentation problemFragmentation problem

After N more GCsAfter N more GCs

Gen1Gen1 PP PP

After GC XAfter GC XGen2Gen2 PP PP

Gen1 Gen1 startstart

Before GC XBefore GC XGen1Gen1


PP PP


Gen2Gen2 PP PP

Fragmentation Problem Caused By Fragmentation Problem Caused By PinningPinning



Without Without demotiondemotion


Gen2Gen2


PP PP

With demotionWith demotion


Gen2Gen2


PP PP

Before GCBefore GC After GCAfter GC

Gen2Gen2Seg0Seg0

Seg1Seg1

Gen2Gen2Seg2Seg2

Gen2Gen2Seg3Seg3

Gen2Gen2

Gen1Gen1EphEph PP Gen0Gen0 PP

Without Segment ReuseWithout Segment Reuse

Gen2Gen2Seg0Seg0

Seg1Seg1

Gen2Gen2Seg2Seg2

Gen2Gen2Seg3Seg3

Gen2Gen2

Old Old EphEph

Gen2Gen2 PP PP

Gen1Gen1New New EphEph

Before GCBefore GC After GCAfter GC

Gen2Gen2Seg0Seg0

Seg1Seg1

Gen2Gen2Seg2Seg2

Gen2Gen2Seg3Seg3

Gen2Gen2

Gen1Gen1EphEph PP Gen0Gen0 PP

With Segment ReuseWith Segment Reuse

Gen2Gen2Seg0Seg0

Seg1Seg1

Gen2Gen2Seg2Seg2

Gen2Gen2

Gen2Gen2 PP PPOld Old EphEph

Old Seg3, Old Seg3, New EphNew Eph

Gen2Gen2 Gen1Gen1

What The User Code Can What The User Code Can Do To HelpDo To Help

Best patternsBest patternsPinning for a short time is cheap.Pinning for a short time is cheap.

Create pinned buffers at the beginning that will Create pinned buffers at the beginning that will stay in regions that are very compactedstay in regions that are very compacted

Create pinned buffers that stay together Create pinned buffers that stay together instead of scattered aroundinstead of scattered around

TechniquesTechniquesAllocate a pinned big buffer (byte array on the Allocate a pinned big buffer (byte array on the LOH), give out a chunk at a timeLOH), give out a chunk at a time

Allocate a pool of small buffers and hand one Allocate a pool of small buffers and hand one out when neededout when needed

void M(byte[] b)void M(byte[] b){{ // if the buffer is already in gen2 it’s unlikely to move.// if the buffer is already in gen2 it’s unlikely to move. if(GC.GetGeneration(b) == if(GC.GetGeneration(b) == GC.MaxGeneration || GC.MaxGeneration || // avoid copying if the buffer is too big// avoid copying if the buffer is too big b.Length > 8 * 1024) b.Length > 8 * 1024) {{ RealM(b);RealM(b); }}

// GetBuffer will allocate one if none is free.// GetBuffer will allocate one if none is free. byte[] TempBuffer = Pool.GetBuffer();byte[] TempBuffer = Pool.GetBuffer(); RealM(TempBuffer);RealM(TempBuffer); CopyBackToUserBuffer(TempBuffer, b);CopyBackToUserBuffer(TempBuffer, b); Pool.Release(TempBuffer);Pool.Release(TempBuffer);}}

Managed CachesManaged Caches

Design patternsDesign patternsDon’t do it in the finalizerDon’t do it in the finalizer

Use combination of weak and strong references Use combination of weak and strong references – could convert strong to weak after certain – could convert strong to weak after certain periodperiod

Cache tuningCache tuningHit rateHit rate

Cost to add an itemCost to add an item

Cost to find an itemCost to find an item

Frequency of cleanupFrequency of cleanup

If not implemented carefully could easily cause If not implemented carefully could easily cause more gen2 collections than neededmore gen2 collections than needed

// sweep the cache for dead objects and compacts the list.// sweep the cache for dead objects and compacts the list.static void sweep_WeakRef()static void sweep_WeakRef(){{ // don't bother if no new collection happened since // don't bother if no new collection happened since // we swept last// we swept last if (GC.CollectionCount(0) != gen0_count)if (GC.CollectionCount(0) != gen0_count) gen0_count = GC.CollectionCount (0);gen0_count = GC.CollectionCount (0); elseelse return;return;

lock (list) {lock (list) { for (int i = 0; i < WeakRef_fill;) {for (int i = 0; i < WeakRef_fill;) { if (list[i].Target == null) {if (list[i].Target == null) { // install the last element in the free slot.// install the last element in the free slot. WeakRef_fill--;WeakRef_fill--; if (i != WeakRef_fill) {if (i != WeakRef_fill) { list[i] = list[WeakRef_fill];list[i] = list[WeakRef_fill]; list[WeakRef_fill] = null;list[WeakRef_fill] = null; } } }} elseelse i++;i++; }} }}}}

Why We Don’t Allow Many Why We Don’t Allow Many Customized SettingsCustomized Settings

Only allow very few settings specified Only allow very few settings specified via hosting API or configvia hosting API or config

Advantages for GC to do the tuning Advantages for GC to do the tuning for youfor you

GC has intimate knowledge of memory GC has intimate knowledge of memory when applications runwhen applications run

GC is tested on many platforms + many GC is tested on many platforms + many configurationsconfigurations

Looking AheadLooking AheadChallenges on 64-bitChallenges on 64-bit

More common as people start using More common as people start using 64-bit machines64-bit machines

VM is practically unlimited so physical VM is practically unlimited so physical memory is the limitmemory is the limit

Servers like the SQL Server tend to Servers like the SQL Server tend to not allow pagingnot allow paging

Gen2 could take a long timeGen2 could take a long time

Tools And ResourcesTools And Resources

http://blogs.msdn.com/maoni/http://blogs.msdn.com/maoni/

CLRProfilerCLRProfiler

.NET CLR Memory performance .NET CLR Memory performance counterscounters

The SoS Debugger Extension for The SoS Debugger Extension for windbg/cdbwindbg/cdb

vadumpvadump

Task ManagerTask Manager

clr: garbage collection inside out maoni stephens fun421 software design engineer microsoft...

Documents