garbage collection 介紹

Download Garbage collection 介紹

Post on 15-Jan-2015

756 views

Category:

Technology

3 download

Embed Size (px)

DESCRIPTION

Garbage collection 介紹

TRANSCRIPT

  • 1. Garbage collection

2. 2013/04 taipei.py pdb : http://www.slideshare.net/ya790026/recoverpdb 2013/05 pyconf.tw CPython :http://www.slideshare.net/ya790026/c-python23247730 2013/08 taipei.py python :http://www.slideshare.net/ya790026/python27854881 3. Garbage Collection memory leak, dangling pointer Reference count Mark and sweep 4. memory leakdangling pointermemorymemorymemorymemoryfree 5. Reference Counting Reference count is maintained for each object on the heap. When an object is first created and a reference to it is assigned to a variable, the object's reference count is set to one. 6. Reference Counting When any other variable is assigned a reference to that object, the object's count is incremented. When a reference to an object goes out of scope or is assigned a new value, the object's count is decremented. 7. a = 5000 aa = 5000 b=a aba = 5000 b=a a = 3000baob_ival: 5000 ob_refcnt: 1ob_ival: 5000 ob_refcnt: 1ob_ival: 5000 ob_refcnt: 2ob_ival: 3000 ob_refcnt: 1 8. a = 5000 b=a a = 3000 b = 4000b ob_ival: 5000 ob_refcnt: 0a ob_ival: 4000 ob_refcnt: 1ob_ival: 3000 ob_refcnt: 1 9. Reference Counting Advantage: suitable for real-time environments where the program can't be interrupted for very long. Disadvantage: reference counting does not detect cycles. 10. a = [] b = [] a.append(b) b.append(a) aba = [] b = [] a.append(b) b.append(a) a = None b = None 11. mark and sweep 1. Find the root objects of the system. These are things like the global environment (like the __main__ module in Python) and objects on the stack. 2. Search from these objects and find all objects reachable from them. This objects are all "alive". 3. Free all other objects. 12. Two-Color Mark & Sweep Sweep FreeSweepWhiteBlackNewMark 13. Two-Color Mark & Sweep the algorithm is non-incremental (atomic collection) 14. Tri-Color Incremental Mark & Sweep Initially grey set is all the objects that are reachable from root references but the objects referenced by grey objects haven't been scanned yet. The white setis the set of objects that are candidates for having their memory recycled. The black set is the set of objects that can cheaply be proven to have no references to objects in the white set. 15. FreeSweepBlackMarkSweep After CheckWhiteBarrier backwardMark NewGrayBarrier ForwardMark 16. Tri-Color Incremental Mark & Sweep When there are no more objects in the grey set, then all the objects remaining in the white set have been demonstrated not to be reachable, and the storage occupied by them can be reclaimed. 17. Generational Collectors 1. Most objects created by most programs have very short lives. 2. Most programs create some objects that have very long lifetimes. A major source of inefficiency in simple copying collectors is that they spend much of their time copying the same long-lived objects again and again. 18. External Memory fragment Free memory is separated into small blocks and is interspersed by allocated memory. Although free storage is available, it is unusable because it is divided into pieces that are too small individually to satisfy the demands of the application. 19. External Memory fragment adel b del dbcdacdacWe cant create a variable with four blocks. 20. Compacting and copying Move objects on the fly to reduce heap fragmentation 21. aa table of object handlesObjectbbObject 22. stop and copy The heap is divided into two regions. Only one of the two regions is used at any time. Objects are allocated from one of the regions until all the space in that region has been exhausted. Find out live objects and copy them to the other region. Memory will be allocated from the new heap region until it too runs out of space 23. freeallocatedunusedallocatedunusedunusedCopy live objectsfreeallocated 24. Python garbage collection Python use both of reference count and mark and sweep. mark and sweep only work for containers for solving reference cycles. Containers mean list, dict, instance, etc. python mark and sweep c extentsion root object 25. Python mark and sweep 1. For each container object, set gc_refs equal to the object's reference count. 2. For each container object, find which container objects it references and decrement the referenced container's gc_refs field. 26. Python mark and sweep 3. All container objects that now have a gc_refs field greater than one are referenced from outside the set of container objects. We cannot free these objects so we move them to a different set. 4. Any objects referenced from the objects moved also cannot be freed. We move them and all the objects reachable from them too. 27. Python mark and sweep 5. Objects left in our original set are referenced only by objects within that set (ie. they are inaccessible from Python and are garbage). We can now go about freeing these objects. 28. 1 gc_refs: 1gc_refs: 12 gc_refs: 1gc_refs: 03 gc_refs: 1gc_refs: 0GC_TENTATIVE LY_UNREACHAB LE 29. 4 gc_refs: 1gc_refs: 1 30. 1 gc_refs: 1gc_refs: 12 gc_refs: 0gc_refs: 03 gc_refs: 0gc_refs: 0GC_TENTATIVE LY_UNREACHAB LE 31. 4 gc_refs: 0gc_refs: 0 32. Java Reference Strong reference SoftReference WeakReference PhantomReference 33. Soft Reference The garbage collector may reclaim the memory occupied by a softly reachable object. Its useful for cache. 34. Weak Reference The garbage collector must reclaim the memory occupied by a weakly reachable object. Canonicalizing mappings 35. Phantom Reference Similar with weak reference Whereas the garbage collector enqueues soft and weak reference objects when their referents are leaving the relevant reachability state, it enqueues phantom references when the referents are entering the relevant state. Establish more flexible pre-mortem cleanup 36. Python Reference Strong reference Weak referenceweakref.ref(object[, callback]) 37. Python gc gc.enable() gc.disable() c.isenabled() gc.collect([generation]) gc.set_threshold(threshold0[, threshold1[, threshold2]]) gc.get_count() gc.get_threshold() 38. Python gc gc.set_debug(flags) gc.get_referrers(*objs) gc.get_referents(*objs) gc.garbage 39. In [1]: import gc In [2]: gc.set_debug(gc.DEBUG_STATS) In [3]: gc.collect() gc: collecting generation 2... gc: objects in each generation: 159 2655 7538 gc: done, 10 unreachable, 0 uncollectable, 0.0123s elapsed. 40. >>> ... ... >>> >>> >>> >>> >>> >>> >>> >>>class Finalizable: def __del__(self): pass a = Finalizable() b = Finalizable() a.x = b b.x = a del a del b import gc gc.collect() 41. memory-bound threshold cpu-bound threshold gc gc 42. python gc python gc ExternalMemory fragment python gc atomic 43. New-Garbage-Collector for lua Garbage Collection gc module docs Details on Garbage Collection for Python python source code(Modules/gcmodule.c) 44. PyConf 45. Thank you