memory leak detection douglas boling president boling consulting inc

Memory Leak DetectionMemory Leak Detection

Douglas BolingDouglas BolingPresidentPresidentBoling Consulting Inc.Boling Consulting Inc.www.bolingconsulting.comwww.bolingconsulting.com

SpeakerSpeaker

Douglas Boling Douglas Boling dbolingmedc @ bolingconsulting.comdbolingmedc @ bolingconsulting.com

Author – “Programming MicrosoftAuthor – “Programming MicrosoftWindows CE 3Windows CE 3rdrd Edition” Edition”

Trainer – Classes on Trainer – Classes on Windows CE App DevelopmentWindows CE App Development

Windows CE OAL Windows CE OAL Development Development

Consultant – Work with companies to Consultant – Work with companies to help their help their

application and application and platform platform development effortsdevelopment efforts

MManagementanagementTToolsools

CCommunicationsommunications& & MMessagingessaging

Device Update Agent

Software Update Services

Live Communications Server

Exchange Server

Internet Security and Acceleration Server

Speech Server

Image Update

LLocation ocation SServiceservices

MMultimediaultimedia

MapPoint

DirectX

Windows Media

Visual Studio 2005DDevelopment evelopment TToolsools

MFC 8.0, ATL 8.0

Win32NNativeative

MManagedanaged

SServer erver SSideide

LLightweightightweight

RRelationalelationalSQL Server 2005 Express EditionEDB

DDa

taata

PPro

gra

mm

ing

ro

gra

mm

ing

MM

od

el

od

el

DDevice evice BBuilding uilding TToolsools

HHardware/ardware/DDriversrivers

Windows XP DDK

Windows Embedded Studio

Platform Builder

OEM/IHV SuppliedBSP

(ARM, SH4, MIPS)OEM Hardware and Standard Drivers

Standard PC Hardware and Drivers

SQL Server 2005SQL Server 2005 Mobile Edition

ASP.NET Mobile Controls ASP.NET

.NET Compact Framework .NET Framework

Microsoft Operations Manager

Systems Management Server

AgendaAgenda

Intro Intro

LMemDebugLMemDebug

Application VerifierApplication Verifier

CeLog DataCeLog Data

Memory Leaks…Memory Leaks…

……are deadlyare deadly

Embedded/Mobile devices run Embedded/Mobile devices run constantlyconstantly

They aren’t treated like a PCThey aren’t treated like a PC

System’s are designed with just System’s are designed with just enough memory to do the jobenough memory to do the job

More cost too muchMore cost too much

AgendaAgenda

LMemDebugLMemDebug



LMemDebugLMemDebug

Monitors heap APIsMonitors heap APIs

Provided in Platform BuilderProvided in Platform Builder

Access through ‘Shell’ toolAccess through ‘Shell’ tool

Programmatic LMemProgrammatic LMem

Applications can control LMemDebugApplications can control LMemDebugAppend extra information to allocsAppend extra information to allocs

Set flags per allocationsSet flags per allocations

Set flags globallySet flags globally

Applications can tag memory allocsApplications can tag memory allocs

HeapAllocTrace HeapAllocTrace Easily tag allocs with file and line numberEasily tag allocs with file and line number

Using LMemDebugUsing LMemDebug

Use Shell to query memory statusUse Shell to query memory status

LMem <proc>|* [recent] [hist] [chk] [delta] [info] [<heap>|*] [breaksize <NN>] [breakcnt <NN>] proc : Proc index, name, PID or * (for all processes) recent : Show list of recent allocations hist : Show histogram of alloc size chk : Checkpoint all current allocations delta : Report allocations since checkpoint info : Report any additional info stored with the allocations (stack etc.) heap : Heap index, handle or * for all heaps breaksize <NN> : Break when size is NN breakcnt <NN> : Break when count is NN

LMem OutputLMem Output

Ptr=0x000302C0 Size=32768 Count=15 LineNum=724 File=C:\.. \41MemTest.C

Stack=0x03F7A3C2 coredll.dll+0x3A3C2 Stack=0x03F78BB6 coredll.dll+0x38BB6 Stack=0x1001205C 41MemTest.exe+0x1205C Stack=0x10011704 41MemTest.exe+0x11704 Stack=0x1001133A 41MemTest.exe+0x1133A Stack=0x03F6199C coredll.dll+0x2199C Stack=0x03F61F66 coredll.dll+0x21F66 Stack=0x10011092 41MemTest.exe+0x11092 Stack=0x10013DC4 41MemTest.exe+0x13DC4 Stack=0x03F6B6A0 coredll.dll+0x2B6A0

Output of one mem allocOutput of one mem allocShow alloc call file and line numberShow alloc call file and line number

Stack at time of alloc callStack at time of alloc call

Ptr=0x00030320 Size=212 Count=8 LineNum=3217 File=d:\jameson\private\winceos\osshell\commctrl\toolbar.cpp

Stack=0x03F7A3C2 coredll.dll+0x3A3C2 _HeapAllocTraceStack=0x03F78BB6 coredll.dll+0x38BB6 _LocalAllocTraceStack=0x03E52130 commctrl.dll+0x32130 ?ToolbarWndProc@@...Stack=0x03F647C0 coredll.dll+0x247C0 _xxx_PerformCallBack4Stack=0x0A06D378 gwes.exe+0x6D378 [email protected]=0x0A035BB8 gwes.exe+0x35BB8 [email protected]=0x0A03ECEA gwes.exe+0x3ECEA ?RootToNull@CWindowManager@Stack=0x03F5F5D8 coredll.dll+0x1F5D8 _xxx_CreateWindowExWStack=0x03E49DD4 commctrl.dll+0x29DD4 _CommandBar_CreateStack=0x100114C6 41MemTest.exe+0x114C6 _DoCreateMain

LMem OutputLMem Output

Int_HeapAlloc

How LMem WorksHow LMem Works

ApplicationApplication

Kernel, FileSys,Device,GWESKernel, FileSys,Device,GWES

CoreDLLCoreDLL LMEMDebug.DLLLMEMDebug.DLL

LocalAlloc

HeapAlloc

HeapSize

HeapCreateHeapAllocHeapAllocTraceHeapReAllocHeapFreeHeapSizeHeapDestroy

How LMem WorksHow LMem Works


CoreDLLCoreDLL

Kernel, FileSys,Device,GWESKernel, FileSys,Device,GWES

Writing your Own LMemWriting your Own LMem

CoreDLL heap API has built in hooksCoreDLL heap API has built in hooks

At App init, coredll looks for At App init, coredll looks for LMemDebug.DLLLMemDebug.DLL

Loads the following entry pointsLoads the following entry pointsHeapCreateHeapCreate HeapDestroyHeapDestroy

HeapAllocHeapAlloc HeapAllocTraceHeapAllocTrace

HeapReAllocHeapReAlloc HeapFreeHeapFree

HeapSizeHeapSize

Writing Your Own LMemWriting Your Own LMem

Helper functions will do the workHelper functions will do the workInt_HeapCreateInt_HeapCreate Int_HeapDestroyInt_HeapDestroy

Int_HeapAllocInt_HeapAlloc Int_HeapAllocTraceInt_HeapAllocTrace

Int_HeapReAllocInt_HeapReAlloc Int_HeapFreeInt_HeapFree

Int_HeapSizeInt_HeapSize

Only need to export functions you wantOnly need to export functions you want

You get the source to LMemDebug!You get the source to LMemDebug!

LMemDebug ImprovementsLMemDebug Improvements

Track fragmentation over timeTrack fragmentation over time

Track Page usageTrack Page usage

Track buffer overrunsTrack buffer overruns

Improved UIImproved UIReal time reporting to Dev PCReal time reporting to Dev PC

Graphical histogramsGraphical histograms

LMem SummaryLMem Summary

AdvantagesAdvantagesFairly quickFairly quick

Provides call stack info for allocationsProvides call stack info for allocations

Heap histogramHeap histogram

DisadvantagesDisadvantagesUses debug serial port as outputUses debug serial port as output

Monitors all applications Monitors all applications

Command line interfaceCommand line interface

AgendaAgenda

LMemDebugLMemDebug




Part of the Windows CE Test Kit Part of the Windows CE Test Kit

MonitorsMonitorsHeap allocationsHeap allocations

Handle allocationsHandle allocations

GDI ResourcesGDI Resources

More…More…If you knew how to write a shimIf you knew how to write a shim

Using the App VerifierUsing the App Verifier

Can be used many waysCan be used many waysLocally on the deviceLocally on the device

Using the Platform Builder Shell toolUsing the Platform Builder Shell tool

From a development PC via EthernetFrom a development PC via Ethernet

Output is a log file that is placed on the Output is a log file that is placed on the devicedevice

App Verifier from the PCApp Verifier from the PC

App Verifier from the PCApp Verifier from the PC

AdvantagesAdvantagesEasier to control from PCEasier to control from PC

Automatically brings data to PCAutomatically brings data to PC

DisadvantagesDisadvantagesNeeds ActiveSync or Ethernet connectionNeeds ActiveSync or Ethernet connection

App Verifier on the DeviceApp Verifier on the Device

App Verifier on the DeviceApp Verifier on the Device

AdvantagesAdvantagesNo need for a connection with a PCNo need for a connection with a PC

DisadvantagesDisadvantagesRequires a device U/IRequires a device U/I

Dialog needs a 800x600 screenDialog needs a 800x600 screen

App Verifier with ShellApp Verifier with Shell

Application verifier can be run as a Application verifier can be run as a ‘Shell’ extension‘Shell’ extension

At the shell promptAt the shell promptLoad the extensionsLoad the extensions

To ready the shim for the applicationTo ready the shim for the application

Run the applicationRun the application

Windows CE>loadext AppVerifSh.dll

Windows CE>loadext shim_heap.dll

Windows CE>appverif -m pword.exe -s shim_heap.dll

App Verifier with ShellApp Verifier with Shell

Advantages Advantages No device U/I requiredNo device U/I required

Can run from Can run from DeviceDevice

PC using Platform BuilderPC using Platform Builder

PC using a serial portPC using a serial port

DisadvantagesDisadvantagesNot a polished as the other methodsNot a polished as the other methods

How It WorksHow It Works

Inserts a ‘shim’ between the Inserts a ‘shim’ between the application and specific APIs exported application and specific APIs exported by the operating systemby the operating system

The shim intercepts the heap APIsThe shim intercepts the heap APIsCaptures the stack on every allocCaptures the stack on every alloc

Matches every free with an allocMatches every free with an alloc

Reports allocs that don’t have a freeReports allocs that don’t have a free

The Shim EngineThe Shim Engine


KernelKernel

CoreDLLCoreDLL

Shim 1Shim 1 Shim 2Shim 2

RegistryRegistry

Other DllsOther Dlls

The Shim EngineThe Shim Engine

Application Verifier uses the Shim Application Verifier uses the Shim Engine to monitor calls from the Engine to monitor calls from the application to the operating systemapplication to the operating system

The Shim Engine has hooks in the The Shim Engine has hooks in the kernel loaderkernel loader

When dynamic linking occurs, the shim When dynamic linking occurs, the shim engine looks in the registry to see if a engine looks in the registry to see if a shim should be linked insteadshim should be linked instead

App Verifier SummaryApp Verifier Summary

AdvantagesAdvantagesMonitors on a per process levelMonitors on a per process level

Uses .map file info for stack traceUses .map file info for stack trace

Flexible user interfaceFlexible user interface

DisadvantagesDisadvantagesSometime flaky connectivitySometime flaky connectivity

Not open to developer improvementNot open to developer improvement

AgendaAgenda

LMemDebugLMemDebug



CELog ToolCELog Tool

Generates data for the Kernel TrackerGenerates data for the Kernel TrackerImplemented as a kernel libraryImplemented as a kernel library

The raw data is in a known formatThe raw data is in a known format

Other tools use the same dataOther tools use the same dataReadLogReadLog

MemalyzerMemalyzer

CELog ZonesCELog Zones

CELog records information in many areasCELog records information in many areasCELZONE_INTERRUPT Events related to interrupts.

CELZONE_RESCHEDULE Events related to the scheduler.

CELZONE_MIGRATE Events related to migration of threads between processes.

CELZONE_TLB Events related to the translation look-aside buffer (TLB).

CELZONE_DEMANDPAGE Events related to paging.

CELZONE_THREAD Events related to threads, except for thread switches.

CELZONE_PROCESS Events related to processes.

CELZONE_PRIORITYINV Events related to priority inversion.

CELZONE_CRITSECT Events related to critical sections.

CELZONE_SYNCH Events related to synchronization.

CELZONE_HEAP Events related to heaps.

CELZONE_VIRTMEM Events related to virtual memory.

CELZONE_LOADER Events related to the loader.

CELZONE_MEMTRACKING Events related to memory tracking.

CELZONE_BOOT_TIME Events in the boot process

CELZONE_KCALL Events related to KCALLs.

Zones For Memory TrackingZones For Memory Tracking

By default, almost all zones are enabledBy default, almost all zones are enabledAll but CELZONE_KCALLAll but CELZONE_KCALL

This generates a huge amount of dataThis generates a huge amount of data

For memory trackingFor memory trackingTurn off all zones but Turn off all zones but

CELZONE_HEAP, CELZONE_VIRTMEM, CELZONE_HEAP, CELZONE_VIRTMEM, CELZONE_MEMTRACKINGCELZONE_MEMTRACKING

This dramatically reduces the information being This dramatically reduces the information being saved and flushedsaved and flushed

This is still a fair amount of informationThis is still a fair amount of information

CELog Zone ControlCELog Zone Control

Zones are set in the registryZones are set in the registryOn the deviceOn the deviceOn the PC (if attached via KITL)On the PC (if attached via KITL)

Using ShellUsing Shellmemtrack commandmemtrack command

ProgrammaticallyProgrammatically

dwZoneCE set the zones discusseddwZoneCE set the zones discussed

void CeLogSetZones (DWORD dwZoneUser, DWORD dwZoneCE,

DWORD dwZoneProcess);

Using CELogUsing CELog

Copy the following files to the deviceCopy the following files to the deviceCELog.DLLCELog.DLL

CELogFlush.EXECELogFlush.EXE

Start CELogFlush –z 8000 –n <file>Start CELogFlush –z 8000 –n <file>Enable memtracking events to <file>Enable memtracking events to <file>

Filename shouldn’t have spacesFilename shouldn’t have spaces

Run appRun appThen copy file.clg to PCThen copy file.clg to PC

MemalyzerMemalyzer

This tool is provided in Platform This tool is provided in Platform Builder but not documentedBuilder but not documented

Takes a CELog file as inputTakes a CELog file as input

Parses out the memory log infoParses out the memory log info

Outputs the results to .txt or .htm Outputs the results to .txt or .htm

Running MemalyzerRunning Memalyzer

USAGE: memalyzer.exe [-pid <process handle>, -name <process name>, -fullscan]

[-map <path>] [output_flags] <file name>

Choose one way to specify which process(es) to output:

-pid <handle> - specify process by handle

-name <name> - specify process by name

-fullscan - output data for all processes (DEFAULT)

-map <path> - specify location to read .map files from

Output flags are one or more of the following:

-v - verbose: print full stack frames (cooked output only)

-nounload - do not unload modules

-out:raw - print packets as logged

-out:cooked - print packets after processing

-out:noerror - suppress errors (such as duplicate loads/allocs)

-out:nomods - suppress output of module load/unload data

-out:html - output HTML files instead of plaintext

CELog SummaryCELog Summary

AdvantagesAdvantagesKernel level monitoringKernel level monitoring

Open data collection modelOpen data collection model

Memalyzer understands .map filesMemalyzer understands .map files

DisadvantagesDisadvantagesLogging is globally enabledLogging is globally enabled

The log buffer can be quite largeThe log buffer can be quite large

Logging is a performance hitLogging is a performance hit

Memalyzer isn’t formally supportedMemalyzer isn’t formally supported

SummarySummary

LMemDebug LMemDebug Great for heap only analysisGreat for heap only analysis

Source available Source available

App VerifierApp VerifierMemory, handle, GDI leak monitorMemory, handle, GDI leak monitor

CELog / MemalyzerCELog / MemalyzerHandy for detailed dumpsHandy for detailed dumps

Source availableSource available

QuestionsQuestions

dbolingmedc @ bolingconsulting.comdbolingmedc @ bolingconsulting.com

While at MEDC 2005…While at MEDC 2005…Fill outFill out an evaluation for this session an evaluation for this session

Randomly selected instant Randomly selected instant WINWIN prizes! prizes!

Use Use real technology in a labreal technology in a lab Instructor led Instructor led Reef E/FReef E/F & & Breakers LBreakers L

Self-paced Self-paced Reef B/CReef B/C

VisitVisit the Microsoft Product Pavilion the Microsoft Product Pavilion

in the Exhibit Hall in the Exhibit Hall Shorelines BShorelines B

After The Conference…After The Conference…

DevelopDevelop

BuildBuild

InstallInstall

BuildBuild

JoinJoin

InstallInstall

EnterEnter

JoinJoin

Full-featured trial versions of Windows CE Full-featured trial versions of Windows CE and/or Windows XP Embeddedand/or Windows XP Embedded

Cool stuff & tell us about it: Cool stuff & tell us about it: msdn.microsoft.com/embedded/community

Windows Embedded Partner Program:Windows Embedded Partner Program:www.mswep.com

Windows Mobile 5.0 Eval Kit including Windows Mobile 5.0 Eval Kit including Visual Studio 2005 Beta 2Visual Studio 2005 Beta 2

Mobile2Market Contest and win up to $25000: Mobile2Market Contest and win up to $25000: mobile2marketcontest.com

Microsoft Solutions Partner Program:Microsoft Solutions Partner Program:partner.microsoft.com

http://www.msdn.microsoft.com/embedded/community

http://www.msdn.microsoft.com/embedded/community

http://www.mswep.com/

http://www.mobile2marketcontest.com/

http://www.partner.microsoft.com/

Tools & ResourcesTools & Resources

msdn.microsoft.com/msdn.microsoft.com/ embeddedembedded

microsoft.public.microsoft.public. windowsxp.embeddedwindowsxp.embedded windowsce.platbuilderwindowsce.platbuilder windowsce.embedded.vcwindowsce.embedded.vc

blogs.msdn.com/blogs.msdn.com/ mikehallmikehall

Windows CE 5.0 Eval KitWindows CE 5.0 Eval KitWindows XP Embedded Eval KitWindows XP Embedded Eval Kit

msdn.microsoft.com/msdn.microsoft.com/ mobilitymobility

microsoft.public.microsoft.public. pocketpc.developer pocketpc.developer smartphone.developer smartphone.developer dotnet.framework.compactframeworkdotnet.framework.compactframework

blogs.msdn.com/blogs.msdn.com/ windowsmobilewindowsmobile vsdteamvsdteam netcfteamnetcfteam

Windows Mobile 5.0 Eval KitWindows Mobile 5.0 Eval Kit

WebsitesWebsites

NewsgroupsNewsgroups

BlogsBlogs

ToolsTools

BuildBuild DevelopDevelop

http://www.microsoft.public.dotnet.framework.compactframework/

memory leak detection douglas boling president boling consulting inc

Documents