qt world summit 2019 - kdab€¦ · clang compiler front end for c, c++, objective-c and...

ClangToolingQtWorldSummit2019

PresentedbyKevinFunk

p.2

ClangTooling

ClangTooling

ClangToolingLLVM/ClangOverviewClangcompilerfrontend(staticanalysis)ClangStaticAnalyzer(staticanalysis)clang-tidy(staticanalysis/linter/refactoring)Clazy(staticanalysis,Qtcentric)clang-format(codingstyle)ClangSanitizers(dynamicanalysis)

p.3

ClangTooling

LLVM/ClangOverview

LLVM/ClangOverview

Clangcompilerfrontend(staticanalysis)

ClangStaticAnalyzer(staticanalysis)

clang-tidy(staticanalysis/linter/refactoring)

Clazy(staticanalysis,Qtcentric)

clang-format(codingstyle)

ClangSanitizers(dynamicanalysis)

p.4

LLVMvs.Clang

LLVM/ClangOverview

LLVMFormerlyLowLevelVirtualMachineSetofcompilerandtoolchaintechnologiesinC++Designedforcompile-time,link-time,run-timeoptimizationofprograms

forarbitraryprogramminglanguages

ClangCompilerfrontendforC,C++,Objective-CandObjective-C++

andnowadaysCUDAandOpenCLUsesLLVMasitsbackendPartofitsreleasessinceLLVM2.6.

p.5

LLVM'simplementationofthree-phasedesign

LLVM/ClangOverview

p.6

ClangHighlights

LLVM/ClangOverview

Expressivediagnosticsandfix-ithintsforwarningsanderrors

Library-basedarchitecture,APIallowsforextensivetooling

AllowsintegrationwithIDEs(→QtCreator,KDevelop,LanguageServerProtocol,...)

Non-restrictiveBSDlicense

highlyactiveandskilledcommunityarounditlargeinvolvementofbigplayers(Google/Samsung/Apple/TI/...)

...

Interestingoverviews:Features:http://clang.llvm.org/features.htmlComparisontoothercompilers:https://clang.llvm.org/comparison.html

http://clang.llvm.org/features.html

https://clang.llvm.org/comparison.html

p.7

WhyisLLVM/Clangsopopular?

LLVM/ClangOverview

It'sareal-world,productionqualitycompiler

Hackablecodebase,extensivedocumentationandhelpfromcommunity

AccessiblepublicAPIsfor...InitiatingcodeparsingTraversingtheAbstractSyntaxTree(AST)ofacompilationunitDoingcodetransformations(refactorings)(andmuchmore)

hint:tooling!

p.8

Structureofthetalk

LLVM/ClangOverview

Clangisnotjustthecompilerbinary,therealotmoreindividualtools:ClangStaticAnalyzer(scan-build)clang-tidyclang-format...

AllthesetoolsdependontheClanglibraries

Nextsectionswilldiscusseachtool

p.9

ClangTooling


LLVM/ClangOverview







p.10

Term:StaticAnalysis


Staticcode(program)analysis:istheanalysisofcomputersoftwarethatisperformedwithoutexecutingprograms

onthecontrary:analysisperformedonexecutingprogramsisknownasdynamicanalysis

p.11

Whatdowewanttocatch?


CatchcodesegmentsthatarevalidC++,but(probably)notwhatyoumeanttowrite.

typosunintentionaloperatorusageunintentionalimplicitconversionPotentialdataover/underflowstautologicalcomparisonsdeadcodealotmore...

...atcompiletime!

p.12

Clangcompiler:Examples


p.12



ExampleCode(clang_warnings_01.cpp)ExampleCode(clang_warnings_01.cpp)

1 #include<cstddef>23 intmain(int,char**)4 {5 std::size_tbigstuff=-1;6 returnbigstuff<0;7 }

p.12




1 #include<cstddef>23 intmain(int,char**)4 {5 std::size_tbigstuff=-1;6 returnbigstuff<0;7 }

$clang++-Weverythingclang_warnings_01.cppclang_warnings_01.cpp:6:28:warning:implicitconversionchangessignedness:'int'to'std::size_t'(aka'unsignedlong')[-Wsign-conversion] std::size_tbigstuff=-1; ~~~~~~~~ ^~clang_warnings_01.cpp:7:21:warning:resultofcomparisonofunsignedexpressionfalse[-Wtautological-unsigned-zero-compare] returnbigstuff<0; ~~~~~~~~^~2warningsgenerated.

p.13

Clangcompiler:Examples(cont'd)


p.13




1 intmain(intargn,char**)2 {3 if(argn=1)4 return-1;5 }

p.13




1 intmain(intargn,char**)2 {3 if(argn=1)4 return-1;5 }

$clang++-Weverythingclang_warnings_02.cppclang_warnings_02.cpp:5:10:warning:usingtheresultofanassignmentasaconditionwithoutparentheses[-Wparentheses]if(argn=1) ~~~~~^~~clang_warnings_02.cpp:5:10:note:placeparenthesesaroundtheassignmenttosilencethiswarningif(argn=1) ^ ( )clang_warnings_02.cpp:5:10:note:use'=='toturnthisassignmentintoanequalitycomparisonif(argn=1) ^ ==1warninggenerated.

p.14

Example3


p.14

Example3



1 /**2 *@paramfoosomenumber3 *@returnsomenumber4 */5 voidmyfunc(intnum)6 {7 }89 intmain(){}

p.14

Example3



1 /**2 *@paramfoosomenumber3 *@returnsomenumber4 */5 voidmyfunc(intnum)6 {7 }89 intmain(){}

$clang++-Weverythingclang_warnings_03.cppclang_warnings_03.cpp:3:5:warning:'@return'commandusedinacommentthatisattachedtoafunctionreturningvoid[-Wdocumentation] *@returnsomenumber ~^~~~~~~~~~~~~~~~~~clang_warnings_03.cpp:2:11:warning:parameter'foo'notfoundinthefunctiondeclaration[-Wdocumentation] *@paramfoosomenumber ^~~clang_warnings_03.cpp:2:11:note:didyoumean'num'? *@paramfoosomenumber ^~~ num

p.15

Verdict:StaticanalysisviaClangcompiler


Thesewarningsarealreadybuilt-intotheclangcompilerNoneedtorunextratoolsNoneedtore-parsesourceforanalysis

Usefulfordetectingsimplecontrolflow/dataflowissues

-Weverythingistoonoisy(i.e.enables-Wc++98-compat)insteadneedtoexplicitlyenablewantedchecksusefulchecks:

-Wsign-conversion-Wall-Wextra-Wc++0x-compat-Winline-pedantic-Wredundant-decls-Wloop-analysis-Wstring-compare-Wshadow-Wunused-variable-Wundef-Wnon-virtual-dtor-Wdocumentation-Wshorten-64-to-32-Wused-but-marked-unused-Wdisabled-macro-expansion-Wparentheses-Wsometimes-uninitialized-Wconditional-uninitialized-Wfloat-equal-Wswitch-enum-Warray-bounds-Wcovered-switch-default-Wunreachable-code-Wnon-literal-null-conversion-Wtautological-compare-Wcovered-switch-default

p.16

ClangTooling


LLVM/ClangOverview







p.17

ClangStaticAnalyzer


Detectsbugswithoutexecutingtheprogram

Deeperdata/controlflowanalysisthanClangcompilersimilartocppcheckandCoverity

Slowerthancompilation

Higherrateoffalsepositives

Runsfromthecommand-line(scan-build),resultsinawebview

p.18

ClangStaticAnalyzer(cont'd)


Checkers:CoreCheckersC++CheckersDeadCodeCheckersSecurityCheckersUnixCheckers

Listofavailablecheckers:scan-build-h

p.19

Usageonsingletranslationunits


Runonsinglefile(withtextoutput):

ExampleCode(clang_sa_warnings_01.cpp)ExampleCode(clang_sa_warnings_01.cpp)

1 intmain()2 {3 int*x=newint;4 }

$clang++--analyze-Xanalyzer-analyzer-output=textclang_sa_warnings_01.cppclang_staticanalyzer_warnings_01.cpp:3:10:warning:Valuestoredto'x'duringitsinitializationisneverread int*x=newint; ^ ~~~~~~~clang_staticanalyzer_warnings_01.cpp:3:10:note:Valuestoredto'x'duringitsinitializationisneverread int*x=newint; ^ ~~~~~~~clang_staticanalyzer_warnings_01.cpp:4:1:warning:Potentialleakofmemorypointedtoby'x'}^clang_staticanalyzer_warnings_01.cpp:3:14:note:Memoryisallocated int*x=newint; ^~~~~~~(...)2warningsgenerated.

p.20

Usageonsingletranslationunits(cont'd)


Runonsinglefile(withhtmloutput):$clang++--analyze-Xanalyzer-analyzer-output=htmlclang_sa_warnings_01.cpp

p.21

Usageonwholeprojects


Forwholeprojects,scan-buildisrequired:

Runningscan-buildRunningscan-build

$→cdtobuilddir$→cleanbuilddir$scan-buildcmake/path/to/source$scan-buildmake...nowcompilesandanalyzessourcecode...

Howdoesitwork?scan-buildoverridestheCCandCXXvariablestouseafakecompilerinsteadfakecompilerexecutesoriginalcompilerandthenthestaticanalyzeronthetranslationunittheoutputofscan-buildisasetofHTMLfiles,viewableinbrowser

p.22

Usageonwholeprojects--viewingresults


Viewingresul tsViewingresul ts

$...$scan-buildmake...scan-build:Run'scan-view/tmp/scan-build-XYZ'toexaminebugreports.$scan-view/tmp/scan-build-XYZ

scan-viewwillopenawebbrowserinthisformat:

p.23

ClangStaticAnalyzer:Extra


Mostusefulscan-buildoptions:-htolistavailableoptionsandcheckers-enable-checker[checker_name]toenableindividualcheckers-otospecifiytheHTMLoutputdir

Windowsusers:Usescan-build.batinstead

p.24

Demo


Demo:Viewingscan-buildresultsinbrowser

p.25

ClangTooling


LLVM/ClangOverview







p.26

clang-tidy


Detectbugpronecodingpatterns

Enforcecodingconventions

Advocatemodernandmaintainablecode

CheckscanbemoreexpensivethancompilationDependsonhowinvolvedthecheckeris

EasytoextendwithcustomchecksE.g.toenforecompanycodepolicies

p.27

But,wait...


Differenttoolsforstaticanalysis(Clang,ClangSA,clang-tidy),whatthe...?!

Explanationfromthedevelopers:Clangdiagnostic:

ifthecheckisgenericenoughtargetscodepatternsthatmostprobablyarebugs(ratherthanstyleorreadabilityissues)canbeimplementedefficientlyandwithextremelylowfalsepositiverate

Clangstaticanalyzercheck:ifthecheckrequiressomesortofcontrolflowanalysis

clang-tidycheck:isagoodchoiceforlinter-stylecheckschecksthatarerelatedtoacertaincodingstylechecksthataddresscodereadability,etc.

p.28

clang-tidy:Introtochecks


Hasseveralbuilt-inchecksListthemwithclang-tidy--list-checks-checks='*'

CanrunchecksfromClangStaticAnalyzer(!)

ExtensiblebywritingcustomchecksChecksareorganizedinmodulesCanbelinkedintoclang-tidyAlsosee:http://clang.llvm.org/extra/clang-tidy/Contributing.html

http://clang.llvm.org/extra/clang-tidy/Contributing.html

p.29

clang-tidy:Checkscategories


boost-*:ChecksrelatedtoBoostlibrary.

cert-*:ChecksrelatedtoCERTSecureCodingGuidelines.

cppcoreguidelines-*:ChecksrelatedtoC++CoreGuidelines.

clang-analyzer-*:ClangStaticAnalyzerchecks.

google-*:ChecksrelatedtotheGooglecodingconventions.

llvm-*:ChecksrelatedtotheLLVMcodingconventions.

misc-*:Checksthatwedidn’thaveabettercategoryfor.

modernize-*:Checksthatadvocateusageofmodernlanguageconstructs.

mpi-*:ChecksrelatedtoMPI(MessagePassingInterface).

performance-*:Checksthattargetperformance-relatedissues.

readability-*:Checksthattargetreadability-relatedissuesthatdon’trelatetoanyparticularcodingstyle.

p.30

clang-tidychecks:Someusefulchecks


google-explicit-constructor

performance-unnecessary-value-param

readability-inconsistent-declaration-parameter-name

hicpp-explicit-conversions

hicpp-use-equals-default

hicpp-use-equals-delete

hicpp-undelegated-constructor

modernize-*(mostofthemareuseful!)

p.31

clang-tidy:Checkonefile


p.31



ExampleCode(clang_tidy_warnings_01.cpp)ExampleCode(clang_tidy_warnings_01.cpp)

1 enumThings{Dog,Frog};23 intbar(Thingsthing)4 {5 intret;6 switch(thing){7 caseDog:caseFrog:ret=0;8 default:ret=-1;9 }

10 returnret;11 }

p.31



ExampleCode(clang_tidy_warnings_01.cpp)ExampleCode(clang_tidy_warnings_01.cpp)

1 enumThings{Dog,Frog};23 intbar(Thingsthing)4 {5 intret;6 switch(thing){7 caseDog:caseFrog:ret=0;8 default:ret=-1;9 }

10 returnret;11 }

$clang-tidyclang_tidy_warnings_01.cpp--1warninggenerated.clang_tidy_warnings_01.cpp:7:30:warning:Valuestoredto'ret'isneverread[clang-analyzer-deadcode.DeadStores] caseDog:caseFrog:ret=0; ^clang_tidy_warnings_01.cpp:7:30:note:Valuestoredto'ret'isneverread

p.32

clang-tidy:Checkwholeproject


Prerequisite:NeedsaCompilationDatabaseinbuilddir

Notethatclang-tidyissingle-threaded!

Needswrapperscripttoberuninparallel:run-clang-tidy.py

Runningrun-clang-tidyonexampleprojectRunningrun-clang-tidyonexampleproject

$cmake-DCMAKE_EXPORT_COMPILE_COMMANDS=ON..$run-clang-tidy.pyEnabledchecks: clang-analyzer-core.CallAndMessage clang-analyzer-core.DivideZero ...

...runsclang-tidyoneachtranslationunit...clang-tidy-8....../clang_sa_warnings_01.cppclang_sa_warnings_01.cpp:3:10:warning:Valuestoredto'x'duringitsinitializationisneverread[clang-analyzer-deadcode.DeadStores] int*x=newint; ^...2warningsgenerated.

clang-tidy-8....../clang_warnings_02.cpp...

https://github.com/llvm-mirror/clang-tools-extra/blob/master/clang-tidy/tool/run-clang-tidy.py

p.33

clang-tidy:Refactoronefile


Torefactor,runclang-tidywith-fixoption!

p.33




ExampleCode(clang_tidy_modernize_01.cpp)ExampleCode(clang_tidy_modernize_01.cpp)

1 structBase2 {3 virtual~Base(){}4 virtualvoidfoo()=0;5 };67 structDerived:publicBase8 {9 virtualvoidfoo(){}

10 };

p.33






10 };

Clang-tidyrunClang-tidyrun

$clang-tidy-checks='-*,modernize-use-override'clang_tidy_modernize_01.cpp-fix---std=c++11

p.33






10 };

Clang-tidyrunClang-tidyrun

$clang-tidy-checks='-*,modernize-use-override'clang_tidy_modernize_01.cpp-fix---std=c++11

Resul tsinPatchResul tsinPatch

@@-6,5+6,5@@structBase

structDerived:publicBase {- virtualvoidfoo(){}+ voidfoo()override{} };

p.34

clang-tidy:Refactorwholeproject


Prerequisite:NeedsaCompilationDatabaseinbuilddir$cmake-DCMAKE_EXPORT_COMPILE_COMMANDS=ON..

$run-clang-tidy.py-header-filter='.*'-checks='-*,modernize-use-override'-fixEnabledchecks: modernize-use-override ...

...clang-tidy-8....../clang_tidy_modernize_01.cpp.../clang_tidy_modernize_01.cpp:9:18:warning:preferusing'override'or(rarely)'final'insteadof'virtual'[modernize-use-override] virtualvoidfoo(){} ~~~~~~~~ ^ override

1warninggenerated.

Importantnoterun-clang-tidy.pywillonlyapplychangesattheendofthefullrun!

p.35

JSONCompilationDatabase


Afilecalledcompile_commands.json

Containsinformationhowtotranslationunitsarebeingcompiled

ExampleJSONCompilationDatabaseExampleJSONCompilationDatabase

{ "directory":".../build/icemon", "command":"/usr/bin/c++ -DQT_CORE_LIB-I....../fakemonitor.cc"}{ "directory":".../build/icemon", "command":"/usr/bin/c++ -DQT_CORE_LIB-I....../hostinfo.cc"}...

p.36

CompilationDatabase:HOWTOcreate?


Buildsystemspecific,somebuildsystemscangeneratethem

VeryeasywithCMake:via-DCMAKE_EXPORT_COMPILE_COMMANDS=ON

Forbuildsystemswhichdon'tgenerateone:MakeuseofBear(Unixonly!)

Simplyrun:bearmakeLastresort:Writeascriptyourself!

Alsosee:Seriesofblogposts:https://www.kdab.com/tag/clang/

https://github.com/rizsotto/Bear

https://www.kdab.com/tag/clang/

p.37

Demo


Demo:Refactoringwholeprojectwithclang-tidy

p.38

Extendingclang-tidy


NotcoveredinthisslidedeckExcellentblogseriesbyStephenKelly:

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-1-extending-clang-tidy/https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-2-examining-the-clang-ast-with-clang-query/https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-3-rewriting-code-with-clang-tidy/

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-1-extending-clang-tidy/

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-1-extending-clang-tidy/

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-2-examining-the-clang-ast-with-clang-query/

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-2-examining-the-clang-ast-with-clang-query/

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-3-rewriting-code-with-clang-tidy/

https://devblogs.microsoft.com/cppblog/exploring-clang-tooling-part-3-rewriting-code-with-clang-tidy/

p.39

ClangTooling


LLVM/ClangOverview







p.40

StaticCodeAnalysis:Clazy


AClangplugin

FindscommonbugsandperformancepitfallsinQtandC++code

Intro:www.kdab.com/use-static-analysis-improve-performance

Projecthomepage:https://github.com/KDE/clazy

Cross-platform,currentlysupportedaremacOS,MSVC2015+andLinux

LGPLv2licensed,aKDEproject

IntegratedinandshippedwithQtCreatorIDE

https://www.kdab.com/use-static-analysis-improve-performance/

https://github.com/KDE/clazy

p.41

Clazy:Examples


p.41

Clazy:Examples


ExampleCode(1 )ExampleCode(1 )

1 #ifndefGL_FRAMEBUFFER_SRB2 #defineGL_FRAMEBUFFER_SRGB0x8DB93 #endif

p.41

Clazy:Examples




ClazyRun(1 )ClazyRun(1 )

qplatformbackingstore.cpp:4:9:warning:Possibletypoindefine.GL_FRAMEBUFFER_SRBvsGL_FRAMEBUFFER_SRGB[-Wclazy-ifndef-define-typo]

p.41

Clazy:Examples






Examplecode(2 )Examplecode(2 )

1 classBase2 {3 public:4 virtual~Base()=default;5 };

p.41

Clazy:Examples






Examplecode(2 )Examplecode(2 )

1 classBase2 {3 public:4 virtual~Base()=default;5 };


main.cpp:36:1:warning:PolymorphicclassBaseiscopyable.Potentialslicing.[-Wclazy-copyable-polymorphic]

p.42

Clazy:Examples(cont'd)


ExampleCodeExampleCode

1 voidtest(QObject*obj)2 {3 inta=1;4 autof=[&a](){/*...*/};5 obj->connect(obj,&QObject::destroyed,[a](){/*...*/});6 obj->connect(obj,&QObject::destroyed,[&a](){/*...*/});7 }

ClazyRunClazyRun

$clazy-isystem/usr/include/qt/-std=c++11tests/lambda-in-connect/main.cpp

p.42



ExampleCodeExampleCode

1 voidtest(QObject*obj)2 {3 inta=1;4 autof=[&a](){/*...*/};5 obj->connect(obj,&QObject::destroyed,[a](){/*...*/});6 obj->connect(obj,&QObject::destroyed,[&a](){/*...*/});7 }

ClazyRunClazyRun

$clazy-isystem/usr/include/qt/-std=c++11tests/lambda-in-connect/main.cpp

1 tests/lambda-in-connect/main.cpp:15:46:warning:capturinglocalvariable2 byreferenceinlambda[-Wclazy-lambda-in-connect]3 obj->connect(obj,&QObject::destroyed,[&a](){});4 ^

p.43



ExampleCode(clazy_qstring_al locations.cpp)ExampleCode(clazy_qstring_al locations.cpp)

1 #include<QtCore/QString>23 boolisFoo(constQString&string)4 {5 returnstring=="foo";6 }78 QStringfoo()9 {

10 return"foo";11 }

ClazyRunClazyRun

$exportCLAZY_CHECKS=level0,level1,level2$clazy-isystem...-std=c++11clazy_qstring_allocations.cpp

p.43



ExampleCode(clazy_qstring_al locations.cpp)ExampleCode(clazy_qstring_al locations.cpp)

1 #include<QtCore/QString>23 boolisFoo(constQString&string)4 {5 returnstring=="foo";6 }78 QStringfoo()9 {

10 return"foo";11 }

ClazyRunClazyRun

$exportCLAZY_CHECKS=level0,level1,level2$clazy-isystem...-std=c++11clazy_qstring_allocations.cpp

1 clazy_qstring_allocations.cpp:5:12:warning:QString(constchar*)beingcalled2 [-Wclazy-qstring-allocations]3 clazy_qstring_allocations.cpp:10:12:warning:QString(constchar*)beingcalled4 [-Wclazy-qstring-allocations]

p.44

Clazy:ExamplesusingFixits


Someerrorscanbefixedautomatically

OnlydothistocodetrackedbyaVCS1 $clazy-isystem/usr/include/x86_64-linux-gnu/qt5-std=c++11-fPIC2 -Xclang-plugin-arg-clazy-Xclangqstring-allocations3 -Xclang-plugin-arg-clazy-Xclangexport-fixes4 -cclazy_qstring_allocations.cpp5 ...generatesa.yamlfileinthecurrentdir...6 $clang-apply-replacements.7 $gitdiff

1 @@-2,10+2,10@@23 boolisFoo(constQString&string)4 {5 - returnstring=="foo";6 + returnstring==QLatin1String("foo");7 }89 QStringfoo()

10 {11 - return"foo";12 + returnQStringLiteral("foo");13 }

p.45

Clazy:Runningitonaproject


Theclazyexecutableneedstobeawareofcommand-lineargumentsNotfeasibletodothismanuallyforawholeproject

Theclazy-standaloneexecutableinfersthatinfofromaJSONCompilationdatabase

JustlikeotherClangtools,suchasclang-tidy

Examples:

RunningClazyonal l .cppfilesinaprojectRunningClazyonal l .cppfilesinaproject

$find.-name"*.cpp"|xargsclazy-standalone-checks=level2-p$BUILD_DIR

T ip:RunningClazyonal l filesinaprojectusingT ip:RunningClazyonal l filesinaprojectusingjqjq

$jq".[]|.file"$BUILD_DIR/compile_commands.json|xargsclazy-standalone-checks=level2-p$BUILD_DIR

p.46

ClangTooling


LLVM/ClangOverview







p.47

clang-format


MotivationDeveloperswastetimeonformattingConsistentcodingstyleisimportant

clang-formatprovidesAutomaticformattingSupportsdifferentstyleguidesFine-tuningofformattingrules

p.48

clang-format(cont'd)


Similartomostothercodeformatters

Canberunonindividualfilesorsubdirs

Parametrizationviacommand-lineorconfigfile

Lessflexiblewithregardstoparameterizationasotherformattersforinstanceuncrustifyisextremelyconfigurable

Offersscriptforpatchreformatting(clang-format-diff.py)

p.49

clang-format:Examples


p.49



ExampleCode(clang_format_example.cpp)ExampleCode(clang_format_example.cpp)

1 intmain()2 {3 inti=0;4 }

p.49





clang-formatrunclang-formatrun

clang-format-style='Mozilla'-iclang_format_example.cpp

p.49





clang-formatrunclang-formatrun

clang-format-style='Mozilla'-iclang_format_example.cpp

Resul tsinResul tsin

1 @@-1,4+1,5@@2 -intmain()3 +int4 +main()5 {6 -inti=0;7 + inti=0;8 }

p.50

ClangTooling


LLVM/ClangOverview







p.51

Sanitizers


FasterrorcheckerssupportedbyGCC4.9+andClang3.1+

Successfullyappliedtolargeapplications,e.g.GoogleChrome,Firefox...

Availablesanitizers:address:memoryerrordetectorthread:detectdataracesundefined:checkcodeforundefinedbehaviorAndmore,withmixedmaturitylevel

p.52

SanitizerInternals


Sanitizersarecompileraddons:

Theyinsertchecksormodifycodetospotcommonprogrammingerrors.

ThesecheckscomewithasignificantCPUandmemoryoverhead.But,itismuchlowerthanValgrind'svirtualmachine.

Threadedcodeisnotartificiallyserializedandcanexecuteinparallel.

ReadtheASanpaperformoreinformationontheinternals:Serebryany,Konstantin,etal."AddressSanitizer:AFastAddressSanityChecker."USENIXAnnualTechnicalConference.2012.APAwww.usenix.org/system/files/conference/atc12/atc12-final39.pdf

https://www.usenix.org/system/files/conference/atc12/atc12-final39.pdf

p.53

EnablingSanitizers


Compileyourcodewith-fsanitize={address,leak,undefined,thread}.

It'salsousefultoapplythistoyourdependencies,e.g.Qt.Otherwise,someerrorscannotbedetected.

EnablingsanitizersforQtglobally:Configurewith-sanitize{address,leak,undefined,thread}.AllcodecompiledagainstthisQtwillinheritthesanitizersettings.

EnablingsanitizersinindividualQMakeprojects:qmake-rCONFIG+="sanitizersanitize_address"...

EnablingsanitizersinCMakeprojects:LeveragetheExtraCMakeModules(ECM): api.kde.org/ecm/module/ECMEnableSanitizers.htmlMakesanitizersavailableinyourCMakeLists.txtwith: include(ECMEnableSanitizers)Enablesanitizersvia: cmake-DECM_ENABLE_SANITIZERS="address;undefined"...

http://api.kde.org/ecm/module/ECMEnableSanitizers.html

p.54

SanitizersonWindows


RequiresClangforWindows.Windows10FallCreatornotsupportedbyClang5.WaitforClang6orcompileClangfromsources.

Ensurecompiler-rtusesthedynamicruntime(/MD)

Sanitizersdonotworkwithdebugruntimes

SanitizerreportsshowupinDbgView

Createacustommkspeccalledwin32-clang-msvc-sanitizersCopytheqplatformdefs.hfromwin32-clang-msvc.Createacustomqmake.conf:

1 include(../win32-clang-msvc/qmake.conf)23 CONFIG+=force_debug_info4 QMAKE_CXXFLAGS_RELEASE_WITH_DEBUGINFO+=-fsanitize=address,undefined5 #optionally:disablecompileroptimizations6 QMAKE_CXXFLAGS_RELEASE_WITH_DEBUGINFO-=-O27 QMAKE_CXXFLAGS_RELEASE_WITH_DEBUGINFO+=-Og89 #linkagainsttheASANruntime,thepathbelowisthecompiler-rtinstallprefix

10 LIBS+=/LIBPATH:C:/Qt/src/compiled/clang/lib/clang/6.0.0/lib/windows/11 LIBS+=-lclang_rt.asan_dynamic-x86_64-lclang_rt.asan_dynamic_runtime_thunk-x86_64

UseitwithQMake:qmake-specwin32-clang-msvc-sanitizers

p.55

AddressSanitizer(ASan)


Use-fsanitize=addresstodetect:Out-of-boundsaccessestoheap,stackandglobalsUse-after-freeUse-after-return(tosomeextent)Double-free,invalidfreeMemoryleaks(NotsupportedonWindows)

Passing-fsanitize=leakonlyactivatesthelow-overheadleakchecking

Note:Applicationwillterminatewhenthefirsterrorisdetected

Documentation:github.com/google/sanitizers/wiki/AddressSanitizerclang.llvm.org/docs/AddressSanitizer.html

https://github.com/google/sanitizers/wiki/AddressSanitizer

http://clang.llvm.org/docs/AddressSanitizer.html

p.56

AddressSanitizer(ASan)-Example


1 int*i=newint[2];2 intread=i[2];3 i[3]=1;//andwrite

Demo:debugging/ex_invalid_readwrite

p.56

AddressSanitizer(ASan)-Example


1 int*i=newint[2];2 intread=i[2];3 i[3]=1;//andwrite

Runningtheexamplecompiledwith-fsanitize=addressyields:1 ==32417==ERROR:AddressSanitizer:heap-buffer-overflow2 onaddress0x60200000de98atpc0x000000400857bp0x7ffd6ee14f40sp0x7ffd6ee14f303 READofsize4at0x60200000de98threadT04 #00x400856inmain.../ex_invalid_readwrite/ex_invalid_readwrite.cpp:456 0x60200000de98islocated0bytestotherightof8-byteregion7 [0x60200000de90,0x60200000de98)8 allocatedbythreadT0here:9 #00x7fd8761a3a62inoperatornew[](unsignedlong).../asan_new_delete.cc:62

10 #10x400817inmain.../debugging/ex_invalid_readwrite/ex_invalid_readwrite.cpp:3

Ifnosymbolsareshown,tryexportingASAN_SYMBOLIZER_PATH=/usr/bin/llvm-symbolizerASAN_OPTIONS=symbolize=1


p.57

AddressSanitizer(ASan)-ErrorRecovery


Executionisterminatedafterthefirsterror.

Thisisjustdefaultbehavior.

Enableerrorrecovery:Compilewith-fsanitize-recover=all.SetenvironmentvariableASAN_OPTIONS=halt_on_error=0.

1 ==32417==ERROR:AddressSanitizer:heap-buffer-overflow2 onaddress0x60200000de98atpc0x000000400857bp0x7ffd6ee14f40sp0x7ffd6ee14f303 READofsize4at0x60200000de98threadT04 #00x400856inmain.../ex_invalid_readwrite/ex_invalid_readwrite.cpp:456 ...78 ==13213==ERROR:AddressSanitizer:heap-buffer-overflow9 onaddress0x60200000efdcatpc0x000000400900bp0x7fffd0a77a30sp0x7fffd0a77a20

10 WRITEofsize4at0x60200000efdcthreadT011 #00x4008ffinmain.../ex_invalid_readwrite/ex_invalid_readwrite.cpp:10

Note:Errorsafterthefirstonemaybespurious.


p.58

LeakSanitizer(LSan)


Low-overheadmemoryleakdetector

Documentation:github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer

https://github.com/google/sanitizers/wiki/AddressSanitizerLeakSanitizer

p.58

LeakSanitizer(LSan)




1 ==7668==ERROR:LeakSanitizer:detectedmemoryleaks23 Directleakof100byte(s)in1object(s)allocatedfrom:4 #00x7fd8c003fb18inoperatornew[](unsignedlong)/build/gcc/src/gcc/libsanitizer/asan/asan_new_delete.cc:1075 #10x558d36621f3binfoo()../../ex_leak/ex_leak.cpp:96 #20x558d3662203einmain../../ex_leak/ex_leak.cpp:167 #30x7fd8bf0dfee2in__libc_start_main(/usr/lib/libc.so.6+0x26ee2)


p.58

LeakSanitizer(LSan)




1 ==7668==ERROR:LeakSanitizer:detectedmemoryleaks23 Directleakof100byte(s)in1object(s)allocatedfrom:4 #00x7fd8c003fb18inoperatornew[](unsignedlong)/build/gcc/src/gcc/libsanitizer/asan/asan_new_delete.cc:1075 #10x558d36621f3binfoo()../../ex_leak/ex_leak.cpp:96 #20x558d3662203einmain../../ex_leak/ex_leak.cpp:167 #30x7fd8bf0dfee2in__libc_start_main(/usr/lib/libc.so.6+0x26ee2)

UsefultodetectleaksinunittestsExitcodewillindicatefailurewhenleaksaredetected

Suppressionfilescanbeusedtoignorethird-partyorsystemlibrariesexportLSAN_OPTIONS=suppressions=/absolute/path/to/file

BacktracescanbreakwhenencounteringthirdpartylibrariesThedefaultunwinderrelieson-fno-omit-frame-pointersWorkaroundthisviaLSAN_OPTIONS=fast_unwind_on_malloc=true


p.59

UndefinedBehaviorSanitizer(ubsan)


Use-fsanitize=undefinedtodetectundefinedbehavior,suchas:InvalidbitshiftingIntegeroverflowNullpointerdereferencingOut-of-boundsaccessonstackarraysPointeralignmentAndmore

CanbecombinedwithASan:-fsanitize=address,undefined

Doesnotterminatewhenanerrorisfound

Backgroundinformation: developerblog.redhat.com/2014/10/16/gcc-undefined-behavior-sanitizer-ubsan

http://developerblog.redhat.com/2014/10/16/gcc-undefined-behavior-sanitizer-ubsan/

p.60

UndefinedBehaviorSanitizer(ubsan)-Example


Assumingargc=1,whereisundefinedbehaviorinthefollowingcode?1 intfoo=23<<(argc*32);23 intbar=std::numeric_limits<int>::min()*argc;4 unsignedintasdf=-bar;

Demo:debugging/ex_undefined_behavior

p.60

UndefinedBehaviorSanitizer(ubsan)-Example


Assumingargc=1,whereisundefinedbehaviorinthefollowingcode?1 intfoo=23<<(argc*32);23 intbar=std::numeric_limits<int>::min()*argc;4 unsignedintasdf=-bar;

Runningtheexamplecompiledwith-fsanitize=undefinedyields:1 main.cpp:7:11:runtimeerror:shiftexponent32istoolargefor32-bittype'int'23 main.cpp:12:18:runtimeerror:negationof-2147483648cannotberepresented4 intype'int';casttoanunsignedtypetonegatethisvaluetoitself

Hint:SetUBSAN_OPTIONS=print_stacktrace=1

Demo:debugging/ex_undefined_behavior

p.61

Lab:Sanitizers


Compileandrunthelab.

Itcrashesinstantly.

Usethesanitizerstofindthebug.

Fixthebugandrepeatthesteps.

Finally,trysettingabirthdayin1750,andchecktheageinseconds.

Lab:debugging/lab_dialog

p.62

ThreadSanitizer(TSan)


Use-fsanitize=threadtodetect:DataracesThreadleaksDestructionoflockedmutexDeadlocks

Documentation:https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManualhttp://clang.llvm.org/docs/ThreadSanitizer.html

Note:Onlysupportedon64bitx86machinesrunningLinux.

https://github.com/google/sanitizers/wiki/ThreadSanitizerCppManual

http://clang.llvm.org/docs/ThreadSanitizer.html

p.63

ThreadSanitizer(TSan)-PreparingQt


CustomQtbuildwithThreadSanitizersupportisnecessaryforproperinterceptionofQMutexetal.:

1 configure\2 -platformlinux-clang-libc++\3 -sanitizethread\4 -debug...

GCCandlibstdc++shouldalsoworkinprinciple.Inpractice,Clang'sTSanwithlibc++ismorereliable.

Alternatively,replace-debugwith-release-force-debug-info.

p.64

ThreadSanitizer(TSan)-Example


1 voidrace(int*counter,intincrement){*counter+=increment;}23 intmain()4 {5 intcounter=0;6 {7 std::vector<std::future<void>>futures;8 constautothreadsToStart=std::thread::hardware_concurrency();9 for(unsignedi=0;i<threadsToStart;++i)

10 futures.push_back(std::async(std::launch::async,race,&counter,i));11 }12 std::cout<<counter<<std::endl;13 return0;14 }

1 WARNING:ThreadSanitizer:datarace(pid=4563)2 Readofsize4at0x7ffeb2e34adcbythreadT2:3 #0race(int*,int)ex_datarace/main.cpp:1(main+0x0000004017ad)4 ...5 Previouswriteofsize4at0x7ffeb2e34adcbythreadT1:6 #0race(int*,int)ex_datarace/main.cpp:1(main+0x0000004017c5)7 ...

Demo:debugging/ex_datarace

p.65

Sanitizersvs.Valgrind


Sanitizer Valgrind heapbufferoverflow Address Memcheck danglingpointer Address Memcheck

mismatchednew/delete Address Memcheck memoryleak Address,Leak Memcheck

uninitializedmemory Memory(WIP) Memcheck stackbufferoverflow Address SGCheck(experimental) integeroverflow Undefined - alignmentfault Undefined -

datarace Thread Helgrind,DRD deadlock Thread Helgrind,DRD

Sanitizersaremuchfasteranduselessmemory.

NorecompilationisrequiredforValgrind.AlllibrarieslinkedintoyourapplicationwillbecheckedbyValgrind.

p.66

Summary

ClangTooling

ClangcompilerUsefulstaticanalysisathigh-levelwarninglevelduringnormalcompilation

StaticanalyzerDeepcontrolflowanalysis,slowHTML-formattedoverviewoverissues

clang-tidy(dependsonCMakecompdb)C++linter,Abilitytofixcodeautomatically

ClazyClangpluginprovidingQt/C++specificdiagnosticsSimilarusagepossiblelikeforclang-tidy

clang-formatSimple-to-usecodeformatter,limitedflexibility

ClangSanitizersDynamicanalysistoolsDetectingerraticbehavioratruntime

p.67

Installation

ClangTooling

Linux:Grabitviayourpackagemanager

Windows:GetviaLLVMwebsite

macOS:Homebrew:$brewinstall--with-toolchainllvm(or?)

Downloadsforbasicallyallplatforms:http://releases.llvm.org/download.html

http://releases.llvm.org/download.html

p.68

Evenmoretools

ClangTooling

Notdiscussed,butalsouseful:Include-what-you-use

Projectwebsite:https://include-what-you-use.org/StripsunnecessaryincludesfromsourcefilesVerdict:Niceintheory,butbreakswhenusingforwardingheaders(heyQt!)

clang-queryUseClangASTmatcherstoquerycodeforsymbolsGoodintro:https://eli.thegreenplace.net/2014/07/29/ast-matchers-and-clang-refactoring-toolsVerdict:Nicetool,butalsosuperslowfortoo-broadqueries.Toolmayfreeze.

clang-queryexampleclang-queryexample

clang-query>matchifStmt(hasCondition(binaryOperator(hasOperatorName("==")).bind("op")))

Match#1:

/tmp/iflhsptr.c:2:7:note:"op"bindshere if(p==0){ ^~~~~~

https://include-what-you-use.org/

https://eli.thegreenplace.net/2014/07/29/ast-matchers-and-clang-refactoring-tools

https://eli.thegreenplace.net/2014/07/29/ast-matchers-and-clang-refactoring-tools

p.69

Links

ClangTooling

Entrypoint:http://clang.llvm.org/Importantforsettingupyourproject(→compdb,etc.)

http://clang.llvm.org/docs/HowToSetupToolingForLLVM.htmlClangStaticAnalyzer:

http://clang-analyzer.llvm.org/scan-build.html/http://clang-analyzer.llvm.org/

clang-tidy:http://clang.llvm.org/extra/clang-tidy.html

Clazy:https://github.com/KDE/clazy

clang-format:http://clang.llvm.org/docs/ClangFormat.html

http://clang.llvm.org/

http://clang.llvm.org/docs/HowToSetupToolingForLLVM.html

http://clang-analyzer.llvm.org/scan-build.html

http://clang-analyzer.llvm.org/

http://clang-analyzer.llvm.org/

http://clang.llvm.org/extra/clang-tidy.html

https://github.com/KDE/clazy

http://clang.llvm.org/docs/ClangFormat.html

p.70

Questions?

[email protected]

KDABisaproviderforexpertQt,OpenGLandC++services

www.kdab.com

mailto:[email protected]

qt world summit 2019 - kdab€¦ · clang compiler front end for c, c++, objective-c and...

Documents