prg –programming essentials...2 administration 2 03/12/2017 michal reinštein, czech technical...

1

PRG– PROGRAMMINGESSENTIALS1

Lecture5– Modules,Namespaceshttps://cw.fel.cvut.cz/wiki/courses/be5b33prg/start

MichalReinšteinCzechTechnicalUniversityinPrague,

FacultyofElectricalEngineering,Dept.ofCybernetics,CenterforMachinePerceptionhttp://cmp.felk.cvut.cz/~reinsmic/

reinstein.michal@fel.cvut.cz

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

2

ADMINISTRATION2

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttps://cw.fel.cvut.cz/wiki/help/common/plagiarism_cheating

PLAGIARISMWARNINGhttps://cw.fel.cvut.cz/wiki/help/common/plagiarism_cheating

3

RECAP: MOREABOUTPYTHON3

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttps://www.youtube.com/watch?v=arxWaw-E8QQ&t=1s

• Themethodsandvariablesarecreatedonstackmemory• Theobjectsandinstancesarecreatedonheapmemory• Newstackframeiscreatedoninvocationofa

function/method• Stackframesaredestroyedassoonasthe

function/methodreturns• MechanismtocleanupthedeadobjectsisGarbagecollector• EverythinginPythonisobject• Pythonisdynamicallytypedlanguage

4

RECAP: LISTS4

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html

• Listsaremutable (wecanchangetheirelements)• Stringsareimmutable (wecannotchangetheirelements)• Useslicingprinciples(indexesinbetweencharacters/items)

5

RECAP:SLICING5

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/strings.html

• A substring ofastringisobtainedbytakinga slice• Slicealisttorefertosomesublist oftheitemsinthelist• Theoperator [n:m] returnsthepartofthestringfromthen’thcharactertothem’thcharacter,includingthefirstbutexcludingthelast(indicespointing between thecharacters)

• Sliceoperator [n:m] copies outthepartofthepaperbetweenthe n andm positions

• Resultof[n:m] willbeoflength(m-n)

6

RECAP: STRINGSvs.LISTS6

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html

• Variables a and b refertostringobjectwithletters "banana”• Useis operatororid functiontofindoutthereference• Stringsare immutable,Pythonoptimizesresourcesbymakingtwonamesthatrefertothesamestringvaluerefertothesameobject

• Notthecaseoflists:a and b havethesamevalue(content)butdonotrefertothesameobject

Strings

Lists

7

RECAP: LISTS– ALIASING,CLONING7

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html

• Ifweassignonevariabletoanother,bothvariablesrefertothesameobject

• Thesamelisthastwodifferentnames wesaythatitis aliased (changesmadewithonealiasaffecttheother)

• RECOMMENDATION:avoidaliasingwhenyouareworkingwithmutableobjects

• Ifneedtomodifyalistandkeepacopyoftheoriginalusethesliceoperator(takinganysliceof a createsanewlist)

8

RECAP: LISTPARAMETERS8

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html

• Passingalistasanargumentpassesareference tothelist,notacopyorcloneofthelist

• Soparameterpassingcreatesanalias

9

MODULES9

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Amodule isafilecontainingPythondefinitionsandstatementsintendedforuseinotherPythonprograms

• Modulesareoneofthemainabstractionlayersavailableandprobablythemostnaturalone

• Abstractionlayersallowseparatingcodeintopartsholdingrelateddataandfunctionality

• Themostnaturalwaytoseparatethesetwolayersistoregroupallinterfacingfunctionalityintoonefile,andalllow-leveloperationsinanotherfile

• Donewiththe import and from... import statements

10

MODULES10

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

WheredoesPythonlookforimportedmodules?• Alistnamed path inmodule sys containsallthelocations(andcanbemodified)

• ThefirstitemonthelististhedirectorywiththeprogramWhathappenswhenPythonimportsamodule?• Pythonsearchesthemoduleamongalreadyimportedmodulesin sys.modules

• Ifthemoduleisnotfoundin sys.modules,Pythonsearcheslocationsin sys.path, executesthemodule onceitisfound,andrecordsthatthemodulehasalreadybeenimported

• Pythoncreatesnamesinlocalnamespace fortheimportedmodule,orforallthevariables,functions,etc.importedfromthemodule

11

MODULES11

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Modulesthatcanbebothrun andimported• Specialvariablecalled __name__ insideeachmoduleitcontains:• thenameofthemodule(ofthe.py file)whenthemoduleisimported

• string "__main__" whenthe.py fileisrunasaprogram(script)

• Variable __name__ isdefinedinboththecallingnamespaceandinthenamespaceofthemodule

12

MODULES12

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://docs.python-guide.org/en/latest/writing/structure/#modules

• Togainaccesstosymbolsdefinedinamodule(i.e.inadifferentnamespace)modulehastobeimported(3ways)

• Thestatementsloadamodule,createanameinthecurrentnamespace,andbindthenametotheloadedmodule

13

MODULES– RANDOMNUMBERS13

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Toincluderandomdecision-makingprocess• Totakesamplesfromprobabilitydistributions• Toplayagameofchancewherethecomputerneedstothrowsomedice,pickanumber,orflipacoin…

• Toshuffleadeckofplayingcardsrandomly…• Inmodellingandsimulations:weathermodels,environmentalmodels,MonteCarlomethod

• ForencryptingbankingsessionsontheInternet

14

MODULES– RANDOMNUMBERS14

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• The randommethodreturnsafloatingpointnumberintheinterval[0.0,1.0)— thesquarebracketmeans“closedintervalontheleft”andtheroundparenthesismeans“openintervalontheright” – 0.0ispossible,butallreturnednumberswillbestrictlylessthan1.0.

• Itisusualto scale theresultsaftercallingthismethodtogetthemintoanintervalsuitableforapplication.

• EXAMPLE:scalingtoanumberintheinterval[0.0,5.0)(uniformlydistributednumbers— numberscloseto0arejustaslikelytooccurasnumberscloseto0.5orcloseto1.0)

15

MODULES– RANDOMNUMBERS15

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• The randrangemethodgeneratesanintegerbetweenitslower andupper argument

• Therandrangemethodsamesemanticsas range(sothelowerboundisincluded,buttheupperboundisexcluded)

• Allthevalueshaveanequalprobabilityofoccurring(i.e.theresultsare uniformly distributed).

• Randrange alsotakesanoptionalstepargument(likerange)

• EXAMPLE:Weneededarandomoddnumberlessthan100

16

MODULES– RANDOMNUMBERS16

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

Thisexampleshowshowtoshuffle alist.(shuffle cannotworkdirectlywithrangeobjectsolist typeconverterfirstisnecessary)

• Randomnumbergeneratorsarebasedona deterministic algorithm— repeatable andpredictable

• Called pseudo-random generators(notgenuinelyrandom)• Eachtimeyouaskforanotherrandomnumber,you’llgetonebasedonthecurrentseedattribute,andthestateoftheseed

17

MODULES– RANDOMNUMBERS17

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Repeatabilityfordebugging andforwritingunittests(programsthatdothesamethingeverytimetheyarerun)

• Forcingtherandomnumbergeneratortobeinitializedwithaknownseedeverytime(oftenthisisonlywantedduringtesting/back-testing)

• Withoutthisseedargument,thesystemprobablyusessomethingbasedontheOStime.

18

MODULES– RANDOMNUMBERS18

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• EXAMPLE:generatealistcontaining n randomints betweenalowerandupperbound

• NOTE:thatwegotaduplicates intheresult(oftenthisiswanted,e.g.ifwethrowadiefivetimes,wewouldexpectsomeduplicates)

19

MODULES– RANDOMNUMBERS19

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

Howtotakecareofduplicates?

• Ifyouwanted5distinctmonths,thenthisalgorithmiswrong• Inthiscaseagoodalgorithmistogeneratethelistof

possibilities,shuffle it,andsliceoff thenumberofelementsyouwant:

20

MODULES– RANDOMNUMBERS20

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Allowingduplicates (usuallydescribedaspullingballsoutofabag withreplacement)

• Noduplicates (usuallydescribedaspullingballsoutofthebag withoutreplacement)

• Algorithm“shuffleandslice”isnotidealforcaseofchoosingfewelementsfromaverylargedomain

(Supposetheneedforfivenumbersbetween1and10million,withoutduplicates.Generatingalistoftenmillionitems,shufflingit,andthenslicingoffthefirstfivewouldbeaperformancedisaster.)

21

MODULES– RANDOMNUMBERS21

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

Choosewiselythealgorithmbasedontheinputdata!

22

MODULES– TIME22

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

Howefficientandreliableisourcode?• Onewaytoexperimentisto

timehowlongvariousoperationstakeandwhatthememoryrequirementsare(relatedtoalgorithmcomplexity:https://people.duke.edu/~ccc14/sta-663/AlgorithmicComplexity.html )

• The timemodulehasafunction clock thatisrecommended• Whenever clock iscalled,itreturnsafloatingpointnumber

representinghowmanysecondshaveelapsedsinceyourprogramstartedrunning(variesaccordingtoOS!)

23

MODULES– TIME23

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• EXAMPLE:Generatingandsumminguptenmillionelementsinunderasecond

• Proprietaryfunctionruns57%slowerthanthebuilt-in one.

24

MODULES– TIME24

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttps://people.duke.edu/~ccc14/sta-663/AlgorithmicComplexity.html

25

MODULES– MATH25

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Themathmodulecontainsmathematicalfunctionstypicallyfoundonacalculator,includingmathematicalconstantslike pi and e

• Functions radians and degrees toconvertangles• Mathematicalfunctionsarepure anddonothaveanystate

26

MODULES– CREATINGMODULES26

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Createownmodules – savescriptasafilewith .py extension• EXAMPLE:functionremove_at inascriptissavedasafilenamed seqtools.py

• Themodulemustbefirstimportedbeforeuse(.pyisthefileextensionandisnotincludedinthe importstatement)

• RECOMMENDATION:breakupverylargeprogramsintomanageablesizedpartsandkeeprelatedpartstogether

27

MODULES– NAMESPACES27

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Namespaceisamapping fromnamestoobjects• Namespace isacollectionofidentifiersthatbelongtoamodule,function,oraclass

• Namespaceissetofsymbolsusedtoorganizeobjectsofvariouskindssothatwecanrefertothembyname

• Namespacespermitprogrammerstoworkonthesameprojectwithouthavingnamingcollisions(allownamereuse)

• Oftenhierarchically structured• Eachnamemustbeunique initsnamespace• NamespaceisverygeneralconceptnotlimitedtoPython• Eachmodulehasitsownnamespace – wecanusethesameidentifiernameinmultiplemoduleswithoutcausinganidentificationproblem

28

MODULES– NAMESPACES28

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

HowarenamespacesdefinedinPython?• Packages(collectionsofrelatedmodules)• Modules(.pyfilescontainingdefinitionsoffunctions,classes,variables,etc.)

• Classes,Functions…Whatisthedifferencebetweenprogramsandmodules?• Botharestoredin.pyfiles.• Programs (scripts)aredesignedtoberun(executed)• Modules (libraries)aredesignedtobeimportedandusedbyotherprogramsandothermodules

• Specialcase:.pyfileisdesignedtobebothaprogramandamodule(itcanbeexecutedaswellasimportedtoprovidefunctionalityforothermodules)

29

MODULES– NAMESPACES29

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

30

MODULES– NAMESPACES30

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Functionsalsohaveownnamespaces• Functionscanread(read-only)variableintheouterscope• EXAMPLE:thethree n‘sabovedonotcollidesincetheyareeachinadifferentnamespace— threenamesforthreedifferentvariables

31

MODULES,NAMESPACES,FILES31

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Pythonhasaconvenientandsimplifyingone-to-onemapping:onemoduleperfile – givingrisetoonenamespace

• Pythontakesthemodulenamefromthefilename,andthisbecomesthenameofthenamespace

• EXAMPLE:math.py isafilename,themoduleiscalledmath,anditsnamespaceismath (inPythontheconceptsaremoreorlessinterchangeable)

• Inotherlanguages(e.g.C#)onemodulecanspanmultiplefiles,oronefiletohavemultiplenamespaces,ormanyfilestoallsharethesamenamespace

32

MODULES,NAMESPACES,FILES32

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• RECOMMENDATION:keeptheconceptsdistinctinyourmind• Filesanddirectoriesorganize where codeanddataarestored• Namespacesandmodulesareaprogrammingconcepts:helpusorganizehowwewanttogrouprelatedfunctionsandattributes.

• Namespacesarenotabout“where”tostorethings,andshouldnothavetocoincidewiththefilestructures

• Ifthefile math.py isrenamed,itsmodulenameneedstobechanged, import statementsneedtobechanged,andthecodethatreferstofunctionsorattributesinsidethatnamespacealsoneedstobechangedaccordingly

33

MODULES– SCOPE33

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Ascope isatextualregionofaPythonprogramwhereanamespaceisdirectlyaccessible

Whattypesofscopescanbedefined?• Localscope referstoidentifiersdeclaredwithinafunction(theseidentifiersarekeptinthenamespacethatbelongstothefunction,andeachfunctionhasitsownnamespace)

• Globalscope referstoalltheidentifiersdeclaredwithinthecurrentmodule,orfile

• Built-inscope referstoalltheidentifiersbuiltintoPython(thoselike range and min thatcanbeusedwithouthavingtoimportanything)

34

MODULES– SCOPE34

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

Whatarethescopeprecedencerules?• Thesamenamecanoccurinmorethanoneofthesescopes,buttheinnermost,orlocalscope,willalwaystakeprecedenceovertheglobalscope,andtheglobalscopealwaysgetsusedinpreferencetothebuilt-inscope

• Namescanbe“hidden”fromuseifownvariablesorfunctionsreusethosenames

• EXAMPLE:variables n andm arecreated justforthedurationoftheexecutionoff sincetheyarecreatedinthelocalnamespaceoffunction f(precedencerulesapply)

35

MODULES– THEDOTOPERATOR35

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html

• Variablesdefinedinsideamodulearecalled attributes ofthemodule

• Attributesareaccessedusingthe dot operator (.)• Whenadottednameisuseditisoftenreferredtoitasa fullyqualifiedname

36

MOTIVATION– DATASCIENCE36

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttps://www.kaggle.com/surveys/2017

37

MOTIVATION– DATASCIENCE37

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

sourcehttps://www.kaggle.com/surveys/2017

38

REFERENCES38

03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague

Thislecturere-usesselectedpartsoftheOPENBOOKPROJECTLearningwithPython3(RLE)

http://openbookproject.net/thinkcs/python/english3e/index.htmlavailableunderGNUFreeDocumentationLicense Version1.3)

• Versiondate:October2012• byPeterWentworth,JeffreyElkner,AllenB.Downey,andChrisMeyers

(basedon2ndeditionbyJeffreyElkner,AllenB.Downey,andChrisMeyers)

• Sourcerepositoryisat https://code.launchpad.net/~thinkcspy-rle-team/thinkcspy/thinkcspy3-rle

• Forofflineuse,downloadazipfileofthehtmlorapdfversionfrom http://www.ict.ru.ac.za/Resources/cspw/thinkcspy3/