prg –programming essentials...2 administration 2 03/12/2017 michal reinštein, czech technical...
TRANSCRIPT
1
PRG– PROGRAMMINGESSENTIALS1
Lecture5– Modules,Namespaceshttps://cw.fel.cvut.cz/wiki/courses/be5b33prg/start
MichalReinšteinCzechTechnicalUniversityinPrague,
FacultyofElectricalEngineering,Dept.ofCybernetics,CenterforMachinePerceptionhttp://cmp.felk.cvut.cz/~reinsmic/
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
2
ADMINISTRATION2
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttps://cw.fel.cvut.cz/wiki/help/common/plagiarism_cheating
PLAGIARISMWARNINGhttps://cw.fel.cvut.cz/wiki/help/common/plagiarism_cheating
3
RECAP: MOREABOUTPYTHON3
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttps://www.youtube.com/watch?v=arxWaw-E8QQ&t=1s
• Themethodsandvariablesarecreatedonstackmemory• Theobjectsandinstancesarecreatedonheapmemory• Newstackframeiscreatedoninvocationofa
function/method• Stackframesaredestroyedassoonasthe
function/methodreturns• MechanismtocleanupthedeadobjectsisGarbagecollector• EverythinginPythonisobject• Pythonisdynamicallytypedlanguage
4
RECAP: LISTS4
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html
• Listsaremutable (wecanchangetheirelements)• Stringsareimmutable (wecannotchangetheirelements)• Useslicingprinciples(indexesinbetweencharacters/items)
5
RECAP:SLICING5
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/strings.html
• A substring ofastringisobtainedbytakinga slice• Slicealisttorefertosomesublist oftheitemsinthelist• Theoperator [n:m] returnsthepartofthestringfromthen’thcharactertothem’thcharacter,includingthefirstbutexcludingthelast(indicespointing between thecharacters)
• Sliceoperator [n:m] copies outthepartofthepaperbetweenthe n andm positions
• Resultof[n:m] willbeoflength(m-n)
6
RECAP: STRINGSvs.LISTS6
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html
• Variables a and b refertostringobjectwithletters "banana”• Useis operatororid functiontofindoutthereference• Stringsare immutable,Pythonoptimizesresourcesbymakingtwonamesthatrefertothesamestringvaluerefertothesameobject
• Notthecaseoflists:a and b havethesamevalue(content)butdonotrefertothesameobject
Strings
Lists
7
RECAP: LISTS– ALIASING,CLONING7
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html
• Ifweassignonevariabletoanother,bothvariablesrefertothesameobject
• Thesamelisthastwodifferentnames wesaythatitis aliased (changesmadewithonealiasaffecttheother)
• RECOMMENDATION:avoidaliasingwhenyouareworkingwithmutableobjects
• Ifneedtomodifyalistandkeepacopyoftheoriginalusethesliceoperator(takinganysliceof a createsanewlist)
8
RECAP: LISTPARAMETERS8
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/lists.html
• Passingalistasanargumentpassesareference tothelist,notacopyorcloneofthelist
• Soparameterpassingcreatesanalias
9
MODULES9
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Amodule isafilecontainingPythondefinitionsandstatementsintendedforuseinotherPythonprograms
• Modulesareoneofthemainabstractionlayersavailableandprobablythemostnaturalone
• Abstractionlayersallowseparatingcodeintopartsholdingrelateddataandfunctionality
• Themostnaturalwaytoseparatethesetwolayersistoregroupallinterfacingfunctionalityintoonefile,andalllow-leveloperationsinanotherfile
• Donewiththe import and from... import statements
10
MODULES10
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
WheredoesPythonlookforimportedmodules?• Alistnamed path inmodule sys containsallthelocations(andcanbemodified)
• ThefirstitemonthelististhedirectorywiththeprogramWhathappenswhenPythonimportsamodule?• Pythonsearchesthemoduleamongalreadyimportedmodulesin sys.modules
• Ifthemoduleisnotfoundin sys.modules,Pythonsearcheslocationsin sys.path, executesthemodule onceitisfound,andrecordsthatthemodulehasalreadybeenimported
• Pythoncreatesnamesinlocalnamespace fortheimportedmodule,orforallthevariables,functions,etc.importedfromthemodule
11
MODULES11
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Modulesthatcanbebothrun andimported• Specialvariablecalled __name__ insideeachmoduleitcontains:• thenameofthemodule(ofthe.py file)whenthemoduleisimported
• string "__main__" whenthe.py fileisrunasaprogram(script)
• Variable __name__ isdefinedinboththecallingnamespaceandinthenamespaceofthemodule
12
MODULES12
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://docs.python-guide.org/en/latest/writing/structure/#modules
• Togainaccesstosymbolsdefinedinamodule(i.e.inadifferentnamespace)modulehastobeimported(3ways)
• Thestatementsloadamodule,createanameinthecurrentnamespace,andbindthenametotheloadedmodule
13
MODULES– RANDOMNUMBERS13
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Toincluderandomdecision-makingprocess• Totakesamplesfromprobabilitydistributions• Toplayagameofchancewherethecomputerneedstothrowsomedice,pickanumber,orflipacoin…
• Toshuffleadeckofplayingcardsrandomly…• Inmodellingandsimulations:weathermodels,environmentalmodels,MonteCarlomethod
• ForencryptingbankingsessionsontheInternet
14
MODULES– RANDOMNUMBERS14
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• The randommethodreturnsafloatingpointnumberintheinterval[0.0,1.0)— thesquarebracketmeans“closedintervalontheleft”andtheroundparenthesismeans“openintervalontheright” – 0.0ispossible,butallreturnednumberswillbestrictlylessthan1.0.
• Itisusualto scale theresultsaftercallingthismethodtogetthemintoanintervalsuitableforapplication.
• EXAMPLE:scalingtoanumberintheinterval[0.0,5.0)(uniformlydistributednumbers— numberscloseto0arejustaslikelytooccurasnumberscloseto0.5orcloseto1.0)
15
MODULES– RANDOMNUMBERS15
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• The randrangemethodgeneratesanintegerbetweenitslower andupper argument
• Therandrangemethodsamesemanticsas range(sothelowerboundisincluded,buttheupperboundisexcluded)
• Allthevalueshaveanequalprobabilityofoccurring(i.e.theresultsare uniformly distributed).
• Randrange alsotakesanoptionalstepargument(likerange)
• EXAMPLE:Weneededarandomoddnumberlessthan100
16
MODULES– RANDOMNUMBERS16
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
Thisexampleshowshowtoshuffle alist.(shuffle cannotworkdirectlywithrangeobjectsolist typeconverterfirstisnecessary)
• Randomnumbergeneratorsarebasedona deterministic algorithm— repeatable andpredictable
• Called pseudo-random generators(notgenuinelyrandom)• Eachtimeyouaskforanotherrandomnumber,you’llgetonebasedonthecurrentseedattribute,andthestateoftheseed
17
MODULES– RANDOMNUMBERS17
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Repeatabilityfordebugging andforwritingunittests(programsthatdothesamethingeverytimetheyarerun)
• Forcingtherandomnumbergeneratortobeinitializedwithaknownseedeverytime(oftenthisisonlywantedduringtesting/back-testing)
• Withoutthisseedargument,thesystemprobablyusessomethingbasedontheOStime.
18
MODULES– RANDOMNUMBERS18
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• EXAMPLE:generatealistcontaining n randomints betweenalowerandupperbound
• NOTE:thatwegotaduplicates intheresult(oftenthisiswanted,e.g.ifwethrowadiefivetimes,wewouldexpectsomeduplicates)
19
MODULES– RANDOMNUMBERS19
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
Howtotakecareofduplicates?
• Ifyouwanted5distinctmonths,thenthisalgorithmiswrong• Inthiscaseagoodalgorithmistogeneratethelistof
possibilities,shuffle it,andsliceoff thenumberofelementsyouwant:
20
MODULES– RANDOMNUMBERS20
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Allowingduplicates (usuallydescribedaspullingballsoutofabag withreplacement)
• Noduplicates (usuallydescribedaspullingballsoutofthebag withoutreplacement)
• Algorithm“shuffleandslice”isnotidealforcaseofchoosingfewelementsfromaverylargedomain
(Supposetheneedforfivenumbersbetween1and10million,withoutduplicates.Generatingalistoftenmillionitems,shufflingit,andthenslicingoffthefirstfivewouldbeaperformancedisaster.)
21
MODULES– RANDOMNUMBERS21
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
Choosewiselythealgorithmbasedontheinputdata!
22
MODULES– TIME22
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
Howefficientandreliableisourcode?• Onewaytoexperimentisto
timehowlongvariousoperationstakeandwhatthememoryrequirementsare(relatedtoalgorithmcomplexity:https://people.duke.edu/~ccc14/sta-663/AlgorithmicComplexity.html )
• The timemodulehasafunction clock thatisrecommended• Whenever clock iscalled,itreturnsafloatingpointnumber
representinghowmanysecondshaveelapsedsinceyourprogramstartedrunning(variesaccordingtoOS!)
23
MODULES– TIME23
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• EXAMPLE:Generatingandsumminguptenmillionelementsinunderasecond
• Proprietaryfunctionruns57%slowerthanthebuilt-in one.
24
MODULES– TIME24
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttps://people.duke.edu/~ccc14/sta-663/AlgorithmicComplexity.html
25
MODULES– MATH25
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Themathmodulecontainsmathematicalfunctionstypicallyfoundonacalculator,includingmathematicalconstantslike pi and e
• Functions radians and degrees toconvertangles• Mathematicalfunctionsarepure anddonothaveanystate
26
MODULES– CREATINGMODULES26
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Createownmodules – savescriptasafilewith .py extension• EXAMPLE:functionremove_at inascriptissavedasafilenamed seqtools.py
• Themodulemustbefirstimportedbeforeuse(.pyisthefileextensionandisnotincludedinthe importstatement)
• RECOMMENDATION:breakupverylargeprogramsintomanageablesizedpartsandkeeprelatedpartstogether
27
MODULES– NAMESPACES27
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Namespaceisamapping fromnamestoobjects• Namespace isacollectionofidentifiersthatbelongtoamodule,function,oraclass
• Namespaceissetofsymbolsusedtoorganizeobjectsofvariouskindssothatwecanrefertothembyname
• Namespacespermitprogrammerstoworkonthesameprojectwithouthavingnamingcollisions(allownamereuse)
• Oftenhierarchically structured• Eachnamemustbeunique initsnamespace• NamespaceisverygeneralconceptnotlimitedtoPython• Eachmodulehasitsownnamespace – wecanusethesameidentifiernameinmultiplemoduleswithoutcausinganidentificationproblem
28
MODULES– NAMESPACES28
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
HowarenamespacesdefinedinPython?• Packages(collectionsofrelatedmodules)• Modules(.pyfilescontainingdefinitionsoffunctions,classes,variables,etc.)
• Classes,Functions…Whatisthedifferencebetweenprogramsandmodules?• Botharestoredin.pyfiles.• Programs (scripts)aredesignedtoberun(executed)• Modules (libraries)aredesignedtobeimportedandusedbyotherprogramsandothermodules
• Specialcase:.pyfileisdesignedtobebothaprogramandamodule(itcanbeexecutedaswellasimportedtoprovidefunctionalityforothermodules)
29
MODULES– NAMESPACES29
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
30
MODULES– NAMESPACES30
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Functionsalsohaveownnamespaces• Functionscanread(read-only)variableintheouterscope• EXAMPLE:thethree n‘sabovedonotcollidesincetheyareeachinadifferentnamespace— threenamesforthreedifferentvariables
31
MODULES,NAMESPACES,FILES31
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Pythonhasaconvenientandsimplifyingone-to-onemapping:onemoduleperfile – givingrisetoonenamespace
• Pythontakesthemodulenamefromthefilename,andthisbecomesthenameofthenamespace
• EXAMPLE:math.py isafilename,themoduleiscalledmath,anditsnamespaceismath (inPythontheconceptsaremoreorlessinterchangeable)
• Inotherlanguages(e.g.C#)onemodulecanspanmultiplefiles,oronefiletohavemultiplenamespaces,ormanyfilestoallsharethesamenamespace
32
MODULES,NAMESPACES,FILES32
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• RECOMMENDATION:keeptheconceptsdistinctinyourmind• Filesanddirectoriesorganize where codeanddataarestored• Namespacesandmodulesareaprogrammingconcepts:helpusorganizehowwewanttogrouprelatedfunctionsandattributes.
• Namespacesarenotabout“where”tostorethings,andshouldnothavetocoincidewiththefilestructures
• Ifthefile math.py isrenamed,itsmodulenameneedstobechanged, import statementsneedtobechanged,andthecodethatreferstofunctionsorattributesinsidethatnamespacealsoneedstobechangedaccordingly
33
MODULES– SCOPE33
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Ascope isatextualregionofaPythonprogramwhereanamespaceisdirectlyaccessible
Whattypesofscopescanbedefined?• Localscope referstoidentifiersdeclaredwithinafunction(theseidentifiersarekeptinthenamespacethatbelongstothefunction,andeachfunctionhasitsownnamespace)
• Globalscope referstoalltheidentifiersdeclaredwithinthecurrentmodule,orfile
• Built-inscope referstoalltheidentifiersbuiltintoPython(thoselike range and min thatcanbeusedwithouthavingtoimportanything)
34
MODULES– SCOPE34
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
Whatarethescopeprecedencerules?• Thesamenamecanoccurinmorethanoneofthesescopes,buttheinnermost,orlocalscope,willalwaystakeprecedenceovertheglobalscope,andtheglobalscopealwaysgetsusedinpreferencetothebuilt-inscope
• Namescanbe“hidden”fromuseifownvariablesorfunctionsreusethosenames
• EXAMPLE:variables n andm arecreated justforthedurationoftheexecutionoff sincetheyarecreatedinthelocalnamespaceoffunction f(precedencerulesapply)
35
MODULES– THEDOTOPERATOR35
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttp://openbookproject.net/thinkcs/python/english3e/modules.html
• Variablesdefinedinsideamodulearecalled attributes ofthemodule
• Attributesareaccessedusingthe dot operator (.)• Whenadottednameisuseditisoftenreferredtoitasa fullyqualifiedname
36
MOTIVATION– DATASCIENCE36
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttps://www.kaggle.com/surveys/2017
37
MOTIVATION– DATASCIENCE37
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
sourcehttps://www.kaggle.com/surveys/2017
38
REFERENCES38
03/12/2017 MichalReinštein,CzechTechnicalUniversityinPrague
Thislecturere-usesselectedpartsoftheOPENBOOKPROJECTLearningwithPython3(RLE)
http://openbookproject.net/thinkcs/python/english3e/index.htmlavailableunderGNUFreeDocumentationLicense Version1.3)
• Versiondate:October2012• byPeterWentworth,JeffreyElkner,AllenB.Downey,andChrisMeyers
(basedon2ndeditionbyJeffreyElkner,AllenB.Downey,andChrisMeyers)
• Sourcerepositoryisat https://code.launchpad.net/~thinkcspy-rle-team/thinkcspy/thinkcspy3-rle
• Forofflineuse,downloadazipfileofthehtmlorapdfversionfrom http://www.ict.ru.ac.za/Resources/cspw/thinkcspy3/