introduciton to python

Creating Big Data: Introduction to Python

Moshe [email protected]

© All rights reserved: Moshe Kaplanhttp://top-performance.blogspot.com

Moshe Kaplan> Scale Hacker:Architecture (Sys and Data)R&DSolving Problems> Blogs:http://top-performance.blogspot.comhttp://blogs.microsoft.co.il/vprnd

2


About Me: It’s all About Scale

3


INTRODUCTIONBackground

4

http://geekandpoke.typepad.com/geekandpoke/2008/07/one-day-in-the.html


Wny Python?Simple and readable

Indentation: hard to messOOP and functionalDynamic and auto memory managementCan be packagedCross platformWide number of librariesWorks great w/ Java, C, .Net

5


Who is Behind?Python Software Foundation~1M USD annual budget Created at 1989Top 10 from 2008

6

https://en.wikipedia.org/wiki/Python_(programming_language)


Who is Using?

7

http://www.dhruvb.com/blog/images/slides/python/using-python.png


Python Version2.X Existing Projects3.X New ProjectsNot so backwards compatible

8


Materials

9

https://www.tocode.co.il10% DiscountCoupon Code: PyApplied


IDEsYou can use Notepad++

10


HELLO WORLDBackground

11

© All rights reserved: Moshe Kaplanhttp://top-performance.blogspot.com12

http://www.dhruvb.com/blog/presentations/python/#/2

© All rights reserved: Moshe Kaplanhttp://top-performance.blogspot.com13

http://www.dhruvb.com/blog/presentations/python/#/2


VARIABLES, NUMBERS AND STRINGSPython

14


C:\>PythonType "help", "copyright", "credits" or "license" for more information.>>> a = "hello">>> print(a)hello>>> print(type(a))<class 'str'>>>> a = 1>>> print(type(a))<class 'int'>>>>

Variables

15

Auto Declaration

Auto type assignment


NumbersInteger (int)FloatingComplex (a+ib)

16

http://www.slideshare.net/TahaniAlManie/python-3-programming-language


StringsImmutable (cannot modify)Char is a single char stringStarts w/ 0 and support –N

17

>>> a = "hello">>> a[0]'h'>>> a[-1]'o'


Strings Methods

18



Strings Operators

19



LISTS AND TUPLESPython

20


Lists BasicsMutableSupport multi typesAgain 0–N

21

>>> l = [1, 'b', 'ccc']>>> print(l[0])1>>> print(l[1:2]);['b']>>> l[2] = 3>>> print(l)>>> del(l[0])>>> print(l)['b', 3]


Lists MagicSupports list in a listFor each:

22

>>> a = [1, 2, 3]>>> [x ** 2 for x in a][1, 4, 9]



TuplesSimilar functionality to listsImmutable (cannot modify)Faster than listsPerfect for functional programming

23

>>> t = (1, 'b', 'ccc')>>> t[0] = 2Traceback (most recent call last):File "<stdin>", line 1, in <module>

TypeError: 'tuple' object does not support item assignment


Dictionaries BasicsKey/Value pairs (JSON like)Mutable (cannot modify)

24

>>> d = {'name':'moshe', 'course':'PySpark'}>>> print(d['name'])moshe>>> print(d.keys())dict_keys(['name', 'course'])>>> print(d.values())dict_values(['moshe', 'PySpark'])>>> print(d.items())dict_items([('name', 'moshe'), ('course', 'PySpark')])


Dictionaries Methods

25



FUNCTIONSPython

26


Functions BasicsFlexible parameters:

27

>>> def a(name = "moshe", course = "pycourse"):... print(name + ": " + course)...>>> a()moshe: pycourse>>> a("roni")roni: pycourse>>> a(course = "spark")moshe: spark


Variable Length Parameters and RefsNot ByVal and not ByRef:

Mutuable: Object Reference (array)Immutable: Change pointer each time (string)

28

>>> def f(*tup):... for t in tup:... print(t)...>>>>>>>>> f(1, "b", "ccc")1bccc

is vs ==


IF STATEMENTS AND LOOPSPython

29


ifTrue or False conditions (no null)Non Null TrueNon Zero True

30

>>> b = True>>> if b:... print(1)... elif b != True:... print(2)... else:... print(3)1>>> x = 'correct' if b == True else 'wrong'


For Loops: Strings

31

>>> for letter in 'python':... print(letter)python


For Loops: For Each

32

>>> for x in range(1, 5):... print(x)...1234


Whilecontinue: Well, continuebreak: Well, breakpass: Void, due to indentation

33

>>> while (x < 5):... print(x)... x = x + 101234


INPUT AND BASIC TERMINAL APPSPython

34


Filesf = open(“file_name” [, access_mode] [,buffering])

“r” read“w” recreate“a” append“b” add for binary reading

i = f.read() All at onceline = f.readline() Single linelines = f.readlines() All at once to a listf.write(string)f.close()

35


I/OImmutable (cannot modify)Char is a single char stringStarts w/ 0 and support –N

36

shuffelBlocks = (raw_input("Shuffel Blocks (y/n): ") == 'y')print(shuffelBlocks)assert(type(a) == type(1))


CLASSES AND ENTERPRISE LEVEL PROGRAMMINGPython

37


Classes

38

> Python class.pyMoshe is earning too muchRoni is earning 100Total Employees: 2

Static variableConstructor

Object var


InheritanceBy default Everything public__attribute Turn to privateReflection++:Multiple inheritance is supported

39

class shape:...

class Circle(shape):...


Modules and ImportsUse 3rd party code:

FunctionsClassesAttributes

40

>>> import module1 [, module2]>>> from module3 import name1 [, name2]

Only specific attributes


Creating ModulesSimple headerAnd we are done…

41

"""file my_module.pyModule 'my_module': my special module"""def my_func(a):if __name__ == '__main__':

print my_func(1)


Packaging

42

def is_number(s):try:

int(s)return True

except ValueError:return False

if __name__ == '__main__':COLOR_BREAK = 'blue'COLOR_HEADER_FOOTER = 'darkCyan';


Python CompilerSee also .pyc file in your stack

43


Python Integration

44


EXCEPTIONSPython

45


Try and ExceptCommon:

NameErrorTypeErrorIndexErrorKeyError

46

try:print(getattr(moshe, "course"))

except NameError:print("no attribute1")


TESTINGPython

47


How to Do that?

48

import my_moduleimport unittestclass my_module_test(unittest.TestCase):

def test_method1(self):self.assertFalse(method1(1))

if __name__ == '__main__':unittest.main()


PREPARE FOR SPARKPython

49


Functions are Objects

50

# Evil Code…>> log, exp = exp, log>> r = [ -1, 0, 1]>> map(abs, r)[1, 0, 1]>> def sum(x, y): return x+y>> reduce(sum, r)0>> def large(x): return x > 0>> filter(large, r)[1]


Thank You !

Moshe [email protected]

054-2291978

introduciton to python

Technology