implementing c++ semantics in python
TRANSCRIPT
![Page 1: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/1.jpg)
1© All rights reserved
Implementing C++ Semantics in Python
Tamir Bahar
![Page 2: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/2.jpg)
From C++ Import MagicImplementing C++ Semantics in Python
2
@tmr232
Tamir Bahar (He/Him)
![Page 3: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/3.jpg)
Pre-COVID Hairstyle
3
![Page 4: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/4.jpg)
Before we start, a few questions
• Who uses C++?
• Who uses C++ their main language?
• Who uses Python?
• Who uses Python as their main language?
4
![Page 5: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/5.jpg)
Your Questions• Try to write down slide numbers
5
![Page 6: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/6.jpg)
But Why?
C++
• Low level
• "Expert oriented"
• Slowly becoming "Pythonic"
Python
• High Level
• Beginner friendly
• Less footguns
6
![Page 7: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/7.jpg)
Resource Management
• In C++, all resources are equal
• Python is garbage-collected
• Memory is handled by the language
• Other resources are handled by the programmer
• Files, sockets, locks, DB connections, etc.
7
![Page 8: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/8.jpg)
_tmp.__exit__(<exception-info>)
print(reader.read())
_tmp = FileReader(path)reader = _tmp.__enter__()
Context Managers
8
with FileReader(path) as f:
print(f.read())
End of blockReturn
Exception
![Page 9: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/9.jpg)
Context Managers, continued
9
class FileReader:def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):self.close()
...
<exception-info>
![Page 10: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/10.jpg)
Real Code - Archive Reader
10
class ArchiveReader:def __init__(self, path: str):
self.data = {}with ZipFile(path) as zipfile:
for name in zipfile.namelist():with zipfile.open(name) as f:
self.data[name] = f.read()
def read(self, name):return self.data[name]
![Page 11: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/11.jpg)
Real Code - Archive Reader
11
class ArchiveReader:def __init__(self, path: str):
self.data = {}with ZipFile(path) as zipfile:
for name in zipfile.namelist():with zipfile.open(name) as f:
self.data[name] = f.read()
def read(self, name):return self.data[name]
reader = ArchiveReader("corecpp.zip")print(reader.read("2021"))
Hello CoreC++!
![Page 12: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/12.jpg)
Archive Reader, continued
• Archives got larger
• Time to open archive grows
• Can no longer unzip entire archive in memory
• Need to hold open ZipFile in our Archive Reader
12
![Page 13: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/13.jpg)
Big Archive Reader
13
class BigArchiveReader:def __init__(self, path: str):
self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def __enter__(self):return self
def __exit__(self, exc_type, exc_val, exc_tb):self.zipfile.close()
Create
Use
Destruct
![Page 14: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/14.jpg)
Big Archive Reader, continued
• Context managers
change interface
• Interface changes
propagate
• Usage
• Composition
14
with BigArchiveReader("corecpp.zip") as big_reader:print(reader.read("2021"))
reader = ArchiveReader("corecpp.zip")print(reader.read("2021"))
...
def __exit__(self, exc_type, exc_val, exc_tb):self.big_reader.close()
...
![Page 15: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/15.jpg)
C++ To The Rescue!
15
![Page 16: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/16.jpg)
Destructors
• C++'s solution to the resource-management problem
• 3 main properties
• Automatic
• Composable
• Implicit
16
![Page 17: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/17.jpg)
Automatic Invocation
17
{auto reader = FileReader(path);
std::cout << reader.read() << '\n';}
End of blockReturn
Exception reader.~FileReader();
![Page 18: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/18.jpg)
Seamless Composition
18
class ArchiveReader {...
};
auto reader = ArchiveReader(path);}
~ArchiveReader();
class BigArchiveReader {ZipFile zipfile;...
};
auto big_reader = BigArchiveReader(path);}
~BigArchiveReader();
~ZipFile();
![Page 19: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/19.jpg)
Implicit Interfaces
19
{auto object = Object();
}
{auto object = Object();
}
With Destructors Without Destructors
![Page 20: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/20.jpg)
Implicit Interfaces
• No change in interface or usage
• No change propagation
20
{auto object = Object();
}
{auto object = Object();
}
With Destructors Without Destructors
![Page 21: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/21.jpg)
Our Goal - From This• 11 lines• 4 are resource
management
21
class BigArchiveReader:zipfile: ZipFile
def __init__(self, path: str):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def __enter__(self):return self
def __exit__(self, exc_type, exc_val, exc_tb):self.zipfile.close()
![Page 22: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/22.jpg)
Our Goal - To This
22
class BestArchiveReader:zipfile: ZipFile
def BestArchiveReader(self, path: str):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
• 7 lines• 0 are resource
management
![Page 23: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/23.jpg)
Our Goal
• From interface pollution
• To normal objects
23
with BigArchiveReader("corecpp.zip") as big_reader:print(reader.read("2021"))
best_reader = BestArchiveReader("corecpp.zip")print(best_reader.read("2021"))
![Page 24: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/24.jpg)
24
Don't try this at work!
Hacks Ahead!
![Page 25: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/25.jpg)
class Greeter:def __init__(self, name):
self.name = nameprint(f"Hello, {self.name}!")
def __enter__(self):return self
def __exit__(self, e_type, e_val, e_tb):print(f"Goodbye, {self.name}.")
def main():with Greeter(1):
print("We have a greeter!")
Greetings!
25
Hello, 1!We have a greeter!Goodbye, 1.
![Page 26: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/26.jpg)
Automatic
26
![Page 27: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/27.jpg)
Stacking Dtors
27
def main():with Greeter(1):
print("Hello, Greeters!")Hello, 1!Hello, Greeters!Goodbye, 1.
![Page 28: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/28.jpg)
Stacking Dtors
28
def main():with Greeter(1):
with Greeter(2):print("Hello, Greeters!")
Hello, 1!Hello, 2! Hello, Greeters! Goodbye, 2.Goodbye, 1.
![Page 29: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/29.jpg)
Stacking Dtors
29
def main():with Greeter(1):
with Greeter(2):with Greeter(3):
print("Hello, Greeters!")
Hello, 1!Hello, 2!Hello, 3!Hello, Greeters!Goodbye, 3.Goodbye, 2.Goodbye, 1.
![Page 30: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/30.jpg)
Stacking Dtors
30
def main():with Greeter(1):
with Greeter(2):with Greeter(3):
with Greeter(4):print("Hello, Greeters!")
Hello, 1!Hello, 2!Hello, 3!Hello, 4!Hello, Greeters!Goodbye, 4.Goodbye, 3.Goodbye, 2.Goodbye, 1.
![Page 31: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/31.jpg)
Stacking Dtors
31
def main():with Greeter(1):
with Greeter(2):with Greeter(3):
with Greeter(4):print("Hello, Greeters!")
![Page 32: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/32.jpg)
Stacking Dtors
32
def main():with Greeter(1):
with Greeter(2):with Greeter(3):
with Greeter(4):print("Hello, Greeters!")
![Page 33: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/33.jpg)
A Proper Stack
class DtorScope:def __init__(self):
self.stack = []
def __enter__(self):return self
def __exit__(self, exc_type, exc_val, exc_tb):while self.stack:
self.stack.pop().__exit__(exc_type, exc_val, exc_tb)
def push(self, cm):self.stack.append(cm)
33
![Page 34: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/34.jpg)
A Proper Stack, continued
def main():with DtorScope() as dtor_stack:
greeter1 = Greeter(1)dtor_stack.push(greeter1)
greeter2 = Greeter(2)dtor_stack.push(greeter2)
34
Hello, 1!Hello, 2!Goodbye, 2.Goodbye, 1.
![Page 35: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/35.jpg)
Implicit
35
![Page 36: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/36.jpg)
def main():with DtorScope() as dtor_stack:
greeter1 = Greeter(1)dtor_stack.push(greeter1)
greeter2 = Greeter(2)dtor_stack.push(greeter2)
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
Removing Hiding Boilerplate
36
![Page 37: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/37.jpg)
class Greeter:def __init__(self, name, dtor_stack):
self.name = nameprint(f"Hello, {self.name}!")
dtor_stack.push(self)...
def main():with DtorScope() as dtor_stack:
greeter1 = Greeter(1, dtor_stack)greeter2 = Greeter(2, dtor_stack)
A Layer of Indirection
37
![Page 38: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/38.jpg)
Another Layer of Indirection
38
def main():with DtorScope() as dtor_stack:
greeter1 = Greeter(1, dtor_stack)greeter2 = Greeter(2, dtor_stack)
def main():with DtorScope():
greeter1 = Greeter(1)greeter2 = Greeter(2)
![Page 39: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/39.jpg)
Another Layer of Indirection
39
def main():with DtorScope() as dtor_stack:
greeter1 = Greeter(1, dtor_stack)greeter2 = Greeter(2, dtor_stack)
def main():with DtorScope():
greeter1 = Greeter(1)greeter2 = Greeter(2)
![Page 40: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/40.jpg)
_dtor_stack = []
def get_dtor_stack():return _dtor_stack
def push_dtor(cm):return get_dtor_stack()[-1].push(cm)
class DtorScope:def __init__(self):
get_dtor_stack().append(self)
...
def __exit__(self, exc_type, exc_val, exc_tb):get_dtor_stack().pop()...
...
Globals to the Rescue!
40
![Page 41: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/41.jpg)
class Greeter:def __init__(self, name, dtor_stack):
dtor_stack.push(self)
self.name = nameprint(f"Hello, {self.name}!")
...
Globals to the Rescue!
41
![Page 42: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/42.jpg)
class Greeter:def __init__(self, name):
push_dtor(self)
self.name = nameprint(f"Hello, {self.name}!")
...
Globals to the Rescue!
42
![Page 43: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/43.jpg)
class Greeter:def __init__(self, name):
push_dtor(self)
self.name = nameprint(f"Hello, {self.name}!")
...
Globals to the Rescue!
43
def main():with DtorScope():
greeter1 = Greeter(1)greeter2 = Greeter(2)
Hello, 1!Hello, 2!Goodbye, 2.Goodbye, 1.
![Page 44: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/44.jpg)
Moving Out
45
def main():with DtorScope():
greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
![Page 45: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/45.jpg)
Moving Out
46
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
with DtorScope():main()
![Page 46: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/46.jpg)
Moving Out
47
def call(f, *args, **kwargs):with DtorScope():
return f(*args, **kwargs)
call(main)
Perfect Forwarding
![Page 47: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/47.jpg)
Moving Out
48
def cpp_function(f):
def _wrapper(*args, **kwargs):with DtorScope():
return f(*args, **kwargs)
return _wrapper
scoped_main = cpp_function(main)
scoped_main()
Closure
f is captured
Holds the closure
![Page 48: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/48.jpg)
Moving Out
49
def cpp_function(f):
def _wrapper(*args, **kwargs):with DtorScope():
return f(*args, **kwargs)
return _wrapper
main = cpp_function(main)
main()
Rebind the name "main"
![Page 49: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/49.jpg)
@cpp_functiondef main():
greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
Moving Out
50
def cpp_function(f):
def _wrapper(*args, **kwargs):with DtorScope():
return f(*args, **kwargs)
return _wrapper
main = cpp_function(main)
main()
Decorator syntax
![Page 50: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/50.jpg)
Moving Out• Declarative• Clean• Explicit
51
@cpp_functiondef main():
greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
![Page 51: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/51.jpg)
Methodic Pause
52
![Page 52: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/52.jpg)
Import HacksWhere things get hairy
53
![Page 53: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/53.jpg)
Basic File Structure
54
from cpp import cpp_function
from greeter import Greeter
@cpp_functiondef main():
greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
Import
Usage
![Page 54: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/54.jpg)
Wouldn't it be Nice?
55
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
Import
Magic!
![Page 55: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/55.jpg)
Modest Beginnings
56
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
magic()
main()
Magic!
![Page 56: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/56.jpg)
Making Magic
57
def magic():calling_module = get_calling_module()decorate_module_functions(calling_module)
![Page 57: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/57.jpg)
Making Magic
58
def magic():calling_module = get_calling_module()decorate_module_functions(calling_module)
import inspect
def get_calling_module():stack_frame = inspect.stack()[2].framemodule = inspect.getmodule(stack_frame)return module
Call Stack
Calling Module
magic()
get_calling_module()
2 places up the callstack
![Page 58: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/58.jpg)
Making Magic
59
def magic():calling_module = get_calling_module()decorate_module_functions(calling_module)
def decorate_module_functions(module):for name, value in inspect.getmembers(module):
if not inspect.isroutine(value):continue
if inspect.getmodule(value) != module:continue
setattr(module, name, cpp_function(value))
All module members
Only functions
Defined in the module
![Page 59: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/59.jpg)
For My Next Trick...
60
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
magic()
main()
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
![Page 60: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/60.jpg)
For My Next Trick...
61
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
magic()
main()
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
![Page 61: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/61.jpg)
Import Mechanism
63
from cpp import magic
Is cpp in cache?
Create cpp module object
Add cpp module to cache
Execute cpp moduleIs magic in
cpp?
magic = cpp.magic magic = cpp.__getattr__("magic")
Global Module Cachesys.modules
Name Binding
![Page 62: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/62.jpg)
Import Mechanism
64
from cpp import magic
Is cpp in cache?
Create cpp module object
Add cpp module to cache
Execute cpp moduleIs magic in
cpp?
magic = cpp.magic magic = cpp.__getattr__("magic")
Global Module Cachesys.modules
Name Binding
![Page 63: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/63.jpg)
Import is a Function Call
65
def _magic():calling_module = get_calling_module()decorate_module_functions(calling_module)
def __getattr__(name):if name != "magic":
raise AttributeError()
_magic()
![Page 64: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/64.jpg)
Where is the Magic?
66
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
Hello, 1!Hello, 2!
![Page 65: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/65.jpg)
Where is the Magic?
67
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
from cpp import magic
main()
![Page 66: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/66.jpg)
Parallel Import
68
import importlib.utilimport sys
def import_by_path(name: str, path: str):spec = importlib.util.spec_from_file_location(name, path)module = importlib.util.module_from_spec(spec)sys.modules[name] = modulespec.loader.exec_module(module)return module
Create cpp module object
Add cpp module to cache
Execute cpp module
![Page 67: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/67.jpg)
Parallel Import
69
def _magic():calling_module = get_calling_module()
name = calling_module.__name__path = calling_module.__file__imported_module = import_by_path(name, path)
decorate_module_functions(imported_module)
Parallel Import
![Page 68: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/68.jpg)
Parallel Import, continued
70
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
![Page 69: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/69.jpg)
Parallel Import, continued
71
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
RecursionError:maximum recursion depth exceeded while calling a Python object
_magic()
![Page 70: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/70.jpg)
Break the Loop
72
IMPORT_FLAG = "__magically_imported__"
def import_by_path(name: str, path: str):...sys.modules[name] = module
setattr(module, IMPORT_FLAG, True)
spec.loader.exec_module(module)return module
def _magic():...if hasattr(calling_module, IMPORT_FLAG):
return
imported_module = import_by_path(name, path)
decorate_module_functions(imported_module)
Set
Check
![Page 71: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/71.jpg)
Break the Loop, continued
73
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
![Page 72: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/72.jpg)
Break the Loop, continued
74
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
main()
IndexError: list index out of range
def push_dtor(cm):return get_dtor_stack()[-1].push(cm)
Decoration happens here
![Page 73: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/73.jpg)
Main Function
75
def _magic():...
if imported_module.__name__ == "__main__":sys.exit(imported_module.main())
![Page 74: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/74.jpg)
Real Magic
76
from cpp import magic
from greeter import Greeter
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
Hello, 1!Hello, 2!Goodbye, 2.Goodbye, 1.
![Page 75: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/75.jpg)
Methodic PauseQuestions?
77
![Page 76: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/76.jpg)
Greetings, Again
78
class Greeter:def __init__(self, name):
push_dtor(self)
self.name = nameprint(f"Hello, {self.name}!")
def __enter__(self):return self
def __exit__(self, exc_type, exc_val, exc_tb):print(f"Goodbye, {self.name}.")
Boilerplate
![Page 77: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/77.jpg)
Base Class
79
class Greeter(CppClass):def Greeter(self, name):
self.name = nameprint(f"Hello, {self.name}!")
def _Greeter(self):print(f"Goodbye, {self.name}.")
C++ Style Ctor & Dtor
Sorry, no ~
![Page 78: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/78.jpg)
Base Class, implementation
80
class CppClass:def __init__(self, *args, **kwargs):
push_dtor(self)
ctor = getattr(self, self.__class__.__name__, None)if ctor:
ctor(*args, **kwargs)
def __enter__(self):return self
def __exit__(self, exc_type, exc_val, exc_tb):dtor = getattr(self, "_" + self.__class__.__name__, None)if dtor:
dtor()
Only call if exists
![Page 79: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/79.jpg)
Decorated Methods
81
def decorate_object_methods(obj):for name, value in inspect.getmembers(obj):
if name.startswith("__"):continue
if not inspect.isroutine(value):continue
setattr(self, name, cpp_function(value))
class CppClass:def __init__(self, *args, **kwargs):
...decorate_object_methods(self)
No special methods
Only functions
![Page 80: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/80.jpg)
Progress Check
82
class Greeter(CppClass):def Greeter(self, name):
self.name = nameprint(f"Hello, {self.name}!")
def _Greeter(self):print(f"Goodbye, {self.name}.")
![Page 81: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/81.jpg)
(More) Problems with Inheritance
83
class Greeter(CppClass):def Greeter(self, name):
self.name = nameprint(f"Hello, {self.name}!")
def _Greeter(self):print(f"Goodbye, {self.name}.")
Explicit
![Page 82: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/82.jpg)
Compositionally Speaking
84
class Greeter(CppClass):Greeter_Greeter
CppClass.__init__CppClass.__enter__CppClass.__exit__
class CppClass:__init____enter____exit__
![Page 83: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/83.jpg)
Compositionally Speaking, continued
85
class Greeter:...
Greeter.__init__ = __init__Greeter.__enter__ = __enter__Greeter.__exit__ = __exit__
![Page 84: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/84.jpg)
Decorated Classes
86
def cpp_class(cls):decorate_object_methods(self)
def __init__(self, *args, **kwargs):...
def __enter__(self):...
def __exit__(self, exc_type, exc_val, exc_tb):...
cls.__init__ = __init__cls.__enter__ = __enter__cls.__exit__ = __exit__
return cls
@cpp_classclass Greeter:
...
![Page 85: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/85.jpg)
Decorated Classes
87
def cpp_class(cls):decorate_object_methods(self)
def __init__(self, *args, **kwargs):...
def __enter__(self):...
def __exit__(self, exc_type, exc_val, exc_tb):...
cls.__init__ = __init__cls.__enter__ = __enter__cls.__exit__ = __exit__cls.__cpp_class__ = True
return cls
@cpp_classclass Greeter:
...
A little extra
def is_cpp_class(obj):return hasattr(obj, '__cpp_class__')
![Page 86: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/86.jpg)
More Magic!
88
def decorate_module_classes(module):for name, value in inspect.getmembers(module):
if not inspect.isclass(value):continue
if inspect.getmodule(value) != module:continue
setattr(module, name, cpp_class(value))
def _magic():...decorate_module_classes(imported_module)...
Only classes
Defined in the module
![Page 87: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/87.jpg)
Applied Magic
89
from cpp import magic
class Greeter:def Greeter(self, name):
self.name = nameprint(f"Hello, {self.name}!")
def _Greeter(self):print(f"Goodbye, {self.name}.")
def main():greeter1 = Greeter(1)greeter2 = Greeter(2)
Hello, 1!Hello, 2!Goodbye, 2.Goodbye, 1.
![Page 88: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/88.jpg)
Methodic PauseAny Questions?
90
![Page 89: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/89.jpg)
A Short Recap
• Automatic
• Dtors are called automatically
• Implicit
• Just import magic!
• Functions & classes automatically converted
• main() is automatically called
• Our next stop: Composition
91
![Page 90: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/90.jpg)
Composition
92
![Page 91: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/91.jpg)
Looking Back
93
class BigArchiveReader:zipfile: ZipFile
def __init__(self, path: str):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def __enter__(self):return self
def __exit__(self, exc_type, exc_val, exc_tb):self.zipfile.close()
• 11 lines• 4 are resource
management
![Page 92: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/92.jpg)
Looking Back
94
class BetterArchiveReader:zipfile: ZipFile
def BetterArchiveReader(self, path: str):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def _BetterArchiveReader(self):self.zipfile.close()
• 9 lines• 2 are resource
management
![Page 93: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/93.jpg)
Looking Back, issues
95
class BetterArchiveReader:zipfile: ZipFile
def BetterArchiveReader(self, path: str):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def _BetterArchiveReader(self):self.zipfile.close()
Dtor called twice!
~ZipFile
Scope End
Dtor
![Page 94: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/94.jpg)
Looking Back, issues solution?
96
class BetterArchiveReader:zipfile: ZipFile
def BetterArchiveReader(self, path: str):self.zipfile = ZipFile(path)remove_dtor(self.zipfile)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def _BetterArchiveReader(self):self.zipfile.close()
Remove from Dtor scope
![Page 95: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/95.jpg)
Remove from Dtor Scope
97
class DtorScope:stack: list...def remove(self, cm):
self.stack.remove(cm)
Equality Based
![Page 96: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/96.jpg)
Remove from Dtor Scope
98
class DtorScope:stack: list...def remove(self, cm):
self.stack.remove(IdentityComparator(cm)
)
Identity check
class IdentityComparator:def __init__(self, obj):
self.obj = obj
def __eq__(self, other):return self.obj is other
operator==
![Page 97: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/97.jpg)
Remove from Dtor Scope, continued
99
class BetterArchiveReader:zipfile: ZipFile
def BetterArchiveReader(self, path: str):self.zipfile = ZipFile(path)remove_dtor(self.zipfile)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
def _BetterArchiveReader(self):self.zipfile.close()
Explicit
![Page 98: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/98.jpg)
A Case for Getters & Setters
100
def get_zipfile(self):return getattr(self, "zipfile")
def set_zipfile(self, zipfile):old = getattr(self, "zipfile", None)
if is_cpp_class(old):old.__exit__(None, None, None)
if is_cpp_class(zipfile):remove_dtor(zipfile)
setattr(self, "zipfile", zipfile)
Just return
Destruct old value
Handle new value
![Page 99: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/99.jpg)
Descriptors
101
class BetterArchiveReader:zipfile = CppMember()
def BetterArchiveReader(self, path):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
...
def _BetterArchiveReader(self):self.zipfile.close()
zipfile.__set__(self, ZipFile(path))
zipfile.__get__(self)
BetterArchiveReader.zipfile.__set_name__(BetterArchiveReader, "zipfile")
![Page 100: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/100.jpg)
Descriptors, continued
102
class CppMember:def __set_name__(self, owner, name):
self.private_name = "_" + name
def __get__(self, instance, owner=None):return getattr(instance, self.private_name)
def __set__(self, instance, value):old = getattr(instance, self.private_name, None)
...
setattr(instance, self.private_name, value)
Save & prefix member name
Use member name
![Page 101: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/101.jpg)
Remove the Dtor
103
class BetterArchiveReader:zipfile = CppMember()
def BetterArchiveReader(self, path):self.zipfile = ZipFile(path)
def read(self, name: str):...
def _BetterArchiveReader(self):self.zipfile.close()
Should be implicit
![Page 102: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/102.jpg)
Remove the Dtor, continued
104
def __exit__(self, exc_type, exc_val, exc_tb):
...
for name, value in reversed(inspect.getmembers(self)):if name.startswith("_"):
continue
if not is_cpp_class(value):continue
value.__exit__(None, None, None)
Reverse order
Avoid prefixed members
Has Dtor?
![Page 103: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/103.jpg)
Final Touches
105
class BetterArchiveReader:zipfile = CppMember()
def BetterArchiveReader(self, path):self.zipfile = ZipFile(path)
def read(self, name: str):...
![Page 104: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/104.jpg)
Final Touches
106
class BetterArchiveReader:zipfile = CppMember()
def BetterArchiveReader(self, path):self.zipfile = ZipFile(path)
def read(self, name: str):...
Explicit, we can do better!
![Page 105: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/105.jpg)
Type Annotations• Do nothing• Stored in __annotations__
107
class BetterArchiveReader:zipfile: ZipFile
Type annotation
![Page 106: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/106.jpg)
Type Annotations, continued
108
def create_members(cls):member_names = list(getattr(cls, "__annotations__", {}))
for name in member_names:member = CppMember()member.__set_name__(cls, name)setattr(cls, name, member)
setattr(cls, "__member_names__", member_names)
def cpp_class(cls):...create_members(cls)...
Save for later
Must call manually
![Page 107: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/107.jpg)
def __exit__(self, exc_type, exc_val, exc_tb):
...
for name in reversed(self.__member_names__):value = getattr(self, name, None)
if is_cpp_class(value):value.__exit__(None, None, None)
Type Annotations, continued
109
Traverse only members
![Page 108: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/108.jpg)
Finally - Best Archive Reader
110
class BestArchiveReader:zipfile: ZipFile
def BestArchiveReader(self, path: str):self.zipfile = ZipFile(path)
def read(self, name: str):with self.zipfile.open(name) as f:
return f.read()
• 7 lines• 0 are resource
management
![Page 109: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/109.jpg)
Wrap Up
• Automatic
• Dtors are called when/where needed
• Composable
• Members don't add boilerplate
• Implicit*
• No extra code
• No change in interfaces
• No interface pollution!
111
* Assuming the entire project uses cpp...
![Page 110: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/110.jpg)
Questions?
112
![Page 111: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/111.jpg)
Thanks
• Barak Itkin
• Adi Shavit
• Inbal Levi
113
![Page 112: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/112.jpg)
Extras
114
![Page 113: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/113.jpg)
Extras• Return• This• Member Access Specifiers
115
![Page 114: Implementing C++ Semantics in Python](https://reader030.vdocuments.site/reader030/viewer/2022012714/61ac942b58988a7e5a472b4a/html5/thumbnails/114.jpg)
116