bento lunch talk
TRANSCRIPT
![Page 1: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/1.jpg)
弁当 (BENTO)A packaging solution for python software
![Page 2: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/2.jpg)
WHAT’S PACKAGING ?•Whatever that has to happen between svn/git/etc... and the
end-user
•Different end-users, different workflows: not just tarballs
(maybe those guys don’t need packaging)
![Page 3: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/3.jpg)
DISTUTILS
•De-facto solution since ±2000
•Works well for simple packages:
• Setuptools/distribute
• add package dependencies
![Page 4: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/4.jpg)
DISTUTILS ISSUES
•Make one’s head hurt:
•Hard to extend
•Fragile codebase
•Reliability issues
![Page 5: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/5.jpg)
BENTO
•Declarative description of package: metadata inspection is simple
• Flexible installation scheme: install any file anywhere
• Layered internal architecture: easy to extend
• Scale down and up: simpler for small packages, more flexible for big ones
![Page 6: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/6.jpg)
A SIMPLE BENTO FILE
Name: fooVersion: 0.1Description: Package description.
Library: Packages:! foo, foo.core
![Page 7: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/7.jpg)
BENTOMAKER
• Command-line interface to bento
$ bentomaker configure$ bentomaker build$ bentomaker install# Or more simply$ bentomaker install
![Page 8: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/8.jpg)
ADDING DATA-FILES
Datafiles: pdf_documentation TargetDir: $pdfdoc Files: main.pdf
$ bentomaker configure --pdfdoc=/usr/pdfdoc
# ant-like glob format supportedDatafiles: pdf_documentation TargetDir: $pdfdoc Files: doc/**/*.pdf
![Page 9: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/9.jpg)
CUSTOM INSTALL PATHS
Path: foodir Description: foo directory Default: $datadir/foo
Datafiles: fubar-doc TargetDir: $foodir Files: main.foo
$ bentomaker configure --foodir=/usr/pdfdoc
![Page 10: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/10.jpg)
HOOK FILES
• Can “hook” python files into the build process
from bento.commands import [email protected] hello(context): print "hello"
• Hook are pure python: no auto-import, etc...
$ bentomaker hellohello
![Page 11: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/11.jpg)
FITTING IN THE PYTHON ECOSYSTEM
# Convert a setup.py to bento.info$ bentomaker convert
# setup.pyimport setuptoolsimport bento.distutilsbento.distutils.monkey_patch()# grab metadata from bento.infosetuptools.setup()
Convert distutils package to bento
Distutils compatibility layer (pip-installable !)
![Page 12: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/12.jpg)
INTERNALS
![Page 13: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/13.jpg)
DESIGN PRINCIPLES
• Usable as a library (still a few singleton to remove...)
• Low-coupling of commands: new package format should not care about building phase
•Driven by real packages (numpy, scipy, twisted, ipython, etc...)
![Page 14: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/14.jpg)
CODE ORGANIZATION• bento/core
• ply-based parser (bento/core/parser)
• python representation of bento.info
• bento/commands
• commands and contexts
• bento/distutils
• distutils compat layer
![Page 15: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/15.jpg)
COMMANDS AND CONTEXTS
Context Commandself.run(context)
• Contexts encapsulate a command ‘environment’• Context-command relationship is overridable (hook)• e.g. mechanism to interact with 3rd party build tools
self.pkgself.cmd_argvself.run_node
![Page 16: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/16.jpg)
MORE ABOUT COMMANDS
• Command options defined externally: any command can query any other’s options
• Commands dependencies resolved at runtime
@hooks.commanddef hello(context): print "hello"@hooks.commanddef goodbye(context): print "goodbye"@hooks.startupdef startup(context): context.set_before("goodbye", "hello")
![Page 17: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/17.jpg)
BUILD MANIFEST
• Goal: decouple build and install
• format: a json file containing info about built parts
• Inspired by cabal (‘distutils’ for haskell)
• build command produces a build manifest
• install only knows about build manifest
• Install only install built files
![Page 18: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/18.jpg)
NODE• Every file is represented internally as a node (waf concept)
• nodes are a in-memory representation of the fs
• gives reliable relative paths between files
sdir = os.getcwd()bdir = os.path.join(os.getcwd(), "build")root = create_root_with_source_tree(sdir, bdir)top_node = root.find_node(sdir)build_node = root.find_node(bdir)print top_node.path_from(build_node)
![Page 19: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/19.jpg)
SOME NEAT FEATURES (1)
• Using metadata in the packaged software
Name: fooVersion: 1.0MetaTemplateFile: foo.py.in
# foo.py.inname = $NAMEversion = $VERSION
# foo.py.inname = "foo"version = "1.0"
![Page 20: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/20.jpg)
SOME NEAT FEATURES (2)
• Sane way to load data files
Name: fooVersion: 1.0ConfigPy: __config.py
# __config.pyBINDIR = r”/usr/bin”...
![Page 21: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/21.jpg)
SOME NEAT FEATURES (3)
•Out of tree build (bye bye source tree pollution)
# Convert a setup.py to bento.info$ bentomaker --bento-info=../source_dir/bento.info
![Page 22: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/22.jpg)
A FEW NUMBERS
• bento: ± 8000 LOC (not including ± 2500 LOC for tests)
• numpy.distutils: ±10000 LOC
• distutils: ± 10000 LOC
• numpy: ±2000 LOC for distutils vs ±1000 LOC for bento
• scipy: ±1600 LOC vs ±1100 LOC
![Page 23: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/23.jpg)
DEMO ON NUMPY
![Page 24: Bento lunch talk](https://reader033.vdocuments.site/reader033/viewer/2022042700/559b31461a28ab3e0a8b47e7/html5/thumbnails/24.jpg)
WHERE TO GET IT ?
• Getting bento on github: http://github.com/cournape/Bento.git
• Bento doc: http://cournape.github.com/Bento
• Bento mailing-list: [email protected]