`s in debian or scientific debian: numpy, scipy and beyond · debian, 18 years later ˇ17’000...
TRANSCRIPT
![Page 1: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/1.jpg)
π‘s in Debian or Scientific Debian:NumPy, SciPy and beyond
Yaroslav O. Halchenko
Debian Project,Dartmouth College, USA
EuroScipy 2011, Paris, France
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 1 / 34
![Page 2: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/2.jpg)
Scientists report:
Hanke M and Halchenko YO (2011) Neuroscience runs on GNU/Linux.Front. Neuroinform. 5:8. doi: 10.3389/fninf.2011.00008
![Page 3: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/3.jpg)
Debian: once upon a time
Fellow Linuxers,This is just to announce the imminent completion of a
brand-new Linux release, which I’m calling the Debian LinuxRelease. [. . . ]
Debra’s husband Ian A Murdock, 16/08/1993comp.os.linux.development
non-commercial distro, competitive in the OS market
easy to install
built collaboratively by volunteer software experts
1st major distro developed “openly in the spirit of GNU”
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 3 / 34
![Page 4: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/4.jpg)
Debian: early history
1993 announcement
1994 Debian manifesto
1997 Debian Social Contract with the Free Software community100% Free Software: Debian Free Software Guidelinesgive backdon’t hide problemspriorities: users & Free Software
1998 Debian Constitutionstructure and rules of a Free-Software-compatible democracy
default: do-cracy, consensus + working codedemocracy, when neededscaffolding: DPL, secretary, etc.
2000 I use Debian for the first time
2004 I submit the first bug reportI contribute my first package
2006 I become an official Debian Developer
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 4 / 34
![Page 5: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/5.jpg)
Debian: early history of my life
1993 announcement
1994 Debian manifesto
1997 Debian Social Contract with the Free Software community100% Free Software: Debian Free Software Guidelinesgive backdon’t hide problemspriorities: users & Free Software
1998 Debian Constitutionstructure and rules of a Free-Software-compatible democracy
default: do-cracy, consensus + working codedemocracy, when neededscaffolding: DPL, secretary, etc.
2000 I use Debian for the first time
2004 I submit the first bug reportI contribute my first package
2006 I become an official Debian Developer
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 4 / 34
![Page 6: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/6.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 7: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/7.jpg)
Debian, 18 years later
≈ 17’000 source packages≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)12 stable releases
The latest stable release:
FUD ALERT: 5 years ago
6.0 Squeeze, February 6th 2011oldstable [still supported]: 5.0 Lenny, February 14th 2009
old oldstable [February 15th, 2010]: 4.0 Etch, April 2nd 2007
Released ≈ 2 yearsSecurity support for 3 yearsUpgradable
2 1/2 rolling releases: testing, unstable (sid), experimental133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 8: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/8.jpg)
Debian, 18 years later
≈ 17’000 source packages≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)12 stable releases
The latest stable release:
6.0 Squeeze, February 6th 2011
oldstable [still supported]: 5.0 Lenny, February 14th 2009old oldstable [February 15th, 2010]: 4.0 Etch, April 2nd 2007
Released ≈ 2 yearsSecurity support for 3 yearsUpgradable
2 1/2 rolling releases: testing, unstable (sid), experimental133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 9: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/9.jpg)
Debian, 18 years later
≈ 17’000 source packages≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)12 stable releases
The latest stable release:
6.0 Squeeze, February 6th 2011oldstable [still supported]: 5.0 Lenny, February 14th 2009
old oldstable [February 15th, 2010]: 4.0 Etch, April 2nd 2007
Released ≈ 2 yearsSecurity support for 3 yearsUpgradable
2 1/2 rolling releases: testing, unstable (sid), experimental133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 10: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/10.jpg)
Debian, 18 years later
≈ 17’000 source packages≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)12 stable releases
The latest stable release:
6.0 Squeeze, February 6th 2011oldstable [still supported]: 5.0 Lenny, February 14th 2009
old oldstable [February 15th, 2010]: 4.0 Etch, April 2nd 2007
Released ≈ 2 yearsSecurity support for 3 years
Upgradable
2 1/2 rolling releases: testing, unstable (sid), experimental133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 11: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/11.jpg)
Debian, 18 years later
≈ 17’000 source packages≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)12 stable releases
The latest stable release:
6.0 Squeeze, February 6th 2011oldstable [still supported]: 5.0 Lenny, February 14th 2009
old oldstable [February 15th, 2010]: 4.0 Etch, April 2nd 2007
Released ≈ 2 yearsSecurity support for 3 yearsUpgradable
2 1/2 rolling releases: testing, unstable (sid), experimental133 “derivative” distributions
(source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 12: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/12.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 13: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/13.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
Debian "stable"
Release:Updates:Security support:
when readyevery two months (only critical fixes)one year after next stable (≈ 3 years)
Debian "stable"
Release:Updates:Security support:
when readyevery two months (only critical fixes)one year after next stable (≈ 3 years)
![Page 14: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/14.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
Debian "stable"
Release:Updates:Security support:
when readyevery two months (only critical fixes)one year after next stable (≈ 3 years)
Debian "stable"
Release:Updates:Security support:
when readyevery two months (only critical fixes)one year after next stable (≈ 3 years)
Debian "testing"
Release:Updates:Security support:
never/continuouslydailyyes
Debian "unstable"
Release:Updates:Security support:
nevermultiple times per daynone (implicit)
rele
ase
10 d
ays
![Page 15: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/15.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
Debian "stable"
Release:Updates:Security support:
when readyevery two months (only critical fixes)one year after next stable (≈ 3 years)
e.g. Ubuntu
e.g. Linux Mint
e.g. MEPIS
Debian "stable"
Release:Updates:Security support:
when readyevery two months (only critical fixes)one year after next stable (≈ 3 years)
Debian "testing"
Release:Updates:Security support:
never/continuouslydailyyes
Debian "unstable"
Release:Updates:Security support:
nevermultiple times per daynone (implicit)
rele
ase
10 d
ays
![Page 16: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/16.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 17: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/17.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 18: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/18.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 19: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/19.jpg)
Debian, 18 years later: 133 derivatives
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 20: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/20.jpg)
Debian, 18 years later
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 21: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/21.jpg)
Debian: the Universal OS
≈ 17’000 source packages
≈ 33’000 binary packages (amd64/sid/main)largest n. of ports among mainstream distros (11 official, 4 unofficial)
2 non-Linux ports: GNU/kFreeBSD + (unofficial Hurd)
12 stable releases
2 1/2 rolling releases: testing, unstable (sid), experimental
133 “derivative” distributions (source: distrowatch.com)
≈ 900 active DDs + ≈200 DMs + thousands other contributors
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 5 / 34
![Page 22: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/22.jpg)
More on Debian & its derivatives
Who the bloody hell cares about Debian?Stefano Zacchiroli, Debian project leader, LCA 2011, Australiahttp://upsilon.cc/~zack/blog/posts/2011/01/who_the_bloody_hell_cares_about_Debian/
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 6 / 34
![Page 23: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/23.jpg)
Debian community: DebConf 2011, 262 humanoids
≈ 900 active DDs, ≈200 DMs, >100 Teams
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 7 / 34
![Page 24: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/24.jpg)
Debian community: DebConf 2007-2011, ≈ 900
≈ 900 active DDs, ≈200 DMs, >100 Teams
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 8 / 34
![Page 25: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/25.jpg)
Debian: the community
≈ 900 active DDs, ≈200 DMs, >100 Teams
Thousands of participants
Debian developers from 57 countriesFinland: 5,77 DDs / million
Teams:http://wiki.debian.org/Teams
Internationalization (i10n):http://www.debian.org/international/l10n/po/rank
Different fields of endeavor:
Debian ScienceDebian MedDebichemNeuroDebian. . .
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 9 / 34
![Page 26: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/26.jpg)
Debian’s -ocracies
1 do-ocracy
An individual Developer may make any technical ornontechnical decision with regard to their own work;
Debian Constitution, §3.3.1.1
2 democracy
Each decision in the Project is made by one or more ofthe following:
1. The Developers, by way of General Resolution [...]Debian Constitution, §2
that means:
reputation follows work, easy to have an impact
no benevolent dictator, no oligarchy
no imposed decisionsby who has money, infrastructure, employees, . . .
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 10 / 34
![Page 27: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/27.jpg)
Pythons in Debian
Python2 :
2.5 [in stable], 2.6 [default] and 2.7 [testing, sid]2630 projects out of 16525 (16%) use Python(2)Used by some core tools:dput, git-buildpackage, piuparts, reportbug, ...2.7 as the default in experimentalhttp://bugs.debian.org/622279
17 bugs holding the transition2.7 as the default – the release goal for wheezy
Python3 :
3.1 [in stable], 3.2 [testing, sid]65 packages dependent on Python3
Maintained by :Matthias Klose, Piotr Ozarowski, Scott Kitterman, Tollef FogHeen, Raphael Hertzog, Michael Vogt, Marc Deslauriers, ColinWatson
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 11 / 34
![Page 28: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/28.jpg)
Pythons in Debian
Python2 :
2.5 [in stable], 2.6 [default] and 2.7 [testing, sid]2630 projects out of 16525 (16%) use Python(2)Used by some core tools:dput, git-buildpackage, piuparts, reportbug, ...2.7 as the default in experimentalhttp://bugs.debian.org/622279
17 bugs holding the transition2.7 as the default – the release goal for wheezy
Python3 :
3.1 [in stable], 3.2 [testing, sid]65 packages dependent on Python3
Maintained by :Matthias Klose, Piotr Ozarowski, Scott Kitterman, Tollef FogHeen, Raphael Hertzog, Michael Vogt, Marc Deslauriers, ColinWatson
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 11 / 34
![Page 29: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/29.jpg)
Pythons in Debian
Python2 :
2.5 [in stable], 2.6 [default] and 2.7 [testing, sid]2630 projects out of 16525 (16%) use Python(2)Used by some core tools:dput, git-buildpackage, piuparts, reportbug, ...2.7 as the default in experimentalhttp://bugs.debian.org/622279
17 bugs holding the transition2.7 as the default – the release goal for wheezy
Python3 :
3.1 [in stable], 3.2 [testing, sid]65 packages dependent on Python3
Maintained by :Matthias Klose, Piotr Ozarowski, Scott Kitterman, Tollef FogHeen, Raphael Hertzog, Michael Vogt, Marc Deslauriers, ColinWatson
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 11 / 34
![Page 30: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/30.jpg)
Who cares about Python in Debian
111 teams :
Debian Python Modules Teamhttps://alioth.debian.org/projects/python-modules
207 membersmaintain NumPy and SciPyPython Applications Packaging Teamhttps://alioth.debian.org/projects/python-apps
133 members
549 individuals
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 12 / 34
![Page 31: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/31.jpg)
Who cares about Python in Debian
111 teams :
Debian Python Modules Teamhttps://alioth.debian.org/projects/python-modules
207 membersmaintain NumPy and SciPyPython Applications Packaging Teamhttps://alioth.debian.org/projects/python-apps
133 members
549 individuals
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 12 / 34
![Page 32: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/32.jpg)
π’s in Debian
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 33: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/33.jpg)
π’s in Debian: popcon is growing
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 34: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/34.jpg)
π’s in Debian: NumPy is on 40% of Debian boxes
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 35: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/35.jpg)
π’s in Debian: SciPy is not waiting
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 36: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/36.jpg)
π’s in Debian
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 37: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/37.jpg)
π’s in Debian
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 38: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/38.jpg)
π’s in Debian: the Core
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 39: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/39.jpg)
π’s in Debian: numerical computing
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
http://blends.alioth.debian.org/science/tasks/numericalcomputation
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 40: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/40.jpg)
π’s in Debian: machine/statistical learning
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
http://blends.alioth.debian.org/science/tasks/machine-learning
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 41: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/41.jpg)
π’s in Debian: brain sciences
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
http://neuro.debian.netHalchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 42: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/42.jpg)
π’s in Debian: games
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 43: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/43.jpg)
π’s in Debian
Popularity Contesthttp://popcon.debian.org
Numpy is on 40% of Debian installations
apt-cache rdepends python-numpy
158 packages depend on NumPy40 packages depend on SciPy
Cover variety of applications out-of-the-box
Most recent additions [Debian sid]
FreeNect (Free Kinect driver)PyCUDAPyOpenCLscikits.statsmodels
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 13 / 34
![Page 44: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/44.jpg)
Debian is beneficial for “Upstream”
Debian provides a robust deployment platform, which helps to ...
iron out problemspackaged/tested on the system nearly identical to the othersbinary builds across all supported platforms(optional) package build-time (unit-)testing”stable” release is stable – bugs triaged before the release
deliver133 derivativesMark S.: “200 millions of Ubuntu users in 3 years and 9 months”official Debian mirrors in 46 countrieshttp://www.debian.org/mirror/list
engage more caring handsQA activities: archive rebuilds (FTBFS), package QA toolscentralized transitions
report usage statisticshttp://popcon.debian.org
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 14 / 34
![Page 45: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/45.jpg)
Debian is beneficial for “Upstream”
Debian provides a robust deployment platform, which helps to ...
iron out problemspackaged/tested on the system nearly identical to the othersbinary builds across all supported platforms(optional) package build-time (unit-)testing”stable” release is stable – bugs triaged before the release
deliver133 derivativesMark S.: “200 millions of Ubuntu users in 3 years and 9 months”official Debian mirrors in 46 countrieshttp://www.debian.org/mirror/list
engage more caring handsQA activities: archive rebuilds (FTBFS), package QA toolscentralized transitions
report usage statisticshttp://popcon.debian.org
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 14 / 34
I have always found my friends Debian developers to be prettygood at getting me do boring but useful stuff.
–Gael Varoquaux
![Page 46: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/46.jpg)
Help yourself to help Debian
Have a deterministic version
Be conscious about all licensesAllow for modularity
use (documented) ”standard” build mechanismstreat 3rd party as 3rd party
no forks – forward fixes upstream, request bugfix releasesallow an option to build against system-installed versions
be compatible with recent released versions NumPy/SciPy
Be prepared for feedbackProvide unit-/doc- tests and examples
easy way to run only lightweight portion (for build-time testing)conventional means to run the testsuse tempfile.* instead of the work-treedo not hardcode matplotlib backends (unless required)
Test/use your software on Debian(especially during Debian freeze)
http://neuro.debian.net/blog/2011/2011-08-23_getting_stuff_packaged.html
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 15 / 34
![Page 47: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/47.jpg)
Debian is a rich platform for Python development
Debian facilitates software development by providing out-of-the-box...
Multiple supported versions of Python
Python Editors/IDEs/refactoring tools
vim, emacs (GNU Python mode, python-mode, ropemacs)DrPython, Eric, Geany, gEcrit, PIDA, Spyder, ...pylint, pyflakesrope, bicyclerepair
debugging facilities
pdb, pydb, pudb, winpdbadvanced extensions debugging(in a minute)
easy ways to bootstrap a complete system(in 2 minutes)
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 16 / 34
![Page 48: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/48.jpg)
Advanced extensions debugging facilities
GDB inspect Python stack
Valgrind pin-point segfaults and memory leaks
Profiling GUI kcachegrind, hotshot + kcachegrind-convertershttps://github.com/PyMVPA/PyMVPA/blob/master/tools/profile
DMTCP snapshot lengthy computations
FReD [coming] reversible debugger
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 17 / 34
![Page 49: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/49.jpg)
GDB: inspect Python stack
> gdb --args /usr/bin/python-dbg segfault.py
GNU gdb (GDB) 7.3.50.20110627-cvs-debian
...
Program received signal SIGSEGV, Segmentation fault.
...
(gdb) py <TAB>
py-bt py-down py-list py-locals py-print py-up python
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 18 / 34
![Page 50: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/50.jpg)
GDB: Python stack
(gdb) bt
...
#10 ... /arrayprint.py, line 156, in _leading_trailing ...
#11 ... at ../Python/ceval.c:3836 ...
(gdb) py-bt
#10 Frame 0xf5c6d0, ... /arrayprint.py, line 156 ....
a[-_summaryEdgeItems:]))
#13 Frame 0xf63230, ... ./arrayprint.py, line 162 ...
min(len(a), _summaryEdgeItems))]
(gdb) py-up
#13 Frame 0xf63230 ...
min(len(a), _summaryEdgeItems))]
(gdb) py-down
#10 Frame 0xf5c6d0 ...
a[-_summaryEdgeItems:]))
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 19 / 34
![Page 51: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/51.jpg)
DMTCP: Snapshot your Python
Why?
Stop/resume the lengthy task
across power outages“reversible” debugging
Move the task across identical boxes (PBS, Condor)
Distributed MultiThreaded CheckPointing
Works with Linux kernel 2.6.9 and later
Supports sequential and multi-threaded computations acrosssingle/multiple hosts
Entirely in user space (no kernel modules or root privilege)
Transparent (no recompiling, no re-linking)
DMTCP Team centered around Northeastern U., with collaboratorsfrom MIT and Siberian State U. of Telecom. and Informatics
Available in Debian ≥ wheezy (current testing)
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 20 / 34
![Page 52: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/52.jpg)
DMTCP: Snapshot your Python
Why?
Stop/resume the lengthy task
across power outages“reversible” debugging
Move the task across identical boxes (PBS, Condor)
STANDALONE USAGE
dmtcp checkpoint a.out
dmtcp command --checkpoint
dmtcp restart ckpt a.out *.dmtcp
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 20 / 34
![Page 53: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/53.jpg)
DMTCP: Snapshot your Python
Why?
Stop/resume the lengthy task
across power outages“reversible” debugging
Move the task across identical boxes (PBS, Condor)
Python interface
In the next release
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 20 / 34
![Page 54: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/54.jpg)
DMTCP: Snapshot your Python
Why?
Stop/resume the lengthy task
across power outages“reversible” debugging
Move the task across identical boxes (PBS, Condor)
FReD: Fast Reversible Debugger (WiP)
reverse−step
reverse−continue
Time
Call StackReverse execution
Forward execution
BREAKPTFNC1
FNC2
reverse−finish
reverse−next
http://www.cs.wisc.edu/condor/CondorWeek2011/wednesday_condor.html
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 20 / 34
![Page 55: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/55.jpg)
Bootstrapping a complete Debian system
Why?
build/test/use previous or upcoming Debian (or Ubuntu) release
get clean environment (track dependencies)
mimic user’s setup
debootstrap + schroot: install into any directory
vmdebootstrap: generate a virtual machinehttp://blog.liw.fi/posts/vmdebootstrap/
Fully Automated Installation (FAI): http://fai-project.org/
VirtualBox: install or use pre-crafted virtual appliance
Cloud: http://wiki.debian.org/Cloud
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 21 / 34
![Page 56: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/56.jpg)
Bootstrapping a complete Debian system
Why?
build/test/use previous or upcoming Debian (or Ubuntu) release
get clean environment (track dependencies)
mimic user’s setup
debootstrap + schroot: install into any directory
vmdebootstrap: generate a virtual machinehttp://blog.liw.fi/posts/vmdebootstrap/
Fully Automated Installation (FAI): http://fai-project.org/
VirtualBox: install or use pre-crafted virtual appliance
Cloud: http://wiki.debian.org/Cloud
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 21 / 34
![Page 57: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/57.jpg)
debootstrap + schroot
> debootstrap sid /var/cache/chroots/sid-amd64
> sudo bash -c "cat << EOF >| /etc/schroot/chroot.d/sid-amd64
[sid-amd64]
description=Debian sid (forever unstable) [amd64]
type=directory
location=/var/cache/chroots/sid-amd64
users=YOURLOGIN
aliases=unstable,sid,default
EOF"
> schroot
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 22 / 34
![Page 58: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/58.jpg)
NeuroDebian VM in VirtualBox
http://neuro.debian.net/vm.htmlHalchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 23 / 34
![Page 59: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/59.jpg)
Ways to contribute
http://wiki.debian.org/ProjectNews/HowToContribute
http://raphaelhertzog.com/2011/06/30
reportbug (+ patches)
Internationalization (i18n):http://www.debian.org/doc/manuals/intro-i18n
packagingLuca’s tutorialapt-get install packaging-tutorialhttp://www.lucas-nussbaum.net/blog/?p=640
Bootstrap packaging of Python modules:py2dsc (python-stdeb package)Good night reading: Debian PolicySeek mentor/sponsor-ship: http://mentor.debian.org
Become “Debian Maintainer”:http://wiki.debian.org/DebianMaintainer
Become “Debian Developer”:http://wiki.debian.org/DebianDeveloper
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 24 / 34
![Page 60: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/60.jpg)
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 25 / 34
![Page 61: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/61.jpg)
Acknowledgements
Michael Hanke
Free and opensourcesoftware developers
Debian CommunityPython/NumPy/Scipy
Maintainers
Jim Haxby
Thanks!Yaroslav O. [email protected]
http://www.onerussian.com
about the slides:available at http://neuro.debian.net/#publicationsc© 2011 Yaroslav O. Halchenko,portions are:c© 2010 Stefano Zacchirolic© 2011 Michael Hankeslide style inspired by Stefano Zacchiroli
CC BY-SA 3.0 — Creative Commons Attribution-ShareAlike 3.0
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 26 / 34
![Page 62: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/62.jpg)
How many care about Python
> grep-dctrl -s Maintainer -F Build-Depends python
-o -F Build-Depends python-dev
-o -F Build-Depends python-all
/var/lib/apt/lists/*\_sid\_main\_source\_Sources
| sort | uniq -c | sort -n -r
| wc -l
660
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 27 / 34
![Page 63: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/63.jpg)
Who cares: teams
> grep-dctrl -s Maintainer -F Build-Depends python
-o -F Build-Depends python-dev
-o -F Build-Depends python-all
/var/lib/apt/lists/*\_sid\_main\_source\_Sources
| sort | uniq -c | sort -n -r
| grep -e alioth -e team -e Maintainers -e Debian
| wc -l
111
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 28 / 34
![Page 64: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/64.jpg)
Who cares: teams
> ...
| grep -e alioth -e team -e Maintainers -e Debian | head
232 Maintainer: Debian Python Modules Team <python-mod...
57 Maintainer: Debian Tryton Maintainers <tryton@list...
51 Maintainer: Debian OLPC <[email protected]...
45 Maintainer: Python Applications Packaging Team <py...
40 Maintainer: Debian/Ubuntu Zope Team <pkg-zope-deve...
40 Maintainer: Debian Multimedia Maintainers <pkg-mul...
26 Maintainer: NeuroDebian Team <[email protected]...
26 Maintainer: Debian Science Maintainers <debian-sci...
26 Maintainer: Debian QA Group <[email protected]...
26 Maintainer: Debian Bazaar Maintainers <pkg-bazaar-...
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 29 / 34
![Page 65: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/65.jpg)
Who cares: individuals
> grep-dctrl -s Maintainer -F Build-Depends python
-o -F Build-Depends python-dev
-o -F Build-Depends python-all
/var/lib/apt/lists/*\_sid\_main\_source\_Sources
| sort | uniq -c | sort -n -r
| grep -v -e alioth -e team -e Maintainers -e Debian
| wc -l
549
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 30 / 34
![Page 66: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/66.jpg)
Who cares: individuals
> ...
| head
26 Maintainer: Matthias Klose <[email protected]>
16 Maintainer: David Paleino <[email protected]>
14 Maintainer: Arnaud Fontaine <[email protected]>
13 Maintainer: Jelmer Vernooij <[email protected]>
12 Maintainer: Pierre Chifflier <[email protected]>
12 Maintainer: Josselin Mouette <[email protected]>
11 Maintainer: Georges Khaznadar <[email protected]>
11 Maintainer: Chris Lamb <[email protected]>
11 Maintainer: Alessio Treglia <[email protected]>
10 Maintainer: Scott Kitterman <[email protected]>
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 31 / 34
![Page 67: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/67.jpg)
NumPy: who cared
/usr/share/doc/python-numpy/changelog.Debian.gz
22 Marco Presi (Zufus)
20 Ondrej Certik
13 Kumar Appaiah
9 Sandro Tosi
9 Matthias Klose
2 Tiziano Zito
... 15 more ...
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 32 / 34
![Page 68: `s in Debian or Scientific Debian: NumPy, SciPy and beyond · Debian, 18 years later ˇ17’000 source packages ˇ33’000 binary packages (amd64/sid/main) largest n. ofportsamong](https://reader034.vdocuments.site/reader034/viewer/2022042810/5f97615146298d466a2f3271/html5/thumbnails/68.jpg)
SciPy: who cared
/usr/share/doc/python-scipy/changelog.Debian.gz
15 Marco Presi (Zufus)
14 Ondrej Certik
7 Jose Fonseca
7 Alexandre Fayolle
5 Luca Falavigna
4 Sandro Tosi
3 Varun Hiremath
3 Piotr O_zarowski
2 Matthias Klose
... 5 more ...
Halchenko (Debian, Dartmouth) π‘s in Debian EuroScipy 2011 33 / 34