![Page 1: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/1.jpg)
Lua on NetBSDScripting Operating Systems with Lua
Lourival Vieira Neto <[email protected]>BSDCon BrazilOctober/2015
![Page 2: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/2.jpg)
“Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.”
Greenspun’s tenth rule
![Page 3: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/3.jpg)
“Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp a good scripting language.”
Ierusalimschy’s first Greenspun’s tenth rule
![Page 4: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/4.jpg)
Topics
❏ Introduction❏ Scriptable Operating System
❏ Example❏ Packet Filter Scripting
❏ Why Lua?❏ Kernel-scripting Environment
❏ lua(4)❏ Conclusions
![Page 5: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/5.jpg)
Introduction
![Page 6: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/6.jpg)
Scriptable Operating System
The combination of extensible operating systems with extension scripting languages.
![Page 7: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/7.jpg)
Scriptable Operating System
❏ Motivation❏ Flexibility
❏ Meet new user requirements❏ Configuration of kernel subsystems
❏ Easy development❏ Allow application developers to customize the
kernel
❏ Prototyping❏ Add new features
![Page 8: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/8.jpg)
Scriptable Operating System
❏ Key idea ❏ OS kernel scripting with Lua
❏ Halfway between..❏ Kernel parameters and kernel modules
❏ Halfway between..❏ Domain-specific and system languages
![Page 9: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/9.jpg)
Scriptable Operating System
❏ Two ways of scripting ❏ Extending (a scripting language)
❏ kernel as a library❏ Lua calls kernel
❏ Embedding (a scripting language)❏ kernel as a framework
❏ kernel calls Lua
![Page 10: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/10.jpg)
Use Cases
❏ Embedding❏ Packet filtering❏ Device drivers❏ Process scheduling
❏ Extending❏ Web servers❏ File systems❏ Network protocols
![Page 11: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/11.jpg)
Example
![Page 12: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/12.jpg)
Packet Filter Scripting
❏ Motivation❏ Deep packet inspection
❏ Traffic shaping❏ Intrusion detection/prevention
❏ New features❏ Port knocking❏ Protocols❏ Port stealthing
![Page 13: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/13.jpg)
SSH Version
![Page 14: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/14.jpg)
SSH Version
1. local data = require'data'2.3. function filter(pkt)4. -- convert packet data to string5. local str = tostring(pkt)6.7. -- pattern to capture the software version8. local pattern = 'SSH%-[^-%G]+%-([^-%G]+)'9.
10. -- get the software version11. local software_version = str:match(pattern)12.13. if software_version == 'OpenSSH_6.4' then14. -- reject the packet15. return false16. end17.18. -- accept the packet19. return true20. end
![Page 15: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/15.jpg)
SSH Version
❏ No measurable overhead ❏ 96 Mbps on both cases (on 100 Mbps virtual
NIC)❏ Binding
❏ 217 lines of C code❏ Script (ssh.lua)
❏ 22 lines of Lua code
![Page 16: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/16.jpg)
NPF
❏ The NetBSD Packet Filter❏ Layers 3 and 4❏ Stateful❏ IPv4 and IPv6❏ Extensible
❏ Rule procedures
![Page 17: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/17.jpg)
NPFLua
❏ Binds NPF to Lua❏ Kernel module + parser module
❏ Rule procedure#npf.confprocedure "lua_filter" { lua: call filter}
group default { pass in all apply "lua_filter"}
❏ Script loadingluactl load npf ./filter.lua
![Page 18: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/18.jpg)
Why Lua?
![Page 19: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/19.jpg)
Why Lua?
❏ Extensible extension language❏ Embeddable and extensible❏ C library
❏ Almost freestanding❏ Small footprint
❏ has 240 KB on -current (amd64)
❏ Fast❏ MIT license
![Page 20: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/20.jpg)
Why Lua?
❏ Safety features❏ Automatic memory management❏ Protected call❏ Fully isolated states❏ Cap the number of executed instructions
![Page 21: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/21.jpg)
Why not .... ?
❏ Python❏ has 2.21 MB on Ubuntu 10.10 (amd64)
❏ Perl❏ has 1.17 MB on Ubuntu 10.10 (amd64)
❏ Also..❏ OS-dependent code❏ Hard to embed1
1. twistedmatrix.com/users/glyph/rant/extendit.html
![Page 22: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/22.jpg)
Kernel-scripting Environment: lua(4)
![Page 23: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/23.jpg)
Brief History
❏ 2008 - Lunatik/Linux❏ 2010 - Lunatik/NetBSD
❏ Google Summer of Code❏ Kernel-embedded Lua (mainly)
❏ 2013 - Lua(4)❏ New infrastructure (Marc Balmer)
❏ 2014 - NPFLua❏ 2015 - Ported Lua Test Suite
❏ Google Summer of Code (Guilherme Salazar)
![Page 24: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/24.jpg)
Lua(4)
❏ Kernel-embedded Lua❏ has no floating-point numbers
❏ User Interface❏ luactl
❏ Kernel Programming Interface❏ sys/lua.h
![Page 25: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/25.jpg)
Operation Overview
![Page 26: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/26.jpg)
Conclusions
![Page 27: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/27.jpg)
Conclusions❏ General-purpose and full-fledged programming language for
scripting kernels❏ e.g., pattern matching, hash table
❏ First to provide scripting both by extending and embedding an interpreter
❏ Part of the official NetBSD distribution❏ Impact
❏ A. Graf. PacketScript—a Lua Scripting Engine for in-Kernel Packet. Processing. Master’s thesis, Computer Science Department, University of Basel, July 2010.
❏ M. Grawinkel, T. Suss, G. Best, I. Popov, and A. Brinkmann. Towards Dynamic Scripted pNFS Layouts. In High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:, pages 13–17. IEEE, 2012.
❏ A. Cagney. What happens when a DWARF and a daemon start dancing by the light of the silvery moon? BSDCan 2015 (Talk).
❏ A. Koomsin and Y. Shinjo. lua_syscall: Specializing Operating System Kernels by Using the Lua Language. 6th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys 2015) (Poster).
❏ K. Rytarowski. Moduły Dynamiczne w Kernelu NetBSD. Programista. 5/2015 (Polish Magazine).
❏ A. Koomsin, Y. Shinjo. Running Application Specific Kernel Code by a Just-in-Time Compiler. 8th ACM PLOS 2015.
![Page 28: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/28.jpg)
Questions and Answers
Contact Information❏ Lourival Vieira Neto
❏ Questions?
More Information❏ L. Vieira Neto, R. Ierusalimschy, A. L. de Moura and M. Balmer.
Scriptable Operating Systems with Lua. Dynamic Languages Symposium 2014. URL netbsd.org/~lneto/dls14.pdf.
![Page 29: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/29.jpg)
System Memory Binding: Luadata
![Page 30: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/30.jpg)
Luadata
❏ Regular Lua library❏ Kernel and user space
❏ Binds system memory❏ Memory block (pointer + size)❏ mbuf
❏ Safe❏ Boundary verification
❏ Packed data❏ Declarative layouts
![Page 31: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/31.jpg)
Luadata
❏ Other features❏ Bit fields❏ String fields and conversion❏ Segments (data decomposition)❏ Endianness conversion
![Page 32: Lua on NetBSDlneto/bsdconbr15.pdfLua on NetBSD Scripting Operating Systems with Lua Lourival Vieira Neto BSDCon Brazil October/2015 “Any sufficiently complicated](https://reader034.vdocuments.site/reader034/viewer/2022052500/5f0ed4b17e708231d44125e5/html5/thumbnails/32.jpg)
RTP Encoding
1. local rtp = {2. version = {0, 2},3. extension = {3, 1},4. csrc_count = {4, 4},5. marker = {8, 1},6. type = {9, 7}7. }8.9. -- apply RTP header layout in the payload
10. pld:layout(rtp)11.12. -- if packet is encoded using H.26313. if pld.type == 34 then14. -- reject the packet15. return false16. end