Knit 1

Knit: Component Composition for Systems Software

Alastair ReidAlastair Reid, , Matthew Flatt, Leigh Matthew Flatt, Leigh Stoller, Jay Lepreau, Eric EideStoller, Jay Lepreau, Eric Eide

University of UtahUniversity of Utah

Knit 2

Why Components?

Everyone is writing too much codeEveryone is writing too much code Not enough code reuseNot enough code reuse Hard to reconfigureHard to reconfigure Hard to understandHard to understand Hard to test/verifyHard to test/verify

Exceptions: Click, Scout, Ensemble, Fox, Exceptions: Click, Scout, Ensemble, Fox, MMLite, OSKit, …MMLite, OSKit, …

Knit 3

Why Not Components?

OverheadOverhead RuntimeRuntime Programmer timeProgrammer time

Advanced systems don’t work with CAdvanced systems don’t work with C Complex component interdependenciesComplex component interdependencies

Locking restrictionsLocking restrictions Top/bottom-halfTop/bottom-half Bootstrap sequenceBootstrap sequence

Knit 4

Goal of Knit Project

To make components practical for systems programming

Knit 5

Key to Achieving Goal

Static configuration languageStatic configuration language

Enables error detectionEnables error detectionEnables optimizationEnables optimization

Knit 6

Target#1: The Utah OSKit [SOSP’97]

Approximately 500 components:Approximately 500 components:

Device drivers, bootstrap code, TCP/IP Device drivers, bootstrap code, TCP/IP stacks, filesystems, SNMP, etc. stacks, filesystems, SNMP, etc.

Doesn’t impose architectureDoesn’t impose architecture 101066 lines of code from Linux, FreeBSD, lines of code from Linux, FreeBSD,

NetBSD, Mach, Fluke, etc.NetBSD, Mach, Fluke, etc.

Knit 7

Outline

IntroductionIntroduction The Knit component modelThe Knit component model

Atomic unitsAtomic units Compound unitsCompound units Automatic InitializationAutomatic Initialization Detecting Configuration ErrorsDetecting Configuration Errors

Implementation and PerformanceImplementation and Performance Open issuesOpen issues

Knit 8

Atomic Units [PLDI’98]

serve_cgi

serve_web

int serve_web(…) { if (…) serve_cgi(…); else serve_file(…);}

serve_file

-Ioskit-DKERNEL-DHAVE_CONFIG

Knit 9

Compound Units [PLDI’98]

files

serve_web

processes

serve_file

select_server

serve_cgi

Knit 10

Initialization

x86

main

pthreads

VM init_x86();init_IDE();init_VM();init_threads();init_filesys();init_main();filesys

IDE

Knit 11

Initialization

x86

main

pthreads

VM init_x86();init_IDE();init_VM();init_threads();init_filesys();init_main();filesys

IDE

X

Knit 12

When Can We Break Cycles?

1.1. Component ‘contains’ subcomponentsComponent ‘contains’ subcomponents

2.2. No dependency between initializersNo dependency between initializers

VM threads locks

f_init f g g_init

Knit 13

Automatic Initialization

Knit generates initialization sequenceKnit generates initialization sequence Cycles are resolved by refining Cycles are resolved by refining

initialization dependencies in unitsinitialization dependencies in units

ExperienceExperience 5% of units need dependencies refined5% of units need dependencies refined Programmers find initialization a big winProgrammers find initialization a big win

Knit 14

Detecting Composition Errors

ethernet

panic

console

filesys

threads

Knit 15

Detecting Composition Errors

ethernet

panic

console

filesys

threads

interrupts

locks

Knit 16

Detecting Composition Errors

ethernet

panic

console

filesys

threads

interrupts

locks

Knit 17

Detecting Composition Errors

ethernet

panic

console

filesys

threads

interrupts

locks

Knit 18

Detecting Composition Errors

ethernet

panic

console

filesys

threads

interrupts

locks

Knit 19

Detecting Composition Errors

ethernet

panic

console

filesys

threads

interrupts

locks

Knit 20

Detecting Composition Errors

ethernet

panic

console

filesys

threads

interrupts

locks

Knit 21

Detecting Composition Errors

ethernet

panic

console

filesys

threads context(threads) <= ProcessContext

context(filesys) <= context(threads)

context(console) <= context(filesys)

context(panic) <= context(console)

NoContext <= context(ethernet)

ProcessContext < NoContext

Knit 22

Extensible Constraint System

Constraint system propagates properties Constraint system propagates properties through component interconnectionsthrough component interconnections Knit can detect global errorsKnit can detect global errors

Constraint system is extensibleConstraint system is extensible In context X, don’t do YIn context X, don’t do Y Type system for Modular IP Routers Type system for Modular IP Routers

(e.g., Click)(e.g., Click) ……

Knit 23

Knit

Supports C, assembly and object filesSupports C, assembly and object files Separates interconnections from codeSeparates interconnections from code Automatic initializationAutomatic initialization Extensible constraint systemExtensible constraint system Allows cyclic component dependenciesAllows cyclic component dependencies Allows multiple instances of componentsAllows multiple instances of components Text basedText based

Knit 24

Outline

IntroductionIntroduction The Knit component modelThe Knit component model Implementation and PerformanceImplementation and Performance Open issuesOpen issues

Knit 25

Implementation (Unoptimized)

ld

a.out

symbol_rename

cc

Knit

.o

.o

.c

.unit

Knit 26

Performance

Component cost should not distort system Component cost should not distort system structurestructure

Reduce overhead by eliminating function Reduce overhead by eliminating function callscalls

Knit 27

Click and Clack

Click modular network router from MIT Click modular network router from MIT [SOSP’99][SOSP’99]

ClackClack Re-implementation of Click using KnitRe-implementation of Click using Knit Similar performance to ClickSimilar performance to Click

Many small componentsMany small components

Knit 28

Performance of Clack

100%

79%

65%

0%

25%

50%

75%

100%

Unoptimized Monolithic Optimized

Tim

e pe

r P

acke

t

Knit 29

Open Issues

Is Knit general purpose?Is Knit general purpose? Need more usersNeed more users Need more applicationsNeed more applications

Is the constraint system extensible enough?Is the constraint system extensible enough? Implicit linking vs. explicit linking?Implicit linking vs. explicit linking?

Knit 30

Conclusions

State of the art component system for CState of the art component system for C Targeted at systems codeTargeted at systems code

Automatic initializationAutomatic initialization Detects local and global errorsDetects local and global errors Low performance overheadLow performance overhead

Available ASAP: http://www.cs.utah.edu/flux/