porting valgrind to netbsd and openbsd by masao uebayashi

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Porting Valgrind to OpenBSD and NetBSD

Masao Uebayashi

Tombi Inc.

Sep. 28, 2014

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Agenda

1. Prologue2. Introduction3. Development4. Internal: System call5. Internal: execve(2)6. Internal: Signal7. Issues8. Result9. Summary

10. Future

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Prologue: Questions

1. Have you heard Valgrind?2. Have you used Valgrind?3. Do you want to use Valgrind?4. Do you want to understand Valgrind?5. Do you want to fix Valgrind?6. Do you want to maintain Valgrind?7. Do you want to write Valgrind (from scratch)?

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Prologue: Audience

Use *BSD?

Write code?

Use (want) Valgrind?

Report problems?

Read Valgrind code?

Writeplatform dependent

code?

Writeplatform independent

code?

Yes

No

User

Yes

Yes

Yes

Yes

Yes

Yes

Advanced user

Valgrind user

Helpful Valgrind user

Luser

Advanced helpful Valgrind user

Valgrind porting developer

Valgrind architect

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Prologue: Goals (of this presentation)

1. Involve more people to help development2. Introduce design & explain internal (esp. syscall + execve

+ signal)3. Convince {Open,Net}BSD developers for necessary

changes

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Introduction: Authors' words

From ``Valgrind: A Framework for Heavyweight DynamicBinary Instrumentation'':

▶ A dynamic binary instrumentation (DBI) framework▶ Designed for building heavyweight dynamic binary

analysis (DBA) tools▶ ``Shadow values'' (register and memory)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Introduction: Wikipedia

From Wikipedia (https://en.wikipedia.org/wiki/Valgrind)``Overview'':

Valgrind is in essence a virtual machine usingjust-in-time (JIT) compilation techniques, includingdynamic recompilation.

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Introduction: Design: Emulator

48 89 d148 8b 3c 2448 8d 54 fc 10:

Client Code (.text)%rdx, %rcx)(%rsp), %rdi)0x10(%rsp, %rdi, 8), %rdx)

(mov(movl(ea

RAXRBXRCXRDXRSPRBPRSIRDI

R8R9R10R11R12R13R14R15

Guest State

Flags

YMMRegisters

FPURegisters

Scheduler

Client Data (.data)

Client Address Space

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Introduction: Design: Address space

0x400000

0x38000000

Clienttext + data

Valgrindtext + data

VM_MAXUSER_ADDRESS(0x00007f7fffffc000)

VM_MIN_ADDRESS(PAGE_SIZE == 0x1000)

ValgrindStack

Client + ValgrindHeap

AddressSpaceMng.

GuestState

ClientStack

ValgrindData

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Project

▶ valgrind.org▶ Unofficial FreeBSD branch @ Bitbucket

(https://bitbucket.org/stass/valgrind-freebsd)OpenBSD/NetBSD branches are forked from valgrind-freebsd

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Code: Structure

Valgrind tool processUserland

Emulator (VEX)

OS / ArchitectureDependentCode

HostSystem

CallHandling

ClientSystem

CallHandling

AddressSpace

Management

SignalHandling

Start-UpCode

initimg(execve)

Kernel

theoreticalabstract

realisticconcrete

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Code: Files and lines

Files:▶ 1,032 *.c files▶ 81 *.c files containing ``freebsd''▶ 48 *.[cS] files containing ``#if .*freebsd''▶ 8 *freebsd*.c files▶ 4 *freebsd*.S files

Lines:▶ 499 lines containing ``freebsd'' in *.[cS] files▶ 340 lines containing ``#if .*freebsd'' in *.[cS] files

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Code: #ifdef's

OS and architecture conditional macros:▶ VGO_*: OS▶ VGA_*: Architecture▶ VGP_*: Platform (== Architecture + OS)

How many for FreeBSD:▶ VGO_freebsd: 196 lines▶ VGA_amd64: 53 lines▶ VGP_amd64_freebsd: 56 lines

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Code: portability

From ``Porting Plans'' inhttp://valgrind.org/info/platforms.html:

...Unlike NetBSD or GCC, we are not interested inhaving Valgrind work on every platform in theknown universe: the maintenance burden is too high....

Portable, but not very portable.From ``Out Of Tree'':

x86/FreeBSD: Doug Rabson and others haved done afairly complete port of Valgrind 3.X. The FreeBSDporting team actively maintains the port. Snapshotsof the work in progress are at this FreeBSD page.

*BSD ports will be unlikely to be merged (?)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Phases

StartBuild

(auto* tools, #ifdef)No client

(internal initialization)Empty client

(client initialization)

Runtest program

Debug & fixValgrind

Debug & fixtest program

BetaRelease

InitialDevelopment(Not Working)

AlphaQuality

(SomewhatWorking)

(DeveloperOnly)

Release

BetaQuality(AlmostWorking)

ReleaseQuality

(Working)

ForValgrindDevelopers

ForValgrindUsers

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Development: Debugging

gdb(vgdb)

gdb

gdb ls

valgrind

valgrind ls

gdbserver

gdb serial protocol

ptrace(2)

ptrace(2)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: System call

▶ Interface for userland to interact with kernel▶ open(2), read(2), write(2), ...▶ Implemented as system call instruction▶ Entering kernel mode

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

% objdump -d /usr/lib/libc.a | sed -ne '/<_thread_sys_read>:/,/^$/p'0000000000000010 <_thread_sys_read>:10: b8 03 00 00 00 mov $0x3,%eax15: 49 89 ca mov %rcx,%r1018: 0f 05 syscall1a: 72 e4 jb 0 <_thread_sys_read-0x10>1c: c3 retq

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

% objdump -d /usr/lib/libc.a | sed -ne '/<_thread_sys_mmap>:/,/^$/p'0000000000000000 <_thread_sys_mmap>:

0: 48 83 ec 28 sub $0x28,%rsp4: 31 c0 xor %eax,%eax6: 4c 89 4c 24 08 mov %r9,0x8(%rsp)b: 45 89 c1 mov %r8d,%r9de: 41 89 c8 mov %ecx,%r8d11: 89 d1 mov %edx,%ecx13: 48 89 f2 mov %rsi,%rdx16: 48 89 fe mov %rdi,%rsi19: bf c5 00 00 00 mov $0xc5,%edi1e: c7 04 24 00 00 00 00 movl $0x0,(%rsp)25: e8 00 00 00 00 callq 2a <_thread_sys_mmap+0x2a>2a: 48 83 c4 28 add $0x28,%rsp2e: c3 retq

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: System call: *BSD

syscallmov $123, $rax

:

trap vector

User Text

User Stack

Kernel Text

syscallmov $123, $rax

:

User Text

User Stack

Kernel Text

SS / CS

syscallmov $123, $rax

:

User Text

User Stack

Kernel Text

SS / CSframe reg

trap vector

syscall syscall

trap vector

syscall

syscall instructionsaves SS/CS on stackentering kernel mode

trap vectorsaves registers on stack

(frame reg)

syscallmov $123, $rax

:

User Text

User Stack

Kernel Text

SS / CSframe reg

trap vector

syscall

syscall code runssaving retval

into frame reg

syscallmov $123, $rax

:

User Text

User Stack

Kernel Text

trap vector

syscall

ret instructionrestores context w/ retval

entering user mode

return values(RAX, RDX)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: System call: Valgrind

▶ Valgrind does not rely on host libc▶ Instead, Valgrind is built on top of POSIX-like ``real''

system calls▶ Two purposes: for Valgrind itself, and for client (``client

syscall'')

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: System call: Valgrind: Client

(Next page)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Valgrind schedulerexecutes syscall instruction

then syscall handleris called

Client syscall handlerchecks syscall number

and dispatches

If read sycall is neededset syscall arguments

reading savedmachine state

Enter kerneltrap vector saves SS/CS

syscall handler savesframe reg

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

SS/CSframe reg

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

SS/CSframe reg

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

Syscall code runssaving retval

into frame reg

Return to userlandWRK asm wrapper

saves resultsinto machine state

Return values(RAX, RDX)

Return values(RAX, RDX)

syscall handler

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

syscallmov $123, $rax

:

Client Text

Client Stack

client_syscall()

ML_(do_syscall_for_client_WRK)()

ClientMachine State

Resume Valgrind schedulerClient sees

syscall resultsin machine state

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

.globl ML_(do_syscall_for_client_WRK)ML_(do_syscall_for_client_WRK):

/* save callee-saved regs */pushq %rbpmovq %rsp, %rbppushq %rdi // -8(%rbp) syscallnopushq %rsi // -16(%rbp) guest_statepushq %rdx // -24(%rbp) sysmaskpushq %rcx // -32(%rbp) postmaskpushq %r8 // -40(%rbp) sigsetSzB

1: /* Even though we can't take a signal until the sigprocmask completes,start the range early.If eip is in the range [1,2), the syscall hasn't been started yet */

/* Set the signal mask which should be current during the syscall. *//* Save and restore all 5 arg regs round the call. This is easier

than figuring out the minimal set to save/restore. */

movq $__NR_sigprocmask, %rax // syscall #movq $VKI_SIG_SETMASK, %rdi // howmovq (%rdx), %rsi // sysmasksyscall

jb 7f /* sigprocmask failed */

movq -32(%rbp), %rdxtestq %rdx, %rdxjz 10fmovl %eax, (%rdx) // postmask

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

10:

/* OK, that worked. Now do the syscall proper. */

/* 6 register parameters */movq -16(%rbp), %r11 /* r11 = VexGuestAMD64State * */movq OFFSET_amd64_RDI(%r11), %rdimovq OFFSET_amd64_RSI(%r11), %rsimovq OFFSET_amd64_RDX(%r11), %rdxmovq OFFSET_amd64_R10(%r11), %r10movq OFFSET_amd64_R8(%r11), %r8movq OFFSET_amd64_R9(%r11), %r9/* 2 stack parameters plus return address (ignored by syscall) */movq OFFSET_amd64_RSP(%r11), %r11 /* r11 = simulated RSP */movq 16(%r11), %raxpushq %raxmovq 8(%r11), %raxpushq %rax/* (fake) return address. */movq 0(%r11), %raxpushq %rax/* syscallno */movq -8(%rbp), %rax

/* If rip==2, then the syscall was either just aboutto start, or was interrupted and the kernel wasrestarting it. */

2: syscall3: /* In the range [3, 4), the syscall result is in %rax,

but hasn't been committed to RAX. */

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

/* stack contents: 3 words for syscall above, plus our prologue */setc 0(%rsp) /* stash returned carry flag */

movq -16(%rbp), %r11 /* r11 = VexGuestAMD64State * */movq %rax, OFFSET_amd64_RAX(%r11) /* save back to RAX */movq %rdx, OFFSET_amd64_RDX(%r11) /* save back to RDX */

/* save carry flag to VEX */xorq %rax, %raxmovb 0(%rsp), %almovq %rax, %rdi /* arg1 = new flag */movq %r11, %rsi /* arg2 = vex state */addq $24, %rsp /* remove syscall parameters */call LibVEX_GuestAMD64_put_rflag_c

4: /* Re-block signals. If eip is in [4,5), then the syscallis complete and we needn't worry about it. */

movq $__NR_sigprocmask, %rax // syscall #movq $VKI_SIG_SETMASK, %rdi // howmovq -32(%rbp), %rsi // postmaskmovl (%rsi), %esisyscall

jb 7f /* sigprocmask failed */

/* don't bother oldmask */

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

5: /* now safe from signals */

xorq %rax,%raxmovq -8(%rbp), %rdimovq -16(%rbp), %rsimovq -24(%rbp), %rdxmovq -32(%rbp), %rcxmovq -40(%rbp), %r8movq %rbp, %rsppopq %rbpret

7: /* failure: return 0x8000 | error code */orq $0x8000, %raxmovq -8(%rbp), %rdimovq -16(%rbp), %rsimovq -24(%rbp), %rdxmovq -32(%rbp), %rcxmovq -40(%rbp), %r8movq %rbp, %rsppopq %rbpret

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: System call: Valgrind: read(2)

PRE(sys_read){

*flags |= SfMayBlock;PRINT("sys_read ( %ld, %#lx, %llu )", ARG1, ARG2, (ULong)ARG3);PRE_REG_READ3(ssize_t, "read",

unsigned int, fd, char *, buf, vki_size_t, count);

if (!ML_(fd_allowed)(ARG1, "read", tid, False))SET_STATUS_Failure( VKI_EBADF );

elsePRE_MEM_WRITE( "read(buf)", ARG2, ARG3 );

}

POST(sys_read){

vg_assert(SUCCESS);POST_MEM_WRITE( ARG2, RES );

}

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: execve(2)

▶ Executing a debug target program (client) under Valgrind== execve(2) a program under kernel

▶ Initial process ``image'' == memory + registers +per-process resources (``struct process'' / ``struct proc'')

▶ Memory▶ Text, data, bss: ELF Program Header PT_LOAD entries▶ Stack: Arguments, environments, ``aux'' information (for

dynamic linker)▶ Registers

▶ Parameters to start code (crt or ld.so)▶ Per-process resources

▶ Timer, ... (what else?)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: execve(2): On *BSD

▶ Kernel parses ELF Program Header▶ Kernel maps text, data, bss, and stack▶ Kernel prepares stack context▶ Kernel prepares register context (``struct trapframe'')▶ Rest done by userland (start code (``crt'') or dynamic

linker (``ld.so''))

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

0x400000

text + data + bss(PT_LOAD entries)

Stack

Address Space

Machine State(struct trapframe)

Registers

Flags Co-processor

Per-Process State(struct process/proc)

char **argv

char **env

auxinfo

arg / env strings

argc

RSP

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: execve(2): On Valgrind

▶ ``initimg''▶ Basically simulates ``execve(2)''▶ Memory and registers▶ ELF Program Header parsing▶ Address space mapped by Valgrind (not kernel)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

VG_(ii_create_image)()

trap vector

Valgrind Text

Valgrind Stack

Kernel Text

syscall

subq $8, $rspandq $~15, $rsp

:

Client Text

Client Stack

ClientMachine State

argc, argv[], env[]auxinfo

argv/env strings

ps_strings

setup_client_stack()

VG_(ii_finalise_image)()

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal

▶ From Wikipedia: https://en.wikipedia.org/wiki/Unix_signalSignals are a limited form of inter-processcommunication used in Unix, Unix-like, and otherPOSIX-compliant operating systems.

▶ Signal action: calling registered functions when signaldelivered (kernel calling userland function!)

▶ Signal trampoline: small code fragment to execute signalhandler and/or return to kernel

▶ BSD signal code: Ancient code (no one wants to touch)Signal code is INTERESTING!!!!!

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal: Sync signal: On *BSD

Signal Handler

Text

Stack

mov %rcx, (%rax)

Signal Handler

Text

Stack

mov %rcx, (%rax)

Signal Trampoline

Signal Handler

Text

Stack

mov %rcx, (%rax)

Signal Trampoline

InternalSignal Stack Data

Signal Handler

Text

Stack

mov %rcx, (%rax)

Signal Trampoline

Signal Handler

Text

Stack

mov %rcx, (%rax)

Signal Trampoline

Signal Handler

Text

Stack

mov %rcx, (%rax)

Kernel createssignal context

and callssignal trampoline

Kernel restoresoriginal context

Signal trampolinereads stack dataand dispatches

handler

Handler finishesand returns back

to trampoline

Trampoline callssigreturn()

to exitssignal context

InternalSignal Stack Data

InternalSignal Stack Data

InternalSignal Stack Data

User code

Kernel sendsig Kernel sigreturn

User codeUser handlerUser trampoline User trampoline

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal: Trampoline: FreeBSD

sys/amd64/amd64/sigtramp.S:

NON_GPROF_ENTRY(sigcode)call *SIGF_HANDLER(%rsp) /* call signal handler */lea SIGF_UC(%rsp),%rdi /* get ucontext_t */pushq $0 /* junk to fake return addr. */movq $SYS_sigreturn,%raxsyscall /* enter kernel with args */

0: hlt /* trap priviliged instruction */jmp 0b

ALIGN_TEXTesigcode:

.data

.globl szsigcodeszsigcode:

.long esigcode-sigcode

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal: Trampoline: OpenBSD

sys/arch/amd64/amd64/locore.S:

NENTRY(sigcode)call *%rax

movq %rsp,%rdipushq %rdi /* fake return address */movq $SYS_sigreturn,%raxsyscallmovq $SYS_exit,%raxsyscall.globl _C_LABEL(esigcode)

_C_LABEL(esigcode):

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal: Trampoline: NetBSD

lib/libc/arch/x86_64/sys/__sigtramp2.S:

NENTRY(__sigtramp_siginfo_2)movq %r15,%rdimovq $SYS_setcontext, %raxsyscallmovq $-1,%rdi /* if we return here, something is wrong */movq $SYS_exit, %raxsyscall

END(__sigtramp_siginfo_2)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal: On Valgrind

▶ Strategy▶ Async: Block and poll (sigwaitinfo())▶ Sync: ``Long-jump'' (HARD WORK)

▶ Trampoline▶ Provide from user▶ ``Fake'' return syscall

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

Signal Frame

Kernel createssignal context

and callssignal trampoline

Client signal handlerdo task

Signal trampolinereads stack dataand dispatches

handler

Handler createsclient signal context

and jump toscheduler

Client trampoline calls

client signal handler

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State V Machine State

Signal Frame

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

Signal Frame

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

Signal Frame

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

Signal Frame

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

Signal Frame

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

Signal Frame

sigreturn()

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

sigreturn()

C Signal Handler

C Text

V Stack

mov %rcx, (%rax)

V Signal Handler

V Text

C Stack

C Signal Trampoline

V Translated Code

Signal Trampoline

V Machine State

sigreturn()

Kernel sendsig

Valgrind signal handler

Client signal handler

Valgrind sendsig Valgrind sigreturn

Valgrind trampoline

Clinet trampolineClient trampoline

Client signal handlerreturns to

client trampoline

Client trampolinecalls "fake"

sigreturn syscall

"Fake" sigreturnrestores

client context

Back to schedulerto continue

restoredclient context

Valgrind code

Client code

Valgrind code

Client code

longjmp

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Internal: Signal: Valgrind: Trampoline

.global VG_(amd64_freebsd_SUBST_FOR_sigreturn)VG_(amd64_freebsd_SUBST_FOR_sigreturn):

/* This is a very specific sequence which GDB uses torecognize signal handler frames. */

movq $__NR_fake_sigreturn, %raxmovq %rsp, %rdiaddq $40,%rdisyscallud2

.global VG_(trampoline_stuff_end)VG_(trampoline_stuff_end):

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Issues: procfs vs. sysctl

▶ procfs is heavily used▶ To retrieve memory mapping information, opened file

descriptors, ...▶ *BSD is basically moving away from that▶ Need to extend sysctl (KERN_PROC_VMMAP)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Issues: Versioning

▶ Syscall and other ABIs (execve + start code, signal, ...)are hardcoded

▶ Can't handle multiple OS versions

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Issues: ioctl

▶ Valgrind needs to know all syscall arguments memoryread/write

▶ ``Broken'' (== impossible to become perfect) by design forcorner cases (user-defined ioctl in kernel modules)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Issues: LD_PRELOAD

▶ OpenBSD prohibits LD_PRELOAD for suid'ed programs

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Summary

▶ External project forked from valgrind-freebsd onBitBucket

▶ Basically ported to OpenBSD/amd64 and NetBSD/amd64▶ Supported static and dynamic executables▶ Supported only basic system calls

▶ Need a few changes against base▶ Already found one (potential) bug in OpenBSD's ld.so▶ TODOs

▶ More ground work (execve(2)/fork(2), threading, ...)▶ More system calls (ioctl(2))▶ More architectures (i386, arm, ...)▶ Code clean-ups (esp. around procfs)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

ValgrindUser

ValgrindDeveloper

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

ValgrindUser

ValgrindDeveloper

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Future

▶ Maitainance is NOT fun (but I will do some)▶ Consider alternatives (clang + ASAN)

..........

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

.....

.....

......

.....

......

.....

.....

.

Questions