advanced topics in module design: threadsafety and portability
DESCRIPTION
Advanced Topics in Module Design: Threadsafety and Portability. Aaron Bannert [email protected] / [email protected]. http://www.codemass.com/~aaron/presentations/ apachecon2005/ac2005advancedmodules.ppt. Thread-safe. From The Free On-line Dictionary of Computing (09 FEB 02) [foldoc]: - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/1.jpg)
Advanced Topics in Module Design:Threadsafety and Portability
Aaron [email protected] / [email protected]
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.http://www.codemass.com/~aaron/presentations/ apachecon2005/ac2005advancedmodules.ppt
![Page 2: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/2.jpg)
Thread-safe
From The Free On-line Dictionary of Computing (09 FEB 02) [foldoc]:
thread-safe
A description of code which is either re-entrant or protected from multiple simultaneous execution by some form of mutual exclusion.
(1997-01-30)
![Page 3: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/3.jpg)
APR
The Apache Portable Runtime
![Page 4: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/4.jpg)
The APR Libraries
APR System-level “glue”
APR-UTIL Portable routines built upon APR
APR-ICONV Portable international character support
![Page 5: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/5.jpg)
Glue Code vs. Portability Layer
“Glue Code” Common functional interface
Multiple Implementations eg. db2, db3, db4, gdbm, …
Sockets, File I/O, …
“Portability Layer” Routines that embody portability
eg. Bucket Brigades, URI routines, …
![Page 6: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/6.jpg)
What Uses APR?
Apache HTTPD Apache Modules Subversion Flood JXTA-C Various ASF Internal Projects ...
![Page 7: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/7.jpg)
The Basics
Some APR Primitive Types
![Page 8: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/8.jpg)
A Who’s Who of Mutexes
apr_thread_mutex_tapr_proc_mutex_tapr_global_mutex_t
apr_xxxx_mutex_lock() Grab the lock, or block until available
apr_xxxx_mutex_unlock() Release the current lock
![Page 9: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/9.jpg)
Normal vs. Nested Mutexes
Normal Mutexes (aka Non-nested) Deadlocks when same thread locks twice
Nested Mutexes Allows multiple locks with same thread
(still have to unroll though)
![Page 10: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/10.jpg)
Reader/Writer Locks
apr_thread_rwlock_t
apr_thread_rwlock_rdlock() Grab the shared read lock, blocks for any writers
apr_thread_rwlock_wrlock() Grab the exclusive write lock, blocking new readers
apr_thread_rwlock_unlock() Release the current lock
![Page 11: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/11.jpg)
Condition Variables
apr_thread_cond_t
apr_thread_cond_wait() Sleep until any signal arrives
apr_thread_cond_signal() Send a signal to one waiting thread
apr_thread_cond_broadcast() Send a signal to all waiting threads
![Page 12: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/12.jpg)
Threads
apr_thread_t
apr_thread_create() Create a new thread (with specialized attributes)
apr_thread_exit() Exit from the current thread (with a return value)
apr_thread_join() Wait until another thread exits.
![Page 13: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/13.jpg)
One-time Calls
apr_thread_once_t
apr_thread_once_init() Initialize an apr_thread_once_t variable
apr_thread_once() Execute the given function once
![Page 14: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/14.jpg)
Apache 2.x Architecture
A quick MPM overview
![Page 15: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/15.jpg)
What’s new in Apache 2.x?
FiltersMPMs
Multithreaded Server Native OS Optimizations
SSL EncryptionImproved Proxy and Cachelots more…
![Page 16: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/16.jpg)
What is an MPM?
“Multi-processing Module” Different HTTP server process models Each give us
Platform-specific features Admin may chose suitable:
• Reliability• Performance• Features
![Page 17: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/17.jpg)
Child
Prefork MPM
Classic Apache 1.3 model 1 connection per Child
Pros: Isolates faults Performs well
Cons: Scales poorly
(high memory reqts.)
Parent
ChildChild … (100s)
![Page 18: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/18.jpg)
Child
Worker MPM
Hybrid Process/Thread 1 connection per Thread Many threads per Child
Pros: Efficient use of memory Highly Scalable
Cons: Faults destroy all threads
in that Child 3rd party libraries must
be threadsafe
Parent
ChildChild … (10s)
10s of threads
![Page 19: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/19.jpg)
WinNT MPM
Single Parent/Single Child 1 connection per Thread Many threads per Child
Pros: Efficient use of memory Highly Scalable
Cons: Faults destroy all threads
Parent
Child
100s of threads
![Page 20: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/20.jpg)
The MPM Breakdown
MPM Multi-process
Multithreaded
Prefork Yes No
Worker Yes Yes
WinNT No* Yes
* The WinNT MPM has a single parent and a single child.
![Page 21: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/21.jpg)
Other MPMs
BeOSNetwareThreadpool
Similar to Worker, experimental
Leader-Follower Similar to Worker, also experimental
![Page 22: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/22.jpg)
Apache 2.x Hooks
Threadsafety within the
Apache Framework
![Page 23: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/23.jpg)
Useful APR Primitives
mutexesreader/writer lockscondition variablesshared memory...
![Page 24: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/24.jpg)
Global Mutex Creation
1. Create it in the Parent: Usually in post_config hook
2. Attach to it in the Child: This is the child_init hook
![Page 25: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/25.jpg)
Example: Create a Global Mutex
static int shm_counter_post_config(apr_pool_t *pconf, apr_pool_t *plog, apr_pool_t *ptemp, server_rec *s) { int rv; shm_counter_scfg_t *scfg; /* Get the module configuration */ scfg = ap_get_module_config(s->module_config, &shm_counter_module);
/* Create a global mutex from the config directive */ rv = apr_global_mutex_create(&scfg->mutex, scfg->shmcounterlockfile, APR_LOCK_DEFAULT, pconf);
![Page 26: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/26.jpg)
Example: Attach Global Mutex
static void shm_counter_child_init(apr_pool_t *p,
server_rec *s)
{
apr_status_t rv;
shm_counter_scfg_t *scfg
= ap_get_module_config(s->module_config,
&shm_counter_module);
/* Now that we are in a child process, we have to
* reconnect to the global mutex. */
rv = apr_global_mutex_child_init(&scfg->mutex,
scfg->shmcounterlockfile, p);
![Page 27: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/27.jpg)
Common Pitfall
The double DSO-load problem Apache loads each module twice:
First time to see if it fails at startup
Second time to actually load it
Also reloaded after each restart.
![Page 28: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/28.jpg)
Avoiding the Double DSO-load
Solution: Don’t create mutexes during the first load
1. First time in post_config we set a userdata flag
2. Next time through we look for that userdata flag
if it is set, we create the mutex
![Page 29: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/29.jpg)
What is Userdata?
Just a hash tableAssociated with each poolSame lifetime as its poolKey/Value entries
![Page 30: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/30.jpg)
Example: Double DSO-load
static int shm_counter_post_config(apr_pool_t *pconf, apr_pool_t *plog, apr_pool_t *ptemp, server_rec *s){ apr_status_t rv; void *data = NULL; const char *userdata_key = "shm_counter_post_config";
apr_pool_userdata_get(&data, userdata_key, s->process->pool); if (data == NULL) { /* WARNING: This must *not* be apr_pool_userdata_setn(). */ apr_pool_userdata_set((const void *)1, userdata_key, apr_pool_cleanup_null, s->process->pool); return OK; /* This would be the first time through */ }
/* Proceed with normal mutex and shared memory creation . . . */
![Page 31: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/31.jpg)
Summary
1. Create in the Parent (post_config)
2. Attach in the Child (child_init)
This works for these types: mutexes condition variables reader/writer locks shared memory etc…
![Page 32: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/32.jpg)
Shared Memory
Efficient and portable shared memory for your Apache module
![Page 33: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/33.jpg)
Types of Shared Memory
Anonymous Requires process inheritance Created in the parent Automatically inherited in the child
Name-based Associated with a file Processes need not be ancestors Must deal with file permissions
![Page 34: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/34.jpg)
Anonymous Shared Memory
Parent SharedSegment
1. Parent creates specialAnonymous shared segment
Child
2. Parent callsfork()
Child 3. Children inherit theshared segment.
![Page 35: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/35.jpg)
Example: Anonymous Shmem
static int shm_counter_post_config(apr_pool_t *pconf,
apr_pool_t *plog,
apr_pool_t *ptemp,
server_rec *s)
{
int rv;
...
/* Create an anonymous shared memory segment by passing
* a NULL as the shared memory filename */
rv = apr_shm_create(&scfg->counters_shm,
sizeof(*scfg->counters),
NULL, pconf);
![Page 36: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/36.jpg)
Accessing the Segment
scfg->counters = apr_shm_baseaddr_get(scfg->counters_shm);
Segment is mapped as soon as it is created
It has a start addressYou can query that start address
Reminder: The segment may not be mapped to the same address in all processes.
![Page 37: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/37.jpg)
Windows Portability
Windows can’t inherit shared memory it has no fork() call!
Solution: Just like we did with mutexes:
The “child” process attaches
(hint: to be portable to Windows, we can only use
name-based shared memory.)
![Page 38: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/38.jpg)
Name-based Shared Memory
SharedSegment
1. Process creates file 2. File is mapped to segment
FirstProcess
file system
SecondProcess
3. Second processopens same file
4. Second process thenmaps the same shared segment.
![Page 39: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/39.jpg)
Sharing with external apps
Must use name-based shm Associate it with a file The other programs can attach to that file
Beware of race conditions Order of file creation and attaching.
Beware of weak file permissions(note previous security problem in Apache scoreboard)
![Page 40: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/40.jpg)
Example: Name-based Shmem
static int shm_counter_post_config(apr_pool_t *pconf, apr_pool_t *plog, apr_pool_t *ptemp, server_rec *s) { int rv; shm_counter_scfg_t *scfg; ... /* Get the module configuration */ scfg = ap_get_module_config(s->module_config, &shm_counter_module);
/* Create a name-based shared memory segment using the filename * out of our config directive */ rv = apr_shm_create(&scfg->counters_shm, sizeof(*scfg->counters), scfg->shmcounterfile, pconf);
![Page 41: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/41.jpg)
Example: Name-based Shmem (cont)
static void shm_counter_child_init(apr_pool_t *p,
server_rec *s)
{
apr_status_t rv;
shm_counter_scfg_t *scfg
= ap_get_module_config(s->module_config,
&shm_counter_module);
rv = apr_shm_attach(&scfg->counters_shm,
scfg->shmcounterfile, p);
scfg->counters = apr_shm_baseaddr_get(scfg->counters_shm);
![Page 42: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/42.jpg)
RMM (Relocatable Memory Manager)
Provides malloc() and free()Works with any block of memoryEstimates overheadThread-safeUsable on shared memory segments
![Page 43: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/43.jpg)
Efficiency
Tricks of the Trade
![Page 44: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/44.jpg)
Questions to ask yourself:
Uniprocessor or Multiprocessor?
What Operating System(s)?
How can we minimize or eliminate our critical code sections?
Exclusive access or read/write access?
![Page 45: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/45.jpg)
APR Lock PerformanceMac OS X 10.2.x PowerPC
lower is better
Mac OS X 10.2.x Darwin PowerPC - testlockperf
0
5
10
15
20
25
30
35
40
1 2 3 4 5 6
Threads
time (sec)
normal mutex (orig)
nested mutex (orig)
rwlock (orig)
![Page 46: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/46.jpg)
APR Lock PerformanceLinux 2.4.18 (Redhat 7.3)
lower is better
Linux 2.4.18-3 UP i586 - testlockperf
0
10
20
30
40
50
60
70
80
90
100
1 2 3 4 5 6
threads
time (sec)
normal mutex (rh9)
nested mutex (rh9)
rwlock (rh9)
![Page 47: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/47.jpg)
APR Lock PerformanceLinux 2.4.20 SMP (Redhat 9)
lower is better
Linux 2.4.20-31.9bigmem SMP i686 - testlockperf
0
5
10
15
20
25
30
1 2 3 4 5 6
Threads
time (sec)
normal mutex (rh9)
nested mutex (rh9)
rwlock (rh9)
![Page 48: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/48.jpg)
APR Lock PerformanceSolaris 2.9 x86
lower is better
Solaris 2.9 x86 UP
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
1 2 3 4 5 6
threads
time (sec)
normal mutex (rh9)
nested mutex (rh9)
rwlock (rh9)
![Page 49: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/49.jpg)
Relative Mutex PerformanceComparing Normal Mutexes
lower is better
Normalized Comparison of Normal Mutexes
0
10
20
30
40
50
60
70
80
1 2 3 4 5 6
threads
relative time
10.2.x
rh9 smp bigmem
solaris 2.9 x86
rh72 UP
![Page 50: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/50.jpg)
Relative Mutex PerformanceComparing Nested Mutexes
lower is better
Normalized Comparison of Nested Mutexes
0
5
10
15
20
25
30
1 2 3 4 5 6
threads
relative time
10.2.x
rh9 smp bigmem
Solaris 2.9 x86
rh72 UP
![Page 51: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/51.jpg)
Relative R/W Lock PerformanceComparing Read/Write Locks
lower is better
Comparison of Read/Write Locks
0
2
4
6
8
10
12
14
16
18
1 2 3 4 5 6
threads
relative time
10.2.x
rh9 smp bigmem
Solaris 2.9 x86
rh7.2 UP
![Page 52: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/52.jpg)
R/W Locks vs. Mutexes
Reader/Writer locks allow parallel reads
APR’s nested mutexes are slow
Reader/Writer locks tend to scale much better
SMP hurts lock-heavy tasks
![Page 53: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/53.jpg)
OS Observations
Solaris has very fast and stable locks
Linux struggling but getting faster
NTPL shows improvement in overall thread performance, but
not in lock overhead.
MacOS (Jaguar) is stable and moderately fast
rwlocks could be improved
![Page 54: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/54.jpg)
APR Atomics
Very Fast Operations Can implement a very fast mutex
Pros: Can be very efficient
(sometimes it becomes just one instruction)
Cons: Produces non-portable binaries
(e.g. a Solaris 7 binary may not work on Solaris 8)
![Page 55: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/55.jpg)
Threads
Adding threads to yourApache modules
![Page 56: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/56.jpg)
Why use threads in Apache?
background processingasynchronous event handlingpseudo-event-driven modelshigh concurrency serviceslow latency services
![Page 57: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/57.jpg)
Thread Libraries
Three major types 1:1
one kthread = one userspace thread 1:N
one kthread = many userspace threads N:M
many kthreads ~= many userspace threads
![Page 58: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/58.jpg)
1:1 Thread Libraries
E.g. Linuxthreads NPTL (linux 2.6?) Solaris 9+’s threads etc...
Good with an O(1) scheduler
Can span multiple CPUs
Resource intensive
Userspace
Kernel
kthread1
kthread4
kthread5
kthread6
kthread3
kthread2
thread1
Process
thread2
thread3
![Page 59: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/59.jpg)
1:N Thread Libraries
E.g. GnuPth FreeBSD <4.6? etc...
Shares one kthread Can NOT span multiple
CPUs Not Resource Intensive Poor with compute-
bound problems
Userspace
Kernel
kthread1
kthread4
kthread5
kthread6
kthread3
kthread2
thread1
Process
thread2
thread3
![Page 60: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/60.jpg)
M:N Thread Libraries
E.g. NPTL (from IBM) Solaris 6, 7, 8 AIX etc...
Shares one or more kthreads
Can span multiple CPUs Complicated Impl. Good with crappy
schedulers
Userspace
Kernel
kthread1
kthread4
kthread5
kthread6
kthread3
kthread2
thread1
Process
thread2
thread3
![Page 61: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/61.jpg)
Pitfalls
pool associationcleanup registrationproper shutdownasync signal handlingsignal masks
![Page 62: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/62.jpg)
Bonus: apr_reslist_t
Resource Lists
![Page 63: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/63.jpg)
Resource Pooling
List of ResourcesCreated/Destroyed as neededUseful for
persistent database connections request servicing threads ...
![Page 64: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/64.jpg)
Reslist Parameters
min min allowed available resources
smax soft max allowed available resources
hmax hard max on total resources
ttl max time an available resource may idle
![Page 65: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/65.jpg)
Constructor/Destructor
Registered CallbacksCreate called for new resourceDestroy called when expunging oldImplementer must ensure threadsafety
![Page 66: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/66.jpg)
Using a Reslist
1. Set up constructor/destructor
2. Set operating parameters
3. Main Loop1. Retrieve Resource
2. Use
3. Release Resource
4. Destroy reslist
![Page 67: Advanced Topics in Module Design: Threadsafety and Portability](https://reader035.vdocuments.site/reader035/viewer/2022081603/568148ba550346895db5d409/html5/thumbnails/67.jpg)
Thank You
The End