ldapcon2015 hamano
TRANSCRIPT
![Page 1: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/1.jpg)
WiredTiger Backend forOpenLDAP
Open Source Solution Technology CorporationTsukasa Hamano <[email protected]>
LDAPCon 2015 Edinburgh November 2015
Open Source Solution Technology Corporation 1
![Page 2: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/2.jpg)
WiredTiger Backend for OpenLDAP
About OSSTech
• ID Management leading company inJapan.
• Storage Solution• Open Source Contribution
Open Source Solution Technology Corporation 2
![Page 3: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/3.jpg)
WiredTiger Backend for OpenLDAP
What’s back-wt
• New OpenLDAP Backend• WiredTiger Database
Open Source Solution Technology Corporation 3
![Page 4: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/4.jpg)
WiredTiger Backend for OpenLDAP
About WiredTiger
• Embedded database• High performance• High scalability
Open Source Solution Technology Corporation 4
![Page 5: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/5.jpg)
WiredTiger Backend for OpenLDAP
Lock Free
• Hazard pointer• Optimistic concurrency control
Open Source Solution Technology Corporation 5
![Page 6: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/6.jpg)
WiredTiger Backend for OpenLDAP
Data Structure
Open Source Solution Technology Corporation 6
![Page 7: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/7.jpg)
WiredTiger Backend for OpenLDAP
bdb_next_id()� �int bdb_next_id( BackendDB *be, ID *out )
{
struct bdb_info *bdb=(struct bdb_info*)be->be_private;
ldap_pvt_thread_mutex_lock(&bdb->bi_lastid_mutex);
*out = ++bdb->bi_lastid;
ldap_pvt_thread_mutex_unlock(&bdb->bi_lastid_mutex);
return 0;
}� �Open Source Solution Technology Corporation 7
![Page 8: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/8.jpg)
WiredTiger Backend for OpenLDAP
wt_next_id()� �int wt_next_id(BackendDB *be, ID *out){
struct wt_info *wi = (struct wt_info *)be->be_private;
*out = __sync_add_and_fetch(&wi->wi_lastid, 1);
return 0;
}� �
Open Source Solution Technology Corporation 8
![Page 9: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/9.jpg)
WiredTiger Backend for OpenLDAP
fsync(2) is slow
Open Source Solution Technology Corporation 9
![Page 10: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/10.jpg)
WiredTiger Backend for OpenLDAP
Durability Levels
1. in-memory txn log – fastest but nodurability
2. write txn log file, no sync3. write txn log file, sync per every
commit
Open Source Solution Technology Corporation 10
![Page 11: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/11.jpg)
WiredTiger Backend for OpenLDAP
New Benchmark Tool - lb
• SLAMD is dead• Command line interface• Written in Go
Open Source Solution Technology Corporation 11
![Page 12: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/12.jpg)
WiredTiger Backend for OpenLDAP
Installation of lb
$ export GOPATH=~/go
$ go get github.com/hamano/lb
Open Source Solution Technology Corporation 12
![Page 13: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/13.jpg)
WiredTiger Backend for OpenLDAP
Benchmark Environment
• 12 Core CPU• No RAID Card• SAS Disk
Open Source Solution Technology Corporation 13
![Page 14: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/14.jpg)
WiredTiger Backend for OpenLDAP
ADD Benchmark Script
� �for c in 1 2 4 8 16 32 64 128 256 512; do
lb add -c $c -n 10000 --uuid ldap://targethost/
done� �
Open Source Solution Technology Corporation 14
![Page 15: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/15.jpg)
WiredTiger Backend for OpenLDAP
ADD (nosync) Benchmarks
0
5000
10000
15000
20000
25000
30000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 15
![Page 16: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/16.jpg)
WiredTiger Backend for OpenLDAP
ADD (sync) Benchmarks
0
1000
2000
3000
4000
5000
6000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 16
![Page 17: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/17.jpg)
WiredTiger Backend for OpenLDAP
BIND Benchmark Script� �for c in 1 2 4 8 16 32 64 128 256 512; do
lb bind -c $c -n 100000 \
-D "cn=user%d,dc=example,dc=com" -w secret \
--last 10000 ldap://targethost/
done� �
Open Source Solution Technology Corporation 17
![Page 18: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/18.jpg)
WiredTiger Backend for OpenLDAP
BIND Benchmark Result
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 18
![Page 19: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/19.jpg)
WiredTiger Backend for OpenLDAP
SEARCH Benchmark Script� �for c in 1 2 4 8 16 32 64 128 256 512; do
lb search -c $c -n 100000 \
-a "(cn=user%d)" \
--last 10000 ldap://targethost/
done� �
Open Source Solution Technology Corporation 19
![Page 20: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/20.jpg)
WiredTiger Backend for OpenLDAP
SEARCH Benchmark Result
0
5000
10000
15000
20000
25000
30000
1 2 4 8 16 32 64 128 256 512
Requests
per
second
Concurrency Level
back-wt req/sec
back-bdb req/sec
back-mdb req/sec
Open Source Solution Technology Corporation 20
![Page 21: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/21.jpg)
WiredTiger Backend for OpenLDAP
Tests
$ make -C tests wt
5465 Succeed
Open Source Solution Technology Corporation 21
![Page 22: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/22.jpg)
WiredTiger Backend for OpenLDAP
Tasks
• Hot-backup• alias and glue entry
Open Source Solution Technology Corporation 22
![Page 23: ldapcon2015 hamano](https://reader031.vdocuments.site/reader031/viewer/2022030314/5887f4101a28ab7d5c8b68c9/html5/thumbnails/23.jpg)
WiredTiger Backend for OpenLDAP
Thank You!
Open Source Solution Technology Corporation 23