trivadis techevent 2016 cgroups im einsatz von florian feicht
TRANSCRIPT
BASLE BERN BRUGG DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. GENEVA
HAMBURG COPENHAGEN LAUSANNE MUNICH STUTTGART VIENNA ZURICH
cgroups im EinsatzResource Management mal andersrum
Florian FeichtConsultant
Florian Feicht
cgroups im Einsatz2 26.09.2016
Consultant and Trainer at Trivadis GmbH
Focus on:
– Oracle Database High Availability
– Database Administration
– Backup/Recovery
– Upgrades/Migrations
– Linux Administration
Agenda
cgroups im Einsatz3 26.09.2016
1. Resource Management
2. What are cgroups?
3. cgroups in init.d and systemd environments
4. Configuration
5. Implementation
6. Use Cases
7. Conclusion
Resource Management
cgroups im Einsatz5 26.09.2016
Preserved resources
Operational stability
Reduce of costs
Different possibilities:
– Virtualization
– Application side solutions (Resource Manager)
– …
– cgroups
What are cgroups?
cgroups im Einsatz7 26.09.2016
Linux Kernel feature (since 2.6.24) to limit resources
Process groups
Based on „everything is a file“ unix paradigma
sched_setaffinity Check cgroup
root Unlimited
cgroupCheck cpuset
configuration
mbind,
set_mempolicyCheck cgroup
root Unlimited
cgroupsCheck cpuset
configuration
What are cgroups?
cgroups im Einsatz8 26.09.2016
Consists of resource controller
Base technology for Linux Container systems
Resource
Controller
cgroups
Structure
cgroups im Einsatz10 26.09.2016
Process orientated
No cgroup mount per default
libcgroup is available for administration
[root@doagdb-ol6 ~]# lssubsys -am
cpu
memory
devices
net_cls
cpuset,cpuacct,freezer,blkio,perf_event /mnt/cgroup
Creation
cgroups im Einsatz11 26.09.2016
libcgroup
– cgconfig
– cgcreate
– etc
Manual configuration
mount {
cpuset = /cgroup/cpuset;
cpu = /cgroup/cpu;
}
[root@doagdb-ol6 cpuset]# mkdir db1
[root@doagdb-ol6 cpuset]# ll db1/
-rw-r--r--. 1 root root 0 Apr 19 13:44 cpuset.cpu_exclusive
-rw-r--r--. 1 root root 0 Apr 19 13:44 cpuset.cpus
-rw-r--r--. 1 root root 0 Apr 19 13:44 cpuset.mems
-rw-r--r--. 1 root root 0 Apr 19 13:44 tasks
Structure
cgroups im Einsatz13 26.09.2016
Essential component
Moved from process to application layer
Administration with built-in systemd tools
libcgroup is deprecated
Structure
cgroups im Einsatz14 26.09.2016
system.slice
ora_db2.service ora_pmon_DB2
sshd.service
user.slice
user_<id>.slice session_1.scope
sshd: root
-bash
user_id<id>.slice session_3.scope
sshd: oracle
ora_pmon_DB1
systemd service configuration
cgroups im Einsatz15 26.09.2016
root@doagdb-cgrp-ol7 ~]# systemctl status oracle_db2.service
● oracle_db1.service - Oracle services started by Trivadis
BasEnv ...
CGroup: /system.slice/ora_db2.service
system.slice ora_db2.service ora_pmon_DB2
systemctl daemon-reload
systemctl enable oracle_db2.service
systemctl start oracle_db2.service
Creation
cgroups im Einsatz16 26.09.2016
Manual configuration
– start in user.slice
– Check startup
systemd
– No further configuration necessary
– Implementation is not complete
user.slice user_id<id>.slice session_3.scope ora_pmon_DB1
system.slice ora_db2.service ora_pmon_DB2
libcgroup
cgroups im Einsatz18 26.09.2016
Check existing subsystems
– lssubsys -am
Configuration:
– cgcreate
– cgset
– cgsnapshot
Resource Controller cgroup
cgset -r cpuset.cpus=1 db1
zugeordnete CPU
Manual configuration
cgroups im Einsatz19 26.09.2016
Limitation of resources
Creation of subgroups under depending resource controller
– „Files“ will be created automatically
mkdir /sys/fs/cgroup/cpuset/db1
echo 0 > /sys/fs/cgroup/cpuset/db1/cpuset.cpus
echo 0 > /sys/fs/cgroup/cpuset/db1/cpuset.mems
systemd
cgroups im Einsatz20 26.09.2016
Resource Controller:
– CPU
– Memory
– I/O
– Devices
Configuration using systemctl
systemctl set-property unit-name property=value
system.slice ora_db2.service ora_pmon_DB2
Monitoring I
cgroups im Einsatz21 26.09.2016
/proc/
ps
ps -ax --format args,cgroup
ora_pmon_DB2 1:name=systemd:/system.slice/oracle_db2.service
ora_psp0_DB2 1:name=systemd:/system.slice/oracle_db2.service
cat /proc/2911/cgroup
10:perf_event,blkio,freezer,cpuacct,cpuset:/tvd12cdb
Monitoring II
cgroups im Einsatz22 26.09.2016
Only in systemd environments:
– systemd-cgtop
– systemd-cgls
Path Tasks %CPU Memory Input/s Output/s
/system.slice/oracle_db2.service 42 3.2 1.6G 190.8K 31.8K
├─1 /usr/lib/systemd/systemd --switched-root --system --deserialize 21
├─user.slice
│ └─user-54321.slice
│ └─session-1.scope
│ ├─2846 -bash
└─system.slice
├─oracle_db2.service
│ ├─2575 ora_pmon_DB2
Database start in a cgroup
cgroups im Einsatz24 26.09.2016
cgexec
cgexec -g cpuset,cpuacct,blkio:tvd12cdb sqlplus / as sysdba
Redirect Shell
systemd Service
echo $$ > /mnt/cgroup/tvd12cdb/tasks
init.ora parameter
– Only with manual configuration
Configuration proposal by Oracle I
cgroups im Einsatz25 26.09.2016
MOS: Doc ID 1585184.1 and Doc ID 1928328.1
Provided configuration script: setup_processor_group.sh
– manual cgroup mount will be done
– Resource controller: cpuset
– Persistence is ensured
Configuration proposal by Oracle II
cgroups im Einsatz26 26.09.2016
Change of init.ora Parameter
– Restart necessary
Only on CDB level
Entry in alert.log
Instance started in processor group tvd12cdb (NUMA Nodes: 0
CPUs: 0)
ALTER SYSTEM SET processor_group_name='tvd12cdb' SCOPE=SPFILE;
Configuration proposal by Oracle III
cgroups im Einsatz27 26.09.2016
Advantages
– Easy naming
– Same functionality in init.d and
systemd environments
– cgroup usage entry in alert.log
– cpuset is enabled by default
– Permission for oracle user is set
Disadvantages
– cgroup necessary for DB startup
– Further resource controller have to be
mounted manually
– Check of the script is very limited
Further configuration possibilities
cgroups im Einsatz28 26.09.2016
systemd configuration
– No need to set init parameter
– No cpuset controller possible
Manual configuration
– Init parameter necessary
– Flexible resource controller
– Persistence has to be configured manually
Necessary DB parameter
cgroups im Einsatz29 26.09.2016
No special settings necessary
Ensure correct cgroup for server processes
One listener per database is recommended
– Check startup configuration
– DEDICATED_THROUGH_BROKER_listener_name
ALTER SYSTEM SET dedicated_server=TRUE;
Manual configuration: test database
cgroups im Einsatz31 26.09.2016
Requirements
– Usage of just one CPU
– Maximum of 10MB/s IO read und write
echo 1 > cpuset.cpus
echo "252:0 10485760" > blkio.throttle.read_bps_device
echo "252:0 10485760" > blkio.throttle.write_bps_device
Manual configuration: test database
cgroups im Einsatz32 26.09.2016
Comparison of AWR reports
– Unlimited:
– Limited:
systemd
cgroups im Einsatz33 26.09.2016
BlkIO controller has to be activated
Resource controller cpuset is not available
systemctl set-property oracle_db2.service CPUAccounting=TRUE
systemctl set-property oracle_db2.service CPUShares=500
systemctl set-property oracle_db2.service BlockIOAccounting=TRUE
systemctl set-property oracle_db2.service
BlockIOWriteBandwidth="/dev/sda 10M"
Use Cases
cgroups im Einsatz35 26.09.2016
Database cloud environments
– Service classes by defined cgroups
– Increased security level by device mapping
Test and production database on the same hardware
Resource management in Standard Edition
Assigment of CPU and memory segments (NUMA Systeme)
I/O limitation
Conclusion
cgroups im Einsatz37 26.09.2016
cgroups are:
– Easy to configure
– flexible
– „Licensefree“
Be careful:
– Concept necessary
– Ensure cgroup availability
– Transparent for the database
– Oracle license
Simple resource management for
Oracle databases on Linux
Any Question…?Florian Feicht
Consultant
26.09.2016 SSD in database environments38
@FlorianFeicht
Session Feedback – now
TechEvent September 201639 09.09.2016
Please use the Trivadis Events Mobile App to give session feedback
Use "My schedule" if you registered for this session
Otherwise use "Agenda" and the search function
If the mobile App does not work (or if you have a Windows Phone) use your Mobile
Browser
– URL: http://trivadis.quickmobileplatform.eu/
– Username: <your_loginname> (like svv)
– Password: sent by mail...