· pdf file2007: aix kernel hotpatch support available with aix 6.1 2008: ksplice linux...

2007: AIX Kernel hotpatch support available with AIX 6.1

2008: Ksplice Linux hotpatch support (based on MIT student’s master’s thesis)

• Stops all running processes – can’t continue if a thread in a function to be patched

2011: Ksplice acquired by Oracle, no new subscribers

2014: kGraft: patching by SUSE

• No patches to kernel data structures possible

• Uses both old and new versions of patched functions until complete (all threads exit kernel space)

2014: kpatch: patching by Red Hat

• No patches to kernel data structures possible

• Relies on stack backtraces (questionable reliability)

• Stops all running processes during patching – can’t continue if a thread in a function to be patched

https://lwn.net/Articles/634649/

4

(photo from istockphoto.com)

4

Step 1:

• Prepare the new rootvg to be able to start the surrogate.

• Normal alt-disk-copy / update process – but transparent to admin because this is

for temporary use.

• Customize the new rootvg with the live update environment

Step 2:

• Initiate from AIX using HMC APIs

• Reserve CPU/MEM resources from the HMC

Step 3:

• Initiate from AIX using HMC K2 APIs that interact with VIOS

• Same storage accessible from both LPARs (split the paths)

• Original rootvg is mirrored (mirror will be split later)

• Private vlan required between LPARs for the move

Step 4:

• Some “conditioning” of the surrogate LPAR required

• identical device configuration (same devnos)

• network interface not yet configured until IP takeover

•Prepare scratch filesystem on the current boot rootvg for chrooted

environment

Step 5:

• minimal “WPAR-style” checkpoint of each process

• designated pids are not checkpointed

• app memory moved asynchronously

• Sync and split the mirrored VG on the original

• Import the original vg in the chrooted env.

• all communication is via temporary, private vlan

• force same pids, tids, IPC ids for restarted processes

• original IP address configured before restart

• update runs chrooted in original rootvg on surrogate LPAR

• normal in-place update, but when complete, no reboot required

• Prepare the original rootvg to be the primary boot disk for the surrogate

• Monitor the transfer of mem (async mobility).

• For alt_disk_install compatibility, prepare the mirror of the original rootvg (In

case, it needs to be used to restore the previous level)

Step 6:

• Shutdown the original

• Return cpu and mem, resources to the HMC

• Mirror of original rootvg is available if needed to go back to pre-update state

5

Required lvupdate.data file: /var/adm/ras/liveupdate/lvupdate.data

(Start with the lvupdate.template file in same directory)

general:

kext_check = no

disks:

nhdisk = <hdisk name(s) for surrogate boot>

mhdisk = <hdisk name(s) for new rootvg mirror>

hmc:

lpar_id = <requested new lpar_id>

user = <HMC user name>

management_console = <hostname or IP addr>

Command Line

Authenticate with HMC

hmcauth –u hscroot –a hmc_name

Preview

6

geninstall –k –p –d /tmp IZ12345.140806.epkg.Z

Run Live Update

geninstall –k –d /tmp IZ12345.140806.epkg.Z

SMIT

smitty -> Software Installation and Maintenance -> Install and Update Software ->

Install Software -> enter input device

Change the option: INVOKE live update? to yes

NIM Setup

Define NIM environment:

Generate HMC Password Key

# /usr/bin/dpasswd -f /export/eznim/passwd/hmc_passwd -U hscroot -P abc123

Use Key to Define HMC object

# nim -o define -t hmc -a if1="find_net hmc_object 0" -a net_definition=“ent255.255.255.0 9.1.2.1” -a passwd_file=/export/eznim/passwd/hmc_passwdhmc_object

Define Managed System of NIM Standalone

# nim -o define -t cec -a hw_type=8203 -a hw_model=E4A -a hw_serial=0123456 -a mgmt._source=hmc_object cec1

Exchange SSH keys between HMC and NIM master

# dkeyexch -f /export/eznim/passwd/hmc_passwd -I hmc -H hmc_object

Define the NIM standalone pointing to the CEC

# nim -o define -t standalone -a if1=find_net mac1 0” -a net_definition=“ent255.255.255.0 9.1.2.1” -a net_setting1=“100 full” -a mgmt_source=cec1 -a identity=<lpar_id> client1

Note: NIM Live Update will call hmcauth during the cust operation to authenticate the NIM client with the HMC using the HMC passwd_file.

NIM Usage

6

•From NIM master

•Using a NIM live_update_data resource, lvup

# nim -o cust -a live_update=yes -a live_update_data=lvup -a lpp_source=720lpp -a fileset=IZ12345.140806.epkg.Z client1

•Using the client’s /var/adm/ras/liveupdate/lvupdate.data file

# nim -o cust -a live_update=yes -a fileset=IZ12345.140806.epkg.Z client1

•Preview

# nim -o cust -a live_update=yes -a live_update_data=lvup -a install_flags=“-p” -a lpp_source=720lpp -a fileset=IZ12345.140806.epkg.Z client1

•From NIM client

•Using Separate Operations to Allocate and Run Live_Update

# nimclient -o allocate -a lpp_source=720lpp -a live_update_data=lvup# nimclient -o cust -a live_update=yes -a fileset=IZ12345.140806.epkg.Z

•Allocate and Run Live_Update

# nimclient -o cust -a live_update=yes -a lpp_source=720lpp -a live_update_data=lvup -a fileset=IZ12345.140806.epkg.Z

•Preview

# nimclient -o cust -a live_update=yes -a lpp_source=720lpp -a live_update_data=lvup -a install_flags=“-p” -a fileset=IZ12345.140806.epkg.Z

(diagram from istockphoto.com)

6

Requisites:

System firmware

Ax730_066*

Ax740_043*

Ax770_063

Ax773_056

Ax780_056

Ax810 or later

* Limitation: PowerVC can not seamlessly manage the updated LPAR

Hardware Management Console (HMC)

840

Virtual I/O Server

2.2.3.50

2.2.4

RSCT (if required)

3.2.1.0

PowerHA® (if required)

7.2.0

PowerSC™ (if required)

1.1.4.0

Subsystem Device Driver Path Control Module (SDDPCM) (if required)

TBD

7

I/O restrictions

• Any Coherent Accelerator Processor Interface (CAPI) device must not be

open during the Live Update operation.

• No physical or virtual tape or optical device is supported. These devices

must be removed before the Live Update operation can proceed.

• The mirrorvg utility can mirror up to 3 copies. If the root volume group of the

original partition is already being mirrored with 3 copies, the Live Update

operation cannot proceed.

• The Live Update operation is not supported on diskless AIX clients.

• The Live Update operation is not supported in a multibos environment.

• Data Management API (DMAPI) is not supported by the Live Update

feature.

• Virtual Small Computer System Interface (vSCSI) support for the Live

Update operation is only for those logical unit numbers (LUNs) that are

backed by physical volumes, not logical volumes.

• The vSCSI disk support excludes the option where the vSCSI server

adapter can be mapped to any partition or partition slot.

Security restrictions

• The Live Update operation is not supported when a process is using

Kerberos authentication.

• The Live Update feature does not support PowerSC Trusted Logging.

• The Live Update feature is not supported by an active Department of

Defense (DoD) security profile.

• The Live Update feature is not supported when audit is enabled for a

stopped workload partition (WPAR).

• The Live Update feature does not support Public-Key Cryptography

Standards # 11 (PKCS11). The security.pkcs11 fileset cannot be installed.

• The Live Update feature is not supported by any of the following Trusted

Execution options in the trustchk command:

•TEP=ON

•TLP=ON

•CHKSHLIB=ON and STOP_UNTRUSTD=ON

•TSD_FILES_LOCK=ON

Reliability, availability and serviceability (RAS) restrictions

• System trace of the Live Update operation is not possible if channel 0 is

already in use.

• The Live Update feature is not supported when ProbeVue is running. The

ProbeVue session needs to be stopped to run the Live Update operation.

• User storage keys are not supported in the Live Update environment.

7

• The system dump that is present on the root volume group of the original

LPAR is not available after a successful Live Update operation.

Miscellaneous restrictions

• The interim fix must have the LU CAPABLE attribute, which means the

interim fix must be compatible with the Live Update operation. The emgr

command can display this attribute. Ideally, all the interim fixes can be

applied with the Live Update operation, but there might be some exceptions.

• The destination of the interim fixes must be on the root volume group of the

client partition in either /, /usr, /home, /var, /opt, or /tmp file systems.

• The Network File System (NFS)-mounted executables must not be running

during a Live Update operation.

• Active WPARs must be stopped before the Live Update operation.

• RSCT Cluster Services are stopped during a Live Update operation, and

then restarted before the Live Update operation completes.

• A configuration with 16 MB page support is not allowed. The promoted (16

MB Multiple Page Segment Size (MPSS)) pages by Dynamic System

Optimizer (DSO) are supported by the Live Update operation.

• The Live Update operation is supported when the DSO running, but DSO

optimization is reset by the Live Update operation. The optimization begins

again based on workload monitoring after the Live Update operation.

• The Live Update feature is not supported on a partition that participates in

Active Memory Sharing (AMS).

• The Live Update feature is not supported on a remote restartable partition.

• If an interim fix is installed without the Live Update operation that requires a

restart, the restart must be completed before a subsequent Live Update

operation can be started.


7

Demo video

8

No commitment expressed or implied for future releases. All plans subject to

change without notice.

(Photo by Aaron Sheffield (acsheff.wordpress.com), used by permission)

9

Compare Live Update to LPM … will take time to build this out so it’s widely applicable. Let’s accelerate the process for Live Update! Sign up for the Early Ship Program (see final 2 slides), put 7.2 in a test environment, put some real workloads in there, and wring this out so we can make a No Reboot Required world a reality.


10

Headline

Foot page 1212

A single-console management system to identify and report on multiple

devices and attributes (e.g., machine name, OS, IP address, CVE)

Provides details on physical and virtual servers, PCs, Macs, POS devices,

ATMs, kiosks, etc.: Machine Name, OS, IP Address, Malware incidents etc. All

known CVEs exposed on an endpoint

Gain a high degree of confidence

that your endpoints are patched and

secureBased on many IBM clients reporting more than 98% first pass patch success rate

22

BigFix monitors and enforces the security configuration compliance of all

endpoints on a continuous basis. Any endpoints that are found to be out of

compliance can be automatically remediated and brought back into

compliance whether they are on or off the corporate network or they can be

quarantined to prevent the spread of malware. By ensuring all endpoints are

configured per security policies, attack vectors are greatly reduced thwarting

attack attempts.

BigFix is unique in that the endpoint agents automatically and continuously

enforce policies such as patches, configurations, etc. Once the initial policy is

applied, the IEM agent checks to ensure that the endpoint remains in

compliance. If a patch or configuration is changed, BigFix automatically and

autonomously re-applies the policy, ensuring that users or malware cannot

compromise the endpoint policies.

____________________

Traditional compliance approach:

1. The security team develops compliance policies

2. The security team runs an assessment tool (or tools) against that policy

23

3. The security team forwards findings to operations

4. Operations makes corrections as workload allows, one item at a time using different tools from security (which generates different answers to questions like “how many endpoints do I have?”)

5. Users make changes causing endpoints to fall out of compliance again

6. Start assessment all over again

IBM compliance approach

1. Security and operations work together to formulate policies and service-level agreements (SLAs)

2. Operations implements the baseline (patch, configuration, anti-virus, etc.) across all endpoints in the organization

3. Policy compliance is continuously monitored and enforced at the endpoint; changes are reported immediately

4. The security team can instantly check on the current state of security and compliance anytime

5. Security and operations teams work together to continually strengthen security and adjust to evolving requirements

23

28

A dynamic community of open development

and collaboration is driving the creation of new

technologies and capabilities. Unless you’re

part of the community, it’s virtually impossible

To keep pace and reap the benefits.

The collaborative community provides the

opportunity to take advantage of a broader

range of solutions and applications—that is,

if your technology can handle it. Implementing

a proven, open platform provides investment

longevity and viability

A new generation of Power Systems has arrived, with POWER8 technology

onboard. POWER8 is the first open server platform for innovation. In addition

to providing the platform for the OpenPOWER foundation, these

characteristics are what make POWER open:

- Linux only systems offer the Power platform with better economics, allowing

clients to deploy scale out data and cloud infrastructures with greater

efficiency.

- The availability of Ubuntu brings the fastest growing open cloud OS to

Power.

- Power KVM brings KVM, the open source virtualization solution, to Power

Systems, the first virtualization to be offered on Power in lieu of PowerVM.

PowerKVM is offered to clients who recognize value in running a fully open-

source stack on their systems from top to bottom, and/or already have KVM

or OpenStack deployed in their datacenters and can use PowerKVM to

simplify compatibility with their existing management tools.

30

SRIOV Adapter support provides i/o virtualization in the SRIOV adapter which offloads network processing from the host to the SRIOV adapter. This allows guaranteed Quality of service to set by virtual function(VF). These VFs can be dynamically added and removed from guest VMs.

For guest VMs vCPUs can be added or removed from a running VM and Memory can be added to a running guest VM

Dynamic Micro-Threading – the PowerKVM host dynamically changes the Micro-Threading on the system to work best with the running workloads

Scenario- Shared Processor Pool with 4 cores of capacity

- Two VMs/LPARs, Uncapped, each with 1/10 core Entitled Capacity and 4

Virtual Processors

- Since there is no contention in the shared processor pool, all 8 VPs can run

simultaneously

-Uncapped weight was ONLY considered when there is contention on physical

cores for CPU

-When each partition consumes 2.0 proc units, pool cap kicks in and suspends

both LPs until next dispatch window

Customer Expectation

-LP 1 (Test) receives EC of 0.1 plus 1/100 of remaining pool capacity (3.45%

of the pool capacity)

-LP 2 (Prod) receives EC of 0.1 plus 99/100 of remaining pool capacity

(96.55% of the pool capacity)

FW840 Results

- Production partition receives 3.862 out of the 4.0 processor cap

which matches customer expectations

34

New PowerVM NovaLink facility allows direct connection to the PowerVM host

using a Ubuntu Linux Partition. New bare metal installer allows allow

PowerVM components to be installed on a new system with a few questions

under an hour.

New vNIC is a very efficient virtual NIC that enables Mobility for SRIOV

adapters.

LPM improvements include better checking to improve reliability of LPM.

Better use of Ether-Channel configurations. This support spreads the I/O

better across multiple ether-channel interfaces. Improved concurrency for

LPM. Better reliability which allows moving workloads when one VIOS in a

pair is not available.

Shared Storage Pool improvements include Tiered storage support within a

storage pool. Up to 10 tiers. This allows workloads to be placed on the

appropriate tier of storage. Ability to grow a LUN in a storage pool

dynamically.

· pdf file2007: aix kernel hotpatch support available with aix 6.1 2008: ksplice linux...

Documents