sprint 39 review

ManageIQSprint 39 Review - Sprint End April 19, 2016

April 20, 2016

Overview● Sprint Statistics (Oleg Barenboim)● User Interface (Dan Clarizio)● Providers (Greg Blomquist)● Platform (Gregg Tanzillo)● REST API (Alberto Bellotti)● SmartState (Rich Oliveri)● Automate (Greg McCullough)● Dockerizing ManageIQ (Barak Azulay)● Performance (Dennis Metzger) ● Discussion

Sprint Statistics (O. Barenboim)

440 Pull Requests Merged (new record!)

* Note that some PRs have more than one label.

PR Breakdown by Feature Category (O. Barenboim)

* Note that some PRs have more than one category.

Providers

Sprint Comparison (O. Barenboim)

All Repo Stats (O. Barenboim)

Data source:

https://github.com/ManageIQthese are listed by most recent activity. go into each one that has been updated in the past 20 days and add up the PRs that have been merged

Order descending by #PRs merged

Repository PRs Mergedmanageiq 440

ansible_tower_client 5

azure-armest 2

guides 5

manageiq_docs 1

manageiq-appliance 4

manageiq-appliance-build 4

manageiq-appliance-dev-setup 1

manageiq-ui-self_service 5

manageiq.org 5

miq_bot 2

ovirt 3

Darga is Coming● Today, we are making the Darga branch● Named after Klaus Darga (chess grandmaster)● First Darga Build coming in next few days● Next 2 sprints will be focused on stabilizing Darga● Planning for Darga Release in June● Name of “E” Release

○ Think of Candidate Names now○ Vote during ManageIQ Summit (week of June 6)

User Interface (D. Clarizio)● PRs merged (145+)

○ SSUI (5)

○ Bugs (64)

○ Enhancements (55)

○ Refactoring/Technical Debt (16)

● New functionality

○ New vertical navigation menus (Demo)

○ VM Reconfigure - add/remove disks support (Demo)

○ Orderable Orchestration Templates - create and copy (Demo)

○ Datastore Clusters for VMware vSphere 5+ - new Explorer

○ SlickGrid replaced with Patternfly TreeGrid

○ Patternfly style updates to the Dashboard and other areas

○ Template/Image compliance policy checking (previously only allowed for VMs/Instances)

○ New UI for replication configuration

○ UI updates to support multiple provider endpoints

UI - New Datastore Explorer

UI - Datastore Clusters

UI - SlickGrid replaced with Patternfly TreeGrid

UI - SVG replacement of PNGs

UI - Patternfly Style Updates

https://github.com/ManageIQ/manageiq/blob/master/app/assets/stylesheets/main.scss

UI - Demos● New vertical navigation menus (E. Winchell)

● VM Reconfigure (H. Kataria)

● Orderable Orchestration Templates (H. Kataria)

Providers (G. Blomquist)

VMwareAzure (video)OpenStackNetwork Topology (video)Hawkular MiddlewareContainersAnsible TowerRHEV

Providers (G. Blomquist)

VMware Clustered Datastores in Provisioning

Providers (G. Blomquist)

Azure Subscription (@djberg96)

Providers (G. Blomquist)

Azure Events (@bronaghs) [video]

Providers (G. Blomquist)

OpenStack Ceilometer Events

Providers (G. Blomquist)

Network Topology (@lsmola) [video]

Hawkular - Event Catcher(H. Rupp)

Event CatcherCurrently for deployment error and success

Hawkular - Server reload(H. Rupp)

Power operations to reload+stop MW-servers

Not reloading/stoppingthe provider

Containers - Chargeback(F. Simoncelli)

Containers - Chargeback(F. Simoncelli)

● Chargeback totals per Project (Namespace)○ CPU Cores usage○ Memory usage○ Network usage (when available)

Containers - OpenSCAP(F. Simoncelli)

● Smart State Analysis has been extended with the OpenSCAP support○ Scanning Pod is instantiated in the container cluster○ Regular SSA happens in ManageIQ using WebDAV to communicate to the

Scanning Pod○ OpenSCAP scan runs in the Pod and ManageIQ collects the results○ No interdependency between regular SSA and OpenSCAP

● At any time it’s possible to see the usual OpenSCAP HTML results per Image

● Scans are scheduled at regular intervals and on a new Image discovery

● Out-of-the-Box Policies for Image compliance○ Additional ones can be created by the users○ Results are pushed into OpenShift annotating Images

Containers - OpenSCAP(F. Simoncelli)

Containers - Policies(F. Simoncelli)

● Support introduced for OpenSCAP○ Out-of-the-Box policies for Images SSA

● Can be extended further (Events and Policy entities) for other use cases

Containers - Cloud Cross Linking(F. Simoncelli)

● Already existing for Infrastructure Providers: RHEV, VMWare, OpenStack

● Extended to Cloud Providers (GCE, AWS) using the ProviderID reported by Kubernetes and OpenShift

Containers - Cloud Cross Linking(F. Simoncelli)

● At the moment, requires a Cloud Provider being configured in Kubernetes and OpenShift (GCE, AWS)

● Ongoing effort in cAdvisor and Kubernetes to automate the ProviderID○ This will allow for other Cloud Providers to be

supported (Azure)○ Targeted for OpenShift Origin 1.3

Containers - Dashboard Additions(F. Simoncelli)

● Pods creation and deletion trends● New images usage trend

Containers - Pods Network Metrics(F. Simoncelli)

Add support for Pods Network Metrics● Supported by Heapster in OpenShift Origin 1.2

Containers - More Enhancements(F. Simoncelli)

● Upgrade to new Gems○ Kubeclient 1.1.3 (was 0.8.1)○ OpenShift Client 1.1.0 (was 0.3.0)○ Image Inspector 1.0.1 (was 1.0.0)

● Display labels for Projects and Routes

Containers - Ongoing (F. Simoncelli)

● Continue OpenShift Deployment automation● Use Multi-Endpoints for Hawkular Metrics

○ No additional steps to use OpenShift Metrics○ No dedicated router on Master node

● Auto-Tagging based on Kubernetes Labels○ It will be possible to map Regions and Zones to

specific Tags in ManageIQ○ This can be extended to other Providers

Ansible Tower (G. McCullough)

● Model updates○ Link JobTemplate instances to Inventory○ Link Ansible hosts to VMs (when possible)

● New Automate namespace○ /ManageIQ/ConfigurationManagement/AnsibleTower

● Demos○ AnsibleTower as a Service Type Bill Wei○ VM Custom Button Integration Madhu Kanoor○ VM Provisioning Integration Madhu Kanoor

Providers (G. Blomquist)

Generic Targeted Refresh Process@blomquisg

RHEV Targeted Refresh@pkliczewski

Providers (G. Blomquist)

Providers (G. Blomquist)

Multiple Endpoints (@juliancheal) [video]

Platform (G. Tanzillo)

Enhancements

● Replication

● Chargeback

● Appliance Console

● Rest API (A. Bellotti)

Platform - Replication(G. Tanzillo)

Replication configuration screen

● Global and Remote regions

● Enable replication on remote regions

● Subscribe to remote regions on global

Platform - Replication(G. Tanzillo)

Remote Region

Platform - Replication(G. Tanzillo)

Remote Region

Platform - Replication(G. Tanzillo)

Global Region

Platform - Replication(G. Tanzillo)

Global Region

Platform - Replication(G. Tanzillo)

Global Region

Platform - Chargeback (G. Tanzillo)

Chargeback Rate Tiers

Platform - Appliance Console(G. Tanzillo)

Appliance Console

● Added alias ‘ap’ as shortcut for appliance_console

○ alias ap='/usr/bin/appliance_console'

Platform - Appliance Console(A. Bellotti)

● Supports updating some external authentication settings

○ New menu entry and usage:

Platform - Appliance Console(A. Bellotti)

● External authentication options can also be updated using the Appliance

Console CLI:

# appliance_console_cli --extauth-opts=sso_enabled=true|false

# appliance_console_cli --extauth-opts=saml_enabled=true|false

# applicance_console_cli --extauth-opts=local_login_disabled=true|false

REST API (A. Bellotti)

● Versioning bumped to v2.2.0

● Added support for /api/settings○ Provides access to subset of Settings

○ Driven by config/api_settings.yml

○ Supports GET /api/settings[/:section]

GET /api/settings/product

{ "product": { "maindb": "ExtManagementSystem" }}

REST API (A. Bellotti)

● Now supporting CRUD on tenant quotas● Create: POST /api/tenants/:id/quotas

{“name” => “cpu_allocated”,“value” => 1

}● Queries: GET /api/tenants/:id/quotas

GET /api/tenants/:id/quotas/:quota_id● Updates: POST /api/tenants/:id/quotas/:quota_id - action “edit”

PUT /api/tenants/:id/quotas/:quota_idPOST /api/tenants/:id/quotas - action “edit” to update multiple

● Deletes: POST /api/tenants/:id/quotas/:quota_id - action “delete”DELETE /api/tenants/:id/quotas/:quota_id

SmartState (R. Oliveri)

SmartState Pull Requests:● Total merged: 13● Bugs: 9

○ SCVMM: 3○ RHEV: 2○ VMware: 1○ Azure: 1○ Common: 2

● Enhancements: 4

SmartState (R. Oliveri)

Features:● LVM thin volume support.

○ Thin volumes ignored in the past.○ VMs with root on thin volumes would fail.○ VMs with thin volumes are now fully supported.

● SmartState Analysis for Azure.○ Images○ Instances○ Optimization TBD

Automate (G. McCullough)

● Modeling changes○ quota_source_type moved into instance○ Added Auto-Approval/Email to VM Reconfigure

● VM Provisioning○ Disable Auto-Placement logic for

create_provision_request and Rest API calls■ Caller must provide all required placement fields■ Default: placement_auto = true

Automate (G. McCullough)

Example:"vm_fields" => {

"placement_auto" => "false","vm_name" => "test",

"Placement_availability_zone" => "2", # us-east-1b

"instance_type" => "2", # 'm1.small: M1 General Purpose Small'

"cloud_network" => "2",

"cloud_subnet" => "3",

"security_groups" => "64",

"floating_ip_address" => "4",

"guest_access_key_pair" => "65",

"monitoring" => "basic",

}

Automate (G. McCullough)

● Generic Object○ New models

■ GenericObjectDefinition● Defines the attributes for a GenericObject type.● Relationships support will be added in future sprint

■ GenericObject● Common attributes are saved in generic_objects table

○ name, uid and generic_object_definition● All other attributes stored in custom_attributes

○ Next: Defining Object types and exposing to automate

● Automate - Git support○ Modeling added○ New “Git Repositories Owner” role○ Branch/Tag lookup supported○ Initial import logic○ Next: rake task for import / UI support

Dockerizing ManageIQ (B. Azulay)

● What is this about○ Be able to deliver ManageIQ docker appliance ○ To be able to install & run ManageIQ with a one liner command (on any OS that supports

docker, including Atomic)

● What can developers do with it○ Deliver self built appliance for pre integration ○ Multiple environments on the same host (with separate builtin DBs)○ Test your code in appliance mode ○ Test your code in a clean environment

● How should developers use it○ Simply follow https://github.com/ManageIQ/manageiq/tree/master/docker-assets

● Things you should be aware of○ Monolithic, Systemd, appliance mode, image size ~1.3 G○ You can not run multiple manageiq containers at the same time on the same host (ports

collision)○ First execution takes a bit longer (DB migrations run on first execution), simply wait till the

container listens to port 80 … and you are good (takes couple of minutes … depending your HW)

Performance (D. Metzger)

Pull Requests:● Total merged: 22

○ Bugs: 4○ Enhancements: 18

Performance (D. Metzger)

● Support for sorting Virtual Columns in the database○ Previously sorting on Virtual Columns was done in Ruby

● Metrics Rollups bug fix○ Only load recent (needed) performance records not all records

● Service Tree improvement○ Test at a customer site showed:

■ Reduction in page rendering time from 68 seconds to 10 seconds■ Reduction in number of queries from 3,200 to 860

● RBAC○ Ongoing effort to reduce the number SQL queries and quantity of data

being transferred

Discussion

Next Sprint Review - May 11