ManageIQ

Sprint 44 Review - Sprint End August 1, 2016

August 3, 2016

Overview

● Sprint Statistics (Oleg Barenboim)● Community Update (Carol Chen)● Providers (Greg Blomquist)● User Interface (Dan Clarizio)● Platform (Gregg Tanzillo)● REST API (Alberto Bellotti)● Automate (Greg McCullough)● Performance (Dennis Metzger)● Quality Engineering (Dave Johnson)● Discussion

Sprint Statistics (O. Barenboim)

345 Pull Requests Merged

PR Breakdown by Feature Category (O. Barenboim)

* Note that some PRs have more than one category.

Providers

Sprint Comparison (O. Barenboim)

All Repo Stats (O. Barenboim)

Data source:

https://github.com/ManageIQthese are listed by most recent activity. go into each one that has been updated in the past 20 days and add up the PRs that have been merged

Order descending by #PRs merged

Repositorymanageiq 345integration_tests 52manageiq-ui-self_service 15manageiq.org 8azure-armrest 7wrapanapi 7foreman_api_client 6miq_bot 6manageiq_docs 5manageiq-api-client 5manageiq-providers-amazon 5guides 3manageiq-appliance 3pg-dsn_parser 3ansible_tower_client 2more_core_extensions 2ovirt 2azure-signature 1linux_admin 1manageiq-appliance-build 1ovirt_metrics 1Total 480

Community Update(C. Chen)

● ManageIQ Design Summit 2016 recap (videos, slides, pics, etc.) -

http://manageiq.org/blog/2016/07/manageiq-design-summit-2016-recap-with-

photos-and-videos/

● New blog series: “Last Week in ManageIQ” #LWIMIQ

○ http://manageiq.org/blog/2016/07/last-week-in-manageiq-faster-reports-cloud-networks-and-m

ore/

○ http://manageiq.org/blog/2016/08/last-week-in-manageiq-saying-goodbye-to-rubyrep/

● Manageiq.org website - renewal work ongoing

● Events

Providers(G. Blomquist)

Providers (G. Blomquist)

Support floating IPs during provisioning

Cross-linking with OpenStack instances

Metrics!

RHEV v4 API

Providers - Nuage(G. Blomquist)

Inventory of Managed Cloud Subnets

First PR!

Providers - VMware vSphere(G. Blomquist)

Storage Profiles modeling and inventory

Providers - VMware vCloud(G. Blomquist)

Initial PRs for modeling and inventory (including vApps)

(Video for Sprint 45!)

Providers - Hawkular (A.Bonas)

New features:

● New operations on deployments:○ Add new deployment○ Start/stop deployment

● Performance reports for datasources● Collect more metrics for datasource

Misc:

● Upgrade of Hawkular gem to 2.3.0● Skip unreachable middleware providers when reporting● Add re-checking authentication status functionality/button● Bug fixes

Providers - Pluggability(G. Blomquist)

“Ask, don’t assume”

● SupportsFeatureMixin○ supports :evacuate○ supports :resize○ supports :retire○ supports :regions ○ supports :migrate

User InterfaceDan Clarizio

● PRs merged (132)○ SSUI (15)○ Bugs (78)○ Enhancements (32)○ Backported to Darga (52)

● Refactoring○ Converted to TreeBuilder - Snapshot, Policy, Policy RSOP, C&U Build

Datastores and Clusters/Hosts, Automate Results○ Quadicons○ Explorer Presenter RJS removal○ CodeMirror version updated (used for text/yaml editors)

User Interface

● Enhancements○ Both UIs updated to latest PatternFly and Angular PatternFly○ Self Service UI language selections separated from Operations UI○ Internationalization

Virtual Columns

Toolbars

Removed string interpolation (for better localization)Changed to use gettext’s pluralization

○ Ansible Tower Jobs moved to the Configuration tab (from Clouds/Stacks)○ Interactivity added to C3 charts on C&U screens (demo)

User InterfaceAnsible Tower Jobs on Configuration Tab

User InterfaceAnsible Tower Jobs on Configuration Tab

User InterfaceC3 Charts Capacity & Utilization Interactivity

Demo

PlatformGregg Tanzillo

Enhancements and Bug Fixes

● PostgreSQL High Availability

● Tenancy

● Replication

● Additional Enhancements and Bug Fixes

● Rest API

PlatformGregg Tanzillo

PostgreSQL High Availability● repmgr package added to appliance

○ http://repmgr.org/

○ https://github.com/2ndQuadrant/repmgr

○ Provides support for automatic failover

● Maintain list of active standby database servers

○ Queries repmgr schema and persists locally in YAML

● WIP - Standby DB config in Appliance Console

PlatformGregg Tanzillo

PostgreSQL High Availability● New gem pg-dsn_parser for converting DSN to a hash

● Was originally in our ActiveRecord extensions

PlatformGregg Tanzillo

Tenancy

● Work is in progress to support sharing resources across

tenants

● Added parent_id to CloudTenant

○ Pre-requisite for mapping OpenStack tenants to

ManageIQ tenants

PlatformGregg Tanzillo

Replication (rubyrep)

Additional Enhancements and Bug Fixes

● Watermark reports updated to be based on max of daily

max value instead of max of average value

● Nice values added back to worker processes

● Removed hourly checking of log growth and rotation if >

1gb

● Expression refactoring and cleanup with relative dates

and times

PlatformGregg Tanzillo

REST APIAlberto Bellotti

Continuing Refactoring

● 12 Refactoring PR’s merged● ApiController::CollectionConfig● Include backtrace in test error responses● Using response.parsed_body● Clarifying rspecs expectations

REST APIAlberto Bellotti

Added support for arbitrary resource paths

● In preparation for upcoming /api/automate● Supported via :arbitrary_resource_path in collection

options in api.yml

REST APIAlberto Bellotti

Work started on the ManageIQ API Client

● Ruby gem (not published yet)● ManageIQ::API::Client● https://github.com/ManageIQ/manageiq-api-client● Foundation work merged

connectivity, authentication, dynamic classes for collections and resources, actions, searches, error handling, entrypoint details, ...

● Remaining TODOs logged as issues

REST APIAlberto Bellotti

Added support for Arbitration Profiles● New /api/arbitration_profiles collection● Full CRUD for Arbitration Profiles

GET /api/arbitration_profiles - Query

POST /api/arbitration_profiles - Create

{

“name” : “aws arbitration default”,

“provider” : { “id” : 5 }

}

POST /api/arbitration_profiles/:id - action “edit” to Edit single resource

{

“action” : “edit”,

“resource” : { “availability_zone” : { “id” : 24 }

}

POST /api/arbitration_profiles - action “edit” to Edit multiple resources

DELETE /api/arbitration_profiles/:id - Delete single resource

POST /api/arbitration_profiles - action “delete” to Delete multiple resources

REST APIAlberto Bellotti

Enhancement to /api/users for settings● Users can now update their own e-mail address and settings in addition to

password● /api entrypoint updated to return user’s settings

GET /api

{

“name” : “API”,

…

“settings” : {

“locale” : “en”,

“display” : {

“timezone” : “UTC”,

“theme” : “red”,

…

},

…

}

}

REST APIAlberto Bellotti

Added support for Arbitration Settings● New /api/arbitration_settings collection● Full CRUD for Arbitration Settings

GET /api/arbitration_settings - Query

POST /api/arbitration_settings - Create

{

“name” : “test_settings”,

“display_name” : “Test Settings”

}

POST /api/arbitration_settings/:id - action “edit” to Edit single resource

POST /api/arbitration_settings - action “edit” to Edit multiple resources

DELETE /api/arbitration_settings/:id - Delete single resource

POST /api/arbitration_settings - action “delete” to Delete multiple resources

REST APIAlberto Bellotti

Added support for Cloud Networks queries● New /api/cloud_networks collection● cloud_networks subcollection for /api/providers

● Querying Cloud Networks as follows:

GET /api/cloud_networks

GET /api/cloud_networks/:id

GET /api/providers/:id/cloud_networks

GET /api/providers/:id/cloud_networks/:id

GET /api/providers/:id/cloud_networks/:cloud_networks_id?attributes=security_groups

AutomateGreg McCullough

● Simulation ○ Updated defaults:

■ Entry-point: System/Process/Request (Previous value of “Automation”)■ Execute Method: Enabled

○ RBAC filtering applied to Object Attributes

AutomateGreg McCullough

● Infrastructure Provision○ Update memory values for VM provisioning dialogs

■ 1, 2, 4, 8, 12, 16, 32 GB

● Service Provisioning○ Expose the number_of_vms when building the provision request for a

service.○ The number_of_vms can be set it multiple places:

■ Catalog Item■ Single Service Item Dialog■ Bundle Service Dialog

AutomateGreg McCullough

AutomateGreg McCullough

● Service Dialogs○ Support for “visible” flag for dynamic fields

AutomateGreg McCullough

● Expose Compliance and ComplianceDetail models to the Automation. (PR #9716)○ This allows for a number of use cases where it is beneficial to programmatically access the

most recent compliance status for a resource, and list the details of each condition tested.

● New associations on VmOrTemplate and Host models:○ expose :compliances○ expose :last_compliance

● New Service Models○ Compliance

■ expose :compliance_details○ ComplianceDetail

■ expose :compliance■ expose :miq_policy

AutomateGreg McCullough

● Generic Object○ Service models created for:

■ GenericObject■ GenericObjectDefinition

● Continued efforts:○ Automate model changes for Pluggable providers○ Creating Service BluePrints○ Support for Git backed Automate domains

Performance(Dennis Metzger)

Capacity and Utilization (C & U) - Scheduling Improvements

● Building on enhancements from previous sprint

● Scheduling reduced to 43 from 119 seconds in local test environment

● Analyzed a large environment that was unable to run C & U ○ Appliances were consuming large amounts of swap and becoming unresponsive

○ After applying updates, the result was zero use of swap and memory use decreased

greater than 50%

Quality Engineering(D. Johnson)

● New integration_tests and wrapanapi project repos● Working on our next iteration of the test automation (fw30)

○ Simplifying use and improving API consistency○ Restructuring test parametrization around available MIQ

endpoints■ Same test whether it is run against the Web UI, REST,

rake, database (Write Once, Run Many)○ Restructuring UI navigation

● More Middleware support and test coverage● More Azure support and test coverage● Various fixes to help with test failures

Discussion

Sprint 45 Review - August 24