making glance tasks work for you - openstack summit may 2015 vancouver

Making Glance tasks work for you

Brian Rosmaita

• Brian Rosmaita – @br14nr – Freenode: rosmaita

About me

2 www.rackspace.com


• Compute Control Plane Product Manager at Rackspace

About me

3 www.rackspace.com



• OpenStack ATC since Folsom

About me

4 www.rackspace.com



• OpenStack ATC since Folsom • Glance driver

About me

5 www.rackspace.com



• OpenStack ATC since Folsom • Glance driver • Happy to be here, hope you’re enjoying the summit

About me

6 www.rackspace.com

Glance at Rackspace

7 www.rackspace.com

Glance at Rackspace

8 www.rackspace.com

http://launchpad.net/glance

• This talk was originally planned to be given along with Nikhil Komawar, the Glance Project Technical Lead

The old bait-and-switch

9 www.rackspace.com

• This talk was originally planned to be given along with Nikhil Komawar, the Glance Project Technical Lead – Unfortunately, he can’t be here today


10 www.rackspace.com


• “Making Glance tasks work for you”




• “Making Glance tasks work for you” – Actually, the focus won’t be on how to implement your own tasks




• “Making Glance tasks work for you” – Actually, the focus won’t be on how to implement your own tasks – More about what tasks are, why you should consider using them




• “Making Glance tasks work for you” – Actually, the focus won’t be on how to implement your own tasks – More about what tasks are, why you should consider using them – … and some requests for feedback



• Ops Feedback – Thursday May 21, 2015 11:50am - 12:30pm – Room 218

Glance design session tomorrow


• Ops Feedback – Thursday May 21, 2015 11:50am - 12:30pm – Room 218 – Please attend if you can – If you can’t, put suggestions on this etherpad: http://etherpad.openstack.org/p/liberty-glance-ops-feedback

Glance design session tomorrow


MAKING GLANCE TASKS WORK FOR YOU


• I was told that to have an effective presentation, you need to start with a compelling story

18

This presentation

www.rackspace.com

• I was told that to have an effective presentation, you need to start with a compelling story

• I will give you one!

19

This presentation

www.rackspace.com

• I was also told that it’s good if your presentation includes some controversy

20

This presentation

www.rackspace.com

• I was also told that it’s good if your presentation includes some controversy

• I’ll give you some of that, too!

21

This presentation

www.rackspace.com

CONTROVERSY!


• Wednesday May 20, 2015 2:40pm - 3:20pm

• Room 222 • https://etherpad.openstack.org/p/liberty-what-is-glance

23

The direction of the Glance project

www.rackspace.com

• Glance’s role in OpenStack: Image service

24

The Glance mission statement

www.rackspace.com

• Glance’s role in OpenStack: Image service

• On July 10, 2014, the following mission statement was merged:

To provide a service where users can upload and discover data assets that are meant to be used with other services, like images for Nova and templates for Heat

– https://review.openstack.org/#/c/98002/

25

The Glance mission statement

www.rackspace.com

• Images API – v1 changed to SUPPORTED status – v2 now considered CURRENT

26

Recent Developments in Glance

www.rackspace.com


• Metadefs catalog – Added in Juno – /v2/metadefs

27


www.rackspace.com



• Catalog Index Service (Experimental API) – Added in Kilo – /v0.1/search

28


www.rackspace.com



• Catalog Index Service (Experimental API) – Added in Kilo – /v0.1/search

• Artifacts (Experimental API) – Scheduled for Liberty – /v0.1/artifacts

29


www.rackspace.com

• Wednesday May 20, 2015 2:40pm - 3:20pm

• Room 222 • https://etherpad.openstack.org/p/liberty-what-is-glance

30

The direction of the Glance project

www.rackspace.com

• Wednesday May 20, 2015 3:30pm – 4:10pm • Room 222 • https://etherpad.openstack.org/p/catalog-index-service-liberty

31

Catalog Index Service – Future and Scope

www.rackspace.com

MAKING GLANCE TASKS WORK FOR YOU


• A compelling story

Outline


• A compelling story • Glance tasks

Outline


• A compelling story • Glance tasks • Controversy!

Outline


• A compelling story • Glance tasks • Controversy! • Concluding thoughts

Outline


• A compelling story • Glance tasks • Controversy! • Concluding thoughts • Questions and discussion

Outline


THE COMPELLING STORY


• End users would like to bring their own images into your cloud

39

The compelling story

www.rackspace.com

• End users would like to bring their own images into your cloud

• But there are a few complications …

40


www.rackspace.com

• Some end users don’t quite understand what the OpenStack Image Service is

41


www.rackspace.com

• Some end users don’t quite understand what the OpenStack Image Service is – You can upload as many JPEGs as you

want, but you’ll never be able to boot a VM from them

42


www.rackspace.com

• Some end users do understand what the OpenStack Image Service is … but they are not nice people

43


www.rackspace.com

• Some end users do understand what the OpenStack Image Service is … but they are not nice people – Someone may upload malicious images

to share malware with other users

44


www.rackspace.com

• Some end users do understand what the OpenStack Image Service is … but they are not nice people – Someone may upload malicious images

to share malware with other users – Someone may upload a malicious image

to try and attack the hypervisor itself

45


www.rackspace.com

• Some end users have really slow connections

46


www.rackspace.com

• Some end users have really slow connections – Upload will take a long time

47


www.rackspace.com

• Some end users have really slow connections – Upload will take a long time – Lots of really slow, long-running uploads

can tie up the image service

48


www.rackspace.com

• Some end users have really slow connections – Upload will take a long time – Lots of really slow, long-running uploads

can tie up the image service – The Image Service is kind of important

for Nova

49


www.rackspace.com

• How to get information back to users?

50


www.rackspace.com

• How to get information back to users? – The Image status field is not very

descriptive

51


www.rackspace.com

• How to get information back to users? – The Image status field is not very

descriptive – The uploaded thing might not be a VM

image, anyway … do we really want to create an “image” that’s not an image?

52


www.rackspace.com

• We want to find a way for end users to: – Upload their custom VM images into an

OpenStack cloud

53


www.rackspace.com


OpenStack cloud – In such a way that the uploaded data can

be verified as a VM image

54


www.rackspace.com



be verified as a VM image – And scanned for malware or exploits

55


www.rackspace.com



be verified as a VM image – And scanned for malware or exploits – Using an interface that’s common across

OpenStack installations

56


www.rackspace.com




OpenStack installations – Yet is customizable as to the actual

workflow

57


www.rackspace.com




OpenStack installations – Yet is customizable as to the actual

workflow – And which provides useful feedback to

the end user

58


www.rackspace.com

• And … – End users may want to download images

to move them to another cloud for various reasons

59


www.rackspace.com



– The same problems present themselves

60


www.rackspace.com



– The same problems present themselves – A provider may want to pre-process an

image before it’s handed over to an end user

61


www.rackspace.com





– The end user may have a slow connection, etc.

62


www.rackspace.com






– We’re dealing with another long-running, asynchronous image-related activity

63


www.rackspace.com






– We’re dealing with another long-running, asynchronous image-related activity

– Would like a way to handle this direction, too

64


www.rackspace.com

• And … – End users may want to move images

from one isolated cloud region to another

65


www.rackspace.com


from one isolated cloud region to another – End users may want to move images

from one OpenStack cloud to another OpenStack cloud

66


www.rackspace.com


from one isolated cloud region to another – End users may want to move images

from one OpenStack cloud to another OpenStack cloud

– Yet another long-running, asynchronous image-related activity

67


www.rackspace.com

GLANCE TASKS


• Provide a common API across OpenStack installations – /v2/tasks

Glance tasks



• Workflow is customizable per cloud provider

Glance tasks




• Don’t create an image until there’s a high probability of success

Glance tasks





• Provide a way to deliver meaningful, helpful error messages

Glance tasks






• Concept of expiration built in

Glance tasks






• Concept of expiration built in • Frees the normal upload/download path for trusted users

Glance tasks


• “New Upload Workflow (“Import”) – https://blueprints.launchpad.net/glance/+spec/new-upload-

workflow

Glance tasks blueprints



workflow

• Import blueprint full specification – https://wiki.openstack.org/wiki/Glance-tasks-import




workflow


• Tasks API proposal – https://wiki.openstack.org/wiki/Glance-tasks-api




workflow


• Tasks API proposal – https://wiki.openstack.org/wiki/Glance-tasks-api

• Tasks product package – https://wiki.openstack.org/wiki/Glance-tasks-api-product



Glance task schema (abbreviated)


{ "name": "task", "properties": { "expires_at": { "description": "Datetime when this resource would be subject to removal”}, "input": { "description": "The parameters required by task, JSON blob”}, "message": { "description": "Human-readable informative message only included when appropriate (usually on failure)”}, "result": { "description": "The result of current task, JSON blob”}, "status": { "description": "The current status of this task", "enum": [ "pending”, "processing”, "success”, "failure” ]}, "type": { "description": "The type of task represented by this content", "enum": [ "import” ]}} }

Glance task list


{ "tasks": [ { "created_at": "2015-04-26T22:19:08Z", "expires_at": "2015-04-28T22:31:52Z", "id": "5f068894-a3a4-480a-97a9-c26bc1e1b5d7", "owner": "123456", "schema": "/v2/schemas/task", "self": "/v2/tasks/5f068894-a3a4-480a-97a9-c26bc1e1b5d7", "status": "success", "type": "import", "updated_at": "2015-04-26T22:31:52Z" } ] }

Glance task detail


{ "created_at": "2015-02-26T22:19:08Z”, "updated_at": "2015-02-26T22:31:52Z", "id": "5f068894-a3a4-480a-97a9-c26bc1e1b5d7”, "owner": ”123456", "schema": "/v2/schemas/task”, "self": "/v2/tasks/5f068894-a3a4-480a-97a9-c26bc1e1b5d7", "expires_at": "2015-02-28T22:31:52Z”, "input": { "image_properties": { "name": "my-imported-image" }, "import_from": "imports/whatever.vhd" }, "message": null, "result": { "image_id": "debb2825-981f-4a1b-ad52-4f5b3a8a70aa" }, "status": "success", "type": "import" }

Glance task detail


{ "created_at": "2015-04-21T13:34:33Z", "updated_at": "2015-04-21T13:34:45Z”, "id": "27b3a5fc-e714-41d7-ba59-f9ca31f74d94", "owner": "123456", "schema": "/v2/schemas/task", "self": "/v2/tasks/27b3a5fc-e714-41d7-ba59-f9ca31f74d94", "expires_at": "2015-04-23T13:34:45Z”, "input": { "image_properties": { "name": "Not a VM image" }, "import_from": "some-container/somefile.pdf" }, "message": "The image cannot be imported. The file is not a valid VHD.", "status": "failure", "type": "import" }

TASKS IN THE BIG PICTURE


• Glance only stores “real” images

Tasks address: MANAGEABILITY



• Keep Glance highly available for Nova




• Keep Glance highly available for Nova

• Security – Screen images for vulnerabilities



• Tasks enable you to process images asynchronously according to current resource availability

Tasks address: SCALABILITY


• Tasks help manage end users’ expectations

Tasks address: USABILITY



• Tasks can reduce user frustration




• Tasks can reduce user frustration • Tasks can provide better error feedback



THE FUTURE OF GLANCE TASKS


• Wednesday May 20, 2015 5:20pm – 6:00pm

• Room 222 https://etherpad.openstack.org/p/ liberty-glance-intercloud-image-sharing

92

Intercloud image sharing

www.rackspace.com

USING GLANCE TASKS


• Juno – Eventlet-based task executor

Using Glance Tasks (Operator Side)


• Juno – Eventlet-based task executor

• Kilo – Taskflow-based task executor

Using Glance Tasks (Operator Side)


Glance tasks: pros


• Flexible

Glance tasks: pros


• Flexible • Customizable

Glance tasks: pros


• Flexible • Customizable • Provide a common interface across OpenStack installations

Glance tasks: pros


Glance tasks: cons


• Flexible

Glance tasks: cons


• Flexible • Customizable

Glance tasks: cons


• Flexible • Customizable • Provide a common interface across OpenStack installations

Glance tasks: cons


• Alternative upload paths – Administrators use classic upload – Nova uses classic upload – End users use the import task

Tasks are flexible …


• Problem – How does a particular user discover what

upload path is available for her to use?

Tasks are flexible …


Tasks are customizable …


{ "type": "import", "input": { "image_properties": { "name": "My excellent custom image" }, "import_from": "exports/excellent-image.vhd" } }

Tasks are customizable …


"input": { "description": "The parameters required by task, JSON blob", "type": "object" }, "result": { "description": "The result of current task, JSON blob", "type": "object" }

Tasks provide a common interface …


"input": { "description": "The parameters required by task, JSON blob", "type": "object" }, "result": { "description": "The result of current task, JSON blob", "type": "object" }

REQUEST FOR FEEDBACK


• Tasks provide a common interface, but the interface hides differences among cloud providers

Questions



• What’s the best way to make these differences available to end users? – Documentation? – Programmatically?

Questions



• What’s the best way to make these differences available to end users? – Documentation? – Programmatically?

• How do users discover what upload facility is available to them?

Questions


THANK YOU

RACKSPACE® | 1 FANATICAL PLACE, CITY OF WINDCREST | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM

© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

WE’RE HIRING!

bit.ly/RackerTalent

Expo Hall Booth P-11 Python OpenStack Engineers C, C++ Linux Systems Engineers Ruby DevOps Engineers Java Frontend & Backend Developers

C#, .NET Software Developer in Test JavaScript, CSS, HTML iOS/Android Development

Twisted, Backhone Data Scientist Angular.JS, Ember.js, Node.js Field Sales Specialist

Restful/JSON/XML Strategic Account Executive Closure, Scala, Erlang

Hadoop, MongoDB, MySQL Solution Architect Data Visualization

THANK YOU

RACKSPACE® | 1 FANATICAL PLACE, CITY OF WINDCREST | SAN ANTONIO, TX 78218 US SALES: 1-800-961-2888 | US SUPPORT: 1-800-961-4454 | WWW.RACKSPACE.COM

© RACKSPACE LTD. | RACKSPACE® AND FANATICAL SUPPORT® ARE SERVICE MARKS OF RACKSPACE US, INC. REGISTERED IN THE UNITED STATES AND OTHER COUNTRIES. | WWW.RACKSPACE.COM

making glance tasks work for you - openstack summit may 2015 vancouver

Technology

glance tasks work

glance project technical

brian rosmaita

folsom glance driver

nikhil komawar

old bait

br14nr freenode

rackspace openstack