creating biomoby workflows in taverna mark wilkinson edward (eddie) kawas icapture centre, st....

85
Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

Upload: mavis-patrick

Post on 04-Jan-2016

217 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

Creating BioMoby WorkflowsIn Taverna

Mark WilkinsonEdward (Eddie) Kawas

iCAPTURE Centre, St. Paul’s HospitalVancouver, BC

Page 2: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

Downloading Taverna

Page 3: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Taverna can be obtained from:http://taverna.sourceforge.net

Page 4: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Once at the site, click on Download

Page 5: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Download the version appropriate for your operating system.

Page 6: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Between releases, the Moby functionality may be updated and you can find instructions on how to acquire those updates from:http://biomoby.open-bio.org/index.php/moby-clients/taverna_plugin

Page 7: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

Running

• For a more comprehensive guide on Running and using Taverna, please refer to http://taverna.sourceforge.net/usermanual/manual.html

Page 8: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Assuming that you have downloaded and unzipped Taverna, you can run it by double-clicking on runme.bat (windows) or executing runme.sh (Unix/Linux/OS X)

Page 9: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Taverna’s splash screen

Page 10: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Once Taverna has loaded, you will see 3 windows:– Advanced Model Explorer

– Workflow Diagram– Available Services

Page 11: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The Advanced model explorer is Taverna’s primary editor and allows you to load, save and edit any property of a workflow.

Page 12: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Workflow diagram contains a read only graphical representation of your workflow.

Page 13: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The Available services window lists all of the services available to a workflow designer.

Page 14: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Under the node ‘Biomoby @ …’ Moby services and Moby data types are represented.

• The Object ontology is available as children of MOBY Objects node

• Services are sorted by service provider authority

Page 15: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• If you wish to use registries other than the default one, you can add a new Moby ‘Scavenger’ by choosing to ‘Add new Biomoby scavenger…’

Page 16: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Enter the registry’s location and click okay.

Page 17: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

Creating Workflows

• I have the workflow saved and would like to offer it for download.

Page 18: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We will start by adding the Object ontology node Object to our workflow.

Page 19: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The Advanced model explorer now shows that we have a processor called Object– Object has 3 input ports: id,

namespace and article name– Object has 1 output port:

mobyData• The Workflow diagram illustrates

our processor

Page 20: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We can discover services that consume our data type, context click on ‘Object’ and choose ‘Moby Object Details’

Page 21: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• A window will pop up that tells you what services Object feeds into and is produced by

Page 22: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Expanding the Feeds into node results in a list of service provider authorities• Expanding an authority, for example, bioinfo.icapture.ubc.ca, reveals a list of

services

Page 23: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We will choose to add the service called ‘MOBYSHoundGetGenBankFasta’ to our workflow.

Page 24: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• A look at the state of our current workflow.

Page 25: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• And graphically.• The service consumes Object, with article name identifier,

and produces FASTA, with article name fasta.

Page 26: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• To discover more services, context click on the service that outputs the data type that you would like to discover consuming services for and choose Moby Service Details.

Page 27: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The resultant window displays the services inputs and outputs.• There are also tool tips that show up when your mouse hovers over any

particular input or output that tells you what namespaces the data type is valid in

Page 28: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Context clicking on an output reveals a menu with 3 options.

– A brief search for services that consume our datatype

– A semantic search for services that consume our datatype

– Adding a parser to the workflow that understands our datatype

Page 29: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The result of choosing to add a parser for FASTA to our workflow.• The parser allows us to extract:

– The namespace and id from FASTA– The namespace and id from the child String– The textual content from the child String

Page 30: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The result of choosing to conduct a brief search for services that consume FASTA

Page 31: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We will add the service getDragonBlastText to our workflow by choosing ‘Add service -…’ from the context menu

Page 32: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The current state of our workflow shown graphically.

Page 33: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• A more complex view of our workflow

Page 34: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Finding services that consume NCBI_BLAST_Text starts by viewing the details of the service ‘getDragonBlastText’

Page 35: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Conduct a brief search

Page 36: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Add the service ‘parseBlastText’ to our workflow

Page 37: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Our current workflow

Page 38: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Workflow inputs are added by context clicking on Workflow inputs in the Advanced model explorer and choosing ‘Create New Input…’

Page 39: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The result from adding 2 inputs:– Id– namespace

Page 40: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The workflow input id will be connected to Object’s input port ‘id’

Page 41: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Workflow after connecting the workflow input ‘id’

Page 42: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The workflow input namespace will connect to Object’s input port ‘namespace’

Page 43: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Workflow after connection the workflow inputs.

Page 44: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Workflow outputs are added by context clicking on Workflow outputs in the Advanced model explorer and choosing ‘Create New Output…’

Page 45: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The result from adding 2 workflow outputs:– moby_blast_ids– fasta_out

Page 46: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The output moby_blast_ids will be connected to parseBlastText’s output port Object(Collection –’hit_ids’)

Page 47: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The output fasta_out will be connected to Parse_Moby_Data_FASTA’s output port fasta_’content’

Page 48: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC
Page 49: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• To run the workflow, click on ‘Tools and Workflow Invocation’

• Choose ‘Run workflow’

Page 50: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• A prompt to add values to our 2 workflow inputs

Page 51: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• To add a value to the input ‘id’ click on id from the left pane and choose ‘New Input’

Page 52: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Enter 656461 as the id

Page 53: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Choose namespace from the left and click on ‘New Input’

Page 54: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Enter NCBI_gi as the value for namespace• Once you are done, click on ‘Run Workflow’

Page 55: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Our workflow in action

Page 56: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Once the workflow is complete, we can examine the results of our workflow.

Page 57: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• A detailed report is available outlining what happened when and in what order.

Page 58: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We can examine the intermediate inputs and output, as well as visualize our workflow.

Page 59: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• If we choose the Graph tab, our workflow is illustrated.

Page 60: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Intermediate inputs allow us to examine what a service has accepted as input

Page 61: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Similarly, Intermediate outputs allows us to examine the output from any particular service.

Page 62: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Without the parser, FASTA is represented as a Moby message, fully enclosed in its wrapper.

• Non-moby services do not expect this kind of message

Page 63: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Non-moby services expect the just the sequence and using the Parse_Moby_Data processor, we can extract just that

Page 64: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Moby services can interact with the other services in Taverna.

• Let’s add a Soaplab service.

Page 65: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We will choose a nucleic_restriction soaplab service called ‘restrict’

Page 66: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Choose the restrict service and add it to the workflow.

Page 67: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We will connect the output port fasta_’content’ from the service Parse_Moby_Data_FASTA to the input port ‘sequence_direct_data’ from the service restrict

Page 68: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The result of our actions so far.• We will need to add another workflow

output to capture the output of restrict.

Page 69: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Create an output called restrict_out

Page 70: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Connect the output port ‘outfile’ from the service restrict to the workflow output restrict_out

Page 71: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Once the connections have been made, run the workflow again using the same inputs.

Page 72: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The workflow on the left has some extra services added to it.

– FASTA2HighestGenericSequenceObject from the authority bioinfo.icapture.ubc.ca

– runRepeatMasker from the authority genome.imim.es

– A Moby parser for the output DNASequence from runRepeatMasker.

– A workflow output Masked_Sequence

• Add them to your workflow

Page 73: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The service runRepeatMasker is configurable, i.e. it consumes Secondary parameters.

• To edit these parameters, context click on the service and choose ‘Configure Moby Service’

Page 74: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The name of the parameter is on the left and the value is on the right.

• Clicking on the Value will bring up a drop down menu, an input text field, or any other appropriate field depending on the parameter.

Page 75: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• The parameter species contains an enumerated list of possibilities.

• Select human.

• When you have made your selection, you may close the window.

Page 76: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Let’s run the workflow

Page 77: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• We will run our workflow with a list– Click on id in the left pane and then click on New

Input twice

Page 78: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Enter 656461 and 654321 as the ids• Enter NCBI_gi as the value for namespace• Our workflow will now run using each id with the single namespace

Page 79: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Notice how the workflow is running with iterations. This is happening because the Enacter is performing a cross-product on the input

Page 80: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• You can still view intermediate inputs and outputs.• Using the queryIDs, you can track each invocation of a

moby service through the whole workflow

Page 81: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Imagine now that you want to run the workflow using a FASTA sequence that you input yourself (without the gi identifier)

• To do this, context click on getDragonBlastText and choose Moby Service Details– Expand the Inputs node and context click on FASTA(‘sequence’)– Choose Add Datatype – FASTA(‘sequence’) to the workflow

• A FASTA datatype will be added to the workflow and the appropriate links created

Page 82: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Notice the datatype FASTA on the left of the workflow– Since the datatype FASTA

hasa String, a String was also added to our workflow and the appropriate connection was made

• We will now have to add another workflow input and connect it to the String component of FASTA.

Page 83: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• A workflow input ‘sequence’ was added to the workflow and a connection was made from the workflow input to the input port ‘value’ of String.

• We also removed the link between MOBYSHoundGetGenBankFasta and getDragonBlastText by context clicking on the link in the Advanced model explorers’ Data links and choosing to remove the link

• Now when we choose to run our workflow, we will also have the chance to enter a FASTA sequence

Page 84: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Go ahead an enter any FASTA sequence as the input to the workflow input ‘sequence’

• Run the workflow

Page 85: Creating BioMoby Workflows In Taverna Mark Wilkinson Edward (Eddie) Kawas iCAPTURE Centre, St. Paul’s Hospital Vancouver, BC

• Any results can be saved by simply choosing to Save to disk– You will be prompted to enter a directory to save the results.– Each workflow output will be saved in a folder with the same name as a workflow

output and the contents of the folder will be the results• You can also choose Excel, which produces an Excel worksheet with

columns representing the workflow outputs and with rows that represent the actual data.