meta’omic analysis with metaphlan, humann, and lefse curtis huttenhower 08-08-13 harvard school of...
Post on 17-Jan-2016
218 Views
Preview:
TRANSCRIPT
Meta’omic Analysis withMetaPhlAn, HUMAnN, and LEfSe
Curtis Huttenhower
08-08-13Harvard School of Public HealthDepartment of Biostatistics
2
Some setup notes
• Slides with green titles or text include instructions not needed today, but useful for your own analyses
• Keep an eye out for red warnings of particular importance
• Command lines and program/file names appear in a monospaced font.
4
Getting some HMP data
• Check out what’s available
Click “HMIWGS”
5
Getting some HMP data
• Check out what’s available
Click on your favorite body site
6
Getting some HMP data
• Check out what’s availableDon’t click on anything!
7
Getting some (prepped) HMP data
• Connect to the server instead and run:– for S in `ls /class/stamps-shared/chuttenh/7*.fasta`;
do ln -s $S; done
• These are subsamples of six HMP files:– SRS014459.tar.bz2 763577454-SRS014459-Stool.fasta– SRS014464.tar.bz2 763577454-SRS014464-Anterior_nares.fasta– SRS014470.tar.bz2 763577454-SRS014470-Tongue_dorsum.fasta– SRS014472.tar.bz2 763577454-SRS014472-Buccal_mucosa.fasta– SRS014476.tar.bz2 763577454-SRS014476-Supragingival_plaque.fasta– SRS014494.tar.bz2 763577454-SRS014494-Posterior_fornix.fasta
• All six shotgunned body sites from– One subject, first visit– Subsampled to 20,000 reads
8
Who’s there: MetaPhlAn
Gene X
X is a core gene for clade Y X is a unique marker gene for clade Y
9
Who’s there: MetaPhlAn
• Go to http://huttenhower.sph.harvard.edu/metaphlan
Scroll down
10
Who’s there: MetaPhlAn
• You could download MetaPhlAn by clicking here
11
Who’s there: MetaPhlAn
• But don’t! Instead, we’ve downloaded MetaPhlAn already for you by clicking here
12
Who’s there: MetaPhlAn
• And saving just the file metaphlan.py in stamps-software/bin
13
Who’s there: MetaPhlAn
• You could download the bowtie2 database here
14
From the command line...
• But don’t! Instead, get it by:– for S in `ls /class/stamps-shared/chuttenh/*.bt2`; do ln -s $S; done
• To see what you can do, run:– module load stamps– metaphlan.py -h | less– Use the arrow keys to move up and down,
q to quit back to the prompt
15
Who’s there: MetaPhlAn
16
Who’s there: MetaPhlAn
• For future reference: these extra options aren’t necessary if you download the whole “default” MetaPhlAn package– metaphlan.py your_input.fasta > your_output.txt
• To launch your first analysis, run:– metaphlan.py --bowtie2db mpa 763577454-SRS014459-Stool.fasta > 763577454-SRS014459-Stool.txt
17
Who’s there: MetaPhlAn
• What did you just do?• Two new output files:
– 763577454-SRS014459-Stool.fasta.bowtie2out.txt
• Contains a mapping of reads to MetaPhlAn markers– 763577454-SRS014459-Stool.txt
• Contains taxonomic abundances as percentages
18
Who’s there: MetaPhlAn
• less 763577454-SRS014459-Stool.fasta.bowtie2out.txt
19
Who’s there: MetaPhlAn
• less 763577454-SRS014459-Stool.txt
20
Who’s there: MetaPhlAn
• Now finish the job:– metaphlan.py --bowtie2db mpa763577454-SRS014464-Anterior_nares.fasta > 763577454-SRS014464-Anterior_nares.txt
– ...
• Note that you can use the up arrow key to make your life easier!
21
Who’s there: MetaPhlAn
• Let’s make a single table containing all six samples:– mkdir tmp– mv *.bowtie2out.txt tmp– merge_tables.py -l -d *.txt | zero.py > 763577454.tsv
• You can look at this file using less– Note 1: The arguments less -x4 -S will help– Note 2: You can set this “permanently” using export LESS="-x4 -S"
22
Who’s there: MetaPhlAn
• But it’s easier using MeV; go to http://www.tm4.org/mev.html
Click “Download”
23
An interlude: MeV
• Don’t forget to transfer your 763577454.tsv file locally for viewing using scp
• Unzip, launch MeV, and select File/Load data
24
An interlude: MeV
• Click “Browse” to your TSV file, then– Tell MeV it’s a two-color array– Uncheck “Load annotation”– Click on the upper-leftmost data value
25
An interlude: MeV
• “Load” your data, then make is visible by:– Display/Set Color Scale Limits– Choose Single Gradient, min 0, max 10
26
An interlude: MeV
• Finally, to play around a bit:– Display/Set Element Size/whatever you’d like– Clustering/Hierarchical Clustering– Optimize both gene and sample order– And select Manhattan Distance (imperfect!)
27
An interlude: MeV
• If you’d like, you can– Display/Sample-Column Labels/Abbr. Names
28
An interlude: MeV
• MeV is a tool; imperfect, but convenient– You should likely include just “leaf” nodes
• Species, whose names start include “s__”• You can filter your file using:cat 763577454.tsv | grep -E '(Stool)|(s__)' > 763577454_species.tsv
– You can, but might not want to, z-score normalize• Adjust Data/Gene-Row Adjustments/Normalize Genes-Rows
• Many other tools built in – experiment!
29
What they’re doing: HUMAnN
• Back to the task at hand; you could download HUMAnN at: http://huttenhower.sph.harvard.edu/humann
Click here
30
What they’re doing: HUMAnN
• ...but instead we’ve already downloaded it• Expand HUMAnN (no install!)
– tar -xzf /class/stamps-shared/chuttenh/sources/humann-0.98.tar.gz
• Set up a link to the KEGG reference DB:– ln -s /class/stamps-shared/chuttenh/kegg.reduced.udb
• And although you would normally download USEARCH from here:– http://www.drive5.com/usearch/download.html
We’re going to use it preinstalled instead
31
What they’re doing: HUMAnN
• If we weren’t all running this, you’d need to:– Get KEGG – used to be free, now it’s not!
• Fortunately, we have a HUMAnN-compatible distributable version; contact me...
– Index it for USEARCH:• usearch6 -makeudb_usearch kegg.reduced.fasta-output kegg.reduced.udb
• This takes a minute or two, so we’ve precomputed it; thus, forge ahead...
32
What they’re doing: HUMAnN
• Did you notice that we didn’t QC our data at all?– MetaPhlAn is very robust to junk sequence– HUMAnN is pretty robust, but not quite as much
• We’ve already run a standard metagenomic QC:– Quality trim by removing bad bases (typically Q ~15)– Length filter to remove short sequences (typically <75%)
33
What they’re doing: HUMAnN
• Must start from FASTQ files to do this• Quality trim by removing bad bases:
– TrimBWAstyle.py < your_data.fastq > your_trimmed_data.fastq
• Length filter by removing short sequences:– 75% of original length is standard (thus 75nt from 100nt reads)– remove_bad_seqs.py 75 < your_trimmed_data.fastq >
your_filtered_data.fastq
• Now convert your FASTQ to a FASTA:– fastq2fasta.py < your_filtered_data.fastq >
your_filtered_data.fasta
• Some final caveats:– If you’re using paired end reads, match filters!– See my course homeworks at http://huttenhower.sph.harvard.edu/bio508– Aren’t you glad you’re not doing this today?
34
What they’re doing: HUMAnN
• Enter the humann directory– cd humann-0.98
• Run your first translated BLAST search:– usearch6.0.192_i86linux32 -usearch_local ../763577454-SRS014459-Stool.fasta -db ../kegg.reduced.udb -id 0.8 -blast6out input/763577454-SRS014459-Stool.txt
• What did you just do?– less input/763577454-SRS014459-Stool.txt– Recall BLAST’s tab-delimited output headers:
• qseqid sseqid pident length mismatch gapopen qstart qend sstart send evalue bitscore
• Rinse and repeat for the remaining samples
35
What they’re doing: HUMAnN
• Normally you’d need to install SCons from:– http://www.scons.org
• Instead, we’ll use it preinstalled as well, so...
GO!– /class/stamps-software/scons/bin/scons
• You should see a bunch of text scroll by– Note: you can run scons -j8 to parallelize tasks
36
What they’re doing: HUMAnN
• After a minute or two, you should see:
37
What they’re doing: HUMAnN
• This has created four main files:– Two each for pathways (big) and modules (small)– Two each for coverage and relative abundance
• Each is tab-delimited text with one column per sample
• All four are in the output directory:– output/04a-hit-keg-mpt-cop-nul-nve-nve-xpe.txt
• Coverage (a) of pathways (t)
– output/04a-hit-keg-mpm-cop-nul-nve-nve-xpe.txt• Coverage (a) of modules (m)
– output/04b-hit-keg-mpt-cop-nul-nve-nve.txt• Abundance (b) of pathways (t)
– output/04b-hit-keg-mpm-cop-nul-nve-nve.txt• Abundance (b) of modules (m)
• I almost always just use 04b-mpm (module abundances)
38
What they’re doing: HUMAnN
• Let’s take a look:– less output/04b-hit-keg-mpm-cop-nul-nve-nve.txt
39
What they’re doing: HUMAnN
• That’s ugly; it gets much better in Excel– Note: this is very sparse since we’re using a small subset of KEGG– Note: the mock community demo data is included on the right
40
What they’re doing: HUMAnN
• And there’s nothing stopping us from using MeV– Or R, or QIIME, or anything that’ll read tab-delimited text
41
What matters: LEfSe
• All of these analyses give you tables– 16S OTUs– MetaPhlAn Species– HUMAnN Modules (or pathways or genes)
• If you know something about your samples...– Case/control– Different habitats– Different time points
• How can you identify features that change?
42
What matters: LEfSe
• Let’s get all of the HMP species data:http://hmpdacc.org/resources/data_browser.php
Click “HMSMCP”
43
What matters: LEfSe
• Download the MetaPhlAn table for all 700 samples
Right click this irritatingly tiny icon
44
Downloading from the command line
• Instead of saving this, download it by:– Right-click to copy the URL– Run wget <paste URL here>– Note: curl –O <URL> works just as well
45
What matters: LEfSe
• Make sure this file is in your home directory, and expand it:– bunzip2 HMP.ab.txt.bz2
• Look at the result– less HMP.ab.txt
• IMPORTANT!!!– This file’s too big to analyze directly today– ln -s /class/stamps-shared/chuttenh/HMP.ab.filtered.txt
• This is great – tons of data, but no metadata– HUMAnN to the rescue– From the humann-0.98 directory:
python src/metadata.py input/hmp_metadata.dat < ../HMP.ab.filtered.txt > ../HMP.ab.filtered.metadata.tsv
• NOW take a look again
46
What matters: LEfSe
• Let’s modify this file to be LEfSe-compatible• Open it up in Excel
47
What matters: LEfSe
• Delete all of the metadata rows except:– RANDSID and STSite– Save it as tab-delimited text: HMP.ab.filtered.metadata.txt
48
What matters: LEfSe
• Visit LEfSe at: http://huttenhower.sph.harvard.edu/lefse
Click here
49
What matters: LEfSe
• Then upload your formatted table– After you upload, wait for the progress meter to turn green!
1. Click here
2. Then here
3. Then here
4. Then watch here
50
What matters: LEfSe
• Then tell LEfSe about your metadata:
1. Click here
2. Then select STSite
3. Then select RANDSID
4. Then here
51
What matters: LEfSe
• Then select LDA=4, “One-against-all,” and run LEfSe!– You can change other default statistical parameters if desired
1. Click here
4. Then GO!
3. Then here(finds differences in at
least one condition rather than in all conditions)
2. Then here(finds only very
extreme differences)
52
What matters: LEfSe
• You can plot the results as a bar plot– Again, lots of graphical parameters to modify if desired
1. Click here
2. Then here
53
What matters: LEfSe
• In Galaxy, view a result by clicking on its “eye” Click here
54
What matters: LEfSe
55
What matters: LEfSe
• You can plot the results as a cladogram– Lots and lots of graphical parameters to modify if desired
1. Click here
2. Then here
56
What matters: LEfSe
57
What matters: LEfSe
• Finally, you can see the raw data for individual biomarkers– These are generated as a zip file of individual plots
1. Click here
2. Then here
58
What matters: LEfSe
• In Galaxy, download a result by clicking on its “disk”
Click here
Then here
59
What matters: LEfSe
Strep. mitis
Veillonellaceae
Actinobacteria
60
Summary
• MetaPhlAn– Raw metagenomic reads in– Tab-delimited species relative abundances out
• HUMAnN– Quality-controlled metagenomic reads in– Tab-delimited gene, module, and pathway
relative abundances out• LEfSe
– Tab-delimited, stratified relative abundances in– Significantly differentially abundant features out
Ramnik XavierHarry SokolDan Knights
Moran Yassour
Thanks!
61
Nicola Segata Levi Waldron
Human Microbiome ProjectOwen WhiteJoe PetrosinoGeorge WeinstockKaren NelsonLita ProctorDirk Gevers
Kat Huang
Bruce Birren Mark DalyDoyle Ward Ashlee Earl
http://huttenhower.sph.harvard.edu
Joseph Moon
Felix Wong
Tim TickleXochi Morgan
DanielaBoernigen
Rob KnightJesse ZaneveldGreg Caporaso
Mark SilverbergBoyko Kabakchiev
Andrea Tyler
EmmaSchwager
Jim Kaminski
Brian Palmer
Eric Franzosa
Boyu Ren
Ren Lu Koji Yasuda
Sahar AbubuckerBrandi Cantarel
Alyx SchubertMathangi Thiagarajan
Beltran Rodriguez-MuellerErica SodergrenAnthony FodorMarty Blaser
Jacques RavelPat Schloss
Makedonka MitrevaYuzhen YeMihai Pop
Larry ForneyBarbara Methe
Jacques Izard
Katherine Lemon
Wendy GarrettMichelle Rooks
Bruce SandsRuth Ley
Omry Koren
Rob BeikoMorgan Langille
Jeroen RaesKaroline Faust
Interested? We’re recruiting just about everything!
top related