genomedata, segway and segtools: how to use the segway … · 2016-06-07 · # pytables pip install...

21

Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets Max Libbrecht

Upload: others

Post on 12-Aug-2020

16 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Page 1: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze

genomics data sets

Max Libbrecht

Page 2: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Genomedata Segway Segtools

Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets

Page 3: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Genomedata Segway Segtools

Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets

Page 4: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Page 5: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Installing Genomedata# HDF5# Ubuntu/Debian:sudo apt-get install libhdf5-serial-dev hdf5-tools# CentOS/RHEL/Fedora:sudo yum -y install hdf5 hdf5-devel# OpenSUSE:sudo zypper in hdf5 hdf5-devel libhdf5

# Pytablespip install numpypip install numexprpip install cython

# Genomedatapip install genomedata

Page 6: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Loading data into genomedata

genomedata-load-assembly --sizes my_genomedata hg19.sizesgenomedata-open-data my_genomedata my_tracknamezcat input.bedgraph.gz | genomedata-load-data my_genomedata my_tracknamegenomedata-close-data my_genomedata

hg19.sizes:chr1 249250621chr2 243199373chr3 198022430chr4 191154276

Page 7: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Accessing data: command line

$ genomedata-query my_genomedata my_trackname chr1 1000000 1000100fixedStep chrom=chr1 start=10000000.00.00.00.00.00.0...

Page 8: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Accessing data: Python

>>> import genomedata>>> g = genomedata.Genome(“my_genomedata”)>>> g[“chr1”][1000000:1000100, “my_trackname”]array([ 17.89999962, 17.89999962, 17.89999962, 17.89999962, 17.89999962, 17.89999962, 17.89999962, 17.89999962, 17.89999962, 17.89999962], dtype=float32)

Page 9: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Genomedata Segway Segtools

Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets

Page 10: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

DNase1

H3k36me3

RNA-seq

Annotation1 2 3 2 3 2 000

HMMSeg: Day et al. Bioinformatics, 2007ChromHMM: Ernst, J. and Kellis, M. Nature Biotechnology, 2010

Segway: Hoffman, M et al. Nature Methods, 2012

Semi-automated genome annotation algorithms partition and label the genome on the basis of

functional genomics tracks

Page 11: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Segment label

RNA-seq

H3K27me3

DNase1

hidden random variable

observed random variable11

Semi-automated genome annotation algorithms use dynamic Bayesian network models

Page 12: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Installing Segway# GMTKwget http://melodi.ee.washington.edu/downloads/gmtk/gmtk-1.4.0.tar.gztar -xzvf gmtk-1.4.0.tar.gz./configuremakemake installcd ..

# Segwaypip install segway

http://melodi.ee.washington.edu/downloads/gmtk/gmtk-1.4.0.tar.gz

Page 13: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Running Segway

segway train my_genomedata my_traindirsegway identify my_genomedata my_traindir my_identifydir

output: my_identifydir/segway.bed.gz

Page 14: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Model parameters

Number of annotation labels--num-labels=25

Number of EM intializations--num-instances=10

Maximum number of EM training iterations--max-train-rounds=100

Page 15: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Input dataInput tracks--track=GM12878_H3K27ac --track=GM12878_H3K4me3 OR --tracks-from=tracks.txt

tracks.txt: GM12878_H3K27ac GM12878_H3K4me3 Genome coordinates--include-coords=coords.bed

coords.bed: chr1 151158060 151658060 chr10 55483812 55983812

--exclude-coords=blacklist.bed

Training minibatch size--minibatch-fraction=0.01

Page 16: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Controlling segment lengths

Downsampling resolution--resolution=10

Long segments prior--prior-strength=1.0

Weight on transition part of the model--segtransition-weight-scale=10

Page 17: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Genomedata Segway Segtools

Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets

Page 18: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

Installing Segtools

pip install segtools

Page 19: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

segtools-signal-distribution measures relationships between annotation labels and signal tracks

H4K20me1H3K79me2H3K36me3

H3K4me3H3K27ac

H3K4me1H3K9me1

H3K27me3

Page 20: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

segtools-length-distribution measures segment lengths genome coverage

segtools-length-distribution segway.bed.gz

Page 21: Genomedata, Segway and Segtools: How to use the Segway … · 2016-06-07 · # Pytables pip install numpy pip install numexpr pip install cython # Genomedata pip install genomedata

segtools-aggregation measures associations with other genome annotations

segtools-aggregation --normalize --mode=gene segway.bed.gz gencode.gff

Segway Documentation · 2017-07-14 · Segway Documentation, Release 2.0.1 2.3Data selection Segway accepts data only in the Genomedata format. The Genomedata package includes utilities

InstaLooter Documentation · $ pip install --user".[test]" # install only test dependencies $ pip install --user". ... * use these only to quicken download, since fetching the others

UpandRunningwith pytest Hypothesisand tox...Setup Debian/Ubuntu apt install python3-{pytest,hypothesis,tox} Arch/MSYS pacman -S python-{pytest,hypothesis,tox} pip pip install pytest

factor analyzer Documentation - Read the Docs · $ pip install factor_analyzer Alternatively, you can install via condawith: $ conda install -c ets factor_analyzer 3.1factor_analyzer

Stupid Python Tricks - NHMUG · Python has a package manager: pip (pip3) Use pip to install packages from the internet – Automatically determines dependencies needed – Downloads

Release 5.7.0 Twilio Inc. · PDF fileCHAPTER 1 Installation Install from PyPi usingpip, a package manager for Python. pip install twilio Don’t have pip installed? Try installing

PIP 444 QS - Extron · PIP 422 and PIP 444 Picture-in-Picture Processors • Quick Start QS-1 Quick Start — PIP 422 and PIP 444 Installation Step 1 For tabletop placement, install

PYPARIS2018 - Django & Nodepyparis.org/static/slides/Romain Dorgueil-a3f47138.pdf · Dev Tools Install dependencies … $ pip install django-debug-toolbar $ pip install django-extensions

PyREbox Documentation - media.readthedocs.org · To install the python dependencies you can use pip: pip install-r requirements.txt Do not forget to activate your virtual env every

Dynamic instrumentation - GitHub Pages · Let [s install Frida The easiest way to install is installing the python bindings via PyPi. pip install Frida Actually on a fresh Ubuntu

Apache Airflow - Data Vault User Group · Airflow - Inbetriebnahme •Install from pypi using pip pip install apache-airflow •Initialize the database airflow initdb •Start the

Segway Documentation - media.readthedocs.org · Segway accepts data only in the Genomedata format. The Genomedata package includes utilities to convert from BED, wiggle, and bedGraph

Pysiology Documentation - buildmedia.readthedocs.org2.1Download and Installation 2.1.1Using pip (suggested method) Open a terminal and type: pip install pysiology 2.1.2Using setup.py

Genomedata Documentation...Genomedata Documentation, Release 1.4.4 pip install numpy 1.1.3Installing Genomedata With Python 2.6 or 2.7 installed: pip install genomedata Note: The latest

PyTorchTutorial - Princeton University · • 💻💻On your own computer • Anaconda/Miniconda: conda install pytorch -c pytorch • Others via pip: pip3 install torch • 🌐🌐On

PyTestpycon.pk/.../pytest-right_way_to_test_python_code.pdfassert == ‘python’ -> pytest self.assertEquals(‘python’) -> unittest Pytest - Getting Started $ - pip install pytest

Python 101 - California Institute of Technologysrk/Ay3/python_intro.pdf · adding more (third party) packages pip install --upgrade astropy pip install --upgrade aplpy …or download

Classi cation scikit-learn - Allegheny College · 2020. 2. 19. · scikit-learn can be easily installed with pip or conda pip install scikit-learn conda install scikit-learn Janyl

Genomedata Documentation · Genomedata Documentation, Release 1.4.1 pip install numpy 1.1.3Installing Genomedata With Python 2.6 or 2.7 installed: pip install genomedata Note: The

Ansible Overview - Quality Thought€¦ · (L) QUALITY THOUGHT (INDIA) PIP Install (All others) Install Libraries (gcc, python-devel) Install Python SetupTools Install Ansible Install

python-docx-template Documentation Documentation, Release 0.1.5 Quickstart To install: pip install docxtpl Usage: fromdocxtplimport DocxTemplate doc=DocxTemplate("my_word_template.docx")

Queueing & Python · Queueing & Python pip install ciw Geraint Ian Palmer @GeraintPalmer@CiwPython PyCon UK 2016 Ciw

Universidad - pdi.uady.mx 2010... · universitaria, pues en las Universidades ... PIP 2 PIP 3 PIP 11 PIP 14 PIP 13 PIP 4 PIP 5 PIP 6 PIP 7 PIP 9 PIP 8 PIP 10 PIP 12 PIP 1 UADY Programas

Week 02 Python Crash Course 1 · 2019-11-17 · hci + d lab. • Install Python on Windows • Install Python on MacOS • Install Python Packages using PIP • Jupyter Notebook •

django-registration-redux Documentation · Using pip, type: pip install django-registration-redux It is also possible that your operating system distributor provides a packaged version

Quick Reference Guide to DJANGO TERMINAL COMMANDS · INSTALL DJANGO INSIDE VIRTUALENV: pip install django (pip install django==2.04 to install a specific version of django) START

Paolo Melchiorre GeoDjango @pauloxnet with PostGIS...Setup 24 $ pip install djangorestframework # RESTful API $ pip install djangorestframework-gis # Geographic add-on $ pip install

GenomeData - Osaka University...GenomeData解析入門1 ①ヒトゲノムデータの取り扱い ②1000 Genomes Projectデータ ③遺伝統計解析ソフトPLINK実習講義の概要

TENSOR FLOW TUTORIAL - GitHub Pages · INSTALLING Ò Install 64-bit Python 3.5 & pip (or Anaconda3-4.2.0-Windows-x86_6 4) Ò Install virtualenv: É CMD: pip install virtualenv É

virtualization project documentation.docxteamcarramrod.weebly.com/uploads/3/1/6/4/31648567/... · Web viewwith pip: sudo pip install jinja2 pymongo bottle pefile maec==4.0.1.0 django

Developing with Python - Embedded Artists · Pip is a package management system used to install software packages written in Python. Pip can be used to install Python extensions from

Genomedata, Segway and Segtools: How to use the Segway ...€¦ · Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets. Installing

Django, Flask 고민없이 개 발하고 서비스하는 PaaS, IBM … Static file: dj_static.Cling $pip install dj_static $pip freeze > requirements.txt • wsgi.py $cfpush pyconkr-jgkong

aiohttp DocumentationFor smart detection of Content-Type by client API you would like to install chardet also: pip install chardet Optional: To improve performances, you can install

Genomedata, Segway and Segtools: How to use the Segway ... · Genomedata, Segway and Segtools: How to use the Segway pipeline to store and analyze genomics data sets. Installing Segtools