hosting public domain chemicals data online for the community – the challenges of handling...

Hosting public domain chemicals data online for the community – the

challenges of handling materials

Antony WilliamsOpportunities in Materials Informatics, University of Wisconsin-Madison

February 9th, 2015

0000-0002-2668-4821

About Me…• I am NOT a materials chemist• I am an NMR spectroscopist by training• Worked on a LIMS while at Kodak• 10 years in commercial cheminformatics• Built the ChemSpider database as a hobby• Worked on validating compounds on Wikipedia• Manage cheminformatics team for RSC• Believer in the value of social networking and

Open Data for science• Dane Morgan asked me to tell jokes…

I would tell a chemistry joke…

But all of the good ones…

An ambitious idea….

• Let’s map together all online chemistry data and build systems to integrate it

• Heck, let’s integrate chemistry and biology data and add in disease data too if we can

• Let’s extract property data and model it and see if we can extract new relationships – quantitative and qualitative

• Let’s make it all available on the web…for free

What about this….

• We’re going to map the world

• We’re going to take photos of as many places as we can and link them together

• We’ll let people annotate and curate the map• Then let’s make it available free on the web• We’ll make it available for decision making • Put it on Mobile Devices, give it away…

Where is chemistry online?

• Encyclopedic articles (Wikipedia)• Chemical vendor databases• Metabolic pathway databases• Property databases• Patents with chemical structures• Drug Discovery data• Scientific publications

• Compound aggregators• Blogs/Wikis and Open Notebook Science

Chemistry on the Internet…

• Most searching for chemistry on the internet…• Name searching Google/Bing/Yahoo• Name searching Wikipedia• Name searching Wolfram Alpha• Name, name, name, name…searching• Structure searching DOZENS of websites, each

with different information or…

Chemistry on the Internet…

• Most searching for chemistry on the internet…• Name searching Google/Bing/Yahoo• Name searching Wikipedia• Name searching Wolfram Alpha• Name, name, name, name…searching• Structure searching DOZENS of websites, each

with different information or…

• Search ONE website integrating the others!

• ~30 million chemicals and growing

• Data sourced from >500 different sources• Crowd sourced curation and annotation• Ongoing deposition of data from our

journals and our collaborators• Structure centric hub for web-searching

• …and a really big dictionary!!!

• Note…NOT all websites connected

ChemSpider

ChemSpider

ChemSpider

Experimental/Predicted Properties

Literature references

Patents references

RSC Books

Google Books

Vendors and data sources

APIs

APIs

Organic Chemistry is hard…

…it has alkynes of trouble

Flavors of Chemistry

Molfiles 10 9 0 0 1 0 0 0 0 0 1 V2000 31.2937 -9.0366 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 0 26.6526 -9.0366 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 0 31.2937 -7.7066 0.0000 O 0 0 0 0 0 0 0 0 0 0 0 0 30.1161 -9.6877 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 0 25.5096 -9.6877 0.0000 O 0 0 0 0 0 0 0 0 0 0 0 0 28.9731 -9.0366 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 0 27.8163 -9.7016 0.0000 C 0 0 0 0 0 0 0 0 0 0 0 0 26.6664 -7.7066 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 32.4367 -9.6877 0.0000 O 0 0 0 0 0 0 0 0 0 0 0 0 30.1161 -11.0177 0.0000 N 0 0 0 0 0 0 0 0 0 0 0 0 3 1 2 0 0 0 0 4 1 1 0 0 0 0 9 1 1 0 0 0 0 7 2 1 0 0 0 0 5 2 2 0 0 0 0 8 2 1 0 0 0 0 6 4 1 0 0 0 0 4 10 1 6 0 0 0 7 6 1 0 0 0 0M END

Molfiles

• Molfiles are the primary exchange format between structure drawing packages

• Can be different between different drawing packages

• Most commonly carry X,Y coordinates for layout• Can support polymers, organometallics, etc.• Can carry 3D coordinates

SMILES

• SMILES is a common format • Can support polymers,

organometallics, etc.• Does NOT carry X,Y or Z

coordinates for layout so requires layout algorithms – can be problematic!

• Generally different between drawing packages

Stereo

Tautomeric forms

Vendor-dependent SMILESACD/LabsCC(C)CCC[C@@H](C)CCC[C@@H](C)CCCC(\C)=C\CC2=C(C)C(=O)c1ccccc1C2=O

OpenEyeCC1=C(C(=O)c2ccccc2C1=O)C/C=C(\C)/CCC[C@H](C)CCC[C@H](C)CCCC(C)C

ChEMBLCC(C)CCC[C@@H](C)CCC[C@@H](C)CCC\C(=C\CC1=C(C)C(=O)c2ccccc2C1=O)\C

Chemists are good…

The InChI Identifier

InChI

• SINGLE code base managed by IUPAC – integrated into drawing packages. No variability as with SMILES

• InChI Strings can be reversed to structures – same problem as with SMILES – no layout

• Adopted by the community (databases, blogs, Wikipedia) – good for searching the internet

Multiple Layers

Tautomers

Stereo

InChIStrings Hash to InChIKeys

Structure search the web

Exact Search

Skeleton Search

Data Quality/Standardization

• MANY structures meant to be something online are MISREPRESENTED.

• Commonly you will have better success finding information by name searches than structure – with many caveats of course…

• Validating chemical structure representations is laborious work – and it’s shocking to review data…

Data Quality IssuesWilliams and Ekins, DDT, 16: 747-750 (2011)

Science Translational Medicine 2011

Data quality is a known issue

Data quality is a known issue

Substructure # of

Hits

# of

Correct

Hits

No

stereochemistry

Incomplete

Stereochemistry

Complete but

incorrect

stereochemistry

Gonane 34 5 8 21 0

Gon-4-ene 55 12 3 33 7

Gon-1,4-diene 60 17 10 23 10

Only 34 out of 149 structures were correct!

Patent data in public databases

Patent data in public databases

You just can’t trust atoms!

You just can’t trust atoms!They make up everything…

ALL variants of Yohimbine!!!

What’s Methane? OLD PUBCHEM

What ELSE is Methane???

NEW PUBCHEM

Depiction vs Accurate Representation

Depiction vs Accurate Representation

What is the Structure of Vitamin K1?

Standardize

• Use the SRS as a guidance document for standardization

• Adjust as necessary to our needs

Nitro groups

Salt and Ionic Bonds

Ammonium salts

Can we MAKE Quality Data?

• We are building systems for everyone to validate and standardize their data

DICTIONARIES are powerful

• Search all forms of structure IDs• Systematic name(s)• Trivial Name(s)• SMILES• InChI Strings• InChIKeys• Database IDs

• Registry Number

Many Names, One Structure

But big and often noisy

Text-Mining and Markup…

Text-Mining and Markup…

With links out to platforms

Dictionaries are invaluable

Text Mining on IUPAC Names

The N-(β-hydroxyethyl)-N-methyl-N'-(2-trifluoromethyl-1,3,4-thiadiazol-5-yl)urea prepared in Example 6 , thionyl chloride ( 5 ml ) and benzene ( 50 ml ) were charged into a glass reaction vessel equipped with a mechanical stirrer , thermometer and reflux condenser .

The reaction mixture was heated at reflux with stirring , for a period of about one-half hour .

After this time the benzene and unreacted thionyl chloride were stripped from the reaction mixture under reduced pressure to yield the desired product N-(β-chloroethyl)-N-methyl-N'-(2-trifluoromethyl-1,3,4-thiaidazol-5-yl)urea as a solid residue

Text Mining on IUPAC Names

The N-(β-hydroxyethyl)-N-methyl-N'-(2-trifluoromethyl-1,3,4-thiadiazol-5-yl)urea prepared in Example 6 , thionyl chloride ( 5 ml ) and benzene ( 50 ml ) were charged into a glass reaction vessel equipped with a mechanical stirrer , thermometer and reflux condenser .

The reaction mixture was heated at reflux with stirring , for a period of about one-half hour .

After this time the benzene and unreacted thionyl chloride were stripped from the reaction mixture under reduced pressure to yield the desired product N-(β-chloroethyl)-N-methyl-N'-(2-trifluoromethyl-1,3,4-thiaidazol-5-yl)urea as a solid residue

Name to Structure Conversion

Name to Structure Conversion

ChemSpider “Annotations”

• Users can add • Descriptions, Syntheses and Commentaries• Links to PubMed articles• Links to articles via DOIs • Add spectral data• Add Crystallographic Information Files• Add photos• Add MP3 files• Add Videos

Spectral Data

• Spectral data to be deposited in standard formats – JCAMP or images

• All spectra available at: http://www.chemspider.com/spectra.aspx

• Data are deposited on a regular basis• Students

• Chemical vendors• Growing collection now

Student Submissions

Data on ChemSpider

Spectral Data EXTRACTION

ORIGINAL

EXTRACTED

It’s exactly the WRONG WAY!

• We should NOT be mining data out of future publications

• Structures should be submitted “correctly” • Spectra should be digital spectral formats,

not images• ESI should be RICH and interactive,

preferably with OPEN DATA

An Adventure into the World of Small but significant contribution..

ChemSpider SyntheticPages

Micropublishing with Peer Review(a chemical synthesis blog?)

Multi-Step Synthesis

Interactive Data

Chemistry data is of value?

• Reference databases generate hundreds of millions of dollars/euros per year

• So much data generated that could go public• Maybe 5% of all data generated is published• There is no “Journal of Failed Experiments”• Funding agencies start to demand Open Data• Scientists want funding but also recognition

A shift to Openness

How will I get recognized?

• Who in the room has an ORCID?

Deposition of Research Data

• If we manage compounds, syntheses and analytical data…

• If we have security and provenance of data…

• If we deliver user interfaces to satisfy the various use cases…

• Then we have delivered electronic lab notebooks for chemistry laboratories. ELNs are research data repositories

Recognition: need to have Impact

Quantitating scientists?

National Information Standards Organization and “Altmetrics”

http://www.niso.org/apps/group_public/download.php/13295/niso_altmetrics_white_paper_draft_v4.pdf

What are we building?

• We are building the “RSC Data Repository”

• Containers for compounds, reactions, analytical data, tabular data

• Algorithms for data validation and standardization • Flexible indexing and search technologies• A platform for modeling data and hosting existing

models and predictive algorithms

Compounds

Reactions

Analytical data

Crystallography data

Deposition of Data

• Developing systems that provides feedback to users regarding data quality• Validate/standardize chemical compounds

• Check for balanced reactions• Checks spectral data

• EXAMPLE Future work• Properties – compare experimental to pred.• Automated structure verification - NMR

So we know about ORGANICS

• Comment – you don’t know all of the challenges until you start to work in the area!

• We, and cheminformatics companies, have solved MANY, but not all of the issues regarding organic chemistry management

• The majority of our approaches do not map to materials • No standard ways to represent compounds• No InChI for materials

Questions to consider…

• Organics are hard enough!

• What are your best dictionaries of materials?

• We have chemical ontologies. Status for materials?

• Is open annotation of your databases possible?

• What standards do you have for materials data exchange?

Polymorphism is common

Known Challenges

• Many materials are non-stoichiometric

• How to represent composite materials (e.g. supported catalysts)?

• Methods to distinguish novelty in materials (equivalent to diversity in organic structures)?

• Many more I will learn at this workshop..?

Collaboration is key

Internet Data

The Future

Commercial SoftwarePre-competitive Data

Open ScienceOpen DataPublishersEducators

Open DatabasesChemical Vendors

Small organic moleculesUndefined materialsOrganometallicsNanomaterialsPolymersMineralsParticle boundLinks to Biologicals

Thank you

Email: williamsa@rsc.orgORCID: 0000-0002-2668-4821 Twitter: @ChemConnectorPersonal Blog: www.chemconnector.com SLIDES: www.slideshare.net/AntonyWilliams