a tale of 3 databases

A Tale of 3 DatabasesChris Skardon

@cskardon | chris@tournr.com

Before We Begin

Contents• Prologue• Chapter 1 – SQL• Chapter 2 – RavenDB• Chapter 3 – Neo4j• Epilogue

Prologue

What is Tournr?• Competition running site• Any competition, • Sport (running, swimming) • Game (scrabble, chess)• Other…

• Help organisers and competitors have a permanent record• Unified Ranking

Chapter 1SQL

WISAThe traditional entry point for a .NET developer

• Windows• Internet Information Services (IIS)• MS SQL Server 2012• Entity Framework (ORM)

• Asp.NET

The Good• Lots of common knowledge• SQL is well-known• Standard• Libraries• Language• Skills

• Cheap!

INTERLUDE – The Generalist• The Swiss-Army Knife of development• Good at lots of things• But you’re not going to want to cut a

tree down with it

• I can write SQL• But I’m no expert

The Bad• Changes to the model involve writing SQL• Lots of SQL

• Running scripts against DBs, • then resetting, • then…

• Slooooooow turnaround• Feels so 2000

6 Months Later…• Adding features to Tournr was becoming a bit onerous• Lookup Tables to more Lookup Tables• Playing around with RavenDB• *PING* Maybe a good idea for Tournr?

Chapter 2Document DB (RavenDB)

WISA to WIRA• Windows / IIS / RavenDB / Asp.NET• Took about a month, maybe a smidge more• This is partly due to a lack of users – so migration not an issue

• Threw away a lot of code (this is good btw)

The Good• RavenDB has 1st class .NET integration• Really – it’s awesome• Supports LINQ

• Adding new features – very quick• Tournr would not exist without RavenDB• Simple to setup (dev wise at least)• Embeddable • Testing against an actual DB instance

• Dynamic Indexing

Example

using (var session = RavenSession) {     

var tournament = session.Load<Tournament>(id);     

tournament.Name = "New Name";     session.SaveChanges(); }

INTERLUDE – Disclaimer• Struggled with TECH X eh? That’s because:• You’re doing it wrong…• You don’t understand…• You’re being vindictive…• <insert your own reason here>

• Probably totally correct (except the vindictive one), and this is really MY experiences!

The Bad• The documentation is sucky• Niche DB

• Getting help is sometimes hard• Especially for edge cases• Hiring

• Forced into a way of architecting your code• Ayende’s way or the Highway

• Manual linking of documents• Documents grow• Hosting

A year or so later…• Tournr was doing well• Adding features was going OK, but…• Beginning to hit the wall of the design• Duplicates appearing in the documents (by design obvs)

• Drawing out a new feature – looked like a graph• But I’m not using a Graph DB• I could though right?

Chapter 3Graph DB (Neo4j)

Am I being foolish?• A question for every day• Negatives:• Switching DBs - you can’t add new stuff whilst doing it• I have users this time

• Positives:• I’ve been using Neo4j for 3 years or so now, so have

experience• Fits the model better

JUST DO IT!

WIRA to WINA• Windows / IIS / Neo4j / Asp.NET• Nuget -> Neo4jClient• Actually:• Azure Web Apps / GrapheneDB / Asp.NET• Is AWAGA as good?

Membership• Important Quick Win• MVC3 Membership -> MVC5 Identity• A pain in itself• The only ‘architectural’ change aside from DB changes• Why?

• No existing Neo4j Membership implementation• An existing Neo4j Identity implementation

• Switched over, and it worked• No-one more surprised than me (seriously)

The “Process”1. Remove RavenDB2. Break build entirely3. Fix build4. Start putting Model 1 into place5. Decide to go with Model 26. Pull out Model 1 code7. Replace with Model 2 code8. Repeat

The ‘Models’• Evolved over time• First model pretty much thrown out

• Getting to market was priority• Ease of writing queries / code over correctness

Model 1

USER PERSONALDETAILS: HAS_PERSONAL_DETAILS

Model 2

USER

PersonalDetails {get;}Email { get; }

Points of change – New stuff• Custom Json Serializers• Not needed before

• Cypher Queries• All DB access was changed• Single biggest area of changes

Points of change – Removed stuff• Controller DB Access code• Ravens approach not really appropriate for Neo4j

So… It compiles – job done!• Looks like it’s working…• Uh oh – problems• Missing / Broken functionality• Performance is SHOCKINGLY bad• 30+ seconds to load a page• Don’t worry Neo4j fans! it’s my fault…

Problems you say… why?• Change of mindset required• First initial code change – simple, and dumb• Multiple queries = SLOW• Always try to get the most out of one query• Literally – if a controller action required 2 calls, make that 1

The Good• Quick & Easy to install• Good modelling • Adding new features quick• Documentation• Cypher

MATCH (t:Tournament)-[:HAS_COMPETITOR]->(c:Competitor)WHERE t.Name = ‘Awesome Comp #1’ RETURN c

The Bad• Niche DB• Support• Hiring

• Cypher can cause muddles• Hosting

EpilogueOr “Was it worth it?”

Model

Comp

750m

User

1.5Km

IS_OWNER

IS_COMPETING

IS_COMPETINGHAS_EVENT

HAS_EVENT

To

Code - Max

IL Cyclomatic ComplexityMax LOC per MethodMethods per Type

Code - Averages

IL Cyclomatic ComplexityLOC per MethodMethods per Type

Future Development• With Model – easier to ‘see’• Some frustrations over how to do some things

Appendix 1Or “Where the book analogy shows signs of being tired”

What should you take from this?• Converting from SQL / N.E.Other Database• Totally Doable

• Is it easy?• Yes and no

• Is it worth it?• Depends on your use-case• Does a graph model provide a closer analogy to your work

space?• Would I do it again?• Yes

Further Reading

Links!• RavenDB – http://ravendb.net/• Neo4j – http://neo4j.org/• Tournr – https://www.tournr.com/• XClave - http://www.xclave.co.uk/

The End.Questions?

@cskardon | chris@tournr.com