a tale of 3 databases
TRANSCRIPT
A Tale of 3 DatabasesChris Skardon
@cskardon | [email protected]
Before We Begin
Contents• Prologue• Chapter 1 – SQL• Chapter 2 – RavenDB• Chapter 3 – Neo4j• Epilogue
Prologue
What is Tournr?• Competition running site• Any competition, • Sport (running, swimming) • Game (scrabble, chess)• Other…
• Help organisers and competitors have a permanent record• Unified Ranking
Chapter 1SQL
WISAThe traditional entry point for a .NET developer
• Windows• Internet Information Services (IIS)• MS SQL Server 2012• Entity Framework (ORM)
• Asp.NET
The Good• Lots of common knowledge• SQL is well-known• Standard• Libraries• Language• Skills
• Cheap!
INTERLUDE – The Generalist• The Swiss-Army Knife of development• Good at lots of things• But you’re not going to want to cut a
tree down with it
• I can write SQL• But I’m no expert
The Bad• Changes to the model involve writing SQL• Lots of SQL
• Running scripts against DBs, • then resetting, • then…
• Slooooooow turnaround• Feels so 2000
6 Months Later…• Adding features to Tournr was becoming a bit onerous• Lookup Tables to more Lookup Tables• Playing around with RavenDB• *PING* Maybe a good idea for Tournr?
Chapter 2Document DB (RavenDB)
WISA to WIRA• Windows / IIS / RavenDB / Asp.NET• Took about a month, maybe a smidge more• This is partly due to a lack of users – so migration not an issue
• Threw away a lot of code (this is good btw)
The Good• RavenDB has 1st class .NET integration• Really – it’s awesome• Supports LINQ
• Adding new features – very quick• Tournr would not exist without RavenDB• Simple to setup (dev wise at least)• Embeddable • Testing against an actual DB instance
• Dynamic Indexing
Example
using (var session = RavenSession) {
var tournament = session.Load<Tournament>(id);
tournament.Name = "New Name"; session.SaveChanges(); }
INTERLUDE – Disclaimer• Struggled with TECH X eh? That’s because:• You’re doing it wrong…• You don’t understand…• You’re being vindictive…• <insert your own reason here>
• Probably totally correct (except the vindictive one), and this is really MY experiences!
The Bad• The documentation is sucky• Niche DB
• Getting help is sometimes hard• Especially for edge cases• Hiring
• Forced into a way of architecting your code• Ayende’s way or the Highway
• Manual linking of documents• Documents grow• Hosting
A year or so later…• Tournr was doing well• Adding features was going OK, but…• Beginning to hit the wall of the design• Duplicates appearing in the documents (by design obvs)
• Drawing out a new feature – looked like a graph• But I’m not using a Graph DB• I could though right?
Chapter 3Graph DB (Neo4j)
Am I being foolish?• A question for every day• Negatives:• Switching DBs - you can’t add new stuff whilst doing it• I have users this time
• Positives:• I’ve been using Neo4j for 3 years or so now, so have
experience• Fits the model better
JUST DO IT!
WIRA to WINA• Windows / IIS / Neo4j / Asp.NET• Nuget -> Neo4jClient• Actually:• Azure Web Apps / GrapheneDB / Asp.NET• Is AWAGA as good?
Membership• Important Quick Win• MVC3 Membership -> MVC5 Identity• A pain in itself• The only ‘architectural’ change aside from DB changes• Why?
• No existing Neo4j Membership implementation• An existing Neo4j Identity implementation
• Switched over, and it worked• No-one more surprised than me (seriously)
The “Process”1. Remove RavenDB2. Break build entirely3. Fix build4. Start putting Model 1 into place5. Decide to go with Model 26. Pull out Model 1 code7. Replace with Model 2 code8. Repeat
The ‘Models’• Evolved over time• First model pretty much thrown out
• Getting to market was priority• Ease of writing queries / code over correctness
Model 1
USER PERSONALDETAILS: HAS_PERSONAL_DETAILS
Model 2
USER
PersonalDetails {get;}Email { get; }
Points of change – New stuff• Custom Json Serializers• Not needed before
• Cypher Queries• All DB access was changed• Single biggest area of changes
Points of change – Removed stuff• Controller DB Access code• Ravens approach not really appropriate for Neo4j
So… It compiles – job done!• Looks like it’s working…• Uh oh – problems• Missing / Broken functionality• Performance is SHOCKINGLY bad• 30+ seconds to load a page• Don’t worry Neo4j fans! it’s my fault…
Problems you say… why?• Change of mindset required• First initial code change – simple, and dumb• Multiple queries = SLOW• Always try to get the most out of one query• Literally – if a controller action required 2 calls, make that 1
The Good• Quick & Easy to install• Good modelling • Adding new features quick• Documentation• Cypher
MATCH (t:Tournament)-[:HAS_COMPETITOR]->(c:Competitor)WHERE t.Name = ‘Awesome Comp #1’ RETURN c
The Bad• Niche DB• Support• Hiring
• Cypher can cause muddles• Hosting
EpilogueOr “Was it worth it?”
Model
Comp
750m
User
1.5Km
IS_OWNER
IS_COMPETING
IS_COMPETINGHAS_EVENT
HAS_EVENT
To
Code - Max
IL Cyclomatic ComplexityMax LOC per MethodMethods per Type
Code - Averages
IL Cyclomatic ComplexityLOC per MethodMethods per Type
Future Development• With Model – easier to ‘see’• Some frustrations over how to do some things
Appendix 1Or “Where the book analogy shows signs of being tired”
What should you take from this?• Converting from SQL / N.E.Other Database• Totally Doable
• Is it easy?• Yes and no
• Is it worth it?• Depends on your use-case• Does a graph model provide a closer analogy to your work
space?• Would I do it again?• Yes
Further Reading
Links!• RavenDB – http://ravendb.net/• Neo4j – http://neo4j.org/• Tournr – https://www.tournr.com/• XClave - http://www.xclave.co.uk/
The End.Questions?
@cskardon | [email protected]