@ Couchbase Connect16

LINQing to data:Easing the transition

from SQL

Welcome

Brant BurnettSoftware Development Team Lead

Couchbase Community Expert

Welcome

Quick Facts .Net Data Modeling Basics Querying with LINQ Advanced LINQ Features for Couchbase Efficient Indexing for LINQ Queries Change Tracking + Read Your Own Write Q & A

The Agenda

Celebrating 12 Year Anniversary

Team of 50 in Roxboro, NC

Sister company is Palace Pointe, a 100k sq. ft. Entertainment Venue for which we were developed as an in-house system

Over 600 users across the US and abroad

FEC’s, Waterparks, Trampoline Parks, Amusement Parks, Skating Rinks, Bowling Centers, Zoos & Museums

Quick Facts

Point of Sale

Admissions & Ticketing

Party, Group & Event Bookings

Online Sales & Party Reservations

Time Clock & Labor Management

& More!

Quick Facts

.Net since inception in 2004Customers use a local client/server SQL app, but these integrate with our cloud-based productsStarted using Couchbase Server 1.8 for online stores in 2012Cache SQL Server query results, persisted shopping cartsNeeded scalability and stabilityNew products operate using a pure Couchbase Server 4.5 database layer

Quick Facts

New Cloud Platform Data Flow

Data Data Data

IndexQuery

Web Servers

Users

Remote Application Servers

How to build your POCOs for consistenc

y

Nesting documents

Nesting arrays

Including reference primary

keys

.Net Data Modeling Basics

.Net Data Modeling Basics

public class Airline{ public string Callsign { get; set; } public string Country { get; set; } [JsonProperty("iata")] public string IATA { get; set; } [JsonProperty("iaco")] public string IACO { get; set; } public int Id { get; set; } public string Name { get; set; } public string Type { get; set; }}

Key: airline_10{ "callsign": "MILE-AIR", "country": "United States", "iata": "Q5", "icao": "MLA", "id": 10, "name": "40-Mile Air", "type": "airline"}

// JSON decorators not shown to simplify displaypublic class Airport{ public string AirportName { get; set; } public string City { get; set; } public string Country { get; set; } public string FAA { get; set; } public Coordinate Geo { get; set; } public string ICAO { get; set; } public int Id { get; set; } public string Timezone { get; set; } public string Type { get; set; }}

public class Coordinate{ public int Altitude { get; set; } public double Latitude { get; set; } public double Longitude { get; set; }}

Key: airport_1254{ "airportname": "Calais Dunkerque", "city": "Calais", "country": "France", "faa": "CQF", "geo": { "alt": 12, "lat": 50.962097, "lon": 1.954764 }, "icao": "LFAC", "id": 1254, "type": "airport", "tz": "Europe/Paris"}

.Net Data Modeling BasicsNesting Subdocuments as Properties

Note: Subdocuments don’t need “type” attributes, just the root

document

// JSON decorators not shown to simplify displaypublic class Route{ public string Airline { get; set; } public string AirlineId { get; set; } public string DestinationAirport { get; set; } public double Distance { get; set; } public string Equipment { get; set; } public int Id { get; set; } public List<Schedule> Schedule { get; set; } public string SourceAirport { get; set; } public int Stops { get; set; } public string Type { get; set; }}

public class Schedule{ public int Day { get; set; } public string Flight { get; set; } public TimeSpan UTC { get; set; }}

Key: route_10000{ "airline": "AF", "airlineid": "airline_137", "destinationairport": "MRS", "distance": 2881.617376098415, "equipment": "320", "id": 10000, "schedule": [ { "day": 0, "flight": "AF198", "utc": "10:13:00" }, { "day": 0, "flight": "AF547", "utc": "19:14:00" } ], "sourceairport": "TLV", "stops": 0, "type": "route"}

.Net Data Modeling BasicsNesting Arrays as Lists

Note: List<T> or similar list construct

[DocumentTypeFilter(TypeString)]public class Airline{ public const string TypeString = "airline";

public string Callsign { get; set; } public string Country { get; set; } [JsonProperty("iata")] public string IATA { get; set; } [JsonProperty("iaco")] public string IACO { get; set; } public int Id { get; set; } public string Name { get; set; }

// Type is now read only to maintain consistency public string Type => TypeString;}

Key: airline_10{ "callsign": "MILE-AIR", "country": "United States", "iata": "Q5", "icao": "MLA", "id": 10, "name": "40-Mile Air", "type": "airline"}

.Net Data Modeling BasicsGood Practices And LINQ Improvements

Note: DocumentTypeFilter automatically applies a WHERE type = ‘x’ predicate

Making Type read only ensures it’s always saved correctly

Key: route_10000{ "airline": "AF", "airlineid": "airline_137", "destinationairport": "MRS", "distance": 2881.617376098415, "equipment": "320", "id": 10000, "schedule": [ { "day": 0, "flight": "AF198", "utc": "10:13:00" }, { "day": 0, "flight": "AF547", "utc": "19:14:00" } ], "sourceairport": "TLV", "stops": 0, "type": "route"}

Key: airline_137{ "callsign": "AIRFRANS", "country": "France", "iata": "AF", "icao": "AFR", "id": 137, "name": "Air France", "type": "airline"}

.Net Data Modeling Basics

Note: To support joining, include the document key (or have a

way to build it)

Querying with LINQ

Basic and Advanced Queries

Joining Documents by Primary

Key

Unnesting Arrays

Querying with LINQ

public ActionResult BasicQuery(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from p in db.Query<Airline>() orderby p.Name select p;

return View(query);}

SELECT `Extent1`.* FROM `travel-sample` as `Extent1`WHERE (`Extent1`.`type` = 'airline')ORDER BY `Extent1`.`name` ASC

Note: DocumentTypeFilter (slide 12) automatically added the type predicate

Querying with LINQEasily add filters, sorts, projections and pagination to the query

public ActionResult AdvancedQuery(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = (from p in db.Query<Airline>() where p.Callsign.StartsWith("A") orderby p.Name select new AirlineModel {Callsign = p.Callsign, Name = p.Name}).Take(10);

return View(query);}

SELECT `Extent1`.`callsign` as `callsign`, `Extent1`.`name` as `name`FROM `travel-sample` as `Extent1`WHERE (`Extent1`.`type` = 'airline') AND (`Extent1`.`callsign` LIKE 'A%')ORDER BY `Extent1`.`name` ASCLIMIT 10

Querying with LINQJoin to other documents using their primary key

public ActionResult Join(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from route in db.Query<Route>() join airline in db.Query<Airline>() on route.AirlineId equals N1QlFunctions.Key(airline) where route.SourceAirport == "ATL" && route.DestinationAirport == "ABE" orderby airline.Name, route.Stops select new RouteModel { AirlineName = airline.Name, Stops = route.Stops, Schedule = route.Schedule };

return View(query);}SELECT `Extent2`.`name` as `airlineName`, `Extent1`.`stops` as `stops`, `Extent1`.`schedule` as `schedule`FROM `travel-sample` as `Extent1`INNER JOIN `travel-sample` as `Extent2` ON KEYS `Extent1`.`airlineid`WHERE (`Extent1`.`type` = 'route') AND (`Extent2`.`type` = 'airline')AND ((`Extent1`.`sourceairport` = 'ATL') AND (`Extent1`.`destinationairport` = 'ABE'))ORDER BY `Extent2`.`name` ASC, `Extent1`.`stops` ASC

Querying with LINQUNNEST in N1QL

{ "airline": "AF", "airlineid": "airline_137", "destinationairport": "MRS", "distance": 2881.617376098415, "equipment": "320", "id": 10000, "schedule": [ { "day": 0, "flight": "AF198", "utc": "10:13:00" }, { "day": 0, "flight": "AF547", "utc": "19:14:00" } ], "sourceairport": "TLV", "stops": 0, "type": "route"}

[ { "day": 0, "destinationairport": "MRS", "flight": "AF198", "sourceairport": "TLV", "utc": "10:13:00" }, { "day": 0, "destinationairport": "MRS", "flight": "AF547", "sourceairport": "TLV", "utc": "19:14:00" }]

SELECT route.sourceairport, route.destinationairport, schedule.day, schedule.flight, schedule.utcFROM `travel-sample` AS routeUNNEST route.schedule AS scheduleWHERE route.type = 'route'

Querying with LINQUsing UNNEST with a second FROM clause

public ActionResult Unnest(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from route in db.Query<Route>() from schedule in route.Schedule where route.SourceAirport == "ATL" && route.DestinationAirport == "ABE" orderby schedule.Day, schedule.UTC select new UnnestedScheduleModel { Airline = route.Airline, Day = schedule.Day, UTC = schedule.UTC };

return View(query);}SELECT `Extent1`.`airline` as `airline`, `Extent2`.`day` as `day`, `Extent2`.`utc` as `utc`FROM `travel-sample` as `Extent1`INNER UNNEST `Extent1`.`schedule` as `Extent2`WHERE (`Extent1`.`type` = 'route')AND ((`Extent1`.`sourceairport` = 'ATL') AND (`Extent1`.`destinationairport` = 'ABE'))ORDER BY `Extent2`.`day` ASC, `Extent2`.`utc` ASC

Advanced LINQ Features for Couchbase

NULL != MISSING

UseKeys – Get documents by their primary

key

UseIndex – Provide index hints to the

query engine

Asynchronous querying

Advanced LINQ Features for CouchbaseHandling undefined JSON attributes – IsMissing, IsNotMissing,

IsValued, IsNotValued

public ActionResult IsMissing(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from p in db.Query<Airline>() where N1QlFunctions.IsMissing(p.IACO) select p;

return View(query);}

SELECT `Extent1`.*FROM `travel-sample` as `Extent1`WHERE (`Extent1`.`type` = 'airline') AND `Extent1`.`iaco` IS MISSING

Advanced LINQ Features for CouchbaseSelect documents directly using their primary key – UseKeys

public ActionResult UseKeys(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from p in db.Query<Airline>() .UseKeys(new[] {"airline_137", "airline_10765", "airline_1316" }) select p;

return View(query);}

SELECT `Extent1`.*FROM `travel-sample` as `Extent1`USE KEYS ['airline_137', 'airline_10765', 'airline_1316']WHERE (`Extent1`.`type` = 'airline')

Advanced LINQ Features for CouchbaseProvide Query Plan hints – UseIndex

public ActionResult UseIndex(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from p in db.Query<Route>().UseIndex("def_sourceairport") where p.SourceAirport == "ATL" orderby p.DestinationAirport select p;

return View(query);}

SELECT `Extent1`.*FROM `travel-sample` as `Extent1`USE INDEX (`def_sourceairport` USING GSI)WHERE (`Extent1`.`type` = 'route') AND (`Extent1`.`sourceairport` = 'ATL')ORDER BY `Extent1`.`destinationairport` ASC

Advanced LINQ Features for CouchbaseRun Queries Asynchronously – ExecuteAsync

public async Task<ActionResult> Async(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from p in db.Query<Airline>() orderby p.Name select p;

return View( await query.ExecuteAsync());}

Advanced LINQ Features for CouchbaseRun Aggregate Queries Asynchronously – ExecuteAsync

public async Task<ActionResult> AsyncAggregate(){ var db = new BucketContext(ClusterHelper.GetBucket("travel-sample"));

var query = from p in db.Query<Route>() select p;

return View( await query.ExecuteAsync( p => p.Average(q => q.Distance)));}

ExecuteAsync() can even run any immediate execution method (aggregates, First, Single, Explain) by passing the method as a

lambda expression

Efficient Indexing For LINQ Queries

Differences between SQL and

N1QL indexes

Indexing on “type”

attribute for speed

Indexing DateTime attributes

Efficient Indexing For LINQ Queries

AirlineSQL Table

AirportSQL Table

travel-sample Bucket

Airline Indexes

Airport Indexes

Bucket Indexes

Note: Remember that GSI indexes are similar to SQL indexes, but not the same

Efficient Indexing For LINQ Queries

/* Use predicate to only index documents of a certain type */CREATE INDEX `airport_sourceairport` ON `travel-sample` (`sourceairport`)WHERE `type` = 'airport'

/* Index the same attribute across multiple document types by including type first */CREATE INDEX `def_type_id` ON `travel-sample` (`type`, `id`)

/* A good practice is to create a fallback in case other indexes aren't used */CREATE INDEX `def_type` ON `travel-sample` (`type`)

Note: Be sure to index on the “type” attribute if you’re using [DocumentTypeFilter(“…”)]

Efficient Indexing For LINQ QueriesIndexing DateTime attributes – STR_TO_MILLIS

var cutoffDateTime = new DateTime(2010, 1, 1, 0, 0, 0, DateTimeKind.Utc);var query = db.Query<Beer>() .Where(e => e.Updated <= cutoffDateTime);

/* LINQ automatically wraps all DateTime constants and properties in STR_TO_MILLIS *//* STR_TO_MILLIS converts an ISO8601 string to a Unix numeric representation *//* It also handles the time zone specifier */SELECT `Extent1`.* FROM `beer-sample` as `Extent1`WHERE (`type` = 'beer')AND (STR_TO_MILLIS(`Extent1`.`updated`) <= STR_TO_MILLIS("2010-01-01T00:00:00Z"))/* So STR_TO_MILLIS must also be used in the index, or the index cannot be used */CREATE INDEX `beer_updated` ON `beer-sample` (STR_TO_MILLIS(`updated`))WHERE `type` = 'beer'

Change Tracking + Read Your Own Write

Document POCO

considerations

Updating, inserting, and

deleting documents

Using MutationState to read your own writes

Note: Linq2Couchbase change tracking is currently in developer preview

Change Tracking + Read Your Own Write[DocumentTypeFilter(TypeString)]public class Route{ public const string TypeString = "route";

// Read only property to return the calculated primary key // This is used when saving the document [Key] public string Key => TypeString + "_" + Id;

// Properties must be virtual for changes to be tracked public virtual string Airline { get; set; } // some properties not shown for clarity...

// Lists now use IList<T> instead of List<T> public virtual IList<Schedule> Schedule { get; set; }

// some properties not shown for clarity...

public string Type => TypeString;}

Update Your Document Models To Support Proxies

Change Tracking + Read Your Own WriteUnit of Work – BeginChangeTracking, SubmitChanges

_db.BeginChangeTracking();

// Query must return the document class, without a select projectionvar query = from p in _db.Query<Airline>() where p.Id == id select p;

// Query must execute after call to BeginChangeTrackingvar airline = query.FirstOrDefault();if (airline == null){ return HttpNotFound();}

airline.Name = model.Name;airline.Callsign = model.Callsign;

// Save the changes, if any_db.SubmitChanges();

Change Tracking + Read Your Own WriteDocument INSERT/DELETE – Save, Remove

_db.BeginChangeTracking();

// To delete a document when SubmitChanges is called_db.Remove(airline);

// To insert a document when SubmitChanges is called_db.Save(new Airline() { Id = 1, Name = model.Name, Callsign = model.Callsign});

// Save the changes_db.SubmitChanges();

Note: Save and Remove execute immediately if called before BeginChangeTracking

Change Tracking + Read Your Own WriteDo I have the latest changes? – MutationState, ConsistentWith

_db.SubmitChanges();

// MutationState represents the changes made by SubmitChanges// Pass into the query to ensure the changes are indexed before the query executes// This will slow the query, but much less than using RequestPlus consistency// Couchbase Server 4.5 is required for this featurevar query = from p in _db.Query<Airline>().ConsistentWith(_db.MutationState) orderby p.Name select p;

Why use Couchbase, N1QL and LINQ?

More scalable and performant than traditional SQL in the cloud

Easy transition for .Net developers trained on SQL and LINQ,and still produces unit testable code

More logical, nested data models rather than dozens of subsidiary tables

Schema-less JSON increases flexibility as your system evolves, leaving schema enforcement in your data access layer

Conclusion

What's next?

Source and Documentation:https://github.com/couchbaselabs/Linq2Couchbase

Example project:https://github.com/brantburnett/Couchbase.Linq.Example

Questions:https://forums.couchbase.com/c/net-sdk - @btburnett3Email - bburnett@centeredgesoftware.comTwitter - @btburnett3

Conclusion

Q & A

Thank YouVisit us at centeredgesoftware.com