1. Firebird ppeerrffoorrmmaannccee ccoouunntteerrssiinn ddeettaaiillssDmitry Yemanovmailto:dimitr@firebirdsql.orgFirebird Projecthttp://www.firebirdsql.org/

2. TThhaannkk yyoouuFIREBIRD INTERNATIONAL CONFERENCE '2014 2 3. AAnnaallyyssiinngg bboottttlleenneecckkssDiskDatabaseTemporary filesFIREBIRD INTERNATIONAL CONFERENCE '2014 3 4. AAnnaallyyssiinngg bboottttlleenneecckkssDiskDatabaseTemporary filesMemoryStatic (cache)Dynamic (pools)SharedFIREBIRD INTERNATIONAL CONFERENCE '2014 4 5. AAnnaallyyssiinngg bboottttlleenneecckkssCPU, waits, etcReal execution timeUser/kernel timeWas time spent for work or waiting?What operations were performed?FIREBIRD INTERNATIONAL CONFERENCE '2014 5 6. LLeeggaaccyy ppeerrffoorrmmaannccee ccoouunntteerrssPage levelFetches, reads, marks, writesShared in SS, per connection in CSAvailable via APIFIREBIRD INTERNATIONAL CONFERENCE '2014 6 7. LLeeggaaccyy ppeerrffoorrmmaannccee ccoouunntteerrssPage levelFetches, reads, marks, writesShared in SS, per connection in CSAvailable via APIRecord levelSelected (seq/idx), inserted, updated, deleted etcPer connection and per tableAvailable via APIFIREBIRD INTERNATIONAL CONFERENCE '2014 7 8. IImmpprroovveemmeennttss iinn FFiirreebbiirrdd 22..xxMulti-level aggregated countersDatabase (SS only)ConnectionTransactionStatementNested PSQL call (procedure / trigger)FIREBIRD INTERNATIONAL CONFERENCE '2014 8 9. IImmpprroovveemmeennttss iinn FFiirreebbiirrdd 22..xxMulti-level aggregated countersDatabaseConnection 1 Connection 2Transaction 1 Transaction 2Statement 1 Statement 1FIREBIRD INTERNATIONAL CONFERENCE '2014 9 10. IImmpprroovveemmeennttss iinn FFiirreebbiirrdd 22..xxNew interfacesMonitoring tablesTrace/auditFIREBIRD INTERNATIONAL CONFERENCE '2014 10 11. PPaaggee lleevveell ccoouunntteerrssPage readsFrom disk to the page cache (physical reads)Usually means a cache missFIREBIRD INTERNATIONAL CONFERENCE '2014 11 12. PPaaggee lleevveell ccoouunntteerrssPage readsFrom disk to the page cache (physical reads)Usually means a cache missPage fetchesAccess page in cache (logical reads)Includes both cache hits and cache missesCorresponds to a shared page lock / latchFIREBIRD INTERNATIONAL CONFERENCE '2014 12 13. PPaaggee lleevveell ccoouunntteerrssPage readsFrom disk to the page cache (physical reads)Usually means a cache missPage fetchesAccess page in cache (logical reads)Includes both cache hits and cache missesCorresponds to a shared page lock / latchCache hit ratio = 1 - reads / fetches ???FIREBIRD INTERNATIONAL CONFERENCE '2014 13 14. PPaaggee lleevveell ccoouunntteerrssPage writesFrom the page cache to disk (physical writes)At transaction commit/rollbackCache is full of dirty pagesAsynchronous notification is received (CS/SC)Immediately after modificationFIREBIRD INTERNATIONAL CONFERENCE '2014 14 15. PPaaggee lleevveell ccoouunntteerrssPage writesFrom the page cache to disk (physical writes)At transaction commit/rollbackCache is full of dirty pagesAsynchronous notification is received (CS/SC)Immediately after modificationPage marksAccess page in cache (logical writes)Corresponds to an exclusive page lock / latchFIREBIRD INTERNATIONAL CONFERENCE '2014 15 16. RReeccoorrdd lleevveell ccoouunntteerrssSequential record readsRecords retrieved through a full table scanIncludes sweepIndexed record readsRecords retrieved positionallyIncludes bitmap index scans, index navigationalwalks, DBKEY based retrievalsFIREBIRD INTERNATIONAL CONFERENCE '2014 16 17. RReeccoorrdd lleevveell ccoouunntteerrssRecord inserts, updates, deletesPretty obvious, huh?FIREBIRD INTERNATIONAL CONFERENCE '2014 17 18. RReeccoorrdd lleevveell ccoouunntteerrssRecord inserts, updates, deletesPretty obvious, huh?Record backoutsLatest (uncommitted) version removedHappens after savepoint rollback, explicit or implicitMay happen after transaction rollbackFIREBIRD INTERNATIONAL CONFERENCE '2014 18 19. RReeccoorrdd lleevveell ccoouunntteerrssRecord purgesOld versions removed while keeping the primaryversion in placeOutdated versions found while chasingFIREBIRD INTERNATIONAL CONFERENCE '2014 19 20. RReeccoorrdd lleevveell ccoouunntteerrssRecord purgesOld versions removed while keeping the primaryversion in placeOutdated versions found while chasingRecord expungesWhole version chains removed, along with theprimary versionRecord is deleted and nobody is interestedFIREBIRD INTERNATIONAL CONFERENCE '2014 20 21. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxRecord repeated readsRecord is retrieved multiple timesBEFORE triggersSort-based updates/deletesFIREBIRD INTERNATIONAL CONFERENCE '2014 21 22. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxRecord repeated readsRecord is retrieved multiple timesBEFORE triggersSort-based updates/deletesRecord locksRecord is selected usng WITH LOCK clauseFIREBIRD INTERNATIONAL CONFERENCE '2014 22 23. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxRecord waitsAttempts to update/delete/lock recordowned by a concurrent active transactionTransaction is in the WAIT modeFIREBIRD INTERNATIONAL CONFERENCE '2014 23 24. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxRecord waitsAttempts to update/delete/lock recordowned by a concurrent active transactionTransaction is in the WAIT modeRecord conflictsUnsuccessful attempts to update/delete/lock recordowned by a concurrent active transactionUPDATE CONFLICT is reportedFIREBIRD INTERNATIONAL CONFERENCE '2014 24 25. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxBackversion readsVersions chased while finding a visible oneMeans old snapshotsFIREBIRD INTERNATIONAL CONFERENCE '2014 25 26. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxBackversion readsVersions chased while finding a visible oneMeans old snapshotsFragment readsFragmented recordsMeans extra page fetches/readsFIREBIRD INTERNATIONAL CONFERENCE '2014 26 27. NNeeww rreeccoorrdd lleevveell ccoouunntteerrss iinn vv33..xxMON$TABLE_STATS MON$RECORD_STATSMON$STATS_IDMON$STATS_GROUPMON$TABLE_NAMEMON$RECORD_STATS_IDMON$STATS_IDMON$STATS_GROUPMON$RECORD_SEQ_READSMON$RECORD_IDX_READSetcFIREBIRD INTERNATIONAL CONFERENCE '2014 27 28. TTOODDOO:: ppaaggee lleevveell ccoouunntteerrssPage writes (expanded)Regular writes (immediate / commit / rollback)Overflow writesAsynchronous writesFIREBIRD INTERNATIONAL CONFERENCE '2014 28 29. TTOODDOO:: ppaaggee lleevveell ccoouunntteerrssPage writes (expanded)Regular writes (immediate / commit / rollback)Overflow writesAsynchronous writesPage waitsHow many times page requests waitedShows contention inside the page cacheFIREBIRD INTERNATIONAL CONFERENCE '2014 29 30. TTOODDOO:: iinnddeexx ccoouunntteerrssPossible metricsIndex scansNode inserts / deletesBucket splits / mergesKeys scanned / compared while searchingReported per indexMON$INDEX_STATSFIREBIRD INTERNATIONAL CONFERENCE '2014 30 31. TTOODDOO:: tteemmppoorraarryy ssppaaccee ccoouunntteerrssOperation metricsReads / writes resolved through the cacheReads / writes redirected to temp filesI/O amount metricsBytes read / written through the cacheBytes read / written from/to temp filesFIREBIRD INTERNATIONAL CONFERENCE '2014 31 32. TTOODDOO:: ttiimmee ssttaattiissttiiccssElapsed timeTotal inside the engineSpent in the user spaceSpent in the system / kernel spaceFIREBIRD INTERNATIONAL CONFERENCE '2014 32 33. TTOODDOO:: ttiimmee ssttaattiissttiiccssElapsed timeTotal inside the engineSpent in the user spaceSpent in the system / kernel spaceWait timeDatabase (and maybe temp files) I/OPage cache waitsTransaction waitsFIREBIRD INTERNATIONAL CONFERENCE '2014 33 34. TTOODDOO:: iinntteeggrraattiioonnCollecting the statisticsStatsD, CollectDReporting / analyzingGraphite / GrapheneOther toolsNagios, Cacti, ZabbixFIREBIRD INTERNATIONAL CONFERENCE '2014 34 35. QQuueessttiioonnss??mailto:dimitr@firebirdsql.orgFIREBIRD INTERNATIONAL CONFERENCE '2014 35