sqlite: light, open source relational database management system
DESCRIPTION
Overview of SQLite benefits including compatibility with R, Python, and ability to work with converted Microsoft Access databases.TRANSCRIPT
![Page 1: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/1.jpg)
Light, Open Source RDBMS
SQLite
Tanner Jessel
Spring 2014IS 592 - Big Data Analytics
School of Information SciencesCollege of Communication & Information
![Page 2: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/2.jpg)
• Big Data in a Little Package– Portable, No Server– Writes directly to
common media– Cross-platform
(Mac, Windows, Linux)
What is SQLite?
![Page 3: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/3.jpg)
• Portable– Small Gadgets (Windows
Phone, Android, iOS)• (500 KB, 300 KB)
• Embeddable– Computer Applications
(Chrome, Firefox, Skype)
• Single file database – Triggers– Tables– Indices– Views
What is SQLite?
![Page 4: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/4.jpg)
• Not a full database application– No forms– No reports– No saved queries
What it is not…
![Page 5: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/5.jpg)
*Free* • No Server (no mess)• Command Line Interface• GUI Interface• Works with R• Works with Python• Good for students to learn advanced
SQL queries, command line interaction• Good for small database projects
Why should you be interested?
![Page 6: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/6.jpg)
Alter, analyze, attatch, begin, commit, create (table, trigger, view, virtual table), delete, select, insert…
SQLite Query Language
“Mostly” SQL
![Page 7: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/7.jpg)
“Database Interface R driver for SQLite. This package embeds the SQLite database engine in R and provides an interface compliant with the DBI package. The source for the SQLite engine (version 3.7.17) is included.”
–Comprehensive R Archive Network (CRAN)
http://cran.r-project.org/web/packages/RSQLite/
RSQLite
![Page 8: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/8.jpg)
Sqlite3
http://www.sqlite.org/cli.html
Command Line Shell
$ sqlite3 myexampledbSQLite version 3.8.4 2014-02-11 16:24:34Enter ".help" for usage hints.sqlite> CREATE table tbl1(one varchar(10), two smallint);sqlite> INSERT into tbl1 values('hello!',10);sqlite> INSERT into tbl1 values('goodbye', 20);sqlite> SELECT * from tbl1;hello!|10goodbye|20sqlite>
![Page 9: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/9.jpg)
*Free*• Feature Rich GUI• Cross-platform– Windows – Mac– Linux– Solaris
• http://sqlitestudio.pl/
SQLite Studio
![Page 10: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/10.jpg)
• Rstudio Console> install.packages("RSQLite")> library(RSQLite)> dbDIR <- "/Users/apple/Documents/IS592-Big-Data-Analytics/SQLite-Presentation"> dbFileName <- paste(dbDIR,"classHOBOData2013.db3", sep="/")>drv <- dbDriver("SQLite”)>con <- dbConnect(drv, dbname = dbFileName)
Connect to Data
![Page 11: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/11.jpg)
• Rstudio Console> allDf <- dbGetQuery(con, "select * from HoboTable ")>View(allDf)
Load Data to Frame
![Page 12: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/12.jpg)
Pendant monitor data from SQLite database in data frame
SQLite Table Data in R Studio
15,186 rows
![Page 13: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/13.jpg)
>summary(allDf)
obsNum serialNo recNo dateTime temperatureC Min. : 1 Length:15186 Min. : 1.0 Length:15186 Min. :14.42 1st Qu.: 3797 Class :character 1st Qu.: 422.0 Class :character 1st Qu.:23.58 Median : 7594 Mode :character Median : 844.0 Mode :character Median :23.87 Mean : 7594 Mean : 844.5 Mean :26.43 3rd Qu.:11390 3rd Qu.:1266.0 3rd Qu.:25.12 Max. :15186 Max. :1722.0 Max. :59.81 intensityLight Min. : 0.0 1st Qu.: 0.0 Median : 10.8 Mean : 2524.8 3rd Qu.: 21.5 Max. :231468.2
![Page 14: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/14.jpg)
Pendant monitor data from SQLite database in plot
>plot(allDf$temperatureC)
Temperature Data
![Page 15: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/15.jpg)
• Rstudio Console> temperatureF <- (allDf$temperatureC * (9/5.0))+32.0
Transformation C to F
![Page 16: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/16.jpg)
• Rstudio Console> temperatureF <- (allDf$temperatureC * (9/5.0))+32.0> newDf <- cbind(allDf,temperatureF)> dbWriteTable(con, "newTable", newDf)
Update Data Frame
![Page 17: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/17.jpg)
Pendant data .db3 file now has two tables
SQLite Table Data Modified by R
15,186 rows, new TemperatureF column added
![Page 18: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/18.jpg)
>view(hotDf)
serialNo dateTime temperatureC1 10081427 2013-06-04 12:32:30 37.8242 10081427 2013-06-04 12:32:40 37.9353 10081427 2013-06-04 12:32:50 37.9354 10081427 2013-06-04 12:33:00 37.9355 10081427 2013-06-04 12:33:10 38.0466 10081427 2013-06-04 12:33:20 38.0467 10081427 2013-06-04 12:33:30 38.4908 10081427 2013-06-04 12:33:40 38.4909 10081427 2013-06-04 12:33:50 38.60210 10081427 2013-06-04 12:34:00 38.602
![Page 19: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/19.jpg)
Quality Assurance / Control
• Import SQLite3import sqlite3conn=sqlite3.connect('/Users/apple/Documents/IS592-Big-Data-Analytics/SQLite-Presentation/classHOBOData2013.db3')c=conn.cursor()
![Page 20: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/20.jpg)
Quality Assurance / Control
• Find, list “errors”• Whatever you
define them to be• > 100 F in
Antarctica, for example
c.execute('SELECT obsNum,serialNo,temperatureF FROM newTable \ WHERE temperatureF > 100 OR temperatureF < 10')
![Page 21: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/21.jpg)
Quality Assurance / Control
• Check for Errors• Nice printed
messsage with your selected “errors”
• python QAQCdemo.py
#Fetch all the query resultsmyListOfNames=c.fetchall()# print them outprintprint("Range Errors for Temperature:")
for myTuple in myListOfNames: print("obsNum: "+str(myTuple[0])+" SerialNo: "+str(myTuple[1])+" Temperature: "+str(myTuple[2]))
conn.commit()conn.close()
![Page 22: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/22.jpg)
'SELECT obsNum,serialNo,temperatureF FROM newTable WHERE temperatureF > 100 OR temperatureF < 10'
>100 F: SQLite/Python QAQC
1,538 records where the temperature was over 100 F
![Page 23: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/23.jpg)
Access SQLite
Convert Access to SQLitetjessel:~ apple$ cd documentstjessel:documents apple$ cd mdb-sqlite-1.0.2/tjessel:mdb-sqlite-1.0.2 apple$ ant distBuildfile: /Users/apple/Documents/mdb-sqlite-1.0.2/build.xml
prepare:
compile: [javac] /Users/apple/Documents/mdb-sqlite-1.0.2/build.xml:45: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
dist: [one-jar] ma
![Page 24: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/24.jpg)
Further reading
http://www.sqlite.org/books.html
![Page 25: SQLite: Light, Open Source Relational Database Management System](https://reader033.vdocuments.site/reader033/viewer/2022061209/548cd3c9b47959672b8b47ea/html5/thumbnails/25.jpg)
Questions?
@mountainsol
http://www.slideshare.net/mountainsol/sqlite