mobappdev (fall 2014): structured data persistence on android devices with sqlite
TRANSCRIPT
MobAppDev
Structured Data Persistence on Android Devices with SQLite
Vladimir Kulyukin
www.vkedco.blogspot.com
Outline
● Persisting of Structured Data with SQL Databases
● Lite Intro to SQLite● SQLite Database Inspection● SQLite by Example● Creation & Population of SQLite Databases
Persistence of Structured Data with
SQL Databases
SharedPreferences: Pros & Cons
● Pros Lightweight Easy to code and debug
● Cons Slow with large data volumes Not helpful with highly structured data when key/value
pairs are insufficient Entire file needs to be read and parsed to access data Takes up more space because XML is used for storage
SQL Databases: Pros & Cons● Pros
Flexible with structured data Scalable: can handle large data volumes Robust: can handle incremental changes Responsive: can handle flexible queries
● Cons Heavyweight: more code & more memory Maintenance: harder to code & debug
Three-Tier Data Management Model
XML
Java Objects SQLite DB
GUI
Object Inflation
Objects to SQLite Records
Object Visualization
SQLite Records to Objects
Database Design
● Standard database design practices also apply on Android● A key design principle is to manage space well: reduction
redundancy via database normalization● The objective of database normalization is to decompose relations
with anomalies to produce smaller, well structured relations● Database design is more an art than a science: takes experience &
practice● Helpful resource: en.wikipedia.org/Database_normalization
Tables & Data Types
● Database is a persistent collection of structured data● Data are structured because they are organized in tables● A Table is similar to an Excel spreadsheet except that its
columns have standard SQL data types (e.g., TEXT, INTEGER, FLOAT, etc.)
● In terms of Object-Oriented Modeling, each table represents an Class (e.g., Book, Author) and each column represents an Attribute
● Each table row represents an Object of a specific Class
Primary Keys● In many tables, the ID column represents the
primary key● The primary key is the key that uniquely
identifies the row entry object● When design an SQL table, it is typical (not
necessary!) to specify the primary key as an integer and mark it as auto-incrementable: this way every inserted row is guaranteed to have a unique primary key
Lite Intro to SQLite
What is SQLite?● SQLite is a relational database management system
(RDBMS)● SQLite is a compact C library that is part of the Android
software stack● Since it is a library, it can be integrated into each
application without external dependencies: this simplifies transactions and synchronizations, minimizes latency
● If you know or have worked with SQL, you will have no problem with SQLite
Where to Look for Application-Specific SQLite Databases
● Every application can create its own database over which it has complete control
● SQLite databases should be considered only when you need to manage complex, structured data
● Databases are stored in /data/data/<package_name>/databases/● For example, if the package is org.vkedco.mobappdev.simple_db_app_01,
then look for the databases in
/data/data/org.vkedco.mobappdev.simple_db_app_01/databases/
SQL vs SQLite Data Typing
● SQL engines typically use static typing: datatype of a value is determined by the column in which the value is stored
● SQLite engines use dynamic typing: datatype of a value is associated with the value not the column in which it is stored
● Basic SQL vs. SQLite tradeoff: security and safety (SQL) vs. flexibility (SQLite)
● SQLite rule of thumb: do not expose your SQLite API to third parties
SQLite Datatypes
● NULL – the value is NULL● INTEGER – signed integers; signed integer can be stored as
1, 2, 3, 4, 5, 6, or 8 bytes; the storage size depends on the value magnitude
● REAL – float-point numbers; stored as 8 bytes● TEXT – text string; stored according to the database
encoding (e.g., UTF-8)● BLOB – data blob, typically stored as array of bytes
Sample Book Database
SQLite Database
● Our sample book database on Sufi poetry will contain two tables
● The first table consists of five book records● The second table consists of two book authors● The exact tables are given on the next two
slides
Book Table
ID Title Author Translator ISBN Price
1 The Essential Rumi Jalal al-Din Rumi
C. Barks, J. Moyne
9780062509581 17.51
2 The Illuminated Rumi Jalal al-Din Rumi
C. Barks 9780767900027 25.04
3 A Year with Rumi: Daily Readings
Jalal al-Din Rumi
C. Barks 9780060845971 14.92
4 A Year with Hafiz: Daily Contemplations
Hafiz D. Ladinsky 9780143117544 12.24
5 The Gift Hafiz D. Ladinsky 9780140195811 12.24
- Book is a Class - ID, Title, Author, Translator, ISBN, Price are Attributes - Rows 1 – 5 are Book Objects
Author Table
ID Name BirthYear DeathYear Country
1 Jalal al-Din Rumi 1207 1273 Persia
2 Hafiz 1325 1389 Persia
- Author is a Class - ID, Name, BirthYear, DeathYear, Country, are Attributes - Row 1 is an Author Object - Row 2 is an Author Object
Command Line DB Access
Suppose that you have created a database such as the book database shown in the previous slides (we will learn how to do this shortly) and want to interact with it. Here is how:
1. Open a command line
2. CD into platform-tools directory
3. Connect to the device (e.g., adb -e shell)
4. CD into the database directory: cd /data/data/<package>databases>
5. Execute: sqlite3 <database_name> (e.g. sqlite3 book_info.db)
Now you can interct with book_info.db via sqlite shell
6. Execute .quit to quit
Connecting to ADB
You can use adb -e shell to connect to the emulator;
Then you can use standard Linux commands such as ls
CD Into Data/Data/<APP PACKAGE>
1. CD Into data/data/ 2. LS 3. Look for APP PACKAGE 4. CD into it 5. LS to find database names
Discovering TABLE Schemas
● A Table Schema is the specification of column values and types for a given table
● To access table schemas for a DB:
1. Get in to the sqlite shell
2. Execute: .schema
You should see something like:
CREATE TABLE android_metadata (locale TEXT);
CREATE TABLE author (ID integer primary key autoincrement, Name text not null, ...);
CREATE TABLE book (ID integer primary key autoincrement, Title text not null, ...);
Using SQLite3 Interpreter
SQLite by Example
Displaying Tables
Here is how you can display the entire table:
SELECT * FROM <TABLE>;
Examples: SELECT * FROM BOOK; SELECT * FROM AUTHOR;
Column Projections
To project the values of specific columns for each record:
SELECT COLUM_NAME_1, …, COLUMN_NAME_n FROM <TABLE>;
Examples:
SELECT Author, Title FROM BOOK;
SELECT Title, Author FROM BOOK;
Retrieval of Records with Specific Values
To retrieve records with specific values, use the WHERE clause:
SELECT COLUM_NAME_1, …, COLUMN_NAME_n FROM <TABLE> WHERE <COL_VAL_SPEC>;
Examples:
SELECT Title FROM BOOK WHERE ID = 3;
SELECT Title, Author FROM BOOK WHERE ID < 3;
Record Ordering
To order records by attributes, use the ORDER BY clause:
SELECT COLUM_NAME_1, …, COLUMN_NAME_n FROM <TABLE> ORDER BY COL_NAME Asc/Desc;
Examples:
SELECT Title, Author FROM BOOK ORDER BY Price Asc;
SELECT Title, Author FROM BOOK ORDER BY Price Desc;
Creation & Population of
SQLite Databases
Book DB Application
Write an Android application that creates and populates an SQLite database on Sufi poetry from an XML specification of the book title and book author tables. The book titles are displayed in a ListView. When the user clicks on an book title, its database record is displayed in a Toast. The application does OO modeling of SQLite tables as Java classes.
Book DB Application
source code is here
Sample Screenshots
Book Table as Book Class
public class Book {
protected String mTitle;
protected String mAuthor;
protected String mTranslator;
protected String mISBN;
protected float mPrice;
// rest of code
}
public class Book {
protected String mTitle;
protected String mAuthor;
protected String mTranslator;
protected String mISBN;
protected float mPrice;
// rest of code
}
source code is in Book.java
Author Table as Author Class
public class Author {
protected String mName;
protected int mBirthYear;
protected int mDeathYear;
protected String mCountry;
// rest of code
}
source code is in Author.java
XML DB Table Specs
● XML DB table specs can come in a variety of formats and from a variety of resources
● Each XML table spec must describe inflatable row records; table specs inflated into table Java objects
● Table specs give you the flexibility to persist (save) Java into SQLite databases or Object databases
● Source of table specs can vary from local XML files, remote XML files, RSS feeds, etc
XML Specs of Book Title Table
<string-array name="book_title_table">
<item>The Essential Rumi;Jalal adDin Rumi;C. Barks, J. Moyne;9780062509581;17.51</item>
<item>The Illuminated Rumi;Jalal adDin Rumi;C. Barks;9780767900027;25.04</item>
<item>A Year with Rumi: Daily Readings;Jalal adDin Rumi;C. Barks;9780060845971;14.92</item>
<item>A Year with Hafiz: Daily Contemplations;Hafiz;D. Ladinsky;9780143117544;12.24</item>
<item>The Gift;Hafiz;Daniel Ladinsky;9780140195811;12.24</item>
</string-array>
XML source is in strings.xml
XML Specs of Book Author Table
<string-array name="book_author_table">
<item>Jalal adDin Rumi;1207;1273;Persia</item>
<item>Hafiz;1325;1389;Persia</item>
</string-array>
XML source is in strings.xml
Design & Implementation of
Database Adapters
DB XML Specs Table Inflation
● Typically there is a class that creates objects from XML table specs
● In our case, we simply inflate XML string-arrays and create objects from XML strings
● Database adapter classes are intermediaries b/w data and SQLite databases; they typically extend SQLiteOpenHelper
● Each object is then given to the database adapter for SQLite persistence
Population of SQLite DB
● Once we have an adapter class, we can use it to populate the SQLite DB
● This is done in SufiPoetryBooksDBMainAct.java by the method populateBookList()
● This method parses XML, creates Book and Author objects and uses the methods insertUniqueBook() and insertUniqueAuthor() defined in SufiPoetryBooksDBHelper.java
Population of SQLite DB: XML Inflation
// 1. open db
mDBHelper.open();
// 2. Read XML arrays or use an XML parser to parse XML
String[] book_table = mRes.getStringArray(R.array.book_table);
String[] author_table =
mRes.getStringArray(R.array.author_table);
Population of SQLite DB: Population of Book Table
// 3. Inflate book table xml specs into Book objects and place them into SQLite DB
String[] book_entry_parts;
for(String book_entry: book_table) {
book_entry_parts = book_entry.trim().split(XML_ENTRY_SEPARATOR);
mDBHelper.insertUniqueBook(new Book(book_entry_parts[0], // title
book_entry_parts[1], // author
book_entry_parts[2], // translator
book_entry_parts[3], // isbn
Float.parseFloat(book_entry_parts[4]) // price
));
}
Population of SQLite DB: Population of Author Table// 4. Inflate author table xml specs into Author objects and insert them into DB
String[] author_entry_parts;
for(String author_entry: author_table) {
author_entry_parts = author_entry.trim().split(XML_ENTRY_SEPARATOR);
mDBHelper.insertUniqueAuthor(
new Author(author_entry_parts[0], // author's name
Integer.parseInt(author_entry_parts[1]), // author's birth year
Integer.parseInt(author_entry_parts[2]), // author's death year
author_entry_parts[3]) // author's country
);
}
// 5. Close db
mDBHelper.close();
Storing Images in SQLite Databases
Modification of Book DB Application
Let us extend our book db application so that it has an SQLite table that stores book cover images and maps them into book ISBNs.
source is here
BOOK_COVER_IMAGE Table
Data Rows
9780062509581|BLOB_01
9780767900027|BLOB_02
9780060845971|BLOB_03
9780143117544|BLOB_04
9780140195811|BLOB_05
Database Schema
ISBN | Image (BLOB)
Image Blobs from Amazon
BLOB_01 BLOB_02 BLOB_03 BLOB_04 BLOB_05
OO Modeling of Book Cover Image Table
public class BookCoverImage {
protected String mISBN;
protected byte[ ] mCoverRef;
public BookCoverImage(String isbn, byte[] coverRef) {
mISBN = isbn; mCoverRef = coverRef;
}
public String getISBN() { return mISBN; }
public byte[] getCoverRef() { return mCoverRef; }
}
Images Into Byte Arrays Bitmap bmp = null; Resources mRes = cntxt.getResources();
// 1. Construct a bitmap from a source image
bmp = BitmapFactory.decodeResource(mRes, R.drawable.essential_rumi_cover);
// 2. Create a ByteArrayOutputStream
ByteArrayOutputStream bos = new ByteArrayOutputStream();
// 3. Compress the bitmap into the ByteArrayOutputStream
byte[] bitmapbytes = null; bmp.compress(CompressFormat.PNG, 0, bos);
// 4. Get the bytes the bitmap's bytes from the ByteArrayOutputStream
bitmapbytes = bos.toByteArray();
// 5. Recycle the bitmap so that it can be used again
bmp.recycle();
// 6. Close the ByteArrayOutputStream
try { bos.close(); } catch (IOException e) { e.printStackTrace(); }
Images Into Byte Arrays Bitmap bmp = null; Resources mRes = cntxt.getResources();
// 1. Construct a bitmap from a source image
bmp = BitmapFactory.decodeResource(mRes, R.drawable.essential_rumi_cover);
// 2. Create a ByteArrayOutputStream
ByteArrayOutputStream bos = new ByteArrayOutputStream();
// 3. Compress the bitmap into the ByteArrayOutputStream
byte[] bitmapbytes = null; bmp.compress(CompressFormat.PNG, 0, bos);
// 4. Get the bytes the bitmap's bytes from the ByteArrayOutputStream
bitmapbytes = bos.toByteArray();
// 5. Recycle the bitmap so that it can be used again
bmp.recycle();
// 6. Close the ByteArrayOutputStream
try { bos.close(); } catch (IOException e) { e.printStackTrace(); }
Inserting Images into SQLite Databases // Assume that bci is a BookCoverImage object.
byte[] bitmapbytes = bci.getCoverRef();
// we assume that bitmapbytes is not null
ContentValues newBookCoverRef = new ContentValues();
// COVER_IMAGE_ISBN_COL_NAME is a string constant
newBookCoverRef.put(COVER_IMAGE_ISBN_COL_NAME, bci.getISBN());
newBookCoverRef.put(COVER_IMAGE_IMG_COL_NAME, bitmapbytes);
// mDb is a database reference of SQLiteDatabase class.
return mDb.insertWithOnConflict(BOOK_COVER_IMAGE_TABLE,
null, newBookCoverRef,
SQLiteDatabase.CONFLICT_REPLACE);
References● en.wikipedia.org/Database_normalization● http://developer.android.com/tools/help/adb.html● http://developer.android.com/reference/android/database/sqlite/SQLiteDatabase.html