COMP228 [& 327] App Development Session: 2019-2020Lecture Set 3 - Data Persistence

[ last updated: 23 Oct 2019 ] !1Illustration credit: vecteezy.com

!2

In these Slides...

• We will cover...• An introduction to Local Data Storage

• The iPhone filesystem

• Property lists

• UserDefaults

• Data Modelling using Core Data

Storing Data These slides will allow you to understand how data can be modelled and stored/cached locally. This may be

for a local database, or simply saving state or preferences for an

application.

Local Data Storage

Data Persistence and Core Data

!4

Intro to Data and Persistence

• There may be a number of reasons for wanting to read or write data to some form of persistent store• Storing preferences - (why have preferences?)• Storing simple data in general• Storing state• Simple values or container classes• Serialising custom object data

• Managing data• SQLite• Core Data

!5

iPhone File System• Each application sandboxes it’s file system• Ensures security and privacy (how?)• When apps are deleted, all the associated files are deleted too

• Each app maintains its own set of directories• somewhat reminiscent of a UNIX filesystem• Files stored in Caches are not backed up during iCloud backup

• Apps cannot write into their own bundles• This violates the code-signing mechanism• If apps want to include data in the bundle that can later be modified• it must copy the data into your documents directory• then the data can be modified!• mark such data as isExcludedFromBackup• only copy the data on first use

The sandbox file directory for a simple Core Data app

Inside a simple Core Data app

!6

File Paths in your Application• Getting to the Application’s (writeable) home directory• NSHomeDirectory

• Finding directories in the file system• NSSearchPathForDirectoriesInDomains• Creates an array of path strings for the specified directories in the specified domains• Good for discovering where a “known” directory is in the file system

// Documents directorylet documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]

// <Application Home>/Documents/myData.plistlet myDataPath = documentsPath + “myData.plist" //it's actually a stringprint(myDataPath)

Why No Parameter names in the method call?

public func NSSearchPathForDirectoriesInDomains(_ directory: FileManager.SearchPathDirectory, _ domainMask: FileManager.SearchPathDomainMask, _ expandTilde: Bool) -> [String]

!7

• Applications can expose files in their Documents directory to iTunes (when syncing)• Allows users to add and delete files directly to an application

• Capability is set by setting the Application supports iTunes file sharing Property in the App’s info.plist to YES

iTunes File sharing

!8

iTunes File sharing

!9

iTunes File sharing

@IBAction func saveAction() { // Documents directory (writeable by the app) let documentsPath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true)[0]

// <Application Home>/Documents/myData.plist let myDataPath = documentsPath + “/myData.plist"

let myArray = [NSDate.distantFuture, 5, "Hello World! 👾"] as NSArray //Note - an NSArray myArray.write(toFile: myDataPath, atomically: true) }

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array>

<date>4001-01-01T00:00:00Z</date><integer>5</integer><string>Hello World! 👾</string>

</array></plist>

!10

Property Lists• Property lists are structured data used by Cocoa and Core Foundation• Used extensively within iOS, iPadOS and MacOS etc.• Typically XML-based data format

• Provides support for• Primitive data types

• strings• numbers - integers• numbers - floating point • binary data - (NSData)• dates - (NSDate)• boolean values

• Collections - which can recursively contain other collections• arrays - (NSArray)• dictionaries - (NSDictionary)

• Root-level object is almost always either an array or dictionary• Could be a single element plist containing a primitive data type

!11

Property Lists

• A really good way to store small, structured persistent data fragments• Good for:• less than a few hundred kilobytes of data• well defined data elements, that fit the XML data

serialisation

• Bad for:• Large data volumes• Loading is “one-shot”

• Complex objects• Blocks of binary data

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> <plist version="1.0"> <dict> <key>CFBundleDevelopmentRegion</key> <string>en</string> <key>CFBundleExecutable</key> <string>$(EXECUTABLE_NAME)</string> <key>CFBundleIdentifier</key> <string>$(PRODUCT_BUNDLE_IDENTIFIER)</string> <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string> <key>CFBundleName</key> <string>$(PRODUCT_NAME)</string> <key>CFBundlePackageType</key> <string>APPL</string> <key>CFBundleShortVersionString</key> <string>1.0</string> <key>CFBundleVersion</key> <string>1</string> <key>LSRequiresIPhoneOS</key> <true/> <key>UILaunchStoryboardName</key> <string>LaunchScreen</string> <key>UIMainStoryboardFile</key> <string>Main</string> <key>UIRequiredDeviceCapabilities</key> <array> <string>armv7</string> </array> <key>UISupportedInterfaceOrientations</key> <array> <string>UIInterfaceOrientationPortrait</string> <string>UIInterfaceOrientationLandscapeLeft</string> <string>UIInterfaceOrientationLandscapeRight</string> </array> </dict> </plist>

!12

Reading and Writing Property Lists

• Both NSArray and NSDictionary have recursive convenience methods• Reading from a file or URL:

• let tempDict2 = NSDictionary(contentsOf: URL) // uses the NSDictionary class initialiser

• Writing to a file or URL• func write(toFile path: String, atomically useAuxiliaryFile: Bool) -> Bool • func write(to url: URL, atomically: Bool) -> Bool

• Property lists can also be serialised from objects into a static format• Can be stored in the file system (in different formats) and read back later• Uses PropertyListSerialization class

// Reading Data.plist from the resource bundle let filePath = Bundle.main.path(forResource: "Data", ofType: “plist") let tempDict = NSDictionary(contentsOfFile: filePath!)

// Reading WrittenArray.plist from the application’s file directory let tempArray = NSArray(contentsOfFile: “WrittenArray.plist")

Note this example is accessing a file from the App Bundle - an actual part of the App that cannot be modified. You can read from but can’t write to the Bundle.

!13

Example: Writing an Array to Disk

let myArray = [NSDate.distantFuture, 5, "Hello World! 👾"] as NSArray myArray.write(toFile: “writtenArray.plist”, atomically: true)

<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><array>

<date>4001-01-01T00:00:00Z</date><integer>5</integer><string>Hello World! 👾</string>

</array></plist>

Create an array of three items - a date, a value and a string

Call the array’s write toFile method (courtesy of the NSArray type)

The resulting XML file is written in the application’s file space, to be read later

!14

JSON - JavaScript Object Notation• Open standard file format that uses human-readable text to transmit

data objects• Language independent - originally from Javascript• Supports:• Collections of name/value pairs - objects { string : value , string : value , ... }• Ordered list of values - arrays [ value , value , ... ]• Values can be strings, numbers, objects, arrays, “true”, “false”, “null”

• Elements can be nested

• JSONSerialization Class• Converts JSON data into Foundation objects• Converts Foundation objects into JSON data• see isValidJSONObject() for supported types

See http://json.org for full definition

!15

What does it look like?• The JSON structure naturally matches that of the Swift types:

• Dictionaries that contain key-value pairs

• Arrays or Sets that contain lists of values

• (and their bridged Foundation equivalents, NS…)

{ "title": "University of Liverpool Computer Science Programme List", "groups": [

{ "grouptitle": "Agents ART Group", "grouplabel": "agentArt", "members": [ "Katie", "Trevor", "Frans", "Paul D.", "Floriana", "Wiebe", "Dave J.", "Terry", "Valli"]},{ "grouptitle": "Complexity Theory and Algorithmics Group", "grouplabel": "ctag", "members": ["Leszek", "Irina", "Igor", "Prudence", "Michele"]},{ "grouptitle": "Logic and Computation Group", "grouplabel": "loco", "members": ["Michael", "Frank", "Clare", "Ullrich", "Boris", "Alexei", "Sven"]},{ "grouptitle": "Economics and Computation Group", "grouplabel": "ecco", "members": ["Piotr", "Rahul", "Martin"]}

]}

!16

Example using JSON{ "id": "questionnaire-1", "title": "COMP327 Questionnaire", "description": "This is a sample questionnaire", "questions": [ { "name": "question-1", "type": "single-option", "title": "Q 1", "question": "How much have you enjoyed COMP327 practicals so far?", "choices": [ { "label": "not at all", "value": 1 }, { "label": "a little", "value": 2 }, { "label": "somewhat", "value": 3 }, { "label": "quite a lot", "value": 4 }, { "label": "Very much", "value": 5 } ] }, { "name": "question-2", "type": "multi-option", "title": "Q 2", "question": "Which of these COMP327 lectures did you attend?", "choices": [ { "label": "Introduction to COMP327", "value": 1 }, { "label": "Introduction to Core Data", "value": 2 } ] } ]}

Questionnaire:{“id”: String,"title": String,"description": String,”questions": [Array]}

Question:{“name”: String,”type": String,"title": String,"question": String, “choices”:[Array] }

Choices:{“label”: String,”value": Int}

!17

Example using JSONvar theQuestionaire: NSDictionary? = nillet url = URL(string: "https://cgi.csc.liv.ac.uk/~phil/COMP327/questionnaire.json")!let task = URLSession.shared.dataTask(with: url) { (data, response, error) in if error != nil { print(error) } else { if let urlContent = data { do { let jsonResult = try JSONSerialization.jsonObject(with: urlContent, options: JSONSerialization.ReadingOptions.mutableContainers) as AnyObject theQuestionaire = (jsonResult as? NSDictionary) let questionnaireTitle = theQuestionaire!["title"] let questionsArray = theQuestionaire!["questions"] as! NSArray let currentQuestion = questionsArray[0] as! NSDictionary print(currentQuestion["question"] as! String) //print the first question "How much have you enjoyed COMP327 practicals so far?” } catch { print("====\nJSON processing Failed\n=====") } } }}task.resume()

!18

Questionnaire

ID, title, description, questions[ ]

Questions [ ]

[0] [1] [2] …

Questions[0]

name, type, title, question, choices[ ]

choices[ ]

[0] [1] …

choices[0]

label, value

!19

Encoding and Decoding in Swift 4

{"name": "Monalisa Octocat","email": "support@github.com","date": "2011-04-14T16:00:49Z"}

struct Author { let name: String let email: String let date: Date}

struct Author: Codable { let name: String let email: String let date: Date}

!20

Encoding and Decoding in Swift 4

let jsonData = """{"name": "Monalisa Octocat","email": "support@github.com","date": "2011-04-14T16:00:49Z"}""".data(using: .utf8)!

struct Author: Codable { let name: String let email: String let date: Date}

let decoder = JSONDecoder()decoder.dateDecodingStrategy = .iso8601let author = try decoder.decode(Author.self, from: jsonData)

print(author.name)

!21

Encoding and Decoding in Swift 4

{ "url": "https://api.github.com/.../6dcb09", "author": { "name": "Monalisa Octocat", "email": "support@github.com", "date": "2011-04-14T16:00:49Z" }, "message": "Fix all the bugs", “comment_count": 0,}

struct Commit: Codable { let url: URL struct Author: Codable { let name: String let email: String let date: Date } let author: Author let message: String let comment_count: Int}

let decoder = JSONDecoder()decoder.dateDecodingStrategy = .iso8601let commit = try decoder.decode(Commit.self, from: jsonData)

print(commit.author.name)print(commit.url)

let jsonData = """{ "url": "https://api.github.com/.../6dcb09", "author": { "name": "Monalisa Octocat", "email": "support@github.com", "date": "2011-04-14T16:00:49Z" }, "message": "Fix all the bugs", "comment_count": 0,}""".data(using: .utf8)!

!22

Example using JSON// Decodable structures for JSON convertingstruct Questionnaire : Decodable { let id : String? let title : String? let description : String? let questions : [Questions]?}

struct Questions : Decodable { let name : String? let type : String? let title : String? let question : String? let choices: [Choices]?}

struct Choices : Decodable { let label : String? let value : Int?} if let url = Bundle.main.url(forResource: fileName, withExtension: "json") { do { let data = try Data(contentsOf: url) let decoder = JSONDecoder() let questionnaire = try decoder.decode(Questionnaire.self, from: jsonData) print(questionnaire.questions![0].question!) //prints “How much have you enjoyed …” } catch { print("error:\(error)") } }

in Swift 4

let jsonResult = try JSONSerialization.jsonObject(with: urlContent, options: JSONSerialization.ReadingOptions.mutableContainers) as AnyObject theQuestionaire = (jsonResult as? NSDictionary) let questionnaireTitle = theQuestionaire!["title"] let questionsArray = theQuestionaire!["questions"] as! NSArray let currentQuestion = questionsArray[0] as! NSDictionary print(currentQuestion["question"] as! String)

!23

Archiving Objects (Serialisation)

• A serialisation of some object graph that can be saved to disk

• and then be loaded later

• This is what is used by nibs/xibs/Storyboard

• Use the Encodable and Decodable protocols

• Declares two methods that a class must implement so that instances can be encoded or decoded

• Encode an object for an archiver: encode(to:)

• Decode an object from an archive: init(from:)

in Swift 4

!24

Using Web Services

• A lot of data is “in the cloud”• data is available from some web server• data can then be sent back to a web server• A number of APIs available• Google, Flickr, Ebay, etc...• Typically exposed via RESTful services• Uses XML or JSON data formats

• Parsing XML• iOS provides some XML support - e.g. PropertyListDecoder handles a specific format of XML used in

Apple’s plist files.• Several open-source XML parsers available

• JSON is also used by Apple Push Notifications

!25

User Defaults and Settings Bundles• Often applications need to save a small number of settings• e.g. preferences or last used settings

• UserDefaults provides a “registry” for storing values• Generated on a per user / per application basis• Register the different settings• Determines the default values and types of the settings for first use, or if reset

UserDefaults.standard.set("Phil", forKey: "name") //save let nameObject = UserDefaults.standard.object(forKey: "name") //retrieve if let name = nameObject as? String { print(name) } let arr = [1,2,3,4] UserDefaults.standard.set(arr, forKey: "array") //save let arrayObject = UserDefaults.standard.object(forKey: "array") //retrieve if let array = arrayObject as? NSArray { print(array) }

!26

User Defaults and Settings Bundles

• UserDefaults can be set and retrieved at any point in the application• Access the defaults object, and treat as a dictionary

• Values are cached in the application and the OS periodically stores values to the defaults database• (Legacy code may refer to the synchronize() method - you should no longer use this. Let the OS do it)

var lastCell = UserDefaults.standard.object(forKey: "lastSelectedCell")

… //program makes some changes to lastCell here

UserDefaults.standard.set(lastCell, forKey: "lastSelectedCell")

Local Data Storage

Core Data

!28

Core Data• A schema-driven object graph management and persistence framework• Model objects can be saved to the file store...• .. and then retrieved later

• Specifically, Core Data• provides an infrastructure for managing all the changes to your model objects• Provides automatic support for undo and redo

• supports the management of a subset of data in memory at any time• Good for managing memory and keeping the footprint low

• uses a diagrammatic schema to describe your model objects and relationships• Supports default values and value-validation

• maintains disjoint sets of edits on your objects• Can allow the user to edit some of your objects, whilst displaying them unchanged elsewhere

!29

The Core Data Stack

• A collection of Core Data framework objects that access a persistent store• This could be a database, but doesn’t have to be.

• Core Data allows the developer to manage data at the top of this stack• abstracts away the storage mechanism• Allows the developer to focus on• Managed Objects (i.e. records from tables in databases)• Managed Object Context (i.e. work area which manages objects from a Core Data store

Managed Object Model

A collection entity descriptions

Managed Object Store

A collection of managed Objects

Persistent Object Store

A collection of object data

Persistent Store Coordinator

A collection of stores

Store File

!30

Managed Objects

• An object representation of a record from a table in a database• The model object (from the MVC pattern) managed by Core Data• The data used within the application• shapes, lines, groups of elements in a drawing program• artists, tracks, albums in a music database• people and departments in a HR application

• An instance of the NSManagedObject • or a NSManagedObject subclass

!31

Managed Object Contexts

• A context represents a single object space in an application• Responsible for managing a collection of Managed Objects• Managed objects form a group of related model objects that represent an internally consistent view of a data

store• e.g. records and their relationships

• Also responsible for:• the complete life-cycle management• validation of data• relationship maintenance• Undo and Redo of actions

• Instance of an NSManagedObjectContext• or an NSManagedObjectContext subclass

!32

Managed Object Contexts• Managed Objects exist within a Managed Object

Context• New managed objects are inserted into context• Existing records in a database are fetched into a context as

managed objects• Changes to records are kept in memory until they are

committed to the store by saving the context• Includes insertion or deletion of complete objects• Includes manipulation of property values

This managed object context contains two managed objects corresponding to two records in an external database.

Note that Nigel’s salary has been increased, but that the change has not been committed to the database.

Other records exist in the database, but there are no corresponding managed objects.

Managed Object Context

EmployeeName FredSalary 90 000

EmployeeName NigelSalary 60 000

EmployeeName SalaryFred 90 000Julie 97 000Nigel 50 000Tanya 56 000

Unsaved Data

Current Data

!33

Managed Object Model• A Managed Object model represents a schema that describes the

data (and hence the database)• And hence the managed objects in the application

• A model is a collection of entity description objects• Instances of NSEntityDescription• Describes an entity or table in a database, in terms of:• Its name• Name of the class used to describe the entity in the app• Its properties and attributes

Core Data It uses the model to

map between managed objects in your app to records

in the database

Managed ObjectName FredSalary 90 000entityDescription

Entity DescriptionName “Employee”

Managed Object Class NSManagedObjectAttribute name

Attribute salary

Database Table

EmployeeName SalaryFred 90 000Julie 97 000Nigel 50 000Tanya 56 000

!34

Using Core Data: The Basics• Core Data support can be included in several of the project templates• Select “Use Core Data” when creating the project

• Includes additional code within the app delegate implementation file• Manages files in the applications Documents Directory• Provides a persistent store coordinator for the app• Returns the managed object model for the app• Also provides the managed object context for the app

• Also includes a Core Data Model (.xcdatamodeld) file• Known as the managed object model

!35

Getting started with Core Data

• Any use of core data will need access to the Managed Object Context

• Get this from the app delegate

let appDelegate = UIApplication.shared.delegate as! AppDelegate

let context = appDelegate.persistentContainer.viewContext

let request = NSFetchRequest<NSFetchRequestResult>(entityName: “Users")

request.returnsObjectsAsFaults = false

do { let results = try context.fetch(request) if results.count > 0 { for result in results as! [NSManagedObject] { if let username = result.value(forKey: "username") as? String { print(username)

!36

Modelling your data• A data model is a collection of entity and property

description objects that describe the Managed Objects• This model can be described programmatically• Alternatively, the graphical modelling tool can be used

• Different editor styles can be selected• Entity and Attribute properties can be viewed by opening the right-most

pane (the Data Model Inspector)

• Entities can be added to the model• This should be an object of type NSManagedObject

• (see later)

• The entity name doesn’t have to be the same as class that will represent instances of this object

• Attributes can then be defined• complete with name and type

!37

Modelling your data• Here we have a simple data

model - with one entity “Users” and three attributes.

• We can add attributes and define their types. Types available are:• undefined, integer 16, integer

32, integer 64, Decimal, Double, Float, String, Boolean, Date, Binary Data, UUID, URI and Transformable.

• We can also view and edit / interact with the data model graphically

!38

Modelling your data

!39

Fetching objects from the store

Database Table

EmployeeName SalaryFred 90 000Julie 97 000Nigel 50 000Tanya 56 000

Resulting Array

Managed ObjectName JulieSalary 97 000entityDescription

Managed ObjectName FredSalary 90 000entityDescription

Managed Object Store

A collection of managed Objects

Persistent Object Store

A collection of object data

Persistent Store Coordinator

A collection of stores

Fetch Request

Entity (table name) Employee

Predicate (optional) salary > 60 000

SortOrderings (optional) name:ascending alphabetical

To fetch objects you need a managed object context and a fetch request. Often, objects not directly fetched, but that are relevant to it (such as related objects) will also be retrieved.

!40

Creating and Executing a Request

// Create the request let request = NSFetchRequest<NSFetchRequestResult>(entityName: “Events”) request.returnsObjectsAsFaults = false

// Set the sort descriptor request.sortDescriptors?.append(NSSortDescriptor(key: "creationDate", ascending: true))

do { var results = try context.fetch(request) // do stuff here

} catch { print("Couldn't fetch results") }

Create the fetch request, and identify the entity. Provide the name of the entity (“Events”) to the managed context.

Set the sort descriptor; otherwise the order the objects are returned will be undefined. As multiple sort orderings may be specified, these are given in an array.

Execute the request - in this case the results are placed in a mutable array, as the results may be modified within our application. Each element of the array is an NSManagedObject

!41

Deleting Managed Objects

override func tableView(_ tableView: UITableView, commit editingStyle: UITableViewCellEditingStyle, forRowAt indexPath: IndexPath) {

if editingStyle == UITableViewCellEditingStyle.delete { // Delete the managed object at the given index path let itemToRemove = eventsArray[indexPath.row] context.delete(itemToRemove)

// Update the array and table view eventsArray.remove(at: indexPath.row) tableView.deleteRows(at: [indexPath], with: UITableViewRowAnimation.automatic) table.reloadData() //should not be needed - OS should update table

// Commit the change do { try context.save() } catch { print(“changes failed") }

....

In this case, we have received an edit request from the UITableView to delete an entry in the table, that corresponds to an entry we want to delete from our store. Identify the NSManagedObject which is to be deleted. In this case, it is held within the eventsArray, which is also used to fill in entries in the table.

In this app, we also need to delete the entry from our array, and from the table view

Save the context, to push the changes down to the persistent store!

!42

let toInsert = NSEntityDescription.insertNewObject(forEntityName: "ArtWork", into: context!) as! ArtWorktoInsert.artist = artInfo.artisttoInsert.fileName = artInfo.fileNametoInsert.id = Int32(artInfo.id!)!toInsert.info = artInfo.InformationtoInsert.lastModified = stringToDate(artInfo.lastModified!)toInsert.latitude = Double(artInfo.lat!)!toInsert.longtitude = Double(artInfo.long!)!toInsert.location = artInfo.locationtoInsert.locationNotes = artInfo.locationNotestoInsert.title = artInfo.titletoInsert.yearOfWork = artInfo.yearOfWorktoInsert.enabled = Int(artInfo.enabled!)! == 1

!43

import Foundationimport CoreData

@objc(CoreArtwork)public class CoreArtwork: NSManagedObject {

}

import Foundationimport CoreDataextension CoreArtwork { @nonobjc public class func fetchRequest() -> NSFetchRequest<CoreArtwork> { return NSFetchRequest<CoreArtwork>(entityName: "CoreArtwork") } @NSManaged public var artist: String? @NSManaged public var enabled: Bool? @NSManaged public var fileName: String? @NSManaged public var id: Int32? @NSManaged public var info: String? @NSManaged public var lastModified: String? @NSManaged public var latitude: Double @NSManaged public var location: String? @NSManaged public var locationNotes: String? @NSManaged public var longitude: Double? @NSManaged public var title: String? @NSManaged public var yearOfWork: String? }

CoreArtwork+CoreDataClass.swift

CoreArtwork+CoreDataProperties.swift

!44

Modelling your data

!45

Modelling your data

Questions?

Data Persistence and Core Data