using mobile components to fool-proof the ocr experience
TRANSCRIPT
U S I N G M O B I L E C O M P O N E N T S T O F O O L - P R O O F T H E O C R E X P E R I E N C E
K R I S T I N A T H A I I O S S O F T W A R E E N G I N E E R
@ K R I S T I N A T H A I
K R I S T I N A T H A I
M O B I L E C O M P O N E N T S
M i n t P a y r o l lQ u i c k B o o k s
M O B I L E C O M P O N E N T S
I m a g e C a p t u r e
P I NP r o d u c t S u r v e y
D o c u m e n t R e v i e w
F O O L - P R O O F I N G O C R
W H AT I S O C R ?
E N D - T O - E N D O C R F L O W
I M A G E C A P T U R E
C O M P O N E N T
D O C U M E N T R E V I E W
C O M P O N E N T
D ATA E X T R A C T I O N
E N G I N E
D E M O
T H E I M A G E C A P T U R E E X P E R I E N C E
I M A G E C A P T U R E
C O M P O N E N T
D ATA E X T R A C T I O N
E N G I N E
D O C U M E N T R E V I E W
C O M P O N E N T
P e o p l e t a k e t e r r i b l e p i c t u r e s .
T I LT / S K E W
N O T E N O U G H L I G H T I N G
B L U R R I N E S S / S H A K I N G
E D G E D E T E C T I O N
D O C U M E N T D E T E C T I O N
self.documentDetector = [CIDetector detectorOfType:CIDetectorTypeRectangle context:nil options:nil];
NSArray *rectFeaturesArray =[_documentDetector featuresInImage:ciImage
:@{CIDetectorAspectRatio:@2.0}];
NSDictionary *documentCorners = [self getDocumentCornersForFeatures:rectFeaturesArray forVideoBox:clap srcImage:ciImage];
D E T E C T C O R N E R Sfor (CIRectangleFeature *f in rectFeaturesArray) { CGPoint markerPos = CGPointZero; //topLeft markerPos = f.topLeft; //flip the coordinates markerPos = CGPointApplyAffineTransform(markerPos, flipTx); //apply correction-‐for-‐orientation-‐transform markerPos = CGPointApplyAffineTransform(markerPos, correctionTransform); //scale to screen-‐coordinates markerPos = CGPointMake(markerPos.x * xScale, markerPos.y * yScale); topLeft = markerPos; … … …
}
Users follow instructions in creative ways…
D ATA F R O M T H E D ATA E X T R A C T I O N E N G I N E
I M A G E C A P T U R E
C O M P O N E N T
D ATA E X T R A C T I O N
E N G I N E
D O C U M E N T R E V I E W
C O M P O N E N T
<employeeSsn> <ns2:confidence>0.5</ns2:confidence> <ns2:bottom>75</ns2:bottom> <ns2:left>289</ns2:left> <ns2:right>392</ns2:right> <ns2:top>60</ns2:top> </employeeSsn> <employerEin> <ns2:confidence>1.0</ns2:confidence> <ns2:bottom>122</ns2:bottom> <ns2:left>61</ns2:left> <ns2:right>157</ns2:right> <ns2:top>108</ns2:top> </employerEin>
C O N F I D E N C E A N D C O O R D I N AT E S
<?xml version="1.0" encoding="UTF-‐8" standalone="yes"?> <FdpW2 xmlns="http://schema.intuit.com/platform/document/semantic/v1" xmlns:ns2="http://schema.intuit.com/platform/document/common/v1"> <taxYear>2013</taxYear> <employeeSsn>333-‐44-‐5555</employeeSsn> <employerEin>00-‐7904153</employerEin> <EmployerName> <ns2:BusinessNameLine1>Rock Castle Construction, Inc.</ns2:BusinessNameLine1> <ns2:BusinessNameLine2>Construction and Estimates</ns2:BusinessNameLine2> </EmployerName> <EmployerUSAddress> <ns2:AddressLine1>1735 County Road</ns2:AddressLine1> <ns2:City>Bayshore</ns2:City> <ns2:State>CA</ns2:State> <ns2:ZIPCode>94326</ns2:ZIPCode> </EmployerUSAddress>
D O C U M E N T D ATA
B U I L D I N G T H E R E V I E W C O M P O N E N T
I M A G E C A P T U R E
C O M P O N E N T
D ATA E X T R A C T I O N
E N G I N E
D O C U M E N T R E V I E W
C O M P O N E N T
C O N F I D E N C E
<employeeSsn> <ns2:confidence>0.5</ns2:confidence> </employeeSsn>
<employeeSsn> <ns2:confidence>1.0</ns2:confidence> </employeeSsn>
C O N F I D E N C E
I M A G E P O S I T I O N Z O O M I N G
<payerName> <ns2:bottom>75</ns2:bottom> <ns2:left>289</ns2:left> <ns2:right>392</ns2:right> <ns2:top>60</ns2:top> </payerName>
M A P P I N G T H E C O N T E N T T O T H E U I
a Employee’s social security number
123-45-6789
Box ID
Box Title
Box Data
G E N E R I C S I N G L E L I N E U I{ "DrHeaderId": "a", "DrBoxCellType": 1, "DrBoxCellReuseId": "DRW2SingleLineCell", "DrBoxEditCellReuseId": "DRW2SingleLineCell_Edit", "DrHeaderTitle": "Employee's social security number", "DrBoxUserApproved": false, "DrUIElements": { "DrValue1": { "DrXPath": "//ns1:FdpW2/ns1:employeeSsn", "DrIsMandatory": true, "DrRegex": "^\\d{3}-‐\\d{2}-‐\\d{4}$|^\\d{9}$", "DrRegexFailMessage": "Please enter a valid SSN (9 numbers)" } } }
-‐(BOOL)validateInputInTextField:(UITextField*)textField {
BOOL isInputValid = [super validateInputInTextField:textField];
NSDictionary* validationDictForThisTF =
self.inputValidationDictionaryForAllTextFields[@(textField.tag)];
if (!isInputValid) { self.valueTFErrorLabel.text = validationDictForThisTF[kDrRegexFailMessage];
} return isInputValid; }
P R O S A N D C O N S
P R O S A N D C O N S O F T H E G E N E R I C M O D E L
Build once, support many
Expertise Open source model
Longer development
time
?
Maintaining generic model
I N T U I T ’ S O P E N S O U R C E M O B I L E L I B R A R I E S
Makes it easy to build advanced
custom animations on iOS
An Objective-C and Swift
collection for iOS and OS X that stores objects grouped into
sections
Makes it easy to get the device's current
location on iOS
