STUDY DESIGN ARTICLE

Implementation of a SenseMaker® research project among Syrian refugees inLebanonNour Bakhache a, Saja Michaelb, Sophie Roupetzc, Stephanie Garbernd, Harveen Bergquiste,f,Colleen Davisona,g,h and Susan Bartels a,g,h

aDepartment of Public Health Sciences, Queen’s University, Kingston, ON, Canada; bSexualities and Sexual and Reproductive Health andRights, ABAAD Resource Center for Gender Equality, Beirut, Lebanon; cUniversity of Leipzig, Department of Psychology, Leipzig,Germany; dDepartment of Emergency Medicine, Beth Israel Deaconess Milton, Milton, MA, USA; eBrigham and Women’s Hospital,Department of Emergency Medicine, Boston, MA, USA; fDepartment of Emergency Medicine, Harvard Medical School, Boston, MA, USA;gDepartment of Emergency Medicine, Queen’s University, Kingston, ON, Canada; hKingston General Hospital Research Institute,Kingston, ON, Canada

ABSTRACTThe Syrian conflict has displaced over 1.2 million Syrians into Lebanon. As a result ofdisplacement, some Syrian families are turning to child marriage as a coping mechanism.The prevalence of early marriage has reportedly increased and the average age of marriagedecreased during the crisis. The aim of the project was to understand the underlying factorscontributing to child marriage among Syrian refugees in Lebanon using Cognitive Edge’sSenseMaker®. This manuscript explores the process of implementing this novel research toolin a humanitarian setting. Twelve interviewers conducted SenseMaker® interviews withmarried and unmarried Syrian girls, Syrian parents, as well as married and unmarried men.Participants were asked to share a story about the lives of Syrian girls in Lebanon and to self-interpret the narratives by answering follow-up questions in relation to the story provided.Data collection occurred across three locations: Beirut, Beqaa, and Tripoli. In total 1422narratives from 1346 unique participants were collected over 7 weeks. Data collection usingSenseMaker® was efficient, capable of electronically capturing a large volume of quantitativeand qualitative data. SenseMaker® limitations from a research perspective include lack of skiplogic and inability to adjust font size on the iOS app. SenseMaker® was an efficient mixedmethods data collection tool that was well received by participants in a refugee setting inLebanon. The utility of SenseMaker® for research could be improved by adding skip logic andby being able to adjust font size on the iOS app.

ARTICLE HISTORYReceived 13 January 2017Accepted 27 July 2017

RESPONSIBLE EDITORDiana Gil-Gonzalez,University of Alicante,Spain

KEYWORDSChild marriage; earlymarriage; SenseMaker®;Syria; tablet

Background

Since 2011, the Syrian conflict has resulted inmore than250,000 deaths, two million people wounded, and 11million people displaced [1]. The scope of the crisis hasresulted in substantial social and economic challengesfor Syrian refugees, including the 1.2 million Syrians [2]who have been displaced into Lebanon [3].Many Syrianrefugees in Lebanon are living in precarious circum-stances in informal settlements with few legal rights [4].In 2014, approximately 70% of the Syrian refugeesregistered with the United Nations HighCommissioner for Refugees (UNHCR) in Lebanonwere considered poor with increasing debt and a grow-ing reliance on food aid [4]. Furthermore, half of therefugees are children and most face challenges withenrolling in educational institutions [5,6]. Over250,000 school-aged Syrian children are currently outof school [5,6].

Child marriage is defined as ‘any formal or informalunion where one or both parties is below the age of 18’[7]. Child marriage is widely recognized as a human

rights violation, as a form of gender-based violence(GBV), as a violation of the Convention on theElimination of All Forms of Discrimination againstWomen [8], as well as a violation of the Conventionon the Rights of the Child [9–12]. Although child mar-riage occurred in some areas within Syria prior to thewar, with 13.3% of girls under the age of 18 reportedlymarried in 2006, forced displacement has resulted in achange in child marriage trends [13]. More specifically,the prevalence of married girls under 18 years of age hasreportedly increased while the age at marriage hasreportedly decreased over the past 5 years [14–17].

As a result of the difficulties associated with dis-placement, child marriage has been identified as acoping strategy among Syrian refugee households[5,18]. Financial hardship and continuous concernsabout protecting girls from sexual violence lead somefamilies to marry their daughters early as a way tosecure their futures financially and to protect themfrom sexual violence [18]. Child brides are at highrisk for pregnancy and labor complications including

CONTACT Susan Bartels susanabartels@gmail.com Department of Emergency Medicine, Queen’s University, Kingston General Hospital, 76Stuart Street, Empire 3, Kingston, ON, K7L 4V7, Canada

GLOBAL HEALTH ACTION, 2017VOL. 10, 1362792https://doi.org/10.1080/16549716.2017.1362792

© 2017 The Author(s). Published by Informa UK Limited, trading as Taylor & Francis Group.This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0/), which permitsunrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

preterm labor, obstructed or prolonged labor, andobstetrical fistulas [10,11]. Infants born to youngmothers are also at greater risk of low birth weight,neonatal death, and stillbirth [7,10,12,19].Additionally, experiences of physical, psychological,and sexual violence are more prevalent among girlswho marry as children than among those who enterinto marriage as consenting adults [7,10,12].

Our ultimate aim is to reduce the rates of childmarriage among Syrian refugees and to do so we useda novel research tool, Cognitive Edge’s SenseMaker®, tocollect mixed methods data regarding the underlyingfactors that contribute to childmarriage with the goal ofidentifying local interventions and community actionsto address the issue. SenseMaker®, available as an app forsmartphones/tablets and as a browser-based softwareprogram, allows the collection of large quantities ofdata, in a short period of time. Qualitative data is col-lected as brief narratives documented in the form of textor audio recordings. Using a tablet or smartphone,participants share a personal story in response to theirchoice of three open-ended prompting questions, andthen self-interpret their story by answering a series offollow-up questions as they relate to the events in thestory. SenseMaker® allows for the collection of hundredsto thousands of self-interpreted narratives, providingresults that have statistical power while still being richin context. After initial analysis is complete,SenseMaker® data can be presented back to communitymembers through focus group discussions to gatherparticipants’ interpretation of and reaction to theresults. These facilitated focus group discussions canalso serve as a medium to identify feasible and culturallyappropriate interventions that address child marriage.

We choose SenseMaker® for this research projectbecause it offered a mixed quantitative/qualitativeapproach with digital data entry that we believedwould be more efficient. Additionally, given the sen-sitivities of discussing child marriage, we hypothe-sized that SenseMaker®’s open-ended questionsmight provide more revealing and honest responsesthan would be possible with more directed question-ing. Furthermore, because the SenseMaker® self-inter-pretation questions (triads, dyads and stones) allowfor more varied responses than typical categoricalquestions, we believed it might provide a morenuanced understanding of the complexities around

child marriage. And finally, we were interested inreducing researcher bias which was possible withSenseMaker® since the qualitative data is self-inter-preted by the participant.

Thus far, documentation of SenseMaker® useamong refugee populations and in humanitarian set-tings is limited [20]. Therefore, the purpose of thismanuscript is to address this gap by providing ascholarly reflection on the process of implementingthis novel research tool in a humanitarian setting andexamining the challenges and opportunities asso-ciated with doing so. SenseMaker® data analysis willnot be reviewed here and the quantitative/qualitativeresearch results will be presented separately.

Methods

Sampling

To capture varied perceptions of the life of Syrian girls, awide range of participants were targeted including mar-ried and unmarried Syrian girls, Syrian mothers andfathers, the husbands of Syrian child brides, unmarriedmen who may choose to take a child bride, and com-munity leaders such as teachers, health care providers,religious leaders, members of the NGO community, etc.The percentage of Syrian refugees in each of the threetarget locations was used to calculate a purposefulweighted sample size for each geographic area of focus.Although a formal sample size calculation is not appro-priate for this mixed qualitative/quantitative data collec-tion, Cognitive Edge recommends a minimum of 300stories per location. This heuristic allows one to disag-gregate the data based on subgroups while still main-taining at least 50 stories per subgroup. A minimum of300 stories per site also provides enough data points toallow patterns to emerge on visual scanning of the triads,dyads and stones. In addition, we aimed to interview 45community leaders across the three regions. A conveni-ence sample was used and participants were recruited ina targeted fashion to meet thresholds in each participantsubgroup per location. Table 1 shows the number ofparticipants interviewed in each subgroup.

Survey development

The SenseMaker® survey was drafted by team mem-bers with collective expertise on humanitarian crises

Table 1. Number of participants in each subgroup per location.Beirut Beqaa Valley Tripoli Subgroup total Overall total for group

Married Syrian girls 58 72 67 197 427Unmarried Syrian girls 78 70 82 230Syrian mothers 83 88 74 245 443Syrian fathers 63 83 52 198Men married to Syrian girls 70 66 92 228 496Unmarried men 92 79 97 268Community leaders 23 13 20 56 56Total for location 467 471 484 - 1422

2 N. BAKHACHE ET AL.

due to armed conflict, child marriage, and surveydesign. Questions were reviewed iteratively by anexperienced SenseMaker® consultant as well as byteam members from the ABAAD Resource Centerfor Gender Equality in Lebanon and revisions weremade accordingly. The survey began with three open-

ended prompting questions, asking the participant toshare an anonymous story about the experiences ofSyrian girls in Lebanon (Figure 1(a)). After theirstories were recorded, participants were asked toself-interpret their own narratives by responding tothree different categories of SenseMaker® questions:

Figure 1. Categories of SenseMaker® questions. (a) Story prompts: Participants choose one of several prompts about which toshare a narrative. (b) Triads: Participants move a marker between the three options in each triad to provide a more nuancedanswer. (c) Dyads: Participants move a marker between two options using a slider. (d) Stones: Participants choose stonesrelevant to their shared story and plot them on the grid.

GLOBAL HEALTH ACTION 3

triads (Figure 1(b)), dyads (Figure 1(c)), and stones(Figure 1(d)). The survey ended with a series ofmultiple-choice demographic questions in additionto questions used to contextualize the events sharedin the story (e.g. emotional tone, the frequency ofevents). The survey was drafted in English, translatedto Arabic by a Syrian translator, and then back trans-lated to English to check for accuracy. Translationdiscrepancies were resolved by consensus. Once fina-lized, the Arabic survey was uploaded to theCognitive Edge secure server and reviewed for errorsprior to initiation of pilot data collection. The open-access SenseMaker® app was installed on each of thestudy’s tablets (iPad Mini 4) and the survey wasdownloaded into each app.

Piloting

In May 2016, the SenseMaker® survey was pilotedamong 28 participants recruited through theABAAD Resource Center for Gender Equality, acrossthree target locations: Beirut (N = 10), Beqaa (N = 8),and Tripoli (N = 10). Two researchers (NB, CD, SM,or SB) were present at each interview, one to conductthe interview in Arabic, and the other to take noteson the interview process and to record any identifiedproblems. At the end of the interview, participantswere asked questions regarding the survey includingopinion on length, comfort using the iPad, and diffi-culty responding to the different categories ofSenseMaker® questions. The major issues identifiedin the pilot were difficulty with understanding thestones questions, discomfort with voice recording bysome participants, and a tendency to answer thetriads in the extreme margins. Survey revisions weremade based on the pilot and the interviewers receiveddedicated instruction on how to deal with such issuesif they occurred during the data collection.

Training

Twelve interviewers were identified by the ABAADResource Center for Gender Equality based on theirplace of residence (Beirut, Tripoli, or Beqaa), gender,and nationality. There were four interviewers for eachof the three locations with an equal number of malesand females. To adequately capture the perspectivesof both Lebanese and Syrian men, half of the six maleinterviewers were Syrian and half were Lebanese.Thus, each location’s interview team consisted oftwo Syrian females, one Syrian male, and oneLebanese male. All interviewers had completed sec-ondary school, with the majority having completed orin the process of completing a university degree.Furthermore, a majority of the interviewers werealready known to staff at the ABAAD ResourceCenter for Gender Equality through previous

programs. Interviewer training was conducted duringa four-day workshop in Beirut immediately prior toinitiation of data collection in July 2016. The trainingwas led by three of the study researchers (SG, SM andSB). Simultaneous translation with a professionalArabic-English interpreter was provided for all train-ing activities. Day one of training provided an intro-duction to the study, an overview of SenseMaker®, adiscussion around research ethics, and basic trainingon the use of an iPad. The second and third daysconsisted of practicing how to obtain informed con-sent, a detailed review of the SenseMaker® survey,multiple role-playing sessions, and training on themanagement of adverse events and program referrals.The last day of training consisted of reviewing dataupload procedures and relevant troubleshooting, anoverview of logistics including completing study led-gers, distribution of all study materials and a trainingdebrief/feedback session.

All interviewers were trained on the GBV standardoperating procedures for referral, as a mitigationmeasure for adverse events. Additionally, each parti-cipant was offered an ABAAD referral card that con-tained a hotline number for support and forinformation on how to access holistic GBV services.Additionally, interviewers were well informed tobring any adverse events to the immediate attentionof their team lead. No adverse events were reportedduring the study period.

At the end of training, each interviewer receivedan iPad loaded with the SenseMaker® app and ourchild marriage survey, print copies of the trainingmaterials, print copies of data collection ledgers,referral cards and contact lists of study teammembers.

Three ABAAD field program staff members wereassigned as ‘team leads’ for each of the geographicregions to serve as local points-of-contact for assis-tance with identifying study participants, facilitatinglogistics, troubleshooting technological problems andensuring that data collection progressed as planned.Each of these team leaders reported to an ABAADproject manager and received brief individual train-ing from the research team prior to data collection.

Data collection

Data collection was conducted over a period of7 weeks with close oversight by the team leads ineach location in addition to the ABAAD projectmanager and two graduate students based inBeirut. Interviews were primarily conducted inthe participants’ homes, at markets, at cafes, atsupport groups, or other points of contact withservice providers. Shared stories were audio-recorded on iPads and participants then respondedto a series of questions that provided quantitative

4 N. BAKHACHE ET AL.

data on the narrators’ interpretation of the experi-ences shared in the narrative. The project managerhandled logistics, held regular phone meetings witheach team lead and was immediately available viacell phone and WhatsApp to all interviewers ensur-ing that any identified issues were promptlyaddressed.

Several quality measures were regularly trackedduring data collection. For example, location of theinterview was to be recorded through GPS to ensurethat participants were being sampled from diversegeographic locations. The length of each interviewwas also recorded with any interviews less than10 minutes being flagged for review and the asso-ciated audio files checked against the participants’responses. Furthermore, each interviewer was askedto report weekly on the number of unique partici-pants interviewed in each subgroup using an onlinesurvey. Self-reported results were then compared toweekly data summaries from the Cognitive Edge ser-ver. In addition, the database was reviewed weekly toensure there were no duplicate recordings.

The study protocol was reviewed and approved bythe ABAAD Resource Centre for Gender Equality.ABAAD team members also provided extensiveinput when planning study implementation includinglogistics and security considerations.

Results

Facilitation of research

EfficiencySenseMaker® proved to be an efficient data collectiontool with 12 interviewers collecting 1422 self-inter-preted stories (from 1346 unique individuals) over a7 week period. SenseMaker® has several unique fea-tures that contributed to this efficiency: (a) typicallythe stories are relatively brief anecdotes and thereforethey can be collected quickly, particularly if they areaudio recorded; (b) the triads, dyads, stones andmultiple choice questions allow the narrator to inter-pret their own story and through this self-interpreta-tion, SenseMaker® generates quantitative data that isavailable almost immediately after upload to the ser-ver; (c) because the narratives have been interpretedby the participant to promptly generate quantitativedata, it is not necessary to transcribe and translate allthe stories thereby increasing efficiency and decreas-ing costs; and (d) SenseMaker® facilitated close mon-itoring and prompt feedback to interviewersregarding number of participants interviewed ineach subgroup per location, quality of data, numberof stories collected by each interviewer, etc. Based onour prior research experience, it would have beenvery difficult, if not impossible, to collect mixed-

methods data on a sensitive topic from 1346 peopleacross three locations in a 7-week period.

Mixed methods researchSenseMaker® provided rich mixed methods resultssince the accompanying narratives help to contextua-lize the quantitative data generated by the self-inter-pretation questions (triads, dyads and stones[Figure 1]). Using a large number of quantitativedata points allows harvesting the ‘wisdom of thecrowds’ and facilitates pattern recognition. The datais also connected to demographic information and sodifferences in subgroups can be further explored withsubstantial sample sizes. The qualitative stories areavailable to complement and provide depth to thequantitative data. This mixed methods approachallows for triangulation of qualitative and quantitativedata across various participant subgroups and pro-vides both breadth and depth of exploration.

Sensitive topicsSome of the shared Syrian girls’ experiences were of asensitive nature. Direct questions regarding a delicateor polarizing issue such as child marriage could leadparticipants to answer in a way they feel the inter-viewer would agree with, thus introducing socialdesirability bias and potentially limiting the accuracyof the data. In this SenseMaker® survey, child mar-riage was not specifically introduced as a topic andthere were no direct questions on child marriage.Despite this lack of direct questioning on the issue,23% of the shared stories were recorded as beingabout child marriage (331 of 1423) and another 17%of the shared stories were recorded as mentioningchild marriage (247 of 1423).

CostsImplementing a SenseMaker® research project hasunique associated costs, such as obtaining a licenseto use the software as well as fees for the upload andstorage of data on the secure Cognitive Edge server.In this study, approximately 10% of the budget wasdedicated to the use of SenseMaker® although thisincludes full analysis costs (approximately 5% forlicense, set-up and hosting and 5% for analysis). Inaddition to direct SenseMaker® costs, 12 iPads werepurchased for data collection. Although there wereless expensive tablets available, the SenseMaker® appfunctions most reliably on an iOS platform, leadingus to invest in more these expensive devices. TheiPads will remain the property of the study team foruse in future research.

Despite some SenseMaker®-specific expenses, webelieve that SenseMaker® was more cost efficientoverall. The collection of 1422 self-interpreted storiesusing traditional research approaches would have

GLOBAL HEALTH ACTION 5

taken longer thus requiring additional research assis-tant days as well as additional travel-related costs.Furthermore, the costs of uploading and storingdata on the Cognitive Edge server were less than thecosts that would have been required to have aresearch assistant digitally enter all the data. Directdigital entry also offered the advantage of having thedata promptly available for analysis and eliminateddata entry errors.

Technology

TabletsSenseMaker® is a tablet-based data collection tool andso reliance on technology was extensive in this study.Despite working in a humanitarian setting, this use oftechnology presented very few challenges. Beinghighly educated and comfortable with everyday useof technology, the interviewers were comfortable withusing, charging and caring for the iPads as well aswith accessing the Internet to upload data. We usedthe iOS SenseMaker® app in this study althoughAndroid and browser-based versions are alsoavailable.

GPSLocation tracking was inconsistent in the currentstudy with GPS coordinates available for only 27%of interviews. Because the iPads used did not havecellular capability, they lacked internal GPS chips.Since Bluetooth and Wi-Fi were typically turned offto preserve battery power while interviewing in thefield, without the internal GPS chip, GPS satellitecoordinates could not be generated for a majority ofthe interviews. To improve on GPS tracking in thefuture, we would recommend keeping Bluetooth and/or Wi-Fi turned on to the extent possible given con-siderations of battery life. Alternatively, an externalGPS tracking device could be plugged into the iPads,which would facilitate GPS tracking even in theabsence of an internal GPS chip, Bluetooth capabilityor Wi-Fi access.

WhatsAppDuring data collection, questions, concerns andupdates were regularly communicated via the mobilephone messaging app ‘WhatsApp’ [21]. Interviewers,team leads, and the project manager expressed apreference for WhatsApp as a means of communica-tion, and so groups were created for the larger teamof 12 as well as for each regional team. The ABAADproject manager and team leads often sent voicemessages via WhatsApp. Correspondence viaWhatsApp was an important asset for maintaininggood team communication, building cohesivenessamong the group, ensuring timely reporting and

providing support to various levels of the researchteam when issues arose.

Voice recordingThe audio quality of recorded narratives was good toexcellent regardless of location and backgroundnoise. If the participant demonstrated discomfortwith recording the story in his/her own voice, theinterviewer listened to the participants’ story in fulland the recounted it in his/her own voice for thepurpose of recording (clearly stating at the beginningthat the research assistant was recording the story onbehalf of the participant). It is not possible to knowhow often the interviewer recorded the story onbehalf of the participant without listening to all1422 stories, which has not been done to date.

SenseMaker® programmingThe SenseMaker® application itself had a number ofprogramming limitations. First, the survey font sizecould not be altered, resulting in occasional instancesof very small text that proved difficult for someparticipants to read. Second, use of Arabic punctua-tion caused some unanticipated challenges when datawas extracted from SenseMaker®. For instance, Arabiccommas were interpreted as delimiters in the extrac-tion process, causing the exported text to shift andmaking it incomprehensible. This required the inter-view team to audio record the stories and to avoid theuse of Arabic punctuation when they were typingresponses. And third, SenseMaker® cannot currentlyperform skip logic, which results in some participantshaving to answer questions unnecessarily by choosinga ‘non-applicable’ option.

Monitoring

At the end of each week, summary statistics from thesurvey were downloaded from the Cognitive Edgeserver and presented in Tableau, an interactive datavisualization software program [22]. Each weeklysummary included the cumulative number of storiescollected from each of the participant groups in eachof the three geographic locations, the cumulativenumber of interviews conducted by each interviewer,the cumulative number of stories about child mar-riage, length of each recorded interview and a link toallow quality control checks on the audio files.Review of the summary also allowed for easy identi-fication of duplicates in the interviews, which some-times occurred if the Internet signal was weakresulting in more than one attempt to upload thedata. The final report on number of participants persubgroup and by location is provided in Table 1.

In addition to the monitoring components alreadymentioned, team leads were also responsible for pro-viding reminders and following up with their

6 N. BAKHACHE ET AL.

interviewers about whether process measures werebeing followed on a daily basis and whether inter-views were being conducted as per the training pro-tocol. Team leads also held regional meetingsapproximately every 2 weeks, were responsible foroverseeing sampling to ensure that participants wereincluded from geographically diverse areas, andensured that the iPads received maintenance andcare including charging.

SenseMaker® survey

The SenseMaker® survey was translated to the Syriandialect of Arabic using language that was easy to readand understand, to account for varying levels offamiliarity with formal Arabic particularly amongthe adolescent girls who had sometimes not com-pleted primary school. Of the three types ofSenseMaker® questions, the stones (Figure 1(c))proved to be the most difficult for participants. Forexample, many participants would use all the stonesregardless of whether the stone was relevant to theirstory. Furthermore, participants often plotted thestone on only one axis (either the X-axis or theY-axis) and ignored the other axis. These challengeswere recognized in the pilot and interviewers weretherefore trained to use a three-step approach inguiding participants through the stone questions.Step one involved deciding whether a given stonewas relevant to the story shared. Step two involvedasking the participant to plot the stone along theX-axis and step three required the participant tothen plot the stone according to the Y-axis, determin-ing its final coordinates. This stepwise approachfacilitated understanding of this complex style ofquestion.

Another common issue was the participants’ ten-dency to choose extreme responses when answeringthe triads, dyads, and stones. Instead of the typicalcategorical responses that are most common in tradi-tional quantitative surveys, SenseMaker® is designedto provide a more nuanced answer, and a tendencyamong participants to choose extremes limits thatcapability. Proneness to extremes was identified dur-ing piloting, and so interviewers were trained toexplain to participants that having the markerbetween the different options is possible, and wouldallow them to give more complex answers.

In total, 59% of participants (840 of 1422) choose asingle story prompt with distribution between theother two story prompts being more even (24% ver-sus 17%). Although it appears that a single promptresonated with a majority of participants, it is impor-tant to provide alternatives to increase the likelihoodthat at least one prompt will trigger memory of arelevant story for all participants.

Discussion

We found SenseMaker® to be an efficient method ofcollecting mixed quantitative/qualitative data, facili-tating the capture of 1422 self-interpreted narrativesfrom 1346 unique participants by 12 interviewersover a 7 week period in Lebanon. Data was collectedacross three locations in a humanitarian setting.

Several SenseMaker® attributes contribute to itsefficiency. First, rather than long, comprehensivetranscripts as would be collected in more traditionalqualitative research, the narratives captured bySenseMaker® are usually brief anecdotes, which inthis case were audio recorded. This allows the inter-views to be conducted more quickly with each lastingapproximately 25–30 minutes. Second, through theuse of SenseMaker®-specific questions (triads, dyads,and stones) as well as multiple choice questions,participants interpret their own narratives on thetablet, which the SenseMaker® app then uses to auto-matically generate quantitative data that is availablealmost immediately after uploading to the CognitiveEdge server. This eliminates the need to transcribe/translate transcripts and to code the data for identi-fication of patterns and emerging themes, which isoften very labor- and time-intensive. It also offers theadditional advantage of reducing researcher biasesthat are inherent when investigators interpret thenarratives of others. Based on data patterns identifiedin the triads, dyads, and stones, a selection of narra-tives can be transcribed/translated to contextualizethe quantitative data but this is typically a relativelysmall subset of stories and allows the research team tosave transcription/translation costs in addition tosaving time. A final attribute contributing toSenseMaker®’s efficiency is that it facilitated closemonitoring and prompt feedback by collecting theinterviewer identification number, the participantsubgroup, as well as the date and location of theinterview. Because the data is captured digitally andis available promptly after upload to the CognitiveEdge server, this feedback can be provided to theresearch team in a timely fashion – daily if desired,although we found weekly to be sufficient for thepurposes of this study.

Although it proved efficient, the SenseMaker® apphas a number of limitations that might hinder itsmore widespread use for research purposes. Forinstance, within SenseMaker® the fixed font size wassometimes too small to be read comfortably on aniPad and it could not be enlarged. To ensure that fontsize does not compromise legibility for participantsand interviewers, it is recommended thatSenseMaker® be adapted to allow a variety of fontsizes to meet individual needs. Additionally,SenseMaker® did not adapt well to Arabic’s non-Roman characters, which caused some text to shift

GLOBAL HEALTH ACTION 7

during data extraction and rendered it nonsensical.Fortunately, the issue was recognized by the secondday of data collection and was overcome by askingthe interviewers to audio record the stories and toavoid the use of Arabic punctuation when they weretyping responses. Although SenseMaker®’s limitationsin working with Arabic were largely overcome inthis study, we recommend that data extractionprocess be better adapted for use in different lan-guages. Otherwise, its global usability will be limited.SenseMaker® also does not currently have skip logiccapability, which results in some participants havingto answer questions unnecessarily by choosing a‘non-applicable’ option. This is inconvenient andlengthens the survey. Given the availability of numer-ous, less sophisticated survey programs that incorpo-rate skip logic, this is a notable shortcoming ofSenseMaker® and needs to be addressed if the appli-cation is to be taken up more broadly as a researchtool.

Another disadvantage of SenseMaker® is that thenarratives are briefer and sometimes lack the richnessthat might be found with more in-depth qualitativeinterviews. This is largely because the interviewers donot ask prompting or follow up questions. Theimpact of this is countered by having participantsinterpret their stories since they respond to theSenseMaker® questions with the full story in mind,adding additional layers of meaning even though theymay not have articulated all the details in the audiorecording. By collecting a large number of storiesfrom a variety of stakeholders, SenseMaker® allowsresearchers to easily disaggregate the data to appreci-ate a variety of perspectives and this is certainly ofvalue. In future research, however, we may considercombining SenseMaker® and its broad capture fromcommunity members with more in-depth qualitativeinterviews being conducted at a later time with parti-cipants who talk about particular topics of interest (inthis case, child marriage).

Conclusion

In summary, SenseMaker® offered several unique fea-tures that make it an efficient data collection tool tocapture a large amount of mixed quantitative/qualita-tive data in a relatively short time period. It was feasibleto use SenseMaker® in a humanitarian setting and theapplication appeared well suited to collect revealingand open perspectives on a sensitive topic.SenseMaker® also facilitated monitoring capability toeasily track progress and tomaintain quality assurance.However, the qualitative data collected withSenseMaker® tended to be short and less rich thanwould be expected with more traditional qualitativeinterviews and a number of programing limitationsought to be addressed to increase SenseMaker®’s utility

for research purposes including ability to alter fontsize, to be more facile in other languages and incor-poration of skip logic. In conclusion, although there isa capacity for improvement, SenseMaker® was found tobe an efficient mixed-method data collection tool inhumanitarian settings and among refugee populations.

Acknowledgments

We would like to thank the Sexual Violence ResearchInitiative and the World Bank Group for their financialsupport of this work and are indebted to the researchparticipants who shared their stories and perspectivesabout the experiences of Syrian girls in Lebanon. We arealso grateful to Ghida Anani and team members from theABAAD Resource Center for Gender Equality in additionto the team leads (Katia Ghandoura, Clara El Warrak, andSalma Atwi) and the interviewers for their dedication andhard work. And finally, our sincerest thanks toSenseMaker® expert, Laurie Webster (QED Insight) forher management of the data and to Shaimaa Helal as wellas Bernice Ho and Annie Bunting for their assistance in thepreparation of this manuscript.

Author contributions

SB, CD, and SM conceived study design. SB designed studysurvey. NB, CD, SM, or SB conducted the pilot. SM direc-ted field implementation and oversaw recruitment of inter-viewers and team leads. NB, SB, and CD designedinterviewer training with SM, SB, and SG leading thetraining. SR, HB, and NB assisted in quality control duringdata collection. NB drafted initial manuscript with allauthors contributed to writing. All authors read andapproved the final manuscript.

Disclosure statement

No potential conflict of interest was reported by theauthors.

Ethics and consent

Study protocol was approved by the Queen’s UniversityHealth Sciences and Affiliated Teaching HospitalsResearch Ethics Board.

Funding information

Sexual Violence Research Initiative and the World BankGroup’s Development Marketplace for innovation on GBVprevention (in Memory of Hannah Graham).

Paper context

SenseMaker® is a mixed method data collection tool avail-able as an app for smartphones and tablets or as a browser-based software program. SenseMaker® use in humanitariansettings or among refugee populations is not well-docu-mented. This study examines the challenges and opportu-nities of using SenseMaker as a data collection tool amongSyrian refugees in Lebanon. Through these reflections,

8 N. BAKHACHE ET AL.

recommendations for future researchers and the improve-ment of SenseMaker® are provided.

ORCID

Nour Bakhache http://orcid.org/0000-0003-2735-9147Susan Bartels http://orcid.org/0000-0003-2782-3480

References

[1] Syrian Observatory for Human Rights. About 2 mil-lion and half killed and wounded since the beginningof the Syrian Revolution. 2015 [cited 2016 Aug 13].Available from: http://www.syriahr.com/en/?p=35137

[2] UNHCR. Syria regional refugee response – Lebanon.2016 [cited 2016 Aug 13]. Available from: http://data.unhcr.org/syrianrefugees/country.php?id=122

[3] Dahi O. The refugee crisis in Lebanon and Jordan: theneed for economic development spending. ForcedMigr Rev. 2014;1:11.

[4] The World Bank. Syrian refugees living in Jordan andLebanon: young, female, at risk. 2015 [cited 2016 Aug13]. Available from: http://www.worldbank.org/en/news/press-release/2015/12/16/syrian-refugees-living-in-jordan-and-lebanon-caught-in-poverty-trap

[5] UNHCR, WFP, and UNICEF. Vulnerability assess-ment of Syrian refugees in Lebanon 2015 report.2015 [cited 2017 Jul 24]. Available from: http://documents.wfp.org/stellent/groups/public/documents/ena/wfp280798.pdf

[6] The World Bank and UNHCR. The Welfare of SyrianRefugees: evidence from Jordan and Lebanon. 2015[cited 2016 Aug 13]. Available from: http://www.worldbank.org/en/news/feature/2015/12/16/welfare-syrian-refugees-evidence-from-jordan-lebanon

[7] UNICEF. Child protection from violence, exploitationand abuse – child marriage. 2016 [cited 2016 Dec 4].Available from: https://www.unicef.org/protection/57929_58008.html

[8] United Nations. Convention on the elimination of allforms of discrimination against women. 1979 (article16) [cited 2016 Dec 13]. Available from: http://www.un.org/womenwatch/daw/cedaw/text/econvention.htm#article16

[9] United Nations. Convention on the rights of the child.1989 [cited 2016 Aug 13]. Available from: http://www.ohchr.org/en/professionalinterest/pages/crc.aspx

[10] International Women’s Health Program. Early mar-riage. 2009 [cited 2016 Aug 13]. Available from:http://iwhp.sogc.org/index.php?page=early-marriage

[11] Nour NM. Health consequences of child marriage inAfrica. Emerg Infect Dis. 2006;12:1644–1649.

[12] UNICEF. Earlymarriage: a harmful traditional practice. Astatistical exploration. New York: UNICEF; 2005.

[13] Save the Children. Too young to wed: the growing pro-blem of child marriage among Syrian girls in Jordan.London: The Save the Children Fund; 2014.

[14] Abu Shama M Early marriage and harassment ofSyrian refugee women and girls in Jordan: amnestyinternational. 2013 [cited 2016 Aug 13]. Availablefrom: https://www.amnesty.org/en/latest/campaigns/2013/09/early-marriage-and-harassment-of-syrian-refugee-women-and-girls-in-jordan/

[15] FXB Center for Health and Human Rights. Running outof time: survival of Syrian refugee children in Lebanon.Boston: Harvard School of Public Health; 2014.

[16] Global Protection Cluster. Alert: urgent child protec-tion issues in Syria. 2013 [cited 2017 Jul 26]. Availablefrom: http://www.protectingeducation.org/sites/default/files/documents/global_protection_cluster_-_urgent_child_protection_in_syria.pdf

[17] UNICEF. Under Siege: the devastating impact on chil-dren of three years of conflict in Syria. 2014. NewYork: UNICEF; 2014.

[18] Samari G. The response to Syrian Refugee Women’sHealth needs in Lebanon, Turkey and Jordan andrecommendations for improved practice. Humanityin Action. 2014 [cited 2017 Apr 2]. Available from:http://www.humanityinaction.org/knowledgebase/583-the-response-to-syrian-refugee-women-s-health-needs-in-lebanon-turkey-and-jordan-and-recommendations-for-improved-practice/print

[19] Raj A, Saggurti N, Winter M, et al. The effect of maternalchild marriage on morbidity and mortality of childrenunder 5 in India: cross sectional study of a nationallyrepresentative sample. BMJ. 2010;340:b4258.

[20] Abdessamad F. Collecting stories from chaos. UNDP.2015 [cited 2016 Oct 19]. Available from: http://www.undp.org/content/undp/en/home/blog/2015/10/2/Collecting-stories-from-chaos.html

[21] WhatsApp Inc. WhatsApp features. 2016 [cited 2016Aug 13]. Available from: https://www.whatsapp.com/features

[22] Tableau. Data visualization. 2016 [cited 2016 Aug 13].Available from: http://www.tableau.com/stories/topic/data-visualization

GLOBAL HEALTH ACTION 9