introducing framework © : increasing quality and transparency in qda kandy woodfield deputy...

Introducing Framework©: increasing

quality and transparency in QDA

Kandy Woodfield

Deputy Director

Qualitative Research Unit

ESRC Methods Festival: Can Software Enhance the Quality of Qualitative Research?

18th July 2006

Introducing Framework© July 18th 2006 - ESRC Methods Festival

Session coverage

The Framework©approach to QDA

• key features of the approach• developments over time

Our new software package being launched in 2007

• how different to existing CAQDAS packages?• increasing transparency?• enhancing quality?


Framework® approach to QDA

Framework© is an approach to qualitative data management

• developed at the QRU, NatCen in mid 1980’s • it is widely used by applied policy researchers• uses an analytical framework consisting of a series of thematic

matrices (grids)• has applications for all types of QR which involve interpretative

analysis– exploration of the world of research participants by identifying and

describing substantive themes– searching for patterns and explanations


Key features of Framework©

The ‘analytical framework’ is developed inductively to fit the data

• it consists of a series of thematically structured matrices• it can be used with all forms of textual qualitative data

Data is synthesised into a series of matrices, the ‘analytical framework’

• synthesised data is displayed in a grid which facilitates cross-case and within-case analysis

• synthesis retains links back to the original data

Provides a systematic and consistent approach to data handling

Facilitates creative thinking: the act of creating a synthesis activates the analytical brain


Example of Framework matrix

Resettling in the UK a qualitative study of refugees’ experiencesChart 4: Family, social contact and family reunion

Case label4.1 Family left

behind in Africa4.2 Social contact

– local people4.3 Social contact– other refugees

4.4 Help and supportreceived in relation to

social contact

4.5 Familyreunion

4.6 Other

#1

#2

#3

#4

Data is summarised into cells

Matrix display facilitates

between and within case analysis of case data


Developments over time

Originally paper based, hand written summaries• difficult to archive

• hard to team work

• no flexibility in layout or ordering of cases

Moved to spreadsheet host in 1990’s• easy to set up and edit input

• facilitates team work and audit checking

• provides easily archivable dataset

• limited functionality for sorting and ordering

• BUT proved to be a ‘clunky’ host (especially in relation to outputs)

Next step - to search for a new host for ‘Framework’


Framework© -

a new tool for managing,

ordering & interrogating qualitative data

Key features• Windows based single user or networked package

• import text(V.1) documents into Framework© to create project

• each project comprises series of cases

– cases may be individuals or a series of linked data units for longitudinal & other approaches using multiple data sources

• data managed into an analytical framework

• drag and drop facility

– allows analytical frameworks to be modified – data re-assigned automatically


The new Framework©software

Data units can be stored in variety of formats

• as imported text

• with case labels (socio-demographic/classificatory) which can be applied at various points during the process (facilitating pattern searching and sub-group analysis)

• as indexed text (coded) by theme/sub theme

– free indexing: iterative process of deriving new analytical framework (good for grounded theory)

– dynamic indexing: application of index generated manually

– combination of free and dynamic

• as data summarised into themes/sub themes/lower levels


Case data is imported into Framework


Analyst applies data unit labels to allow for latersorting and identification of group based patterns

and themes


You design your analytical framework


Your framework can include different levels of

themes and sub themes


Analyst indexes the data unit


Analyst summarises the data unit into the analytical

framework


The new Framework© software

Analysis facilitated by the process of creating analytical queries • asking the software to produce indexed text extracts, matrix extracts

or notebook extracts by theme or sub theme

– can produce universal output (all cases, all themes) or filter by themes and participants/sub groups

• thematic pattern searching

– allows you to identify where themes have been applied to cases and produces a graphical display to allow you to easily detect where patterns lie in the case/dataset/sub group

• simple text searching

– for word occurrence/word associations and positional location within the dataset

customised outputs to screen/paper, free text or matrices


You can add notes as you go along and retrieve, search or archive

these later


When you are ready to explore the dataset you can create an

analytical query


You can choose what type of data& data units you want Framework

to include


Then you specify which themes or sub themes you are interested in


Framework stores all queries and you can return to these at any time


You leave an audit trail


Framework will also look for thematic patterns across the dataset


How does Framework© differ from other CAQDAS packages?

Framework© will:

enable data synthesis • make it less laborious • free up time for interpretation

allow the analyst to customise outputs • including matrices of summarised data

a new era of ‘code and retrieve’?


How does Framework© differ from

other CAQDAS packages?

The networked version allows multiple team members• to work simultaneously on a shared dataset

– collectively analyse a dataset– share thoughts/analytical queries in real time

The dynamic audit trail facility provides• systematic documentation of changes and developments

– when and by who

Comprehensive project administration system• complies with new expectations relating to data protection


Adding quality?

Spencer et al’s (2003) Quality Framework for the Cabinet Office identified criteria for assessing quality in qualitative studies.

In relation to analysis:

• Evidence of the use of data management methods that preserve context and are suited to the analytical task

• Portrayal of detail, depth and complexity

• Detection of implicit and explicit explanations & underlying factors

• Identification and discussion of patterns of association

• Portrayal of links between the data, interpretation and conclusions

-Evidence of analytical building blocks-Discussion of how explanations/interpretations/conclusions were derived

Framework will support all of these


Adding quality to data management

Quality enhanced by:

In-built system checks• enforce consistent, systematic treatment of data. For example, direct

linking of data to original script/document

Closeness to data • original data always on display

• original data directly linked to summaries and matrices

Dynamic audit trail

• visible account of changes to the analytical pathway including date/ownership of change and rationale for change, can log:

- development of analytical framework- application of that framework to data units- summarisation of data into conceptual categories


Adding quality to the analytical process

Framework has a high degree of operational flexibility, thisfacilitates:

• analytical pluralism allowing users to customise the approach they take depending on their needs

• the development of frameworks/indexes and analytical queries can be driven by the data rather than shaped by the package

• the ability to work with data in multiple formats facilitates creative thinking

• the documentation of analytical pathways in notebook encourages greater reflexivity

• the ability to tailoring analytical queries & customise outputs allows users to be innovative in their approach to the dataset


Adding quality to the analytical process

The package enhances the analytical process by allowing users to:• ask analytical queries of the whole dataset or subgroups within it

- facilitating investigation of patterns and associations

• produce matrix outputs and/or text outputs from all or part of the dataset - allowing the user to select an output format which suits them and the data

- allowing the user to work with data at different levels

• apply typologies and classifications at any point

- enhances the analytical power providing another means of exploring patterns and explanations

• perform simple and complex text searches depending on the analytical task• identify positional relationships between themes/groups/cases• search and explore the notebook for their own and others conceptual thinking


Increasing transparency

Transparency in the analytical process is enhanced by the:

• dynamic audit trail function which provides:- a systematic, visible and chronological account of changes to the analytical

pathway - including ownership of changes

• notes function which provides a means of sharing and documenting analytical developments, allowing the user to record and display:

- the building blocks of emergent findings- the development of classifications or interpretative theories and how

cases/data have been assigned to these- development of concepts, theories and interpretations- how and why particular interpretations have been chosen


The new Framework© software:

launch schedule

Currently in final stages of development and in-house testing•Autumn 2006 beta testing

•Summer 2007 launch of full single user and networked version

•coupled with programme of training sessions

For further information

William O’Connor : [email protected]

Kandy Woodfield: [email protected]

Or see: www.natcen.ac.uk

introducing framework © : increasing quality and transparency in qda kandy woodfield deputy...

Documents

introducing framework

analytical framework

qda framework

data unit slide

new analytical framework

esrc methods festival

data handling

multiple data sources