research article open access the role of the user within the medical device design … ·...

RESEARCH ARTICLE Open Access

The role of the user within the medical devicedesign and development process: medical devicemanufacturers’ perspectivesArthur G Money1*, Julie Barnett2, Jasna Kuljis2, Michael P Craven3, Jennifer L Martin3, Terry Young2

Abstract

Background: Academic literature and international standards bodies suggest that user involvement, via theincorporation of human factors engineering methods within the medical device design and development (MDDD)process, offer many benefits that enable the development of safer and more usable medical devices that are bettersuited to users’ needs. However, little research has been carried out to explore medical device manufacturers’beliefs and attitudes towards user involvement within this process, or indeed what value they believe can beadded by doing so.

Methods: In-depth interviews with representatives from 11 medical device manufacturers are carried out. We askthem to specify who they believe the intended users of the device to be, who they consult to inform the MDDDprocess, what role they believe the user plays within this process, and what value (if any) they believe users add.Thematic analysis is used to analyse the fully transcribed interview data, to gain insight into medical devicemanufacturers’ beliefs and attitudes towards user involvement within the MDDD process.

Results: A number of high-level themes emerged, relating who the user is perceived to be, the methods used, theperceived value and barriers to user involvement, and the nature of user contributions. The findings reveal thatdespite standards agencies and academic literature offering strong support for the employment formal methods,manufacturers are still hesitant due to a range of factors including: perceived barriers to obtaining ethical approval;the speed at which such activity may be carried out; the belief that there is no need given the ‘all-knowing’ natureof senior health care staff and clinical champions; a belief that effective results are achievable by consulting aminimal number of champions. Furthermore, less senior health care practitioners and patients were rarely seen asbeing able to provide valuable input into the process.

Conclusions: Medical device manufacturers often do not see the benefit of employing formal human factorsengineering methods within the MDDD process. Research is required to better understand the day-to-dayrequirements of manufacturers within this sector. The development of new or adapted methods may be requiredif user involvement is to be fully realised.

BackgroundFrom a human factors engineering perspective, ensuringthe development of high quality and well designed med-ical devices that are in tune with patient and user needs,require formal human factors engineering methods (alsoknown as user-centred usability engineering methods) tobe used at every stage of the MDDD process [1].

Employing such formal methods ensures that the devicedesign process considers appropriately the environmentin which the device is to be used, the work patterns ofusers and the specific individual needs of the user,which could be any individual involved in the use of thedevice including health professionals, patients, and laycare givers [2]. In particular, human factors engineeringmethods highlight the importance of considering theneeds of the user when designing and developingdevices at the earliest stage of defining the device con-cept and then at every subsequent stage of the device

* Correspondence: [email protected] of Computer Science and Technology, University ofBedfordshire, Park Square, Luton, LU1 3JU, UKFull list of author information is available at the end of the article

Money et al. BMC Medical Informatics and Decision Making 2011, 11:15http://www.biomedcentral.com/1472-6947/11/15

© 2011 Money et al; licensee BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative CommonsAttribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction inany medium, provided the original work is properly cited.

mailto:[email protected]

http://creativecommons.org/licenses/by/2.0

development process [3]. The importance and value offocusing on user needs has been recognised as having anumber of health related benefits including; improvedpatient safety [4,5], improved compliance and healthoutcomes [6], and higher levels of patient and user satis-faction [7]. Furthermore, employing human factors engi-neering methods throughout the MDDD process hasbeen said to substantially reduce device developmenttime because usability issues are identified and attendedto prior launch, and hence avoid costly design changesand product recalls [8,9].Given the proposed benefits of developing medical

devices according to a human factors engineering per-spective, medical device standards bodies are increas-ingly recognising the importance of these methods andthe important role they play in developing safe andusable medical devices. Currently the two most impor-tant regulations for medical device developers are theEuropean Commission (EC) Medical Device Directive93/42/EEC [10] and the United States (US) Food andDrug Administration (FDA) regulations since medicaldevices must comply with these regulations in order tobe sold throughout Europe and the US [11]. These regu-lations promote the use of human factors engineeringmethods within the MDDD process, for example, theUS FDA regulations specify that medical device develo-pers must demonstrate that human factors principleshave been used in the design of the device so as toensure that use-related hazards have been identified,understood and addressed. Some guidance is also givensuggesting how human factors principles may be inte-grated into the MDDD process [12]. A further standardthat medical device developers have been obliged toadhere in recent years is the International Electrotechni-cal Commission (IEC) standard 60601-1-6 [13], whichhave equivalent European Standards (ES) and BritishStandards (BS), requiring medical device developers toincorporate human factors engineering processes toensure patient safety, stating that “The manufacturershould conduct iterative design and development.Usability engineering should begin early and continuethrough the equipment design and development life-cycle”. More recently, the usability standard ‘IEC 62366:Medical devices - Application of usability engineering tomedical devices’ [14] has superseded IEC 60601-1-6, andextends the requirement for medical device developersto incorporate human factors engineering methods inthe development of all medical devices, not just electri-cal devices. In early 2010 IEC 62366 was harmonised bythe EU Medical Device Directive meaning that it is nowa legal requirement for medical device developers to for-mally address the usability of a device before placing iton the market anywhere in Europe. In order to complywith this standard, medical device developers must

document the process in detail within a Usability Engi-neering File.

Human Factors Engineering MethodsThe MDDD process may be considered to be made upof four key stages. At stage one, user needs are estab-lished and scoping exercises are carried out with users.Stage two aims to validate and refine the concept. Stagethree involves designing the device, and Stage fourinvolved evaluation of the device.Figure 1 presents the medical device development life-

cycle, and the associated user-centred design methodsthat may be used at each respective stage.In some of our earlier work, as part of the Multidisci-

plinary Assessment of Technology Centre for Healthcare(MATCH) research activity, we carried out a rigorousreview of the methods that have been employed in theMDDD process as presented in the academic literature[5,11,15]. MATCH is a UK research initiative betweenfive universities which is funded by the Engineering andPhysical Sciences Research Council (EPSRC) and theDepartment of Trade and Industry (DTI). The aim ofMATCH is to provide support to the healthcare sectorby developing methods to assess the value of medicaldevices at every stage of the MDDD process, with afocus on working with industrial partners to solve realworld problems.To provide some insight into what a selection of these

methods involve, a brief description of the most fre-quently occurring methods within the device develop-ment lifecycle are now provided. For a detaileddescription of all methods and how these may beapplied within the MDDD is provided in [5].Focus groups involve group discussion typically

between 8-10 users, with a moderator guiding and facili-tating the group through relevant topics of discussion.They may be appropriate for use at stage one, two orfour of the device development lifecycle. Focus groupsare used in a wide variety of industry settings and maybe conducted at a comparatively low cost comparedwith other methods such as usability testing.Interviews are one of the most common approaches

employed in user centred research, and may be of valueat stages one, two and four of the development pathway.They can be rapidly deployed, and are relatively inex-pensive to carry out. Interviews enable the researcher toaccess a broad range of opinion regarding a device,whilst also allowing rich and detailed opinions to becollected.Usability testing, typically performed at stage four,

advocates is a function of three criteria: effectiveness,efficiency and satisfaction. Usability testing protocolsinvolve users carrying out specific tasks with a specificdevice, whilst usability measures are taken. Effectiveness


of 12

is concerned with whether the user is able to success-fully accomplish a given task. Efficiency may include acount of the number of user actions or the amount oftime required to carry out a task. Satisfaction may typi-cally be measured subjectively by means of a satisfactionquestionnaire, completed after using a particular device.Heuristic evaluation is a more rapid form of usability

test which may be deployed by developers, perhapsprior to carrying out usability tests. Developers stepthrough the device features and functionality and checkthe extent to which it complies with pre-determined listof characteristics (heuristics/rules of thumb) that mayhave been defined by earlier UCD design activities. Thistype of evaluation would normally be applied at stagefour of the development pathway, once a tangible devicehas been developed.

The challenge for industryAlthough a large number of human factors engineeringmethods are available that may be employed within theMDDD process, previous research indicates that medicaldevice manufacturers often avoid employing such meth-ods due to lack of resources and the perception thatsuch methods are often too resource intensive [16]. Theliterature in the area of user involvement suggests thatthere are a number of risks for manufacturers associatewith a lack of engagement with users within the productdevelopment process. For example, Cooper [17] suggeststhat failure to build in the voice of the customer is oneof the key reasons for the failure of developing effectiveand innovative products. Certainly in the realm of ICTproducts Kujala [18] suggests that attending to user per-spectives increase the likelihood of a successful product.Alongside this however, other work has highlightedsome of the challenges of, and barriers to, involvingusers in the product development process. Brown [19]and Butler [20] both suggest that the volume of datagenerated by field studies with users may be costly, diffi-cult to analyse with no obvious route to informingdevelopment. When considering the factors leading to

successful innovation, van der Panne et al. [21] notethat although it is incontrovertible that good marketresearch is associated with a successful product, the roleof customer involvement in innovation remains conten-tious. They suggest that customer involvement at anearly stage may tend to gravitate towards imitative pro-ducts, being less able to envision or express novelty andthus, “bias innovation efforts towards incremental inno-vation” [[21] p.326]. User preferences may change overtime and engaging with a limited range of users mayresult in over specification of the product [22]. From thedeveloper’s perspective, their criteria for the success ofuser involvement may be different than those (often aca-demics or researchers) who actually do user engagementwork [23,24]. Furthermore, user information that isbased on formal methods of elicitation may be at var-iance with the representations of the user held by devel-opers themselves [22] and may thus not readily beappropriated within the organisational culture and struc-ture of the lead organisation [25]. Many of the aboveexamples are drawn from the more general area of userinvolvement within the product development process,since there is limited research in this area specificallyaddressing such issues within the medical device devel-opment domain. There is a lack of existing primaryresearch that explores the challenges and benefits ofinvolving users specifically within the medical devicedevelopment process, particularly from a medical devicemanufacturer’s perspective. Examples of some of thework that does exist within this domain includes, Gro-cott et al. [26] who carry out pioneering work in thefield, proposing a valuable model of user engagement inmedical device development, and focus on the practicalissues around how user needs may be captured through-out the MDDD process [26]. Shah and Robinson [16]carry out secondary research in the form of a literaturereview of research that has involved users in some wayin the design and development of medical devices,exploring the barriers and benefits to involving users asmay have been reported in the reviewed literature.

Figure 1 Medical device development lifecycle adapted from [5].


of 12

The findings of this study are in line with the perspec-tive offered above that whilst user involvement inMDDD process is likely to benefit the user in terms ofdeveloping devices better suited to the users’ needs,MDDD methods may be perceived as highly resourceintensive and hence often not a feasible option for somemanufacturers.Reflecting on the body of literature above, and in par-

ticular on the perceived benefits and barriers to userinvolvement within the wider domain of technology pro-duct development, it seems that there are a number ofkey factors that impact on effective engagement withusers. Perhaps most notable is the notion that productdevelopers consider user needs research to be dispro-portionately costly, in light of the perceived benefits andpay-off for engaging in such activities. Furthermore, userneeds research is perceived by developers to generateunmanageable volumes of data and there does notappear to be any clear route through to informing thedevelopment of the product once the user needs datahas been collected and analysed. Whilst all of the per-ceived drawbacks, if well-founded, may be notable rea-sons for avoiding engagement with user needs research,it is not clear what the underlying reasons are for thesefactors. For example, a commonly held view by somedevelopers is that the key function of human factorsengineering methods is to serve as a means of facilitat-ing a ‘cake-frosting’ exercise [27], by which ‘superficial’design features may be ‘painted’ onto the device at theend of the development process. Such views of humanfactors engineering methods do not lend themselves topositive engagement or indeed a realisation of the fullcomplexity and pay-off that such methods may poten-tially deliver if they were to be deployed with methodo-logical rigour at appropriate points within the designprocess [28]. Certainly from an academic and/or humanfactors engineer’s point of view, it may be argued thatmany of these factors may be overcome by increasedawareness and better educating industrial developers inhuman factors engineering methods and their applica-tion. However, is it the case that with more training inthe area of user research methods research, productdevelopers may overcome these reservations, recognisethe potential opportunities these methods promise, anddevelop the necessary skills to deploy methods and ana-lyse data in a timely fashion, or are these methodsindeed incompatible within the industrial context?Should user needs research be outsourced and deliveredby the human factors engineering experts, in order toovercome the overhead in acquiring appropriate skillsand level of understanding to actually effectively deploythese methods? Should these methods be adapted tomake them more lightweight and easier to implement,hence making them more fit for purpose? These are all

questions that need to be answered if the goal of incor-porating the user into the product development processis to be realised. More specifically within the MDDDprocess, there are likely to be domain specific factorsand conditions that influence the uptake of suchmethods.Therefore, from a theoretical perspective, human fac-

tors engineering methods are presented as being ofvalue at every stage of the MDDD process, manufac-turers may not actually be employing these methods inpractice. If their full benefits are to be realised, moreprimary research is needed to better understand manu-facturers’ perspectives and motivations regarding suchmethods. Therefore, the aim of this study is to gain firsthand and detailed insights into what medical devicemanufacturers’ attitudes are towards engaging withusers, and what the perceived value and barriers are ofdoing so. Furthermore, we aim to explore which meth-ods are used, and what device manufacturers’ attitudesare towards employing such methods.

Section summaryThus far, we have detailed a range of formal methodsthat may be used to elicit user perspectives as part ofthe process of medical device development and notedthe range of regulatory requirements to involve users.We have suggested that the ‘in principle’ value of sys-tematically seeking user input may be somewhat at var-iance with the day to day experiences of manufacturerswhere user involvement may be seen as a barrier tospeedy and innovative product development. There iscurrently no evidence about this in respect of medicaldevice development. The remainder of this paperaddresses this issue and is structured as follows. In thenext section, we describe a study which involved carry-ing out in-depth interviews with 11 medical device man-ufacturers which explored the perceived value of usersin the MDDD process. In the following section, theresults of the analysis of these interviews are then pre-sented. The paper concludes with suggested recommen-dations for future research and practice in this area.

MethodsEleven interviews were carried out with senior membersof staff in each company. Recruitment was by conveni-ence sampling: participants were identified as a result oftheir employment by companies that were industrialpartners of the MATCH collaboration. To provide caseexamples for discussion, company representatives wereasked to choose one medical device to discuss duringthe interview, which provided an example of the MDDDprocess they engaged in. The majority of the deviceschosen for discussion were intended for use by surgeonswithin a clinical setting. Interviews lasted approximately


of 12

one hour in duration, and were carried out by membersof the MATCH research team. One of the topics of dis-cussion was around device users. Interviewees wereasked about the role the user is perceived to play withinthe design and development of the example medicaldevice, what they saw as the barriers and benefits toinvolving users within the MDDD process, and whathuman factors engineering methods they used (if any)within the MDDD process. All interviews were recordedand researchers also took notes during the interviews.Before beginning, the interviewer explained the purposeand format of the interview to the participant andinformed consent to participate, and for the audiorecording of the interview, was obtained. Table 1 sum-marises the position held within the company for eachinterviewee, the treatment area or clinical use for thedevice, the intended users of the devices, and the actualindividuals consulted in the MDDD process by therespective manufacturers.A thematic analysis of the textual dataset was then

carried out. Detailed descriptions of what the thematicanalysis process involves are available in [29-32]. Inbrief, thematic analysis facilitates the effective and rigor-ous abstraction of salient themes and sub-themes froma complex and detailed textual dataset [33], hence isparticularly suitable in this context. The following stepswere taken to analyse the data collected during theinterview process, Figure 2 provides an overview of this.Initially all recordings of the trial sessions were tran-

scribed into text format. After transcription, the textualdataset was initially perused to conceptualise the over-arching themes that existed within the transcripts at ahigh-level. These were noted in a coding frame, with

each concept assigned a code name, and a descriptionand examples of text that fit each concept. The datasetwas then examined iteratively, enabling themes and sub-themes to be developed further. These were spliced andlinked together, and text relating to each category andsub-category were appropriately labelled. The first andsecond authors coded the data and discussed inconsis-tencies where these arose until a clear consensus of themain themes was reached. When no further refinementof the categorisation could be derived a final group ofcategories and sub-categories, representative of the tran-scripts was produced. The main themes are those drawnfrom multiple contributions and that represent issuesthat are clearly central to the participants themselves.Within these themes, we have explored areas of consen-sus and diversity as they were presented by interviewees.

ResultsAs a result of carrying out a thematic analysis of theinterviews carried out with manufacturers, four highlevel themes emerged relating to manufacturers’ viewsof user involvement within the MDDD process. Theseare as follows: Who is the user?; Methods used; Per-ceived value and barriers to user involvement; The nat-ure of user contributions. The remainder of this sectionpresents the findings of our study according to thesethemes.

Who is the User?Discussions relating to the range of individuals that areconsulted during the MDDD process revealed that therewas a mismatch between the users that were consulted,and those that would actually use the device in practice.

Table 1 Company details and intended device users

Position held Device’s treatment area orclinical use

Intended users Individuals consulted

#1 Operations Mgr & Clin. Affairs Mgr. Orthopaedics Surgeons Surgeons

#2 Operations Mgr Cardiology Health profs. Generalpublic

Health profs.

#3 Scientific Director Oncology Health profs. Physicians Health profs. Trainee healthprofessionals

#4 Director Spectroscopy Health profs. Clinicians Electronics experts

#5 Director of Clin. Research & Clin.Reimbursement Mgr.

Orthopaedics Surgeons Surgeons

#6 Managing Director Orthopaedics Surgeons Surgeons

#7 R&D Mgr. Orthopaedics Surgeons Surgeons

#8 R&D Director Laryngology Gen. health practitioners Purchasing reps. Clinicalchampion

#9 Managing Director Phlebotomy Clinicians Universityresearchers

Clinicians University researchers

#10 Medical Tech. Mgr. Vital signs monitoring Nurses Clinicians Physicians

#11 Research Programme Mgr. Wound care Home patients Surgeons


of 12

Some manufacturers believed that the needs of thepatient do not originate from the patient themselves,and that patients’ needs are better articulated through ahierarchy of health professionals including surgeons and‘clinical champions’.

#2: “The need actually will probably be establishedby the clinical fraternity....All you have to do afterthat is convince the people on down the chain, fromthe hospital clinical researchers right down to the guyin the street who says ‘It’s a good idea to have one ofthese. That’s where the need is identified. It is identi-fied really at the top, and then it’s taught down, ifyou follow me”

In light of this, manufacturers had a preference forseeking input from more senior health care staff overless senior staff, regardless of who would actually usethe device in practice. The assumption was that seniorstaff members were more than capable of speaking onbehalf of less senior staff, even though manufacturersacknowledged that it was likely that senior staff mem-bers would never actually use the device. This was thecase even in scenarios where the patient was seen to bethe main user of the device. For example, Manufacturer#2 who is involved in the development of AutomatedExternal Defibrillators (AEDs), recognised that a signifi-cant proportion of the intended users of the deviceincluded members of the public, however, they did notsee it as necessary to consult members of the public,but rather consulted senior health professionals in theearly stages of device design and development. Inputfrom nurses was also considered to be less desirablethan input from more senior health care staff such as

surgeons. For example, Manufacturer #11 identifiednurses as the main user of their product, however, didnot consider it necessary to consult nurses in the designand development process. It was felt that surgeonsmade the decisions on behalf of nurses and patients,and therefore it was logical to consult them to identifynurse and patient needs, as is articulated below.

#11: “Surgeons may be there making the decision orrecommending which models to buy, but it might bethe nurses who are actually using the unit...the sur-geon has made the decision, but he doesn’t necessa-rily have to actually work with it, so in your case, thecontrollability aspect, the patient actually under-standing how to operate the unit, the clinician hasmade the decision and has prescribed that particulardevice...”

Surgeons were also considered by Manufacturer #11as having sufficient knowledge to act as representativesfor home patients. The motivation for this was as adirect result of the way in which the device is intro-duced and promoted to the patient. Surgeons do themarketing and promoting of the product to the patients,and therefore it was seen as important that the device isprimarily designed according to the surgeons’ require-ments. In the quote below, manufacturer #11 justifiestheir motivation for valuing the surgeons’ opinion overthe patient users.

#11: “but, by and large, most of our market researchis done with the surgeons, not with the end users,rightly or wrongly...but how it will be marketed willbe through the healthcare professional, who will have

Figure 2 Process of thematic analysis.


of 12

to sell it effectively to the patient, show them how touse it.”

Similarly, in the event of the user group being generalhealth professionals, the opinions of a small number of‘clinical champions’ was sought by Manufacturer #8 aswell as purchasing representatives from the healthauthority. It was deemed more important to meet theneeds of the individuals who were responsible for mak-ing purchasing decisions or held most influence overthem, as opposed to focusing on the needs of the indivi-duals that would be using the devices on a day to daybasis. Therefore MDDD activity often seemed to be car-ried out a strong focus on how effort would translate tosales. In the following quote, Manufacturer #4 is askedwho would be consulted in identifying design require-ments for the development of the given medical device:

#4: P1: “...it’s through the doctors to, its [name]’s con-tact with the hospitals out there, now then I doubt itwould be at doctor level, it would be more the man-agerial level of the hospital...”I2: ...so in effect you are capturing a user need in thatway...P: “ ...yes.”

(Note: 1 P: Denotes the participant’s response, 2 I:Denotes the interviewer’s questioning)Manufacturer #1 also focuses on those making/influ-

encing purchasing decisions, which includes manage-ment and administrative staff. Manufacturer #1articulates this shift in focus when asked who they viewas their customer:

#1: “Orthopaedic surgeons really. Although it is usedin patients, it is the orthopaedic surgeons who decidewhat he will use. Sell to orthopaedic surgeons.Increasingly in the UK, we have to sell to the hospitalmanagement to justify why they should be using ourbone graft substitutes as opposed to any other on themarket...”

Manufacturer #8 stated that health professionals areconsidered to be the main user of their device, However,the opinions of purchasing representatives are the pri-mary source used to inform device design and develop-ment, followed up by consultation with a small numberof ‘clinical champions’ (typically high profile surgeonsand well known experts in their field). When questionedon what value the user may add value to the MDDDprocess, although couched in a humorous reply, it wasclear that the usefulness of the user was ideally directlylocated in relation to sales relevant information.

#8: “The most helpful would to give me the yearthree sales figures with absolute confidence. Year-1...[laughs]...”

There seems to be limited overlap between the indivi-duals that will be using the device, and those that areconsulted to inform the MDDD process. In particular,priority is given to those that hold more senior positionswithin the health care system. Therefore, surgeons, doc-tors and clinical champions were seen as more valuablesources for identifying user needs as opposed to thoseindividual that would actually use the device on a dailybasis. Furthermore, from the manufacturers’ point ofview, the motivation for maximising sales seems to haveconflated the distinction between the customer and theuser. Therefore the needs of those that make purchasingdecisions or indeed have most influence over these deci-sions are more salient than the needs of the user. Thisis certainly a more complex picture than the human fac-tors engineering approach, which puts the user’s needsat the centre of the design and development process.

Methods usedGiven the wide range of formal methods that are avail-able to engage with users in the MDDD process, manu-facturers tended to use only a very limited range ofmethods to capture information from users and patients.In line with the analysis above regarding the nature ofthe preferred user to consult, the most typical methodused to gain input from users was via informal discus-sions with senior health professionals. Only one out ofthe eleven manufacturers (Manufacturer #8) stated thatthey regularly used some formal methods throughoutthe MDDD process, such as focus groups and question-naires when developing their airway management device.Interestingly, the initial identification of a need for thisdevice occurred as a result of six members of the com-pany attending a postgraduate university course, whichrequired the use of formal methods in order to exploreand identify new device ideas. It was apparent that man-ufacturers do not feel that they have the time orresources to engage in rigorous formal user data collec-tion methods, instead relying on a range of strategiesincluding gut feel, instinct, and a personal belief thatthey understand the market place in which they operate,in order to identify and develop new devices.The idea of employing formal methods is once again

something that is not regarded as feasible, given theamount of resources available and the fact that manu-facturers believe it is necessary to move quickly in orderto remain competitive with their rivals. The view is thatconsulting a large number of individuals is problematicin itself, as every person that is approached for feedback


of 12

has a different view. Informal methods, however, are seenas offering versatile and rapid solutions, hence the beliefthat relying on gut feel and pressing ahead when themoment feels right is a more feasible and efficient solu-tion. This point is articulated by Manufacturer #8 below:

#8: “The very fact that someone is willing to talkwith you almost means that they have slightly differ-ent view. You can go on asking forever. It’s that bal-ance between have we got sufficient confidence inwhat we have here to move forward vs. just the gen-eration of the information....being confident enough ofits assurance. That’s what you constantly face any-how. I think that’s the dilemma you always have.You can ask the users till the cows come home butyou never get a new product. You ask 100,000 youget 99,999 different opinions!”

Manufacturer #3 further echoes the observation thatformal methods are rarely used for medical device devel-opment, however, informal discussions and observationare seen as more versatile and fit for purpose, :

#3: P:"you’re introducing a medical device to people,first question they’ll ask, is “what will it do to helpme?” Second question will be “how long is this goingto take?” In a busy clinic that’s really important...If ittakes too many clicks, then people won’t use itbecause they are busy enough as it is.”I: have you been able to capture that at all?P: “Through our interaction with users. We haven’tgot a specific mechanism for capturing it.”I: Do you use any formal methods for converting cus-tomer needs into product development?P: “No.”

There was typically the belief that there was little needto consult the actual users formally regarding a newinnovation, but rather contacting what they referred toas a ‘clinical champion’, was sufficient to qualify the fea-sibility and validity of a given new innovation. Forexample, Manufacturer #8, responding to a questionregarding how the feasibility of new device ideas arequalified responded:

#8: “Every project will have a clinical champion.They will typically be involved, sometimes they comedown here to meetings. We have a list of a couple ofdozen clinicians that I can pick the phone up at any-time throughout the world and say “what do youthink of this?”.

When asked whether there are any formal methodsused within their organisation, Manufacturer #6 stated

that formal methods are used within his organisation,however, they are not relevant for this particular pro-duct. Once again, formal methods, in this case example,were considered to be too bureaucratic, time consum-ing, and not applicable given the device and develop-ment scenario. Informal methods, such as ad hocdiscussions with senior health professionals were likelyto identify the majority of user design needs, and werealso more appropriate.

#6:”Yes we do, but I couldn’t apply that to hip repla-cement at this point in time. If the surgeon tells me“that this catheter is a bit too stiff, and could youmake it a bit softer, if you like, or a bit more flexible,yet do exactly the same task?” we will do that - it’sfor everyone’s benefit”

In the above example, a pragmatic approach is takenby manufacturers to make modifications to devices,adopting the belief that if a problem is encountered byone individual, then it is likely to be a problem for themajority, providing it seems to be a reasonable request.The intuition of the manufacturer plays a major part inthe process of developing products that are useful to theuser, and responding to their needs.

Perceived value and barriers to user involvementThere was limited evidence that direct elicitation of userviews was seen by manufacturers as being of value tothe MDDD process. This appeared to be particularly thecase in relation to patient users. For example, Manufac-turer #11, discussing their device that was purely aimedat the home patient market, did not believe that patientinvolvement in the MDDD process was a particularlywise expenditure of resources. This was explicitly linkedto the degree of influence that they have in terms of thelevel of power and influence they have in the levels ofuptake of the device. In response to being askedwhether they would like to involve the patient more inthe MDDD process, manufacturer #11 replied:

#11: “if they are highly powered (i.e. influential interms of clinical decision making), if they have nopower then we have to ethically try to make surethat we don’t harm the patient and [that] what wedo is for their benefit, but appealing to them mayactually be a waste of resources, so we have to makesure we don’t pretend that they have a sway whenthey really don’t, you know? “

Once again, the above statement reinforces the notionthat patients are seen as being at the bottom of the hier-archy of influence, and hence investing resources andeffort to find out their opinion is not considered an


of 12

effective of efficient strategy. Manufacturer #2, who alsosaw their device as being targeted at home market,acknowledged that the patient user is becoming increas-ingly important, particularly as they are now sellingmore directly to the home patient. However, the major-ity of individuals involved in the MDDD process are stillpredominantly senior health professionals as opposed topatients, who may play a stronger role in the clinicaltrials phase, after the device has been designed anddeveloped. Therefore, the role of the patient user is con-sidered to play a passive role, more in terms of verifyingthe value of an already developed device, as opposed tobeing involved in the initial design and developmentstages. This is discussed in more detail in the nextsection.Another factor discouraging manufacturers from

involving patient users in the MDDD process is the pro-spect of having to obtain ethical approval in order tocarry out the research. When asked whether there areany difficulties in obtaining ethical approval for invol-ving patient users in the MDDD process, Manufacturer#3 stated:

#3: “Yes. A huge problem. In my experience if you arenot affecting patient care, patient throughput, youcan get ‘ethics’, but if you are you won’t get it, or youmight but it will take years. So you have to designyour trial so as not to affect patient care.”

Manufacturer #1 also comments on the ethicalapproval process and the R&D committees that must beattended to when employing more formal methodsusers, both patient users and professional users in thecollection of clinical data.

#1: R&D committees. That’s another thing. You starta study and surgeon wants to collect clinical data tobe sure that they want to use this. Their experiencethat this is the product they want to use. You have tonotify the hospital R&D and then suddenly they see itas an R&D project, so they throw on their overheadsand it’s an extra hurdle to go through. They startreviewing it in addition to ethical board. This iswhat has changed in the last few years. In those daysyou notified the larger University hospitals. Increas-ingly has to be approved. Some are straightforward;others ask extra questions that delays process - UKresearch.

This perceived difficulty with obtaining ethicalapproval may be one reason why manufacturers tendnot to use formal methods, or engage in systematicresearch activity in order to inform the MDDD process,particularly at the early stages of development. Indeed

manufacturers appear to actively avoid involving profes-sional and patient users in the process, in fear that theMDDD process could be delayed for years as a result ofthe ethical approval process. As identified in section 3.2,informal discussion with clinical staff is seen as a morerealistic, pragmatic, and feasible route to informing theMDDD process, which can be carried out informallyand hence without ethical clearance. Manufacturer #3goes on to describe the strategy that they use to getaround the challenge of obtaining ethical approval:

#3: “So you take the least hard route, to not affectpatient care in any way whatsoever, and design yourstudy to sit on the back of that. So the study mightnot be optimal, but at least you can do it.”

Setting up device design and development activity so asto avoid the need for user involvement seems to be the‘method’ of choice for some device manufacturers. Despitethis being seen as a potentially sub-optimal approach, per-haps less effective in evaluating the extent to which designinnovations may be accepted by users, it seems a routeworth taking given the alternative of incurring long delaysas a result of the ethical approval process.

The nature of user contributionsThe measures used to evaluate the effectiveness of newlydeveloped devices towards the end of the MDDD processecho the findings presented earlier, that formal methodsemployed in any part of the MDDD process are seen bymanufacturers as having the potential of slowing the processdown, and incurring additional and unnecessary costs andoverheads that otherwise could be avoided if a less formalapproach was taken. Most commonly, manufacturersreported that the success of a new device is typically mea-sured by the absence of receiving customer complaints or‘bad news’ emerging as a result of the device being used inthe field. Manufacturer #1, in response to being asked howthe success of their device is measured responded as follows:

#1: “The absence of bad news. The fact that we haveaccess to the surgeons who are using the product andany adverse effects would be reported and we wouldbe aware early on.”

With regards to seeking patient feedback about the suc-cess of a medical device in providing an effective healthintervention, Manufacturer #3 takes a similar ‘no badnews is good news’ approach to the design and function-ing of their product. When asked whether any patientfeedback is sought about the device, the reply was:

#3: “Not explicitly, we haven’t gone out to get it, butwe get feedback though the users (clinicians). It’s


of 12

non-invasive, so as far as the patients concerned it’snot a problem to use it. No-one has said they don’twant it done or had any problems with that.”

Manufacturer #9, commenting on whether any formalmethods are used for converting user needs into devicedesign requirements, also indicated that they adopt areactive stance to customer suggestions and complaints,which is their default position on such matters.

#9: “Erm, we certainly have regular meetings herewhere we will look at customer feedback, you know,whether it’s customer complaints or customer sugges-tions or whatever, yes so, yes we do have a means todo that...”

Some evidence of formal user involvement did emergefrom the interview data, however, it indicated that themajority of formal user involvement took place at theclinical trials stage, i.e. after the product had been devel-oped, and manufacturers were at the stage of demon-strating its efficacy and clinically effectiveness.Therefore, in effect any considerations in terms ofdesign preference of the medical device that may benefitthe user in terms of their treatment or use of the devicehad already been made prior to this formal user involve-ment. Manufacturer #2 highlights the notion that thepatient user is primarily seen as being of value whenattempting to demonstrate the clinical effectiveness of anew medical device.. Interestingly, patient users are notseen as primarily informing the general design of thedevice at earlier stages of the MDDD process, at leastnot in a formal capacity. When asked whether formaluser methods are employed within the MDDD process,Manufacturer #2 responded:

#2: “There is the formal method of gaining patientdata which means at the moment for example wehave conducted clinical trials, clinical investigations.All sorts of clinical information gathering is going onat the [hospital name]. We are also... we tend togather information first of all on the effectiveness orefficacy of it...Any advance, any variation that islikely to happen to the unit, any proposals for changethat are going to improve the machine are all testedout in the clinical environment.”

Manufacturer #5 reported that the key driver for col-lecting formal user data, relating to the performance ofa device, comes from the possibility that an organisationsuch as the National Institute for Clinical Excellence(NICE) may decide to investigate the efficacy of theirdevice at some unknown point. Therefore, the key driverfor formally collecting user generated data is to fulfil the

potential future requirements of external standards orpurchasing agencies. The motivation to collect userfacing data does not appear to be borne out of an inher-ent belief, on the manufacturer’s behalf, that this userdata would add any significant value in terms of fulfill-ing their own need to develop more effective devices orindeed learn more about the effectiveness and efficiencyof their own device.

#5: I think, if you take an organisation like NICE asa customer, or any of the other health technologyorganisations, one doesn’t know whether your parti-cular intervention is going to be assessed by them,and if so at what point, and one can see that as [theproduct] gains momentum and the NHS starts tolook at what its spending on [this] surgery, then itmay be something that NICE feel, or NICE get direc-ted to take a look at. So it’s a kind of a problem tous to understand when that’s going to happen and itwill certainly be a challenge; we have to thereforemake sure that we are gathering the evidence in casethey do.”

Discussion and ConclusionsIn this study, 11in-depth interviews were carried outwith medical device manufacturers, who were asked tocomment on the role and value they believed that usershave within the MDDD process. Given the small samplesize, it is recognised that the results of this study shouldbe considered as provisional. However, given the limitedexisting research in this area, the findings provide animportant point of reference for further work.The results revealed that manufacturers tend to priori-

tise the views of more senior health professionals abovethose that are less senior as well as patients. Furthermore,manufacturers’ perceptions of the customer and the userhas become conflated, partly due to the strong salesfocus of manufacturers, seeking device design input fromthose individuals who make purchasing decisions, asopposed to the users of the devices. With regards to seek-ing input from the patient user, there was little motiva-tion to engage in such practice which was seen as anineffective use of resources, with patients, and less seniorhealth professionals being perceived as having littleimpact or influence on general device sales, largely dueto the inbuilt culture of patients being ‘taught down’their needs from health care professionals.Only one out of 11 manufacturers claimed to regularly

use formal user centred design methods within theMDDD process. Interestingly, this individual alsoreported that they were familiar with such methods as aresult of being introduced to them within the universitysetting in which they carried out the initial formulation


of 12

and design of the product they were discussing. Thisexperience appeared to have had a lasting effect, as theyreport the ongoing use of formal research methods tothis day. It is therefore possible that a contributory fac-tor to lack of engagement with formal methods is a lackof education, familiarity or confidence in their use.Necessary training has indeed been found to be a factorthat affects the level of uptake of usability methods inmedical device development [34]. Informal methodswere typically preferred by manufacturers, in the formof extemporised discussions with a small number ofesteemed medical experts. There was also a belief that ifa manufacturer wished to be competitive and responsivewithin a fast moving market, formal methods do notoffer the necessary versatility and are more generally notappropriate for purpose, despite the proposed benefitsthat academic literature suggests such methods promiseif applied within the health care sector [35,36]. Researchin the technology design domain suggests it is necessaryto adopt a flexible and evolutionary stance when apply-ing formal methods to cater for the unique context andorganisational cultures that present themselves withinany new design challenge [37]. Similarly, there may bevalue in increasing awareness of the versatility of exist-ing human factors engineering methods, and in specia-list contexts to explore the potential development ofmore agile and tailored methods that cater for manufac-turers’ needs in the context in which they are to beapplied. It has been suggested that, as a consequence ofthe increased pressure from standards agencies to incor-porate formal user methods into the MDDD process,some manufacturers may ‘misuse’ existing human fac-tors methods [38]. The reasons for this are unclear. Is itperhaps in a bid to make methods more fit for purpose,or as a result of not being fully aware of the ways inwhich existing methods should be applied?The perceived barriers to user involvement within the

MDDD process were linked to the notion that manufac-turers seek out those individuals that will be most influen-tial in making purchasing decisions for their products.Consequently, involving patient users appeared to be oflowest priority, since it was believed that they held thelowest level of influence over whether health care organi-sations purchased their products. These findings are sup-ported by our earlier preliminary work relating to barriersto user involvement [39,40]. The examples that did emergeof consulting users via formal methods, tended to occur atthe end of the MDDD process, after the device had beendesigned and developed, where users played a passive rolein aiding the manufacturer in verifying the efficacy of thedevice primarily for the purposes of satisfying externalstandards and purchasing agencies requirements. One keyreason for avoiding user involvement was the difficultythat this presents when attempting to gain ethical approval

for user elicitation studies. Manufacturers conceded that,although avoiding user involvement may perhaps be sub-optimal, it is necessary if they are to remain competitivewithin a fast moving market. Warlow [41] and Stewartet al. [42] both propose that over-regulation of clinicalresearch poses a significant threat to public health and itseems that this study support this. In particular, our find-ings suggest that the seemingly unnecessary bureaucracyassociated with obtaining ethical approval for non-inter-ventional and low-risk studies, that seek to capture useropinions and requirements, leads to manufacturersexcluding the voice of the user from the developmentprocess.This study reveals that the notion of proactively invol-

ving the user within the MDDD process in generalslows down the process. Rather, a reactive ‘no bad newsis good news’ stance is taken, only taking into accountusers input if they are presented in the form of com-plaints or feedback on devices that have already beenreleased into the health care system. The only evidenceof engagement with formal methods of user involvementis apparent when the use of such methods are manda-tory, dictated to manufacturers by standards and pur-chasing agencies. Given the findings of this study, theappropriate employment of formal methods by manufac-turers is unlikely to occur to significant levels withoutdeliberate efforts to encourage and support manufac-turers in doing so. The following recommendations pro-pose where some of these efforts should focus, in orderto achieve increased levels of user engagement by manu-facturers, as is now stipulated by IEC 62366 [14]:

- Research to better understand the requirements ofmanufacturers, in terms what is required fromhuman factors engineering methods in order tomake their use more feasible and accessible inpractice.- Provision of training on the use and benefits ofemploying formal human factors engineering meth-ods at every stage of the MDDD process.- Health care providers should implement formalprocesses to ensure better communication andsynergy between those making purchasing decisionsand the actual users of the devices.- Provisions should be made within the ethicalapproval process that enables medical device manu-facturers to engage more easily with users with mini-mal levels of bureaucracy whilst also ensuring thatall research is conducted in an ethical manner thatprotects healthcare staff and patients.

AcknowledgementsThis study was in part funded by grant number Ref: GR/S29874/01 from theEngineering and Physical Sciences Research Council


of 12

Author details1Department of Computer Science and Technology, University ofBedfordshire, Park Square, Luton, LU1 3JU, UK. 2Department of InformationSystems and Computing, Brunel University, Uxbridge, UB8 3PH, UK.3Department of Electrical and Electronic Engineering, University ofNottingham, University Park, Nottingham, NG7 2RD, UK.

Authors’ contributionsAll authors contributed to the conceptual design of this study. AGM and JBcarried out primary data analysis and drafted the manuscript. MPC and JLMcontributed to data collection and provided domain expertise. All authorscontributed to redrafting the manuscript.

Competing interestsThe authors declare that they have no competing interests.

Received: 28 October 2010 Accepted: 28 February 2011Published: 28 February 2011

References1. Gosbee J: Human factors engineering and patient safety. Quality and

safety in health care 2002, 11:352-354.2. Shah SGS, Robinson I: Medical device technologies: who is the user?

International Journal of Healthcare Technology and Management 2008,9:181-197.

3. Shah SGS, Robinson I, Alshawi S: Developing medical device technologiesfrom users’ perspectives: A theoretical framework for involving users inthe development process. International Journal of Technology Assessment inHealth Care 2009, 25:514-521.

4. Wilson T, Sheikh A: Enhancing public safety in primary care. BritishMedical Journal 324:584-587.

5. Martin JL: Design for patient safety: User testing in the development ofmedical devices National Patient Safety Agency, London, UK; 2010.

6. Sax H, Allegranzi B, Uckay I, Larson E, Boyce J, Pittet D: ’My five momentsfor hand hygiene’:a user-centred design approach to understand, train,monitor and report hand hygiene. Journal of Hospital Infection 2007,67:9-21.

7. Wressle E, Samuelsson K: User Satisfaction with Mobility Assistive Devices.Scandinavian Journal of Occupational Therapy 2004, 11:143-150.

8. De Vito Dabbs A, Myers B, McCurry K, Dunbar-Jacob J, Hawkins R, Begey A,Dew M: User-Centered Design and Interactive Health Technologies forPatients. Computers Informatics Nursing 2009, 27:1-16.

9. Bias RG, Mayhew DJ: Cost-Justifying Usability: An Update for the Internet AgeSan Francisco, CA, USA: Morgan Kaufmann Publishers; 2005.

10. Council Directive 93/42/EEC of 14 June 1993 concerning medicaldevices. Official Journal of the European Communities 1993, L169.

11. Martin JL, Norris JN, Murphy E, Crowe JA: Medical device development:The challenge for ergonomics. Applied Ergonomics 2008, 39:271-283.

12. Sawyer D: Do it by design: an introduction to human factors in medicaldevices. US Department of Health and Human Services Food and DrudAdministration 1996.

13. IEC 2004: General Requirements for Safety of Electrical Medical EquipmentCollateral Standard: Usability , 60601-1-6.

14. ISO/IEC 2008: Medical Devices - Application of Usability Engineering to MedicalDevices. International Organisation for Standardisation Geneva;, 62366.

15. Martin JL, Murphy E, Crowe J, Norris B: Capturing user requirements inmedical device development: the role of ergonomics. PhysioogicalMeasurement 2006, 27:49-62.

16. Shah SGS, Robinson I: Benefits of, and barriers to involving users inmedical device technology development and evaluation. InternationalJournal of Technology Assessment in Health Care 2007, 23:131-137.

17. Cooper RG: The invisible success factors in product innovation. Journal ofProduct Innovation Management 1999, 16:115-133.

18. Kujala S: User involvement: a review of the benefits and challenges.Behaviou and Information Technology 2003, 22:1-16.

19. Brown DS: The challenges of user-based design in a medical equipmentmarket. Field methods casebook for software design John Wiley and Sons,Inc; 1996, 157-176.

20. Butler MB: Getting to know your users: Usability roundtables at Lotusdevelopment. Interactions 1996, 3:23-30.

21. Van der Panne G, Van Beers C, Kleinknecht A: Success and failure ofinnovation: a literature review. International Journal of InnovationManagement 2003, 7:309.

22. Stewart J, Williams R: The Wrong Trousers? Beyond the Design Fallacy:Social Learning and the User. In User involvement in innovation processesStrategies and limitations from a socio-technical perspective. Edited by:Rohracher H. Munich: Profil-Verlag; 2005.

23. Wixon D: Evaluating usability methods: why the current literature failsthe practitioner. Interactions 2003, 10:28-34.

24. Steen M, Kuijt-Evers L, Klok J: Early user involvement in research anddesign projects - A review of methods and practices. 2007.

25. Steen M, de Koning N, Takahashi I: Innovating together with users -Taking the organizational context into account. 2009.

26. Grocott P, Weir H, Ram MB: A model of user engagement in medicaldevice development. International Journal of Health Care Quality Assurance2007, 20:484-493.

27. Gulliksen J, Boivie I, Goransson B: Usability professionals - currentpractices and future development. Interacting with Computers 2006,18:568-600.

28. Boivie I, Gulliksen J, Goransson G: The lonesome cowboy: A study of theusability designer role in systems development. Interacting withComputers 2006, 18:601-634.

29. Boyatzis RE: Transforming qualitative information: Thematic analysis and codedevelopment Thousand Oaks, CA, USA: Sage Publications; 1998.

30. Taylor SJ, Bogdan R: Introduction to Qualitative Research Methods: The Searchfor Meanings New York, NY: Wiley; 1984.

31. Silverman D: Doing Qualitative Research London: Sage Publications; 2005.32. Joffe H, Yardley L: Content and thematic analysis. In Research Methods for

Clinical and Health Psychology. Edited by: Marks DF, Yardley L. London, UK:Sage Publications; 2004:56-66.

33. Dey I: Qualitative Data Analysis: A User-Friendly Guide for Social ScientistsLondon: Routledge; 1993.

34. Garmer K, Liljegren E, Osvalder AL, Dahlman S: Application of usabilitytesting to the development of medical equipment. Usability testing of afrequently used infusion pump and a new user interface for an infusionpump developed with a Human Factors approach. International Journallof Industrial Ergonomics 2002, 29:145-159.

35. Carayon P: Human factors of complex sociotechnical systems. AppliedErgonomics 2006, 37:525-535.

36. ginsburg: Human factors engineering: A tool for medical deviceevaluation in hospital procurement decision-making. Journal ofBiomedical Informatics 2005, 38:213-219.

37. Stewart J, Williams R: The Wrong Trousers? Beyond the Design Fallacy:Social Learning and the User. In User involvement in innovation processesStrategies and limitations from a socio-technical perspective. Edited by:Rohracher H. Munich, Germany: Profil-Verlag; 2005.

38. Engineering human factors is critical. [http://medicaldesign.com/engineering-prototyping/Engineering-human-factors-critical/].

39. Martin JL, Craven MP, Norris BJ: MATCH: A new industry-focusedapproach to medical device development. In Healthcare systemsErgonomics and Patient Safety: Human Factor, a bridge between care andcure. Edited by: Tartaglia R, Venneri F, Parlangeli O, Tomassini CR, Bagnara S.London, UK: Taylor 2005:294-297.

40. Craven MP, Martin JL: How does the Healthcare Industry involve Users inMedical Device Development? - Pointers for UbiHealth, Nottingham,England, 7-10 September 2004. 2004.

41. Warlow C: Over-regulation of clinical research: a threat to public health.Clinical Medicine, Journal of the Royal College of Physicians 2005, 5:33-38.

42. Stewart PM, Stears A, Tomlinson JW, Brown MJ: Regulation - the realthreat to clinical research. British Medical Journal 2008, 337.

Pre-publication historyThe pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1472-6947/11/15/prepub

doi:10.1186/1472-6947-11-15Cite this article as: Money et al.: The role of the user within the medicaldevice design and development process: medical device manufacturers’perspectives. BMC Medical Informatics and Decision Making 2011 11:15.


of 12

http://www.ncbi.nlm.nih.gov/pubmed/12468696?dopt=Abstract

















http://medicaldesign.com/engineering-prototyping/Engineering-human-factors-critical/

http://medicaldesign.com/engineering-prototyping/Engineering-human-factors-critical/



http://www.biomedcentral.com/1472-6947/11/15/prepub

research article open access the role of the user within the medical device design … ·...

Documents