Accred Qual Assur (1997) 2 :381–387Q Springer-Verlag 1997 PRACTITIONER’S REPORT

Reports and notes on experiences with quality assurance, validationand accreditation

Zbigniew Dobkowski Problems of validation of computerised

instruments for accredited chemical

laboratories

Received: 19 November 1996Accepted: 20 March 1997

Z. Dobkowski (Y)Industrial Chemistry Research Institute(ICRI)Rydygiera 8, PL-01-793 Warsaw, PolandTel.: c48-22-633-82-98;Fax: c48-22-633-82-95;e-mail: zbigdo@aquila.ichp.waw.pl

Abstract Some problems of vali-dation of computerised instrumentsare reviewed briefly, taking essen-tial standards and guides into ac-count. The significant role of certi-fied standard reference materials isunderlined. An attitude of sup-pliers towards the validation of in-struments is presented, and pro-ducers’ responsibilities and obliga-tions are discussed. The “black-box” concept is recommended as apreliminary step for the validationof computerised instruments. Twoexamples for gel permeation chro-matography are given that illus-trate a bad manufacturer’s practice(BMP) and good manufacturer’spractice (GMP). In the case ofBMP, a need is expressed for aguide and for regulations thatshould be implemented into thequality assurance system. It has

been proposed that the EURA-CHEM/VAM draft of guidance forqualification/validation of instru-ments should be amended by in-corporating the “black-box” ap-proach as a preliminary procedurefor validation of computerised in-struments, a retrospective valida-tion procedure if the need for cur-rent validation was not foreseen ornot specified, and a procedure (orselection rules) for qualification ofthe supplier. Moreover, the mecha-nisms of inspection to control theobservance of the standardisedrules and commonly recognisedrecommendations should also beconsidered by international qualityorganisations.

Key words Validation ofcomputerised instruments 7Black-box concept for validation

Introduction

The problem of validation and/or qualification of labo-ratory instruments seems to be omitted in generalstandards and guides for quality assurance such as theISO 9000 and equivalent EN 29000 standards and ISO/IEC Guide 25 or EN 45000 standards. These standards,written in broad terms, do not go into details on instru-ments.

There are more detailed recommendations concern-ing laboratory equipment included in the OECD publi-cations [1, 2] and the guideline documents for chemicallaboratories [3, 4]. However, the validation of methods(or testing procedures) and, to some extent, computer-ised systems are mainly considered in these standards[2–4]. It has been noted that computerised systems maybe involved increasingly as an integral part of automat-ed equipment [2]. It is known, however, that for testingaccording to the requirements of the validated methoda validated instrument (or equipment) has to be availa-

382

ble. Therefore, at least two of the most important ele-ments of measurements have to be validated in everylaboratory:– A standard operation procedure, testing procedure,

or method of measurement,– A measuring device or measuring instrument.The problem is of crucial importance for every accre-dited laboratory, but some rather unusual aspects haverecently been noted that seem to be specific for theCentral and East European (CEE) countries’ relationswith so-called Western countries. Some of Westernsuppliers and/or producers of expensive instruments,even those having certificates according to the ISO9000 standards, tend to deliver their out-of-date prod-ucts to the CEE market where the mechanisms for pro-tection of customers are not yet fully established. Sup-pliers refuse to validate their products in the case ofcomplaints of customers, if the results of users’ valida-tion are not satisfactory. Thus, the reputation of ISOstandards and the whole quality assurance system canbe questioned.

Moreover, the problem of equipment validation isincreasing with the sophistication of instrumentation,its computerisation, and reliance on software for dataacquisition and for the data treatment to obtain finalresults of analysis and testing.

The importance of the problem of instrument vali-dation for chemical testing laboratories that were im-plementing the Western European standardised qualityassurance (QA) system has been presented and dis-cussed at conferences of Polish [5, 6] as well as interna-tional [7–10] communities. As a result, the problem ofequipment validation has been tackled by EURA-CHEM in 1995 for chemical laboratories [8, 11] and si-multaneously by the EURACHEM-UK/VAM Work-ing Group [12]. Moreover, some recent publications aretaking the problem into account with the essentialstatement that “the quality process for commercial ana-lytical equipment starts with the selection of the ven-dor” [13].

Therefore, some selected aspects of validation ofcomputerised laboratory instruments are presented anddiscussed in this paper as the problem is becomingmore general and important for every accredited test-ing laboratory.

Present status on instrument validation

Standards and guides

As has been mentioned above, the problem of valida-tion of laboratory instruments seems to be omitted ingeneral standards and guides for quality assurance suchas the ISO 9000 and equivalent EN 29000 standards

and ISO/IEC Guide 25 or EN 45000 standards. Theterm “equipment validation” or “instrument valida-tion” has not appeared in glossaries of quality defini-tions. It has not been included even in the OECD GLPrecommendations [1], the WELAC/EURACHEMGuidance Document [3] or the CITAC Guide 1 [4], ex-cept for the validation of computers.

In the OECD GLP document [1], it is said that theapparatus used for the generation of data and for con-trolling environmental factors should be suitably lo-cated and of appropriate design and adequate capacity.It should be periodically inspected, cleaned, main-tained, and calibrated according to standard operatingprocedures. A record of procedures should be main-tained. However, the method for checking the “appro-priate design” was not given.

The guidelines [3, 4] recommend that the laboratoryequipment should be checked during the audit in orderto determine whether, for example– the equipment in use is suited to its purpose,– instrument calibration procedures are documented,

or– instrument performance checks show that perfor-

mance is within specification.However, the problem is not only to ensure that

equipment performs to specification but also to haveconfidence that it produces valid answers in all situa-tions and for all parameter values.

The clauses concerning the validation of computers[3, 4] can be modified and applied to contemporarycomputerised laboratory instruments, since measuringdevices, together with computers of such instruments,form uniform systems. The computer is an importantand intrinsic part of the measurement process: it con-trols both the instrument performance and data han-dling and has to be validated rigorously together withthe whole instrument.

Recently published documents and papers [2, 12, 13]took the instrument validation problems into account.“The demonstration that a computerised system is suit-able for its intended purpose is of fundamental impor-tance and is referred to as computer validation. Thevalidation process provides a high degree of assurancethat a computerised system meets its pre-determinedspecifications” [2]. It implies that computerised instru-ments should be validated as computers. In the draft ofEURACHEM/VAM guidance [12] “the terms ‘valida-tion’ and ‘qualification’ are used widely and often toconvey the same meaning”, and the whole process ofequipment qualification/validation is covered by sev-eral individual stages, such as design, installation, oper-ational, and performance qualifications/validations.Many requirements and recommendations have beenformulated. However, the problem of how the guide-lines are implemented in practice by producers andvendors still exists.

383

Producers’ declarations on the validation ofinstruments

Recently, most manufacturers of computerised labora-tory instruments have become aware of the problems ofequipment validation, and they usually declare their re-sponsibility for the instrument validation. Some exam-ples are given below.1. Hewlett-Packard was probably the first manufactur-er to issue (in 1993–1994) series of information bro-chures on the validation of liquid chromatographs [14]describing the proposed procedures of validation, in-cluding testing and validation of equipment (both hard-ware and software). Validation parameters are se-lected, depending on the type and scope of validation,e.g. for each column the plate number, peak tailing fac-tor and pre-column pressure is checked. At this stage,not only the correctness of the data is checked, but it isalso demonstrated proved that the equipment used togenerate that data is fit for its task. Thus, factory vali-dated products and tools for on-site validation are of-fered.2. Waters claim their ability to validate the HPLC soft-ware and hardware, and that certified validation spe-cialists are available to help customers to assure thequality of the results [15]. Details, however, have notbeen disclosed.3. Milton Roy offer Softcards (i.e. software cards) forthe performance validation of the spectrophotometers[16]. The programs include: wavelength accuracy, noise,stray radiant energy, photometric linearity/accuracy,zero percent transmission and output devices (such asan internal or external printer, if installed).4. ATI Unicam Analytical Technology also offer theGLP unit for automatic performance checks to main-tain the validity of UV/VIS spectrometers [17]. Thus,the wavelength and absorbance accuracy is checkedwith filters that are certified and traceable to interna-tional standards. Moreover, stray light, noise and driftare also checked as an independent verification of in-strument validation.

It is the task of the users to test any equipmentagainst the manufacturers’ specification and manufac-turers’ validation. Usually it is considered that if themanufacturer’s validation is not confirmed by the user’svalidation results, then market forces should ensurethat poor instruments would fall. However, in somecases such market forces are rather weak or even donot exist. Therefore, it is expected that the quality assu-rance system should create universal rules that help ac-credited laboratories in purchasing high quality instru-ments.

Important elements for validation of instruments

Defining intended use and instrument functions

The definition of validation of computers, given in theguidelines for chemical laboratories [3, 4], can be mod-ified and extended to computerised laboratory instru-ments. This has been done, in fact, in the recent guid-ance of OEDC [2] and the EURACHEM/VAM draft[12].

The definition of validation (of a computer, compu-terised instrument, or equipment) should contain suchelements as checking data for correctness and com-pliance with applicable standards, rules and conven-tions. In the context of equipment, validation involveschecking for correct performance.

Understanding of this term is expressed by some ofthe Hewlett-Packard definitions [14]:– Validation is a formalised procedure requiring an op-

erator to (a) deine what is to be done, (b) test thatwhat has been defined can indeed be done, (c) per-form the operation as defined, and (d) keep com-plete records of what actually occurred for as long asthe operator uses the process.

– Verification is the step in a validation process wheretests check that performance specification can bemet, either in part or as a whole.

– Revalidation is a repetition of validation necessaryafter the measuring process has been changed, e.g. ifthe system is upgraded.In general, it can be said that instrument validation

means the performance check of the instrument.According to all relevant standards and guides [1–4,

12], the equipment in use should be suited to its pur-pose. Therefore, the purpose of instrument functionhas to be precisely defined for the validation proce-dure. It should be defined in terms of what is to bedone.

The degree of validation necessary depends on theexact use of the computerised laboratory instrument.Therefore, for each computerised instrument, the pro-posed use should be defined so that the degree of vali-dation necessary may be established. The performanceparameters that have to be checked for some instru-ments in chemical laboratories are listed in guides [3,4]. These parameters can also be considered for the in-strument validation [2, 12]. The acceptance criteriahave also to be formulated and checked prior to puttingthe instrument into routine use [2].

Calibration and traceability

According to relevant standards and guides [1–4, 12],the essential requirement for laboratory measuring in-

384

GPC/SEC CHROMATOGRAPH

Hardware Software*Sample solution

] Standard solutions(s) Autosampler: Data acquisition*Reference material

solution (input datafor black-box test

sample concentrationand volume, samplingprecision

program

Calibration program(type of calibration

*GPC solvent Pump:constant flow andflow value

polynomial, statisticaltreatment, criteria ofacceptance) *Results:

Columns:bed (type, grains, poresize),number of columns,total resolution

Detector(s):type (RI, UV, VISC,etc.), connections,dead volume,sequence ofmultidetectors

Computer system withmonitor and printer

Data treatmentprogram (rawchromatogramtreatment,algorithms)

Calculation of results(Mn, MV, MW, etc.MW/Mn etc.,algorithms,Mark-Houwinkconstants for MV).

– calibration curve– averages molecular

weights: Mn, MV,MW, etc.

– polydispersity:MW/Mn, etc.

– chromatograms(normalised)

Fig. 1 The “black box” of theGPC chromatograph

struments is an instrument calibration. Usually individ-ual calibration programmes and calibration proceduresare needed for each type of instrument. Instrument cal-ibration procedures should be documented and recordsof calibration satisfactorily maintained. Calibration in-tervals for some instruments in chemical laboratoriesare given in relevant guides [3, 4].

The overall programme for the calibration of meas-uring instruments should ensure that all measurementsare traceable either to national or international meas-urement standards or to certified reference materials[4]. If such a calibrant or certified reference material isnot available, a material with suitable properties andstability should be selected or prepared by the labora-tory and used as a laboratory reference.

The recent draft on qualification/validation of analy-tical instruments [12] considers the problem of calibra-tion of instruments using reference materials or stand-ards traceable to national or international standardsand traceability.

The “black-box” syndrome

It is known that computers, whatever their type, sufferfrom the “black-box” syndrome [3, 4]. Since contempo-rary laboratory instruments are usually computerised,the whole instrument system may be treated as the

“black box”. Such an approach seems to be correct,since most sophisticated instrument hardware cannotbe seen inside the instrument and is usually unknownto the operator. Acess to the hardware is even forbid-den by the warranty and service regulations.

Therefore, the clauses concerning the validation ofcomputers may be applied to computerised laboratoryinstruments. They can be read as follows. Computer-ised laboratory instruments, whatever their type, sufferfrom the “black box” syndrome: an input is made atone end, and an answer is produced at the other. Be-cause what happens inside cannot be seen, it must beassumed that the box is functioning correctly. For thepurpose of validation it is usually acceptable to assumecorrect operation if the computerised laboratory instru-ment (computer) produces the expected answers wheninput with well-characterised parameters.

The input of well-characterised parameters is as-sured by the application of standard reference materi-als or certified reference materials. If such materials arenot available, laboratory reference materials may beapplied. Therefore, the instrument validation using the“black-box” approach can be considered as an ex-tended calibration procedure for the whole instrument,because reference materials are applied both for thecalibration and for the validation of instruments. Such anew and significant role of reference materials, in parti-cular certified reference materials, should be noted.

385

Examples for the validation of gel permeation

chromatographs using the “black-box” approach

The gel permeation chromatography (GPC) method,known also as size exclusion chromatography (SEC), iscommonly used for determination of average molecularweights (Mx), where subscript x denotes number (n),viscosity (v), or weight (w) averages and molecularweight distribution (MWD) of polymers measured asthe polydispersity index Mw/Mn. Exact Mx and MWDdata are extremely important to judge characteristics ofpolymers and their end-use properties. The principle ofthe method is based on the separation of various sizesof macromolecules on a specially prepared (usually po-lymer gel) bed. Therefore, many parameters influencethe final result of measurement (see Fig. 1). A properoperation needs a high level of computerisation for cal-ibration of molecular weight values, data acquisition,treatment of raw data (where various treatment criteriamay be chosen), and final calculation of results (wherevarious algorithms may be applied).

The black-box concept, as recommended in [3], hasbeen used for the validation of computerised instru-ments – GPC chromatograph of Waters (Case a) andShimadzu (Case b). The validation has was performedby the user and the same polymer standard PS DOW1683 was used in both cases.

Case a Results for the Waters 150CV GPC instrument:double detectors (RI detector and single capillaryVISC detector), temperature 40 7C, solvent tetrahydro-furan (THF)

Input data Output data(standard referencematerialPS DOW 1683):

(results):

Mnp1000000B2000 (2%) Mnp115720 (16%)MV not given MVp247760MWp250000B3000 (1%) MWp261385 (5%)MW/Mnp2.50 MW/Mnp2.26 (P10%)[h]p84.90 cm3/g at 30 7C [h]p101.0 cm3/g (19%)Calibration Measured using VISC

optionKp1.65!10P2 cm3/g Kp0.63!10P2 cm3/g

(P62%)ap0.703 ap0.778 (11%)

Note that [h] is the intrinsic viscosity that can be meas-ured using a Waters VISC detector; K and a are con-stants of the Mark-Houwink equation; error values aregiven in parentheses.

It has been found that the output data are not con-sistent with the values of standard parameters. In parti-cular, the errors for the viscometric output data are ex-tremely high. The measured values of constants of the

Mark-Houwink equation are unreliable (not consistentwith literature values as well as with values determinedduring calibration of the same instrument. Thus, the in-strument is not valid for measurements of viscometricparameters as specified by the producer, and the user isnot satisfied. Moreover, the producer refused to per-form the validation of his instrument. Thus, this is anexample of a bad manufacturer’s practice (BMP) thatcontradicts published producer’s declarations [15]. Fur-ther user’s investigations revealed that the hardware, inparticular for the VISC detector, is inadequate [9]. Theabove-described attitude of the producer may be ex-plained by the fact that the use of the single-capillaryviscometer as a sensitive and reliable GPC detector hasbeen questioned. This fact was known to the producerin advance before the sale of his product [18–20], andthe information was not delivered to the user as theguidance [12] recommend. Again, it is a BMP. More-over, the accreditation body (Lloyd), which deliveredthe accreditation certificate to Waters, explained thatthe above-described instrument was produced in 1992and it is out of the scope of the certificate issued in1993. In such a case, however, a retrospective evalua-tion should be performed according to the OECD GLPrecommendations [2].

Case b Results for the Shimadzu C-R4A GPC instru-ment: UV detector, temperature 36 7C, solvent tetrahy-drofuran (THF)Input data Output data(standard referencematerialPS DOW 1683):

(results):

Mnp100000B2000 (2%) Mnp97176 (P3%)MWp250000B3000 (1%) MWp239615 (P4%)MW/Mnp2.50 MW/Mnp2.47 (P1%)

Note that the instrument is not designed for measure-ments of viscometric parameters; error values are givenin parentheses.

According to the instrument validation made by thesame user, the errors for the average molecular weightvalues are acceptable. Thus, the instrument is consid-ered as valid one for the measurements of polymer mo-lecular characteristics, and the user is satisfied. It is anexample of good manufacturer’s practice (GMP).

Responsibility for the instrument validation

There are three possible option: responsibility for theinstrument validation may be placed on (1) the supplier(manufacturer, producer or vendor), (2) user or labora-tory staff, or (3) both of these.

It seems that from the business goals point of viewboth parties should be interested in the instrument val-idation. Of course, producers are responsible for the

386

appropriate design, and, since they know the construc-tion of their products in detail, they have the technicalknow-how, and may well be the most effective peopleto do the job. However, producers must be recognisedas being reliable by the market and/or be accredited bya recognised accreditation body. Moreover, they haveto be honest to their customers. Therefore, it should beemphasised that in every procedure for validation of in-struments the quality process starts with the selectionof the producer and the vendor [13].

There is also an opinion that it is exclusively the jobof the purchaser or user (that is the laboratory staff) totest any equipment against the manufacturer’s specifi-cation and manufacturer’s results of validation (if suchresults exist). It is believed that, if poor quality of theinstrument is found by the user, existing market forcesand/or legal regulations should ensure that such instru-ments would fail to be sold. However, such forces and/or regulations may not exist in regions transformingtheir economy system, e.g. in CEE countries. Then, thequality assurance system would provide tools for elimi-nation of bad manufacturer’s practice (BMP).

It seems that a proper solution is offered by Hew-lett-Packard [14]: instrument validation is performedon a newly implemented system by company personnelthat are not system developers (so-called a-testing),and then, after debugging of the a-testing results, in-strument validation is performed at a later stage of im-plementation by operators employed at the customer’ssite (so-called b-testing). Hewlett-Packard’s procedureshave also been described in [13]. They comply with thecontemporary concept of quality, ensuring customer sa-tisfaction. The only shortcoming of the procedures [13,

14] is that examples of how it works in practice arescarce.

Concluding remarks

Both manufacturers and users are aware that problemsof instrument validation/qualification are increasing. Inparticular, the validation of sophisticated and compu-terised laboratory instruments is becoming very impor-tant.

It is proposed as a general approach to apply theconcept of the “black box” and relevant reference ma-terials for the preliminary stage of instrument valida-tion/qualification that can be performed by the user.

In the case of negative results of user’s validation, amore precise instrument validation should be perform-ed by the producer and the user together according to arecognised, and preferably standardised, procedure.Such a procedure does not yet exist. However, the EU-RACHEM/VAM draft [12] offers a solution.

It is proposed that the EURACHEM/VAM draft[12] should be amended by incorporating the “black-box” approach as a preliminary procedure for valida-tion of computerised instruments. A retrospective vali-dation procedure in the case where the need for currentvalidation was not foreseen or not specified [2] shouldalso be considered, as well as a procedure for qualifica-tion (or selection) of the supplier [13].

Moreover, the mechanisms of inspection to controlthe observance of the standardised rules and commonlyrecognised recommendations should also be consideredby international quality organisations.

References

1. The OECD Series on Principles ofGood Laboratory Practice and Com-pliance Monitoring, No. 1: TheOECD Principles of Good Laborato-ry Practice, Environment MonographNo.45, OECD, Paris 1992

2. The OECD Series on Principles ofGood Laboratory Practice and Com-pliance Monitoring, No. 10: The Ap-plication of the Principles of GLP toComputerised Systems, EnvironmentMonograph No. 116, OECD, Paris1995

3. WELAC/EURACHEM GuidanceDocument (1993) Accreditation forchemical laboratories, guidance onthe interpretation of the EN45000 se-ries of standards and ISO/IEC guide25, 1st edn

4. CITAC/EURACHEM Guide 1(1995) International guide to qualityin analytical chemistry: an aid to ac-creditation, 1st edn

5. Dobkowski Z, Cholinska M (1994)Application of viscometric detector tocharacterization of polymers by theGPC method (in Polish). PolishChemical Society Meeting (sectionlecture), Warsaw 12–15 September1994

6. Dobkowski Z (1994) Contemporaryquality concept for products of chem-ical industry (in Polish). 1st Congressof Chemical Technology (section lec-ture), Szczecin, 19–22 September1994. In: I Kongres TechnologiiChemicznej, Szczecin University ofTechnology, Szczecin 1995, pp 546–551

7. Dobkowski Z (1994) Specific qualityproblems in chemical laboratories. In-ternational workshop on laboratoryquality assurance, accreditation andreference materials (invited lecture),Smolenice (Slovak Republic), Octo-ber 24–26, 1994

8. Dobkowski Z (1995) Validation ofequipment for accredited chemical la-boratories (short presentation). EU-RACHEM Committee Meeting, Lis-bon (Portugal), 4–7 April 1995

9. Dobkowski Z, Cholinska M (1995)GPC single capillary viscometric de-tector – an ingenious achievement ormistaken idea? 10th Bratislava Inter-national Conference on Macromole-cules “Chromatography of polymersand related substances”. Bratislava(Slovak Republic), 18–22 September1995

387

10. Dobkowski Z (1996) Validation ofcomputerized instruments for accre-dited chemical laboratories. 3rd EU-ROLAB Symposium, Berlin (Germa-ny), 5–7 June 1996

11. King B (1995) Problems associatedwith instrument validation (shortnote). EURACHEM Executive Com-mittee Meeting, Noordwijkerhout(The Netherlands), 4 October 1995

12. Bedson P, Sargent M (1996) AccredQual Assur 1 :265–274

13. Huber L (1996) Accred Qual Assur1 :24–34

14. Hewlett-Packard Product Notes:Good Laboratory Practice Part 1:Vendor-validated cgomputer-con-trolled HPLC systems (1992), Part 2:Automating method validation andsystem suitability testing (1993), Part3: Validation and verification ofHPLC hardware (1994)

15. Waters Millennium Technology(1994) Publication B58

16. Milton Roy Product Note (1995) Ap-plication and performance validationSoftcards for Spectronic GenesysSpectrophotometers

17. ATI Unicam (1995) UV Series: GoodLaboratory Practice (leaflet)

18. Yau WW, Jackson C, Barth HG, Ha-ney MA (1991) The ‘Lesec Effect’: isit limited to single-capillary viscome-ters? Waters International GPC Sym-posium Proceedings (1991) quoted byPostma B (1995) Polymer characteri-zation in GPC/SEC with multiple de-tectors

19. Lesec J (1994) J Liq Chrom 17 :1011–1028

20. Lesec J, Millequant M, Havard T(1994) J Liq Chrom 17 :1028–1055