database quality assessment

7/31/2019 Database Quality Assessment

1/17

XXXX

Title: InForm Database Quality Assessment Specification

Sponsor ProjectNumber:

XXXX ProjectNumber:

102294

Sponsor Name: XYZ

This document has been signed electronically on the final pages by the following:

Signatory

Independent ReviewerName:

Role

Signatory

Reviewer Name:Role:


2/17

XXXX

TABLE OF CONTENTS

1. REVISION HISTORY ............................................................................................................................ ..... ..3

2. Purpose ..........................................................................................................................................................4

3. Scope .............................................................................................................................................................4

4. Out of Scope ..................................................................................................................................................4

5. Time Frame .............................................................................................................................................. ..... .4The assessment(s) will be performed at pre-determined time points during the trial as determined by theXXXX Data Operation Lead (DOL) and the DMS during the Study Set up phase of the trial (refer to the DCSfor the time points). This will allow flexibility for each trial based on; study design, study duration, rate ofenrollment, interim analysis and/or final database lock timeline. The Data Operation Lead (DOL) will beresponsible for selecting an Independent Reviewer to initiate all associate tasks..............................................4

6. Required Documents ................................................................................................................................... ...4

7. Required Listings and Reports .......................................................................................................................5

8. Random Subject Sample Selection ................................................................................................................5

9. Procedural Steps ............................................................................................................................................6

10. Database Quality Assessments Reviews .....................................................................................................7

11. Review Meeting/ Result Summary .............................................................................................................11

12. Calculation of the Error Rate .....................................................................................................................13


3/17

XXXX

1. REVISION HISTORY

Version Effective Date Name of author

Summary of change(s)


4/17

XXXX

2. Purpose

The Database Quality Assessment Specification provides the requirements and procedures to ensure that the databasemeets the XXXX quality standard.

3. Scope

This Data Management Operational Guideline describes the process for a regular sample-based assessment of thedata review process. The outcome is an estimate of the overall error rate that measures the level of quality applied inthe data review process.

Data selected for the quality assessment must be clean prior to commencement of the Database Quality Assessment.Un-actioned discrepancies, outstanding queries, and unresolved data clarification issues are not permitted; unless it

pertains to the out of scope activities listed below.

4. Out of Scope

The following reports and queries will be considered out of scope for this Database Quality Assessment and thereforeshould not be included in the quality assessment.

Reports out of scope:o Non-randomized patients (Screen Fails)o

External Vendor Reporto Patient Diaries Reportso Coding Reports

Queries out of scope(determined by the XXXX Data Processing Organization and the DMS during the StudyStart) :

o Coding Querieso SAE Queries with pre-fix SAE in the query Texto External Vendor Data Querieso Autoqueries automatically closed

5. Time Frame

The assessment(s) will be performed at pre-determined time points during the trial asdetermined by the XXXX Data Operation Lead (DOL) and the DMS during the Study Set up


5/17

XXXX

7. Required Listings and ReportsThe following listings/reports will be required to complete Database Quality Assessment:o Standard In-Life Report Listing and Study Specific Listings Off-Line listings per the Edit Check Specificationo All Queries Report.

The following Ad-Hoc Report will be generated to help initiate the sample selection process for the Database QualityAssessment. :o Subject Status Report.

8. Random Subject Sample Selectiono The Project Team will select the Sample size.o Sample size a random sample of the subject population, using the square root of N+1, where N is thetotal number of patients in a study.o Subjects taken for the random sample selection-1181 (total subjects enrolled).o random Sample size - 30., this is obtained by applying N+1.o For this assessment a sample of 18 subjects was taken from the subjects declared clean by the PCDA.o PCDA is responsible for generating and sending it to Independent reviewer and IR in turn forward theSubject Status Report to the designed Clinical Programmer.o The DOL will be responsible for requesting read-only access for the Independent Reviewer (remember to

request Ad-Hoc Capabilities to the request)o Prior to sending the Subject Status Report to the Clinical Programmer, the Independent Review mustmeet with the Primary CDA and Data Operation Lead to identify the clean subjects using the completed subjectlist.o Once the primary CDA and Data Operation Lead have identified the cleaned subjects, the primary CDAmust sign the Subject Status Report to verify that the selected subjects are an accurate reflection of the dataacross the study, which will not compromise the integrity of the test.o The criteria for selecting a clean subject is as follow:

No missing data on the eCRFs No missing eCRFs/Visits No missing no outstanding queries (excluding autoqueries automatically closed , Coding, SAE , Lab queries) All queries have been closed properly.

o After the clean subjects have been identified the Independent Reviewer must send the amended Subject


6/17

XXXX

9. Procedural Stepso Each project must complete the Database Quality Assessment unless otherwise specified by the GlobalProgram Lead or the Project Lead.o The Data Operation Lead (DOL) will be responsible for selecting an Independent Reviewer to initiate allassociate tasks.o Prior to starting the Database Quality Assessment the Independent Reviewer must schedule a meetingwith Data Operation Lead (DOL) and Primary CDA to obtain a full overview of the project. During themeeting the Project Team would explain all discrepancies and outliers associated with the project and theselected patients.o Once all reports, listings, appropriate access rights have been provided to the Independent Reviewer theDatabase Quality Assessment can start.

9.1 Instructions on how to generate the Subject Status Report are listed as followed( Generated by PCDA):

1. To Generate Subject Status report Adhoc reporting was used.

2. Click on the Reports Tab

3. On the Reporting and Analysis Screen click on Ad-Hoc Reporting

4. Select the Clinical Link

5. Click on the Clinical Data By Form

6. Select the Demography Form Folder

a. Drag and Drop Subject Number from the Demography Form Folder

b. Drag and Drop Select Full Baseline Number from the Demography Form Folder

c. Drag and Drop Baseline Screen Number from the Demography Form Folder


7/17

XXXX

12. Click on the blue error under the save file Run Options.

13. Select the Format Excel 2002.

14. Click the Run Icon to generate the report.

15. Remove the Title from the Excel Spreadsheet.

16. Selecting the View Icon on the Toolbar and then select Header and Footer Icon. Create the Header andFooter using the following format:

Header:Protocol Number:MK0462-082Subject Status Report

Footer: XXXX Project Number:102294 Page: [X of Y] Version Number : [X.0]

Effective Date: [DD-MON-YYYY]

17. Filter and pick the Completed, Excluded and Discontinued subjects in the Subject Status for the Studycolumn remove all blank rows.

18. Review the sample Subject Status report in Appendix 1.

19. Generate and Print the report for review.

20. Save the report as a CSV File in PMED.

21. Forward the CSV File to the correct programmer.

10. Database Quality Assessments Reviewso The Database Quality Assessment reviews process will consist of the following assessments:

Review of Standard In-Life Report Listing and Study Specific Listings (section 10.2) Review Missing Datapoints (section 10.3) Review of Queries (section 10.4)


8/17

XXXX

10.1 Testing Operational Instructions

These instructions describe how each task in the Quality Assessment will be performed.

10.2 Review of Standard In-Life Report Listing and Study Specific Listings

When appropriate, the Independent Reviewer will review the In-Life Standard Listings Reports and StudySpecific Listings according to the latest version of the Edit Check Specification. The Primary CDA anddesignated programmers will be responsible for generating both sets of listings for Database QualityAssessment exercise.

10.2.1 Instructional Steps

The Independent Reviewer will review the listings against the Database to ensure all rows have beenactioned correctly for the selected patients. Secondly, the reviewer will evaluate the audit trail (onlinescreen review in database) for any related datapoints and associated queries and responses.

All instances where potential errors have been identified must be documented in the Error Log. (TheError Log is accessible via the MERCK-D Project File folder in PMED.)

C:\MyData\MyDocuments\Merck-D

If discrepancies were noted as an error and was not queried or queried incorrectly, it will be considered atrue error.

Ensure all listings have been signed and dated by the Independent Reviewer and the verifier.

If discrepancies are identified because the In-Life Standard Listings Reports and Study Specific Listingsare not available or were not available at the time of the assessment, they are not considered as errors.


9/17

XXXX

10.4 Review of Queries

10.4.1 Instructional Steps

The PCDA will be responsible for generating and sending the All Queries Report for the selectedsubjects.to the independent reviewer.

The Independent Reviewer will investigate if all queries have been actioned appropriately. This review willinclude:

o Request a list of the CDA staff members that worked on the study that have closed queries from theDOL or the primary CDA.

o Review all manually generated queries by the CDA staff and auto queries closed by the CDA Staff.

o Do not review queries closed by the system unless it was reopened manually by a CDA staff.

o Also, review all remaining open or answered queries.

o Make sure the Auto Closed Queries by the CRA/Site is not included in the review.

o It is required that the query details are reviewed by investigating the audit trail which will help toidentify query details and potential query errors.

o The query errors include the following:

It was queried, but incorrectly

It was queried correctly, but inappropriately answered and closed.

It was queried and/or answered, but remains open.


10/17

XXXX

10.4.2 Instructions on How Generate the All Queries Report (Generated by PCDA)

Generate the Queries Report using the Ad-Hoc reporting tool to populate the All Queries Report:

1. Click on the Reports Tab

2. On the Reporting and Analysis Screen click on Ad-Hoc Reporting Tab

3. Select the InForm Trial Management Folder

4. Select the Site Folder

a. Drag and Drop Site Mnemonic from the Site Folder

5. Select the Subject Folder

a. Drag and Drop Subject Number from the Subject Folder

6. Select the Visit Folder

a. Drag and Drop Visit Mnemonic from the Visit Folder

7. Select the Form Folder

a. Drag and Drop Form Mnemonic from the Form Folder

8. Select the Queries Folder

a. Drag and Drop Query Text from the Queries Folder

b.Drag and Drop Query Status from the Queries Folder

c. Drag and Drop Originator User from the Queries Folder


11/17

XXXX

16. Click the Run Icon to generate the report.

17. Remove the Title from the Excel Spreadsheet.

18. Selecting the View Icon on the Toolbar and then select Header and Footer Icon.Create the Header and Footer using the following format:

Header:Protocol Number:MK0462-082All Queries Report (Added to the Center Section )

Footer: XXXX Project Number: 102294 (Added to the left section in the footer) Page: [X of Y] (Added to the right section in the footer) Version Number : [X.0] (Added to the right section in the footer) Effective Date: [DD-MM-YYYY}(Added to the right section in the footer)

Note: Once the Header and footer are completed then use the Page set-up function toselect the format of the page as Landscape.

19. Then sort the excel spreadsheet according to selected Site/ Subject Number and print eachoutput for the review.

20. Review the sample report in Appendix 2.

11. Review Meeting/ Result Summary

11.1 Feedback of Errors Identified to the Responsible Primary CDA and DOL

Once the assessment has been completed, the Independent Reviewer must review the error log, listings, andreports to ensure that all errors have been documented and accounted for. The Independent Reviewer mustthen schedule the last Review Meeting with the Study team to review the errors that were identified duringthe review in a timely manner. During the meeting the Independent Reviewer refers to the Primary CDAand DOL responsible for the data for each error found:

o If the error is agreed, it will remain on the Error Log.o If the error is not a valid error then it will be documented on the Error Log in the comment section

explaining why it is not a valid error.o If the error is not agreed, the Independent Reviewer will discuss the error with Data Operations Lead to

help identify if this is a valid error.


12/17

XXXX

11.2 Team Responsibilities

Allocate the Quality Assessment tasks to an Independent Reviewer. Adjust the table accordingly. Ensure that the Independent Reviewer is someone other than those who are associated with the project.

Task PCDA/Designee

DOL Programmer Independent Reviewer

Set up and Run Reports inEDC System

Execute programs togenerate the subjectsample

Produce any SAS listings

Perform QualityAssessment

Evaluation of errors withthe Project Team(including PCDA andDOL)

Calculate error rate

Independent review oferror rate calculation

Determine remediation iferror rate is unsatisfactory

or


13/17

XXXX

Correction of Errors

Investigate if all queries have been actioned appropriately. This review will include:o Manually generated queries by the CDA Staff and Auto queries Closed by the CDA staff.o Query details, manually check audit trial in the database, and assess for query error.

If any errors were highlighted during the review, the DOL will investigate if the issues result frominsufficient training of the CDAs or from inappropriate instruction in the Data Cleaning Specification(DCS). If so, initiate adequate further training CDAs, or to update the DCS, as appropriate. Initiatecorrective measure, as appropriate.

If errors have been identified, investigate if error results from inappropriate validation programming orcheck logic in the Edit Check Specification. Modify the program or Edit Check Specification Logic/Test

Data as appropriate. This will need to be communicated to the Programmer/line manager. The CRF Completion Instructions, Edit Check Specification, and the validation procedures will be

updated and amended as required from the results of the review in accordance with the InForm DataCleaning process. If applicable, the revised Edit Check Specification will be resent to the sponsor forapproval.

12. Calculation of the Error Rate

Count the confirmed errors on the listing and record the counts for each table in the Database Quality AssessmentError Rate Calculation and Remediation Form.

Calculate the error rate. Sign and date the Database Quality Assessment Error Rate Calculation and Remediation Form.

12.1 Error Rate Calculation

The overall percentage error-rate is calculated as follows:

The total number of errors for all identified listed items (Numerator) The total number of datapoints reviewed (Denominator) The percentage error rate is computed manually by using the provided formula

[(Numerator) Number of Errors / (Denominator) Number of Datapoints reviewed] X 100

The percentage error-rate for each Form/Dataset is calculated as follows:

The total number of errors identified in the Form/Dataset (Numerator)


14/17

XXXX

12.2 Definition of Numerator and Denominator

Numerator: The total number of errors identified:

What Counts as an Error

Queries errors include items as follows:o It was queried, but incorrectlyo It was queried correctly, but inappropriately answered and closed.o It was queried correctly, but the wrong data correction was accepted resulting in query being closed

incorrectly.o It was queried and/or answered, but remains openo It was queried correctly, but a spelling error was noted that changed the meaning of the query which

counts as an error. (e.g. Hypertension instead of Hypotension)o Autoqueries answer inadequate and inappropriately closed

Missing data (On-Screen Review) In-Life Report Errors include items as follows:

o Report output not addressedo Report output addressed incorrectly (e.g. query issued incorrectly)

Denominator: The total number of datapoints reviewed.

12.3 Correction of Errors for Remediation

Errors will only be corrected in the following circumstances:

The overall database error rate is greater than 0.2 %. The error rate for an individual DCM is greater than 0.2% and remediation is considered essential (e.g.

systematic errors are identified).

Details of remediation will be recorded in the Database Quality Assessment Error Rate Calculation and

Remediation Form.

Documentation of correction of errors will be annotated on listings produced for examination of data forremediation. Each item that has been corrected will be signed and dated on the relevant listing.


15/17

XXXX

Appendix 1:Subject Status Report

To be used for selecting the random samples.

FM-GRO-WW-056-01Effective Date: 12 May 2010

Related to: SOP-GRO-WW-009

Page 15 of 17Version Number: X.X

Confidential Effective Date: Date of last signature


16/17

XXXX

Appendix: 2 All Queries Report

To be used during the review of the Query






17/17

XXXX

Appendix: 3 DBQA Listing

To be used during the review the calculation of the error rate and review process.



