wg5 subgroup2 creation and validation of scripts
DESCRIPTION
WG5 Subgroup2 Creation and validation of scripts. Kevin Kane, PHASTAR Joy Li, FDA. Aims. To create a list of proposed actions To start to produce a flowchart that documents the process of creating and validating scripts - PowerPoint PPT PresentationTRANSCRIPT
WG5 Subgroup2Creation and validation of scripts
Kevin Kane, PHASTARJoy Li, FDA
Aims
• To create a list of proposed actions• To start to produce a flowchart that
documents the process of creating and validating scripts
• To raise list of questions, issues and points to note for other WG5 subgroups and other WGs
Scope
• Other subgroups– What scripts are required– System to house scripts
• Leverage existing standards (CDISC)• Try not to limit to specific scripting languages• Well documented and validated• Can be for outputs or data transformation
Scope
This is only a start of a framework. Much work will need to be done in the future. We hope you will continue to work with us.
Let’s make as much progress as we can – we won’t be able to finalise all details.
Script Creation
• Once a script is loaded, the original author is stored as metadata but does not have any further rights or responsibilities
• Basic set of programming standards would be useful. If they are too detailed, may conflict with individual organisations
• We should develop standard templates for specifications and user guides etc
• Investigate “V Model” further for development process• For minor changes, this should not be a separate script –
should be added as an option• Encourage backward compatibility but not an absolute
requirements
Metadata
• Program name• Language• Program version (auto?)• Platform• Purpose• SDTM/ADaM version/NA
(dropbox)• Keywords• Original Author (auto)• Usage counts• Ratings/feedback• Validation status
• Assumptions• Inputs• Outputs• Requirements• Comments/notes• Reason for change• Bug flag (DB table?)• Current author• Language version • Validation documentation
Definition of validated script
• Script does what it says in specification• Specifications are required
– Design– Inputs– Outputs
• Test under various scenarios: these scenarios become assumptions
• Code review• Validation by experience is not enough• Website/wiki needs a disclaimer• ISSUE: What documentation is required for unvalidated scripts
Process for scripts to be validated
• Upload all validation documentation• Approval by moderator (committee?)• Meets all requirements on validation checklist
Can we learn from SAS Online help web pages?
Script governance - functions
• Approve scripts• Draft specs• Call for Scripts• Template specs• Guidelines• Validation checklist• Library management• Ratings management• Define metadata• Change management• Incentive management
Script governance – documentation required
• Guidelines for creating specs• Define metadata– Overlap between specs and metadata
• Web based database?• Template for user guide• Basic programming standards• Checklist for approval to validated state• Definition of requirements to consider a script
validated
Issues to pass to platform group
• Need to be able to review and comment on scripts. Ideally with quality rating
• Create and store multiple versions• Need scripts to be able to have different states: e.g. validated;
unvalidated; in development• Metadata e.g. program name; language; parameters; bug flag;
variables; outcomes; version number (need to decide list of metadata variables)
• Check-in check-out (not 100% defined- what happens if one person checks out for long time)
• Ability to have multi-person multi-function teams• Can we have a metadata database on a Wiki
Notes
• CDISC CTO suggested we find out about FDA Open Toolbox project. Norman Stockbridge and George Rochester from the FDA (ask Mat)
• We noted that different user groups may have different requirements e.g. FDA reviewers data exploration
Incentives:Results from brainstorm
• Maybe we don’t need any incentive• Encourage people to get a top rating leading to enhanced reputation• Platform records downloads – “most cited script”• Messages to “market”:-
– Reputation factor– This system can save organisations money– This is the same code that the FDA will use
• Could offer a PhUSE discount or award• FDA recommendation to use scripts• Airmiles/points system – bronze/silver/gold• Academic encouragement : get your methodology adopted• Confirm if we need any money. Ask PHARMA???