the measure of success: key metrics for monitoring content quality with brian buehling
TRANSCRIPT
Session Agenda
1. Introduction
2. Current State of Content Monitoring
3. Content Quality Defined
4. Key Metrics for Content Quality
5. Visualization Techniques
Dakota Offices - Chicago, IL USA
● Founded in 1999 to support the growing need for technical publication integration services
● Headquartered in Chicago, USA serving customers in North America, Europe, Middle East and Asia
● Partnered with technical communication software providers
● Added information architecture training services in 2004
● Added production support and hosting in 2010
● Introduced Dakota Content Platform in 2012
daksys.com
Hundreds of Enterprise Implementations
37% International Implementations
Current State: Evolution of CMS Metrics
Metrics:Available with Web analyticsTracks access to technical documentationRelies on user feedbackNo insight into effectiveness
Usage Metrics
Metrics:Available with CMS technologyLed to over complicated workflowsModelled after product lifecycle managementNo insight into usage
Process Metrics
Content Metrics
Metrics:Available with XML database technology
Tracks access to component level metricsBuilds models for effectiveness
Metrics:Focus on resources Basic metrics on what is boughtNo insight into utilization
System Metrics
Current State: Evolution of CMS Metrics
Metrics:Focus on resources Basic metrics on what is boughtNo insight into utilization
System Metrics
● DITA Publishing System
● Single Source Publishing
● Multi Channel Publishing
● XML Publishing
● Structured Writing
● XML Authoring
● Enterprise CMS
● Database Publishing
Many Names, One Concept
“A system that enforces structured writing guidelines using XML, so that information
can be reused across different formats and document outputs.”
CCMS Technical Architecture
Content Metrics: System View
• Number of Technical Writers
• Number of Reviewers
• Number of Users
• Number of Publications
• Number of Distribution Channels
• Ratio of Technical Writers / Publication
Content Metrics: System View
Content Metrics: System View
Metrics:Available with CMS technologyLed to over complicated workflowsModelled after product lifecycle managementNo insight into usage
Metrics:Focus on resources Basic metrics on what is boughtNo insight into utilization
Current State: Evolution of CMS MetricsSystem Metrics Process Metrics
Content Metrics: Process View
20
Content Metrics: Process View
• Average Publishing Cycle Time
• % Time Spent in Each State
• % Time Spent on Each Document
• % Documents Behind Schedule
• Number of Releases / Period
Metrics:Available with Web analyticsTracks access to technical documentationRelies on user feedbackNo insight into effectiveness
Usage Metrics
Metrics:Available with CMS technologyLed to over complicated workflowsModelled after product lifecycle managementNo insight into usage
Process Metrics
Metrics:Focus on resources Basic metrics on what is boughtNo insight into utilization
System Metrics
Current State: Evolution of CMS Metrics
Content Metrics: Usage View
• Usage Feedback / Comments
• Number of Document Accesses
• % Time Spent on Each Document
• Indirect Determination of Effectiveness
Content Metrics: Usage View
Metrics:Available with Web analyticsTracks access to technical documentationRelies on user feedbackNo insight into effectiveness
Usage Metrics
Metrics:Available with CMS technologyLed to over complicated workflowsModelled after product lifecycle managementNo insight into usage
Process Metrics
Content Metrics
Metrics:Available with XML database technology
Tracks access to component level metricsBuilds models for effectiveness
Metrics:Focus on resources Basic metrics on what is boughtNo insight into utilization
System Metrics
Current State: Evolution of CMS Metrics
The Relationships Built Between Engineers and Customers Ignite Award-Winning Content
Content Metrics: Productivity
Who is my most productive writer?
● System View - # of documents?
● Process View – least time spent in editing tasks?
● Usage View – most hits? Highest ratings?
● Content View – weighted percentage of content
Content References
Repurposing
Content Metrics: Content Reuse
● Reduces content creation costs
● Reduces content maintenance costs
● Reduces content translation costs
● Increases content accuracy
Profiled Documents
Dynamic Assembly
Content Metrics: Content Usage
● More relevant info to customers
● Easier creation of new products
● Fresher, real-time information
● Information on demand
Content Metrics: Quality Definition
• Sections – Average depth, length, # of graphics
• Titles – Average # of words, distinct values
• Warnings – Frequency of distinct phrases• Tables – Ratio of Tables to Graphics• Graphics - % of graphics w/o titles• Emphasis - % of Emphasis greater than 3
words• Lists - % lists greater than 7 or less than
3
Content Metrics: Content Templates
Content Metrics: Content Templates
Content Metrics: XQuery
//note[b]
Content Metrics: Normal Data
Content Metrics: Visualization
