MICROCREDENTIALS COMBINING THEORY WITH REAL-WORLD APPLICATION

Many industry sectors are experiencing a shortage of skilled and qualified workers. While professional certifications are one way workers can demonstrate their qualifications, they are time and resource intensive coming to market. Also, some jobs require workers with specific skills, rather than the full range of knowledge and skills required to perform an entire job (traditional certification/traditional job). Professional Testing Inc. 7680 Universal Boulevard Orlando Florida 32819

G O I N G B E Y O N D t h e T R A D I T I O N A L

The recent evolution of micro-credentials is providing “just-in-time” credentialing for workers in emerging and rapidly evolving job markets. The clean-energy sector is one such rapidly growing job sector, with many workers from professions performing many components of many jobs. Developing micro-credentials is one strategy one clean-energy organization is using to develop assessment-based credentials that are focused on specific skills.

Following accepted practices in measurement and assessment-development, and utilizing technology as the research and development platform, micro-credentials that save time and resources are providing a “real-time” solution to qualifying a work-force. The presentation will discuss one approach to developing a micro-credential from the exploratory phase through administration.

PROFESSIONALTESTING INC. - 7680 UNIVERSAL BOULEVARD - SUITE 300 - ORLANDO FLORIDA 32819

+1- 800-330-3776

Online at: www.Proftesting.Com Visit our blog: www.FromTheItemBank.com

1 MICROCREDENTIALS Preparing a Skilled Workforce in “Real Time”

Developing a Micro-Credential

The clean energy sector, similar to many other

industries and professions, has relied on

professional certifications and training programs to

qualify its workforce. While credible, these

“traditional” models typically require a significant

amount of time to develop and get to market,

substantial costs to administer, and rely on the

continuous input of subject-matter experts (SMEs)

to maintain. Often program development and

maintenance activities require resources for SMEs

to meet face-to-face, and for qualified facilitators

such as psychometricians and experts to develop a

body of knowledge (BOK) and an assessment.

IREC approached Professional Testing Inc.1 to

partner with them to develop an alternative

approach to credentialing in the form of a micro-

credential for the clean energy sector, one that is

credible while saving time, money and human

resources. The outcome is to build a prototype

process organizations within the clean energy sector

can use to bring a micro-credential to market. It is

interesting to note that while the clean energy sector

agreed that it needs an alternative approach to

traditional credentialing programs, industry

representatives did not agree on the type of

alterative credential or how it would be developed.

There are several approaches organizations can take

in developing micro-credentials. Finding the right

approach will inform the methodology utilized to

develop the credential. In an exploratory planning

meeting of renewable energy experts from IREC

and assessment experts from Professional Testing

(exploratory panel), two methodologies were

considered: 1) selection of a job-function that

combines functional elements of two different jobs,

each of which has been defined by a job-task

analysis (JTA) resulting in training and/or

certification; and 2) selection of a job-function for

which no job-task analysis has been conducted, but

which is being performed in the workplace. The

exploratory panel selected Option 1 as a starting

point, as JTAs have defined existing jobs and job-

tasks, and are therefore, a good point from which to

launch a micro-credential. As previously noted, the

clean industry sector has several certifications and

training programs from which micro-credentials can

be developed.

Christine D. Niero, PhD Vice-President, Professional

Certification. Professional

Testing, Inc.

Cniero@proftesting.com

Christine DePascle, MS Psychometrician, Professional

Testing, Inc.

Cdepascale@proftesting.com

Reed Castle, Ph.D. EVP & Chief Psychometrician,

Professional Testing, Inc.

Rcastle@proftesting.com

Introduction

Many industry sectors are experiencing a shortage of skilled and qualified workers, individuals who need to

be “job-ready.” Not so coincidentally, many evolving forms of credentialing are clamoring for our attention

to help workers document their skills, and employers to find job-ready personnel. Some of these evolving

forms of credentialing include digital badges, alternative credentials (“alt creds”), verified certificates, nano-

degrees and micro-credentials. For industries experiencing rapid growth, micro-credentials may be a

solution to the challenge of qualifying individuals for jobs. Still largely undefined, but generally agreed to

as a smaller, discrete, specialized opportunity for education and skills assessment, micro-credentials are

useful for both learners wanting to achieve a level of proficiency in a skill, and workers seeking the same to

remain current in and/or advance in their jobs, some of which may already require training or certification.

In the United States, the clean energy sector is one industry that is experiencing rapid growth and has a

shortage of qualified workers who can work across a myriad of job functions. It is also an industry that has

seen the consequences of unqualified workers, both from the poor quality of work produced, low energy

savings, and risks to worker and consumer safety. As a result, professional certifications1 were developed

to enhance the quality of work, but only for core renewable and energy efficiency occupations. Meanwhile,

the industry finds itself at the intersection of expanding jobs for individuals who work full time in clean

energy fields and those working in allied industries that “touch” clean energy. Furthermore, as the industry

has expanded, the need for a means of more quickly developing and documenting the skills and knowledge

of workers has also grown. Developing micro-credentials is a strategy the Interstate Renewable Energy

Council (IREC)1 is using to respond to this challenge of efficiently qualifying a workforce, minimizing the

resources (time, money and personnel) required to do so, all while producing a credible and quality

credentialing option for the clean energy sector.

_____________________________________________

1 Professional certifications are typically comprised of the components associated with awarding a certification, including a certification examination, eligibility for certification and qualifying for the exam, recertification, disciplinary procedures, governance structure and policies. 2 The Interstate Renewable Energy Council (IREC) develops quality and competency standards, accreditation and certification programs for clean energy educators and training programs for a highly-trained, quality-prepared workforce www.irecusa.org. The initiative to develop micro-credentials to address the need to more quickly prepare a skilled workforce is spearheaded by Jane Weissman, IREC’s President and CEO. 3 Professional Testing, Inc. is a full-service provider of assessment, evaluation, certification and accreditation services www.proftesting.com.

02APPROACHES

Selecting an Approach

“Utilizing technology is

essential and may

reduce resources

required of traditional

face-to-face meetings;

while this is a highly

desirable outcome, it

cannot restrict feedback

or quality of input. For

example, not every step

of development may be

able to be optimally

conducted in a virtual

environment.”

There are several approaches organizations can take in developing micro-credentials. Finding the right approach will inform the

methodology utilized to develop the credential. In an exploratory planning meeting of renewable energy experts from IREC and

assessment experts from Professional Testing (exploratory panel), two methodologies were considered: 1) selection of a job-

function that combines functional elements of two different jobs, each of which has been defined by a job-task analysis (JTA)

resulting in training and/or certification; and 2) selection of a job-function for which no job-task analysis has been conducted,

but which is being performed in the workplace. The exploratory panel selected Option 1 as a starting point, as JTAs have defined

existing jobs and job-tasks, and are therefore, a good point from which to launch a micro-credential. As previously noted, the

clean industry sector has several certifications and training programs from which micro-credentials can be developed.

Next, the exploratory panel set out to identify the broad parameters for the content, defined as job functions that could be

supported by functional descriptions of the tasks associated with performing the job function. The job function needed to be

conceptualized as a smaller, discrete, specialized opportunity for skills assessment. The exploratory panel then identified the

steps to take to develop the BOK for the micro-credential, and the foundation upon which an assessment would be based. There

were accepted practices in credentialing that the exploratory panel followed in the development of the micro-credential:

1. The micro-credential must be developed from a credible approach such as a job-task analysis or a modified JTA.

2. The micro-credential must include input from SMEs who represent the breadth and scope of the micro-credential.

3. An assessment must be a component of the micro-credential, and the development of the assessment must follow

acceptable psychometric practices (reliability and validity).

4. Input on content must be provided from industry representatives and SMEs.

The exploratory panel also worked from the following principles:

1. Utilizing technology is essential and may reduce resources required of traditional face-to-face meetings; while this is

a highly desirable outcome, it cannot restrict feedback or quality of input. For example, not every step of development

may be able to be optimally conducted in a virtual environment.

2. Credibility is paramount, so minimizing resources in the development process cannot compromise the credibility

associated with specific processes such as utilizing the appropriate number and type of SMEs.

3. Bringing a micro-credential to market faster does not mean the development process and program integrity can be

compromised—best practices must be followed and adhered to.

The exploratory panel then reviewed the practices for conducting job-task analyses as a starting point for outlining the process

to follow for developing the BOK for a micro-credential. The steps to conducting a JTA were discussed to determine which

steps could be eliminated, compressed or managed by technology in the development of a micro-credential. The panel discussed

the steps essential to developing a smaller, discrete, specialized BOK and foundation upon which to base an assessment.

03 STEPS

The following chart depicts this process and approximates the time line. The chart also identifies the parties

responsible for specific activities: IREC (the industry) and Professional Testing (expertise in developing

credentialing programs.)

04CONSENSUS

Reaching Consensus

To begin this process and to

identify the existing gap,

subject matter experts

(SMEs) representing home

performance professionals

(BA and EA) and those

representing healthy

housing professionals

(those who currently

complete full healthy-home

assessments) were asked to

respond to a function.

Working with industry leaders, IREC garnered support for the micro-credential content, and refined the job function as

the “Healthy Home Evaluator” whose responsibility is to assess and characterize home-based environmental health and

safety hazards and to communicate those risks and hazards to the occupant with the goal of improving health and quality

of life. The Healthy Home Evaluator micro-credential was derived to build upon the foundation of two certifications—

the Building Performance Institute (BPI) certified Building Analyst Professional (BA) and BPI Energy Auditor (EA), and

content was developed to fill the existing gap of healthy-home assessment in those certifications.

Building upon the foundation of BAs and EAs, the competencies required to conduct an in-depth healthy home

environmental risk assessment were determined by subject-matter-experts. To begin this process and to identify the

existing gap, subject matter experts (SMEs) representing home performance professionals (BA and EA) and those

representing healthy housing professionals (those who currently complete full healthy-home assessments) were asked to

respond to a function definition survey. The purpose of this survey was to clarify the definition and scope of the healthy

home evaluator by looking at those aspects of the healthy-home assessment that are performed by healthy housing

professionals that are not typically performed by home performance professionals. The survey was divided into two

primary parts. The first part looked at the various components of a healthy home assessment. The second part looked at

the O*NET4 generalized work activities. The difference in the responses of the home performance professionals and the

healthy housing professionals were analyzed and used as input in the next phases of development.

In a facilitated day-and-a-half face-to-face meeting, SMEs reviewed the results of the function definition survey and

identified the scope of the healthy home evaluator micro-credential. The SMEs were instructed to identify only the

competency requirements not currently performed by BAs that are needed to assess and characterize home-based

environmental health and safety hazards and to communicate those risks and hazards to the occupant with the goal of

improving health and quality of life. A content outline identifying the duties, tasks and KSAs was developed as a distinct

body of knowledge for BAs and EAs resulting in the Healthy Home Evaluator micro-credential, an “add-on” to the BA

and EA credentials assessing additional competencies—the “micro-credential.” Following the review and revision of the

content outline, a validation survey was developed for issuance to a broader range of SMEs. Typically this survey provides

data on the importance and criticality of specific tasks, and the percentage of content on an assessment—exam weighting.

___________________________________________________

4 O*NET is a content model developed by the U.S. Department of Labor (1993) to provide a common language of occupational information and lists generalized work activities across jobs and organizations.

05ASSESSMENT

Assessment

An assessment is considered

to be an important component

of a micro-credential, and as

such follows similar protocols

for development as a

certification examination,

upholding the tenets of

validity, reliability and legal-

defensibility.

The next step in the development of the micro-credential is the assessment. An assessment is considered to be an

important component of a micro-credential, and as such follows similar protocols for development as a certification

examination, upholding the tenets of validity, reliability and legal-defensibility. The SMEs identified as item-writers and

reviewers collectively represent the scope of the micro-credential and write and review items designed to assess the

competency requirements of the Healthy Home Evaluator. Resources such as industry standards and protocols may be

utilized during item writing and may assist SMEs in writing questions at the correct degree of difficulty. All items must

be linked to the content outline, and assess only the competency requirements within the scope of the micro-credential.

In keeping with acceptable industry practices in certification examination development, a criterion-referenced passing

score study is required to determine the pass/fail point for candidates. In essence, the process requires: 1) a methodology

to identify what content is appropriate for the exam, 2) leveraging of SMEs in the item and exam form development

process, 3) a reasonable rationale for the identification of a passing point, and 4) a review of exam and item statistics.

With that said, timeliness and expense are considerations in the development of the micro-credential exam.

For the exam development process, SMEs must be identified by the sponsoring organization with expertise in the micro-

credential content. Because fewer SMEs are used given the narrowed scope of the job function and assessment content

coverage, it is critical to identify those best suited for the role. SMEs are tasked with developing and reviewing the items.

This process can be managed remotely and in a short period of time given the fact the exam has much fewer items than a

traditional certification exam (we estimate 30 – 50 items for the micro-credential, while certification exams typically

contain 100 – 120+ questions.) A management system must be in place where a given SME can draft items and have the

capability to review item submission by other item writers. SMEs must be provided the opportunity to finalize any items

requiring additional review, which can be conducted via webinar. Following this process a form of the exam is developed

to match the test specifications, and should be reviewed and approved by the SMEs prior to administration.

The first administration of the micro-credential is very important as statistical and written feedback (examinee comments

and possibly direct questions to examinees) can be used to make final content and scoring decisions. For example, if an

item is not behaving well statistically, changes to the items are made. If comments are made about the content of an item,

SMEs can make final decisions about the appropriateness of the item—for example, is the item poorly written, or does

the item not meet test specifications. Once the statistical and comment reviews have taken place, the exam is ready to

have a passing standard applied. A traditional modified Angoff procedure works well for this type of assessment process.

This too can be accomplished remotely.

06 LESSONS

Lessons Learned

Micro-Credential vs Traditional Certification

Areas of cost-reduction:

Reducing the number of SMEs for

JTA/Functional Analysis from 1—12 to 3 - 6

Reducing costs of travel by conducting

webinars for any “person-to-person” meeting

Reducing development time from 3 – 6 months

to 8 – 10 weeks

Reducing the cost of facilitation—reducing 3

days of meetings for JTA plus additional days

for item-writing and review and passing score

study, to 4 – 6 hours of webinars

Reducing the cost of psychometric expertise by

reducing number of meeting days, plus the

amount of measurement and analysis required

Reducing the cost of producing an

assessment—fewer questions, item writers self-

train and write items; items are 3 response

multiple-choice vs. typical 4 response; passing

score study requires fewer resources

Typical savings in R & D is about two-thirds from

traditional JTA studies and exam development.

However, micro-credentials are less in scope, and should

require fewer number of test questions than a traditional

certification examination.

Note: Savings related to scheduling face-to-face

meetings may require more attention to management

and follow-up with SMEs.

There are many lessons learned in developing any credential, but many to be learned in

tackling a new concept. First, the comfort level of participants and supporting

organizations will vary, depending on how vested they are in traditional models of

credentialing, and their support will change throughout the process—sometimes they

are on board, and other times they will push back. Knowing the personalities of the

participants and the culture of the participating organizations may help facilitators

anticipate the comfort level of participants and plan around it. Transitioning mind-sets

from what people know and what their organizations do is harder than you think. People

tend to resort to what is familiar when they are unfamiliar with a new process. It’s very

important to remind participants about the purpose and focus of the micro-credential.

In their minds and practices, “chunking” smaller units of knowledge and/or skill may

be counter-intuitive—be prepared to overcome traditions and “how we do things here.”

The concept of “micro” can be interpreted as “less-than” in the market-place, so it is

important to be very descriptive about the purpose and function description of the micro-

credential, in essence, it’s “micro-ness.” When two JTAs are referenced, and two

credentials are used as a foundation for earning the micro-credential, BOTH sides want

their content areas to be included, resulting in too much content and the evolution of a

full-scope certification—it’s easier for people to add to the content list, than to refine it

to its essentials. Take care and time in drafting the function description—you should

“know it when you see it” is a good rule of thumb. Also when using two JTAs, the trust

factor the certification provides as an assurance or measurement of competence may go

down and SMEs may feel the need to re-create existing competencies. In working with

SMEs to develop content, helping them understand “how deep is too deep,” and “how

much is too much” for a micro-credential is a challenge. Frequent reviews of the content

outline to determine redundancies with existing credentials, or within the BOK is useful.

So is asking the question: “Is this really necessary?”

While reducing and/or eliminating face-to-face meetings is critical to refining the use

of resources, there may be instances in which SMEs will want to work face-to-face. For

example, if the concept of a micro-credential is abstract to an organization, a face-to-

face meeting to develop the job functions may be required to establish the foundation

for sponsoring organizations and SMEs. The benefits of this need to be weighed against

the significant costs associated with these meetings. Perhaps this will change with time

as the credentialing world becomes more familiar with micro-credentials. Finally, even

with the support of industry, be prepared for push-back—once they are on board does

not necessarily mean they will stay on board, so learn to “go with the flow.”

07THE RIGHT CHOICE?

Are Micro-Credentials Right for Your Organization?

Certification bodies in emerging fields (energy efficiency), industries that rapidly change (IT or medical), professions that are facing job shortages, or those with

certifications that cross-over several areas of expertise (coaching and safety), may find they are well-suited to developing micro-credentials. Certification bodies

often develop full-scope certification programs first to define the BOK, build a foundation, establish industry standards, and define the job or role. As these

fields continue to develop and additional specialized knowledge and skill areas surface, developing a micro-credential may be the next logical step to providing

learning and assessment opportunities (before the next JTA study) for the certified persons to stay current, and/or for workers in a related field to supplement

their skill-set. Micro-credentials can become options for recertification for certified persons, as well as “stand-alone” opportunities that may extend to and

include related professions (a real-estate agent learns how to assess the value of a home with solar panels). Micro-credentials may also fill a training void in

emerging fields, and in fields that rapidly evolve so credentialed workers can remain current. Micro-credentials may even help workers qualify for certification

and/or meet changing job demands, especially in instances where educational opportunities and training have not caught up. For narrowly focused, in-depth,

specialized knowledge and skill sets, micro-credentials may be a better option than the development of a full-scope certification program, and may not require

the resources it takes to develop a certification program. And in a perfect world, micro-credentials should take less time to bring to market.

There are some advantages for established certification bodies to expand into the development of micro-credentials, so how do you know if this is right for your

organization?

Assess the need a micro-credential may fill—is it resource related, time-sensitive, is there a demand for the content? Currently, what need is not being

filled that a micro-credential will fill?

Assess the reputation of your certification—will micro-credentials dilute it, be perceived as a replacement, considered equal to certification? Will you

create confusion in your industry?

Assess if your certification organization can withstand the internal competition a micro-credential might pose with your other products and services.

What is the likelihood that a competing organization will offer a similar micro-credential—how would that competition measure up?

What is the market for the micro-credential and who will seek it—your certified persons, other certified persons, other related professionals? If the

latter, is there an opportunity to “cross-pollinate or cross-populate” these credentials?

Where can resources be maximized—can you work from an existing JTA, or more than one JTA for a field with multiple credentials?

Where are efficiencies apparent? Can micro-credentials count toward recertification? Can they help people qualify for certification?

What will it take to sustain micro-credentials—are they truly “one-and-done” or will they require continued maintenance, for example, administration

of an assessment?

How will your organization assure the development and administration of credible micro-credentials?

What do you expect to gain as ROI? Will the value be long-term or short-term?

Determining if there is a role for micro-credentials in the credentialing landscape, and what they might look like is a conversation that is sure to continue.

What it reflects is the need and desire for people to qualify for jobs, remain current and relevant with the jobs they have, build their portfolio of skills, and for

organizations to meet this demand with flexibility, creativity and credibility.