3 tips for anyone to design or evaluate training

Share this:

Forget those who embrace training and see themselves as lifelong learners. We have no choice but act in constant learning mode, from lunch and learns to new policy rollout to new software to train-the-trainer to skills development.  As the saying goes “if you’re not getting better, you’re getting worse” and organizations spend on training:  in 2011, United States organizations spent $156.2 billion on training, ~3.2% of payroll.

Toby Elwin, telling ain't training, 3 tips, design training, evaluate training

Today, I’ll be telling you about my brilliance. I am SME, hear me roar!

To CEOs the 2 most important learning metrics map to:

  1. Business strategy impact
  2. Return on Investment (ROI)

For simplicity sake let’s bundle training, education, and learning. I can effectively split hairs in another post, but here let’s keep the 3 together.

The biggest challenge to a training ROI and how people and organizations embrace, accept, or reject training is constant training attempts by people who know little about effective communication and less about what makes effective training.

Let’s address the elephant in the room: the Subject Matter Expert (SME):

  1. An SME does not correlate to trainer
  2. SME attempts to train may do more harm than good

So, we spend a lot on training, before we even factor in people’s time, but as the American Society for Training & Development loves to say, Telling Ain’t Training.

Effective Communication is Effective Training

In my organization development role I advance a very distinct point: communication is training and training is communication. The goal of either: convey information to cause reaction.

Training as communication simplifies the change management game. While many change management efforts count on communication, the majority I review leave training out. Worse, the past 15 years, most projects I work on at least scope some form of communication before project launch, but I can count on one hand the amount of projects that scope training before project launch.

Also, training as communication holds the subject matter expert accountable for effective communication and how and if they are the best source or should deliver communication. Just because you know a topic gives no ability to communicate effectively. Depth of skill does not correlate to ability to train.

Finally, good communication is measurable, good training is measurable.

Outsourced training is a perceived solution that many organizations take. The objective expectation to outsource is an increased return with training subject matter experts, either as a product vendor or a specialized training partner. The trouble I continue to run into: groups who offer training, themselves, have with a quality review of the list below, reveal little evidence as qualified or capable trainers.

Here are 3 tips to evaluate training quality. Use this list to also provide quality assurance to help design training. Effective training needs to cover 3 key areas and how the tips are presented:

  1. Communication Quality
  2. Training Quality
  3. ROI Quality

Tip 1: Communication Quality

Here is a communication quality check list to evaluate how to communicate. Seek these in design and in evaluation:

  1. Know your audience — clearly the easiest maxim for effective communication. What does this mean? There are multiple personas with multiple needs, identify what their needs are. Does your training answer WIIFM? (What’s In It For Me?). You design for personas, for example: Enterprise software rollout communication to meet an executive assistant’s needs is different than a sales representative’s need.
  2. Learning goals — Identify learning goals of the session. You might find a need for multiple sessions. And since you have multiple training community personas the multiple goals relate to how they work and what they need to get their work done [more insight below within Adult Learning Styles].
  3. Agenda — Review the agenda, what are the topics, what will be covered
  4. Tell them what you will say; say it; tell them what you said – An old communications maxim as old as Hammurabbi’s code. Prepare your audience for a topic, communicate about the topic, then recap what (3 take-aways; see point 5, below) you told them.
  5. 3, is a magic number — McKinsey Consulting recommends no more than 3 points in a PowerPoint slide; School House Rocks says 3 is a Magic Number, in music there is a reason they call a 3-piece band a power trio. Keep your points, objectives, goals to 3 and keep things in digestible chunks.

Tip 2: Training Quality

This is a list to gauge if training is set for real results and about learning style. Adult learning styles require a set of diverse needs to meet how they learn best. About 10 years ago I found this 1-page list 30 Things We Know For Sure About Adult Learning from Honolulu Community College, originally written in 1984, by Ron and Susan Zemke.

This list has been a go-to source for me to review the myriad of conditions I want to account for when I design or how I assure design quality. The Zemke’s brilliantly separate the 30 items into (guess what!) 3 categories. Here are samples from each section:

  1. Motivation to Learn
    • The learning experiences adults seek out on their own are directly related – at least in their perception – to the life-change events that triggered the seeking.
    • Adults are generally willing to engage in learning experiences before, after, or even during the actual life change event. Once convinced that the change is a certainty, adults will engage in any learning that promises to help them cope with the transition.
  2. Curriculum Design
    • Programs need to be designed to accept viewpoints from people in different life stages and with different value “sets.”
    • A concept needs to be “anchored” or explained from more than one value set and appeal to more than one developmental life stage.
    • Adults need to be able to integrate new ideas with what they already know if they are going to keep – and use – the new information.
    • Information that has little “conceptual overlap” with what is already known is acquired slowly.
    • Fast-paced, complex or unusual learning tasks interfere with the learning of the concepts or data they are intended to teach or illustrate.
  3. In the Classroom
    • Adults bring a great deal of life experience into the classroom, an invaluable asset to be acknowledged, tapped and used. Adults can learn well – and much – from dialogue with respected peers.
    • Instructors who have a tendency to hold forth rather than facilitate can hold that tendency in check–or compensate for it–by concentrating on the use of open-ended questions to draw out relevant student knowledge and experience.
    • Adults have something real to lose in a classroom situation. Self-esteem and ego are on the line when they are asked to risk trying a new behavior in front of peers and cohorts. Bad experiences in traditional education, feelings about authority and the preoccupation with events outside the classroom affect in-class experience.

You will quickly value this collection on adult learning insight. Print the list, laminate the list, save the list on your computer, tattoo 15 items on the inside of each eyelid; 2 x 15 = 30, see there is that “3” again.

Tip 3: Measure ROI Quality

Here, ROI measures both organization Return on Investment and Return on Involvement.

You can not change what you can not measure. The value of training is on trial, a trial to prove training’s return on investment. With money scarce you need to identify the dollar impact on training. Here is your greatest ally: The Kirkpatrick Four-Level Evaluation Model. Kirkpatrick’s model is the 2nd go-to quality item when I design and when I evaluate other’s design.

What do you evaluate, exactly? Here are the evaluations for each level:

  • Level 1: Reaction — To what degree participants react favorably to the training.
  • Level 2: Learning — To what degree participants acquire the intended knowledge, skills, attitudes, confidence, and commitment based on their participation in a training event.
  • Level 3: Behavior — To what degree participants apply what they learned during training when they are back on the job.
  • Level 4: Results — To what degree targeted outcomes occur as a result of the training event and subsequent reinforcement.

Important each level below builds ability to meet the next. You can not jump to Level 4: Results without accomplishing Levels 1: Reaction through Level 3: Behavior to note that you need to build Level 1, then 2, then 3, to get to Level 4.

training, kirkpatrick, toby elwin, evaluation, level 1, level 2,  4 Levels of Learning Evaluation, learning, training

The Kirkpatrick Model, 4 Levels of Evaluation, Level 1 and Level 2 – select picture for larger view

training, kirkpatrick, toby elwin, evaluation, level 3, level 4,  4 Levels of Learning Evaluation, learning, training

The Kirkpatrick Model, 4 Levels of Evaluation, Level 3 and Level 4 – select picture for larger view

How do you measure? Well, that is a post on survey development and measurement, that will need to wait.

Not What You Know, But What Is No

Some organizations rely on external training partners to deliver their training, the logic comes from expectation that these experts are highly-competent professionals and allow organizatons a discernable training focus to mix and match based on needs.

Outsouring training comes with risk: that they bill as trainers, does not mean they know training. How can you tell they are serioues? A very clear indication if you are working with a professional trainer is to ask if their methods include all 4 of Kirkpatrick’s levels for evaluation. If they say, “no”; move on. If they say, “yes”, ask how they measure as well as a sample of some of their evaluations.

Keep a particularly discerning eye for bad Level 1 Evaluations that focus on the quality of the food served at training, if the instructor wore pastel colors, and if the temperature was set comfortably.

It is not that you evaluate, it is what you evaluate.

You can not change what you can not measure.

The entire point of communication is to build confidence, those with confidence to learn have competence to contribute.

If you want to tuck into evaluation immediately I recommend trainingcheck.com as an excellent source for creating evaluations at different levels with samples and insight for each level.

3, Or 4, Great Learning Sources:

  1. Dr. Don Kirkpatrick — author, consultant, and trainer at Kirkpatrick Partners
  2. Barbara Carnes — author, consultant, trainer, and blogger at Making Training Stick
  3. American Society for Training & Development ASTD 2012 State of the Industry Report
  4. trainingcheck.com Guide to Training Evaluation