Learning analytics: Using talent data to improve business outcomes

Effective evaluation and measurement of learning and development initiatives is critical to maximise the impact of training, identify gaps for improvement and ensure that efforts are aligned to the business' needs. Learning Analytics outlines how analytical approaches can respond to these challenges, the types and benefits of technological solutions and how to ask the right questions of organizational data in order to build a learning organization that boosts performance and competitive advantage. Drawing upon case studies from organizations who have applied such approaches such as The Gap, Hilton Worldwide University and Seagate Technology, Learning Analytics will enable those involved in learning and development to make the business case for their activities and deliver an evidence-based service to their organizations. Alongside updated chapters on learning technology tools and moving beyond learning analytics to talent management analytics, this second edition also features new content on measuring informal learning, increasing data literacy, and framing L&D's contributions through a portfolio evaluation approach.

CONTENTS List of figures and tables xii About the authors xviii Foreword xix How to read this book xxv Acknowledgements xxvii PART ONE The L&D value gap and how to close it 01 The rise of learning analytics 3 Why is all of this important? 6 Standards are coming 7 Data availability 8 Changing the way talent analytics work gets done 9 Providing unique insight into employee behaviour 12 Learning analytics rises 13 Notes 19 02 What is learning analytics? 21 Introduction 21 Learning analytics today: measure for measure, what should be measured? 23 Why measure learning? 25 Most organizations start with the simple: measure training activity and satisfaction 26 Efficiency, effectiveness and business outcomes: closing the learning measurement gap 28 The journey to learning analytics 29 The Four Levels of Evaluation 30 The Return on Investment Methodology 31 Impact Measurement Framework 33 Success Case Method 34 Performance-Based Evaluation 36 Conclusion 44 Notes 46 03 The value-centred learning organization: A new evaluation paradigm 49 Volume is not value 49 We're already delivering value, though... right? 50 Delivering and demonstrating value: the Talent Development Value Framework 52 The Talent Development Value Framework in action 61 Advancing measurement maturity 63 Conclusion 65 Notes 66 PART TWO Establishing sound measurement practices 04 Aligning L&D's value with the C-suite: The Four Value Drivers and Portfolio Evaluation 71 What the C-suite wants from L&D 71 Connecting L&D with the business strategy 73 The Four Value Drivers 74 Building business alignment 76 Translating value drivers to action: Portfolio Evaluation for L&D 78 Immediate benefits of portfolio alignment 83 Additional benefits: portfolio management 85 Change the conversation 90 Conclusion 91 Notes 91

05 Linking learning to business impact 93 What works? 93 Why does it work? 95 Experimental designs 99 Alternatives to experimental designs 103 Alternative designs: practical ways forward 103 Conclusion 114 Notes 116 06 The new leading indicators of success and how to manage them 118 Your training programmes may not be as good as you think they are 118 Scrap learning and how to reduce it 121 Performance improvement 126 Net promoter score 129 Manager support and how to improve it 131 Predictive Learning Impact Model 2.0: Causal Modelling 140 Conclusion 154 Notes 155 07 Developing a sustainable reporting strategy 156 The role of reporting in learning analytics 156 Getting started: design principles 159 Components of an effective reporting strategy 161 Reporting strategy development 170 Critical success factors 172 Perform gap assessment 175 Implementing the strategy 181 Special cases: dashboards and scorecards 183 Monitor the strategy: success indicators 188 Conclusion 189 Notes 190 08 Technology's role in learning measurement 192 What should technology do? 193 Benefits and costs of learning technologies 194 The requirements for a new technology system in the BI space 200 The challenge of self-reported data 206 What is the ROI of technology systems? 209 Applying principles of business intelligence systems to L&D 210 Additional technologies 215 Conclusion 218 Notes 219 09 Benchmarks 221 Comparison to standards provides insights for decision-making 221 Benchmarking improves maturity 222 Why are benchmarks valuable in the L&D space? 224 What benchmarks are available? 225 Benchmarks and statistical significance 228 What does MTM bring to the market beyond benchmarks? 234 How do clients use benchmarks to support decision-making? 235 Conclusion 237 Notes 237

PART THREE Refine the strategy: Evolution, ongoing transformation and innovation 10 Driving alignment from strategy through execution 243 Measure twice, cut once 243 The ADDIE Model: sustaining alignment using a cyclical approach 251 Closing the loop 266 Conclusion 268 Notes 269 11 Optimizing investments in learning 271 Learning and development groups struggle to create value 271 Developing a framework 273 Reporting measures to the business 275 Working with business leaders 285 Continuous improvement and management approaches 285 Principles 288 Less is more 289 Assumptions 291 Moving from reporting to action 291 Conclusion 295 Notes 295 12 Measuring informal learning outcomes 297 Introduction 297 What is informal learning? What is social learning? 299 The new learning landscape 300 Learning from the past: e-learning lessons 304 Organizational ecosystem for informal learning 306 Traps, potholes and pitfalls of informal learning 308 Showing value through measurement 311 What should we measure to show value? 312 Use cases 317 Conclusion 320 Notes 321

13 Beyond learning analytics to talent management analytics 324 The future is for those who can predict it 324 Defining what to measure in talent management 325 Understanding the employee life cycle 328 Integrating data 331 Research on talent analytics 331 It's not the analytics that matter: it's how they are applied 337 Managing data in the analytics proc