While assessment of client and student satisfaction is not a new topic for career development practitioners, the demand placed on units like career services from organization and campus administration continues to grow by leaps and bounds. In the early part of the 20th century, the Carnegie Foundation started a trend that began the widespread use of standardized, objective testing to measure learning in higher education, which, over the years, expanded from classroom based assessment to the evaluation of student services. However, fleeting are the days where simple attendance metrics and evaluation surveys can justify worth and prove effectiveness; career services centers today face growing pressure to clearly document the lasting effect their services have on the transformation of today’s college students. This NCDA monograph reinforces this important theme and lays out effective processes for implementing learning outcomes assessment within your own center.
Summary of Content
This 80 page paperbound monograph includes 5 chapters and 2 appendices. The authors’ purpose was to educate readers about the value of learning outcomes assessment and to demonstrate a step-by-step process that can be used in a career services organization. While Makela and Rooney have focused this work primarily towards higher education professionals, the process laid out is applicable in many other settings.
The authors’ learning outcomes assessment process is called the Assessment of Learning Outcomes for Interventions Cycle, the ALOI Cycle (pronounced “alloy”). The authors use Chapter 1 to show the necessity of shifting the focus from counting individuals or static objects (such as things that career professionals do) to interventions for groups showing the relationship between career services and those who receive them. The key question becomes: In what ways do our interventions help clients to change, develop, and move from Point A to Point B? The authors want to help the reader learn how to develop and refine an “evidence-based practice” based on the best available data about the efficacy of a career intervention.
Chapter 2 sets the stage for embracing a learning framework for assessment in career services as an on-going event. The authors define learning as “the process of transforming experiences, both past and present into acquired knowledge, skills, and values that can be applied to future endeavors” (p. 7). While measuring satisfaction, demographic categories, and the needs of current participants are important, shifting to a learning framework for assessment provides more accountability data to demonstrate the value of career services to the mission of the institution. Makela and Rooney maintain we must define learning holistically and view ourselves as educators and our career development interventions as learning opportunities. They want to shift our focus from processes to interventions, diversify our approach to assessment, connect to the core learning mission of our institutions, and develop a learning outcomes assessment frame of mind.
Chapter 3 lays out the step-by-step process for learning outcomes assessment, complete with clear instructions and multiple examples applied within a career services context. This process includes defining context, brainstorming outcomes, writing outcomes statements, connecting theories and professional standards, prioritizing learning outcomes, evaluating learning outcomes, reflecting on results and processes, and using learning outcomes assessment. This chapter is both logical and a work of art. The authors consistently use sample charts, worksheets, and diagrams to demonstrate each step of the ALOI Cycle so it can be applied to the reader’s unique situation.
In Chapter 4, the authors provide three very different examples of the ALOI Cycle in action, demonstrating that the process can be used for a wide variety of interventions such as office-wide learning outcomes, outcomes of a drop-in resume review service, and individual career counseling for first year students who are exploring majors. In each example, the authors demonstrate how they would proceed through each step systematically in order to provide learning and dialoguing process for all parties involved.
The authors use Chapter 5 to reflect on the importance of (a) creating an organizational culture of learning outcomes assessment that helps clients make informed educational and career decisions, (b) gathering evidence about the effectiveness of possible interventions and how they can be improved, (c) providing evidence of how career services can enhance the mission of institutions, and (d) building a sense of community among career professionals.
Review and Comments
It is difficult to find weaknesses with this monograph. Makela and Rooney have obviously spent considerable time refining their ALOI Cycle learning outcomes assessment process, making it user friendly, and refining the monograph itself.
Their topic is difficult for most counseling and career professionals to enjoy because of the increasing mandates for learning outcomes assessment and the steps necessary to complete the process well. In the past, assessing participant satisfaction levels and gathering demographic and current needs information has been considered adequate, especially if a high percentage of questionnaires or completed surveys were obtained. However, the authors’ make the point that a shift in thinking about outcomes assessment is necessary. It is becoming essential that career services practitioners clearly define and refine an evidence-based practice so it is clearly understood how interventions cause participants to grow, develop, and move from Point A to Point B.
Chapters 1 and 2 lay the groundwork beautifully for the purpose and value of learning outcomes assessment. Chapter 3 teaches the process and Chapter 4 moves the reader through three practical examples. The appendices provide multiple worksheets on each step. The monograph’s systematic way of defining the process steps, numerous examples, worksheets, and diagrams are among its significant strengths.
The authors end with Chapter 5, moving the reader a step further than anticipated towards a vision of an on-going “culture of learning outcomes assessment” where decisions about the efficacy of interventions are made more objectively and the reporting of outcomes can be used to openly claim a rightful place within the mission of the institution.
We found this monograph to be essential reading for those not well experienced in learning outcomes assessment. It serves as a guidebook and an effective learning tool for an individual or a group. We believe it could also become a welcome library addition for any career services professional, a supervisor of career services providers, an instructor within graduate programs teaching career development courses, or a workshop leader.
Learning Outcomes Assessment Step-by-Step: Enhancing Evidence-Based Practice in Career Services can be purchased in the NCDA Career Resource Store .
Joy Evans is the Director of Counseling and Career Services at Gainesville State College. She received her PhD in Counseling Psychology and Human Systems from Florida State University. She can be reached at jevans@gsc.edu.
Phyllis N. Weatherly is the Director of Career and Counseling Center at Southern Polytechnic State University. She can be reached at pweather@spsu.edu.
Kim Meredith on Wednesday 03/06/2013 at 04:42 PM
Thanks Joy and Phyllis for reviewing this monograph. I am going to add it to my library this year.