Higher education institutions interested in assessing student learning outcomes (SLOs) are faced with a number of challenges and questions related to the pros and cons of homegrown or externally developed assessments, such as: Can we sustain the process of developing our own assessments given our current faculty and staff resources? Or, can we align an externally developed assessment to our learning outcomes to ensure that we can obtain meaningful information? At ETS, we want to help inform the answers to these questions and others for institutions that are concerned about creating more efficient assessment processes.
That is why ETS developed the HEIghten® outcomes assessment suite with transparency and efficiency in mind. The suite of computer-delivered, modular assessments for critical thinking, written communication, quantitative literacy, civic competency and engagement, and intercultural competency and diversity, provides valid and reliable data to support educators and demonstrate the impact they’re having in the classroom. The assessments are a comprehensive tool that institutions can use in conjunction with homegrown assessments to meet accountability and improvement initiatives on your campus.
Our ETS Assessment Insights team works with institutions to determine if our SLO assessment suite is aligned with their vision for what students should know, think or do as a result of their curricular and co-curricular activities. We collaborate with assessment leaders on their campuses to ensure they are obtaining useful information about student learning in the most efficient way possible. Here are a few reasons why colleges and universities should consider partnering with ETS when developing student learning outcomes assessments rather than building their own assessment:
Going Your Own Way Can Be Costly
Student learning outcomes assessment can be a resources-intensive endeavor on most college campuses. While developing your own assessment solution can foster enriching conversations amongst faculty regarding student learning, it can be huge drain on faculty resources. If assessment is not episodic but continuous on your campus, you may find that there is a limit to the amount of faculty involvement you can achieve in any given year for all of your assessment activities. Developing and scoring your own assessments for all of your learning outcomes can be an arduous, time-consuming, costly and ultimately unsustainable process.
By collaborating with ETS on assessment efforts, institutions can reduce the burden of continuously developing and scoring learning outcomes assessments. This collaboration does not require the institution to give up those rich conversations that faculty members appreciate. ETS offers several ways for institutions to engage and talk about student learning outcomes when implementing the HEIghten outcomes assessment suite. We have released framework papers that synthesize the literature on measuring specific learning outcomes in higher education. We also allow assessment leaders on your campus to review the tasks in each of the assessments to ensure that they are measuring the skills in ways that resonate with your faculty members and your students. Lastly, we produce scores in ways that promote conversations about student performance on the learning outcomes.
Comparing Apples to Oranges
Developing your own assessment solution ensures a high level of alignment with your institution’s curriculum. However, institutions often want to gauge their student performance on their learning outcomes by comparing different groups of students within the institution, as well as comparing their students to students from similar institutions. These comparisons are often difficult to make when institutions develop their own assessments due to their strict alignment to courses within their own institution.
When developing the HEIghten assessment suite, ETS focused on creating assessment tasks that were not discipline or course specific and would allow for comparisons to be made within and across institutions. These comparisons, when appropriately made, can help institutions build context around student performance on the learning outcomes assessment.
Assessing the Assessment
If you develop your own assessment, there are two things that must be done following each administration: score the assessment, then validate the scores. Institutions often overlook the significant back-end costs associated with scoring hundreds of assessments. Computer-based selected-response assessments can be scored fairly quickly if you have faculty members that are familiar with statistical software. If it is a paper-based assessment, or if you are using a rubric to assess student artifacts, your scoring process often involves training many faculty members on a rubric and then assigning at least two of those faculty members to rate each artifact. When assessing multiple outcomes, this process could involve many faculty members for several hours per assessment.
Validating the scores is a process for ensuring that the scores are precise and meaningful. This may involve conducting statistical analyses of the scores to ensure that the assessment is fair to all students, valid for the purposes of its use, and produces reliable information over time.
As the world’s largest private nonprofit educational testing and assessment organization, ETS removes the burden of the post-administration work from the institution. For most of our assessments, scores are produced almost instantaneously. Information describing their fairness, reliability and validity can be found in the research literature that we produce for each of our assessments.
One Size Does Not Fit All
With assessments, there is rarely a one-size-fits-all solution. Some institutions only want to conduct a single assessment across a small group of students, while others may want to assess the entire student body across a variety of skills. ETS provides institutions with this flexibility, making it easy for institutions to quickly scale up or down based on their needs.
Having a partner like ETS is critical when developing SLO assessments. When a problem or question arises, we are there to help you and your students. As a nonprofit organization, it is our mission to help institutions advance quality and equity in education for all people worldwide. Whatever your institution’s needs are, ETS can assist you in identifying a solution that meets your vision.