Table 1 Principles of Quality Assessment |
|
|
The Nature of Assessment
Assessment leads to improvement. Both the assessor (person giving feedback) and the assessee (performer) must trust the process. Although the assessor gives the feedback to the assessee, the assessee is always in control. The assessee may use the assessor’s feedback for improvement. Although a well-designed assessment process yields high quality improvements in a timely manner, any assessment process can lead to some improvement. Assessment is an area in which assessors can start simple and increase the complexity as the process is better implemented (4.1.4 Assessment Methodology).
One can use assessment to improve a performance or an outcome. For example, a composition instructor might assess a student’s writing by looking at a completed assignment draft (outcome) and finding strengths and areas to improve in the writing. The instructor might also observe the student as he or she writes the paper to assess strengths and areas to improve in using the writing process (to develop the written sample).
Principles of Quality Assessment
Table 1 outlines ten principles for undertaking assessment in any teaching/learning situation. These principles address the mindset under which assessment is conducted, the circumstances surrounding assessment activities, and the nature of the dialogue between the assessor and assessee. A brief discussion of each principle follows.
1. Assessment focuses on improvement, not judgment.
It is important that both the assessee and assessor understand that the purpose of assessment is to add to quality, not to judge the level of quality or to give interesting feedback that will not be used (Stiggins, 1996).
2. Assessment focuses on the performance, not the performer.
Assessment is only about improving a performance. It is not meant to judge the quality of the performance, nor does it in any way judge the qualities of the performer. One may use assessment to give feedback on how a performer’s skills could be improved to in order to improve a performance. It should never be used to point out weaknesses in the performer, because doing so would undermine both the purpose of assessment and the building of trust needed for effective assessment.
3. Assessment is a process that can improve any level of performance.
There are always areas to improve in a performance, regardless of the level of quality, and there are always areas that made the performance as good as it was. So assessment can always be used to give feedback that can be used to improve a performance.
4. Assessment feedback depends on who both the assessor and assessee are.
Although it focuses on the performance alone, assessment is much more effective when both the assessor and assessee understand their own abilities as well as those of the other. This understanding helps in creating realistic performance criteria and feedback that can be used effectively (4.1.6 Performance Levels for Assessors).
5. Improvement based on assessment feedback is more effective when the assessee seeks assessment.
As in most things in life, feedback is useful only when it is valued. One of the components of valuing assessment feedback is the assessee’s desire to obtain it. When the assessee seeks assessment, it is clear that he or she sees the need for improvement and has plans to act on the given feedback.
6. Assessment requires agreed-upon criteria.
Both the assessor and assessee must have a common understanding of what will be assessed. In any performance, the purpose lends itself to numerous areas in which to look for strengths and areas to improve. The involved parties should decide in advance on the criteria that will be used in the assessment. These criteria can focus on the performance itself (performance criteria) and/or the final outcome (outcome criteria). Both types of criteria can be used in assessing a performance or in assessing a product (Pellegrino, Chudowsky & Glaser, 2001). The chosen criteria should focus on areas that both the assessee and assessor believe are important; they must be appropriate to the performance; and they must be appropriate for the assessment abilities of the assessor (Astin, et al., 1992).
7. Assessment requires analyses of the observations.
Once performance criteria are set, the assessor must collect information germane to the set criteria by observing the performance. During the actual performance, or after the information is collected, the assessor must identify the strengths of the performance and why the strengths contribute to the quality of the performance. In addition, the assessor must identify the areas where improvement could occur and how the improvements could be made (1.4.2 Fundamentals of Rubrics).
8. Assessment feedback is accepted only when there is mutual trust and respect.
The assessee must trust in the assessment process and in the assessor’s abilities. The assessor must trust in the assessee’s willingness to accept and use feedback. Often this trust takes time to build, but it builds quickly once the assessee sees improvement. To help build the trust, an assessor should be sure to follow these guidelines in the feedback report:
Use only positive language; for example, “area to improve,” instead of “weakness.
Include no judgmental statements (4.1.2 Distinctions Between Assessment and Evaluation)
Focus only on agreed-upon criteria
Describe real strengths and why they are strengths
Provide substantial supporting evidence for both strengths and areas to improve
Offer specific suggestions about how to improve
Provide interesting and relevant insights
Convey support and encouragement for change
9. Assessment should be used only where there is a strong opportunity for improvement.
It makes sense to carry through an assessment process only if there is the opportunity for improvement. If assessment feedback is given during the performance (formative assessment), the performer has the opportunity to use the feedback to improve the current performance. If the feedback is given at the end of the performance (summative assessment), the feedback can be used to improve future performance. If there are no plans for future performances, summative assessment should not be used.
10. Assessment is effective only when the assessee uses the feedback.
The assessee must have the opportunity and desire to improve in order for the feedback to be used. Not only must there be an opportunity to improve, but there must also be a willingness to implement the suggested improvements. Even if the assessment process might help in identifying needed improvements, there is little point in taking the time to assess if there will be no effort to improve.
Issues that Affect Assessment Quality
A variety of factors influence the quality of an assessment process. These include the skills of the parties involved as well as the resources available for conducting the assessment (Angelo & Cross, 1993).
Factors Related to the Skills of the Parties Involved
Content Expertise of the Assessor
An assessor who is a content expert in a field specific to the performance or outcome will typically give feedback that is more useful than that given by a novice in the content area, assuming that the assessment skills of the two people are equivalent. Understanding how knowledge is constructed within a discipline can help one determine which evidence to collect and how to analyze it. It also helps, as one collects evidence, to understand the content. This advantage does not mean, however, that an assessor must be a content expert in order to provide any useful assessment feedback. For example, it would be helpful for a novice in a content area to assess a performance if one of the criteria is “to reduce the use of technical jargon.”
Assessment Skills of the Assessor
It is as important that an assessor be knowledgeable and skilled in assessment as it is that he or she have expert knowledge about what is being assessed. Experts in the field are not automatically strong assessors. Highly effective assessors
Display respect for the assessee
Work closely with the assessee to set appropriate criteria for the assessment
Assess only those aspects which meet the agreed- upon criteria
Apply keen observation skills that put findings in context
Employ strong recording skills
Collect relevant and high quality evidence
Analyze results to extract important patterns and gain understanding
Generalize findings so they can be transferred to new situations
Offer timely and constructive feedback
Enjoy reflecting/introspecting
Are comfortable in their role, which is solely focused on improving the assessee’s performance
Usefulness of the Assessment Report
Once the assessment process is completed, the assessee is left with the report of the findings. Since the purpose of the assessment is for improvement, it is important that the report outlines in a concise way what was done well and why it was done well (strengths), areas that could be improved, and some strategies for improvement (4.1.9 SII Method for Assessment Reporting). A quality assessment report
Includes only non-judgmental statements
Follows a concise, well-organized format
Focuses on agreed-upon criteria
Describes real strengths and describes why they are strengths
Provides substantial supporting evidence for both strengths and areas to improve
Offers specific suggestions about how to improve
Factors Related to Available Resources
Quality of the Tools Used
An assessor can rely on his or her memory or use elaborate tools to complete an assessment. As a general rule, the more structured the tool, the wider the audience of potential assessors, and the more likely it will be that the assessment report will be more specific. Many of the assessment instruments in the Faculty Guidebook have been implemented in dozens of faculty development workshops and in hundreds of college classrooms. However, there is no need to wait to assess something until the tools are in place to assess everything.
Development and Implementation Costs
The cost of assessment can vary from very little to quite a lot. Elaborate expense can only be justified for educational research questions that have programmatic implications. The Faculty Guidebook suggests many assessment instruments that offer a point of departure that will minimize the cost of developing effective, special-purpose instruments for courses, projects, and institutions.
Time Required to Conduct an Assessment
An assessment may have a complex design or be carried out with little or no preparation. Instructors and administrators need to balance assessment activities with planning and facilitation activities. Often spending 5% of in-class and out-of-class time on assessment is adequate to determine strengths and areas to improve.
Examples of Assessment
Peer Coaching
A great way to get feedback on facilitation skills is to have a second set of eyes in the classroom. An instructor can meet with a trusted colleague before class and outline two or three focus areas to be assessed in the class. The peer coach should avoid becoming a second instructor and should instead keep relevant notes and report them back to the instructor after the class. This method is beneficial both to the assessor and the assessee; the assessee gets valuable feedback, and the assessor can observe teaching strategies that he or she might find valuable. Assuming peer coaching is an ongoing process between the two parties, this would be an example of formative assessment of a performance.
Course Outcome Review
At the end of a course, an instructor can have students review the desired course outcomes listed in the syllabus and estimate how well they completed each outcome (outcome criteria). When students identify outcomes that have and have not been fully met and explain alternative actions that could be taken in future semesters to ensure achievements of each outcome, the instructor can use the information to assess the course instruction and curriculum. This is an example of summative assessment of an outcome.
Assessment of Student Learning
Students can be tested early in a course to determine how well they have learned and retained the skills and concepts they need to carry over from a previous course. The information can be used to revise the content, focus, and teaching of the previous course. This is an example of indirect assessment in which one group is evaluated in order to improve something that affects the quality of the evaluated group’s performance.
Concluding Thoughts
Learning to use assessment widely and frequently is likely to produce a positive, trusting learning environment. The creation of “magical or teachable moments” will stimulate student engagement in the teaching/learning process and promote productive risk-taking. Long-term use of classroom assessment techniques provides opportunities for “raising the bar” for learner performance and shifting responsibility for learning to students (Angelo & Cross, 1993). When instructors model assessment in their daily classroom and professional activities, both instructors and students can improve significantly over the term.
References
Angelo, T. A., & Cross, K. P. (1993). Classroom assessment techniques: A handbook for college teachers. San Francisco: Jossey-Bass.
Astin, A. W., Banta, T. W., Cross, K. P., El-Khawas, E., Ewell, P. T., Hutchings, P., et al. (2003). Nine principles of good practice for assessing student learning. Washington, DC: American Association for Higher Education.
Glaser, R., Linn, R., & Bohrnstedt, G. (1997). Assessment in transition: Monitoring the nation’s educational progress. New York: National Academy of Education.
Pellegrino, J., Chudowsky, N., & Glaser, R. (Eds.). (2001). Knowing what students know: The science and design of educational assessment. Washington DC: National Academy Press.
Stiggins, R. J. (1996). Student-centered classroom assessment (2nd ed.). Old Tappan, NJ: Prentice Hall.