What you will find here
- What is an assessment activity?
- What aspect of the program should be assessed?
- Defining a research question
- What methods of assessment should we use?
- Where can we get help with data / research?
- What makes for a good assessment activity?
- Learning Support / Student Services: How do we assess our programs?
- What about interdisciplinary assessments?
WHAT IS AN ASSESSMENT ACTIVITY?
Learning outcomes clearly state learning goals. Assessment uses these outcomes as a framework for exploring whether students (and, therefore, whether we) are achieving those goals.
In other words: How are students doing – in our courses, services, programs, and institution?
An assessment activity attempts to address some version of this question.
The big picture: Asking the question helps us take the long view of our courses and programs – to look at the “big picture.” Faculty tend to know how last year’s students did in a specific course. What we do not always know, however, is how well this coursework helps students achieve their larger degree or certificate goals, or how effectively we’re contributing to a student’s general education.
Drilling down: While assessment is about looking at the bigger picture, a generic question (i.e., “How are students doing?”) is often not very helpful. Instead, the assessment process gives us a chance to ask a specific question about the bigger picture of student learning.
Some examples of possible inquiries:
- Looking for patterns in student learning to see if there are specific areas of weakness at the course, service, or program level that need addressing (“Which concept in our Political Science courses are students struggling with most?”)
- Exploring ways to address issues we already know exist, such as closing achievement gaps, and assessing what is working (“How can we support students of color to help them achieve important learning outcomes in our discipline?”)
- Evaluating the effectiveness of some new teaching strategy or initiative (“Which of our pedagogical strategies have helped students improve their sentence-level writing?”)
Examples. Here are some examples of what might be considered “assessment” activities – activities intended to clarify or address aspects of student learning.
- What aspect of the degree program needs reinforcement?
- Faculty administer a survey to recently graduated students, soliciting input; AND/OR
- Faculty create a capstone assignment in capstone course; grade collaboratively, using a rubric aligned with PLOs; identify where students need more support; design and implement interventions (new teaching strategies, curriculum changes, more targeted use of the Learning Center, etc.). - How can I better prepare students to apply what they learn in my class to their general knowledge work?
- Faculty collaborate across disciplines to create a shared assignment, focusing on core competency in their discipline, and grade with a shared rubric; OR
- Faculty collaborate with faculty and staff from other disciplines and services to host a student activity (discussion, debate, team game, movie viewing); OR
- Faculty participate in a flex day ILO assessment workshop, to re-envision in an interdisciplinary context.
(back to top)
WHAT ASPECT OF THE PROGRAM SHOULD WE ASSESS?
Standard definition: A program is a series of courses culminating in a degree or certificate.
Defining "program" for the purposes of assessing student learning: The most obvious way to use SLO assessment is to see whether students are achieving the learning goals of a degree or certificate program.
In many institutions, the SLO assessment process focuses on degree or certificate programs. Where many students are focused on achieving a degree or completing a certificate (for instance, in four-year institutions or CTE programs), the program outcomes offer a good vantage point from which to evaluate the effectiveness of the courses. In other words, where lots of students take a degree, it is worth ensuring that the courses in that degree really impart the knowledge, skills, or abilities defined in the outcomes.
However, it is not always practical or useful to focus on the learning goals of a degree or certificate program. Not all disciplines offer a degree or certificate, and in many disciplines, few students complete the degree or program. There may be other endpoints from which faculty want to evaluate the effectiveness of their courses or services. There may also be issues in student learning that require urgent attention, which relate only indirectly to the degree or certificate goals.
Faculty can therefore prioritize whatever aspect of their discipline or service area that needs attention. This could be:
- the learning goals of a degree or certificate program;
- the learning goals of a sequence of courses (for instance, ESL courses)
- the learning goals of a disciplinary core (for disciplines with neither sequenced courses nor a degree/certificate)
- the learning goals of a learning community.
Even where faculty do wish to assess the effectiveness of degrees and certificates, they may not always be able to assess all their degrees and/or certificates in a two-year period or they may need to prioritize a degree, certificate, or sequence of courses for a particular reason. Good reasons to prioritize might include:
- to assess a very popular degree or certificate ("Is our Certificate in Teaching Yoga giving students what they need?")
- to assess a new degree / certificate / sequence of courses ("Does our new Oenology AA give our students the right skills?")
- to focus on improving student learning in one particular program or course outcome ("Program Outcome #2 looks weak; what can we do to improve this?")
- to focus on improving one skill area ("Students are struggling with reading - let's find ways to improve.")
- to evaluate the success of a change in pedagogy or curriculum ("Are the new course requirements improving students' grasp of the material?")
- to evaluate the success of a particular student body in the program ("Have we improved outcomes for our Pacific Islander students?")
- to scrutinize a problematic degree, certificate, or course sequence (low rates of success, retention, etc.) ("Where exactly are students struggling in ENGL 110?")
If your discipline offers popular degree and/or certificate programs, these are good targets for assessment.
However, focus your assessment where it will have the most impact on student learning.
(back to top)
DEFINING A RESEARCH QUESTION
The starting point for any assessment of student learning is: What do we want to find out?
Some questions can always apply. For example, faculty can always ask the kinds of questions that help establish the health of student learning in a chosen program:
- Is the curriculum in the program cohesive, current, complete?
- Do students walk away with the knowledge, skills, and abilities promised in the relevant learning outcomes?
- Are students retaining their knowledge/skills and applying it to other classes?
- Where are there achievement gaps, and what can we do about them?
But some questions may be more precise, and may reflect specific priorities. For example, faculty might want to look at a particular student population, or at the health of a new curriculum, or at some particular interest or concern.
Examples:
- Where do international students need the most academic support in our program?
- We dropped a pre-requisite last year. Has this impacted student learning in the class?
- Since complying with AB 705, we’ve opened up our first-year classes to students with varied levels of preparedness. Are they achieving the same learning outcomes as past first-year students? And if not, which skills specifically need the most support, and what interventions are most effective?
- Three instructors tried three different methods of getting students to read effectively. Which worked best?
WHAT METHODS OF ASSESSMENT SHOULD WE USE?
Suit the method to the question: The best choice of assessment method depends on what you are trying to find out.
Direct methods involve direct observations of student work, and typically (though not necessarily) draw on course assignments. These are best suited to questions about improved student performance (“Did this newly created module on active reading help students understand their assigned readings better?”).
Indirect methods may be better suited to questions about student perception, confidence, or reflection (“Did this program adequately prepare students for transfer work?”).
Success on external exams or employment are typically counted as an indirect measure of student learning. However, in a program intended to prepare students for licensure or employment, it is probably the most relevant measure of the program’s effectiveness.
Methods of assessment include, but are not limited to:| Direct method | Indirect method |
|---|---|
|
|
Capstone courses / assignments
Definition: A capstone is the summative course or assignment in a course of study.
- A capstone assignment requires students to demonstrate all the knowledge, skills, and abilities in a given course.
- A capstone course requires students to demonstrate all the knowledge, skills, and abilities in a given program.
Where there is a capstone course, a key assignment or assignments in that course also give an overview of student learning in the program outcomes.
An embedded assignment can use language that deliberately evokes the program outcomes. Here is an example from ENGL 110 (the relevant part italicized):PROGRAM LEARNING OUTCOMES - ENGL AA-T:
|
| ESSAY: Many of the stories and poems we've read focus on language barriers inside families. What happens to family relationships when parents and children speak a different language? What problems arise, and how do people overcome them? In a well-developed essay (about 2000 words), discuss this question. Your essay should present a coherent thesis, and your support should include a convincing literary analysis of at least one poem and one short story we've read, as well as evidence and/or illustrations from non-fiction works by Judith Harris' The Nurture Assumption, Amy Tan's "My Mother's English," and at least two other sources you find yourself. Your paper should be laid out and cited in MLA format. |
Aligned rubric: A graded assignment can easily yield an SLO score with a rubric aligned to the outcomes.
(back to top)
WHERE CAN WE GET HELP WITH DATA / RESEARCH?
Assessment is framed around your research question, so you should look for the data that will answer your specific question.
Here is a (by no means exhaustive) list:
- student work (from in-class assignments or specially organized student activities);
- student surveys (enrolled students, or those recently graduated, employed or transferred);
- focus groups;
- student achievement data;
- college-wide surveys;
- licensure results;
- follow-up with target programs or employers.
Who can help you find this data?
Faculty and staff: Faculty and staff working with students can use in-class assignments, surveys, focus groups, student activities, contact target programs or employers for feedback, or draw on licensure results.
PRIE: The Office of Planning, Research, Innovation and Effectiveness supports small-scale faculty assessment research in a number of ways:
- Research requests: PRIE has access to College student achievement data and other resources.
- Trainings: PRIE can offer workshops in research basics such as survey design.
- Campus Climate and Satisfaction Surveys: These are administered by PRIE and may include questions relating to outcomes assessment, such as student satisfaction with Learning Centers, support services, and institutional learning outcomes.
(To find out more, visit PRIE's website: http://collegeofsanmateo.edu/prie/).
Marketing: The College's Marketing Office can help with outreach to CSM alumni.
The Student Learning Outcomes Coordinator and Research Inquiry and Institutional Effectiveness Manager: Feel free to contact the SLOAC or the Research Inquiry and Institutional Effectiveness Manager in the Office of PRIE with any questions.
(back to top)
WHAT MAKES FOR A GOOD ASSESSMENT ACTIVITY?
Qualities of a good assessment activity - Here are some questions to bear in mind:
- Does your assessment activity involve collegial conversations? Think of the SLO requirement as a mandate to have regular discussions about student learning, inside and across disciplines. The Assessment Plan should be a departmental discussion item; and institutional outcomes assessments create a forum for interdisciplinary conversations about student learning. Some departments have only one faculty member, so not all discipline-level assessments can be part of a collegial process. However, wherever possible, assessment planning should emerge from collegial discussions.
- Does your assessment activity focus on something specific? Tackle a specific aspect of learning or research question.
- Does your assessment activity address student learning? Raising grades, improving enrollment, boosting productivity, and so on are important, but not necessarily issues of student learning (although improved grades may be the way you measure a proposed improvement in learning).
- Does your assessment activity tell you something that you don't already know? If your SLO activity only repeats what you know already from grading and working with students, try something new.
- Does your assessment activity involve evidence and observation? Learning outcomes assessment should be grounded in data, which can take the form of surveys, experiments, course grades, completion rates, or whatever other instrument suits your purpose.
- Is your assessment activity actionable? Faculty should be able to do something about what they discover.
- Is your assessment activity manageable? Faculty do not have time to launch elaborate research projects. Start with a research question that is feasible to assess in a specified amount of time.
Types of assessment activities - Assessment activities might include:
- Pedagogical experiments: Assessment processes can evaluate new teaching strategies, a curricular innovation, participation in a new initiative, or a small-scale learning community aimed at promoting student learning within or across disciplines.
- Curricular review: Faculty can review SLO maps (for program or institutional outcomes) to ensure degree and certificate curricula are still coherent, current, and comprehensive.
- Research: Faculty can research a particular question about student learning that can't be answered in class (for example, whether an accelerated class has the same success rate as a full-semester class, and why).
- Norming / standardizing faculty assessment of student learning: Faculty can use the assessment process to ensure that grades reflect true student learning, or to minimize subjectivity in grades through norming.
- Improving assessments: Do your assignments truly gauge student learning? Faculty can use the process to reflect on and improve their own assessment instruments.
- Promoting and assessing interdisciplinary learning: Small-scale learning communities, shared activities (movies, debates, etc.) between classes, participation in interdisciplinary workshops, and student activities are all excellent ways to promote and assess student learning beyond the classroom.
Examples: Here are some examples of assessment activities:
"Students are doing poorly in sentence-level writing. Three of our faculty tried three different strategies for improvement and compared notes at the end of the semester to see which one worked better. None was perfect, but one was noticeably less useful. We're discussing this at our next meeting."
"The Biology department is trying culturally relevant approaches to teaching BIOL 100 to tackle achievement gaps in STEM education. We will monitor grades to see if students improve, and administer a short survey at the end of the semester to see if students had a more positive experience. A biology teacher came to an English 165 class as part of a unit on bad science reporting. Students completed a short survey afterwards to report on whether this enhanced their grasp of critical thinking across disciplines."
"We reviewed the alignment of our course-to-program outcomes; turns out no one remembers who wrote the original outcomes, and we don’t like them, so we’re rewriting them to reflect current program needs."
"Two of our faculty participated in the Fall ILO workshop, creating a rubric for Effective Communication, which we used to norm some assignments in our department in Spring to see if our assignments aligned clearly with the ILO."
"With AB705, we're going to have a lot more students repeating our ENGL 105 class. What will these students need? We will survey students repeating classes to find out what they think they need more help with, what got in their way, and what would help them most as they retake the class."
"We wanted to know if the students who completed our AA-T in Culinary Arts felt they were well-prepared for careers or transfer. Working with Marketing, we got ahold of 23 recent graduates and surveyed them. We discovered that we are doing much better at preparing students for transfer than for careers. We are adding a unit to two of the required courses directly aimed at workplace issues."
For more ideas,
- Contact the SLOAC (email listed at http://collegeofsanmateo.edu/sloac/)
- Contact the Research Inquiry and Institutional Effectiveness Manager at PRIE (https://www.collegeofsanmateo.edu/prie/contactus.php)
- Review the 2019 Assessment Report (https://collegeofsanmateo.edu/sloac/docs/2019_assessment_report.pdf)
(back to top)
LEARNING SUPPORT AND STUDENT SERVICES: HOW DO WE ASSESS STUDENT OUTCOMES?
The goal of Learning Support and Student Services assessment is to ask questions that will help improve student outcomes, gather data that will answer those questions, and analyze and act upon the findings.
Student Services: SLOs or SAOs? Not all student services outcomes are learning outcomes. Which sort of outcome will best help you evaluate the success of your service?
SLOs = Student Learning Outcomes. A learning outcome describes the knowledge, skills, or abilities students should take away with them due to interacting with the college. For student services, this might be a workshop, information session, or other engagement intended to leave the student with specific knowledge, skills, or abilities.
An SLO is appropriate where faculty or staff are imparting information and success rests on student retention of the information. Examples:
-
- A workshop on instructional material (i.e., grammar, math etc.)
- An orientation to communicate the resources of a center
- A workshop to inform students about financial aid, campus resources etc.
Methods typically include exit quizzes, surveys, etc.
Where a student support service offers instructional courses (e.g., Counseling), the course SLOs are assessed like any other instructional course.
SAOs = Service Area Outcomes. SAOs describe what students have been able to do as the result of an interaction with the college. Examples:
-
- Complete an application or form
- Get access to a health or wellness counselor
- Complete their registration entirely online
Here is a side-by-side comparison:
| SLO | SAO |
|---|---|
| Question: Did this service succeed? = Did students learn something? | Question: Did this service succeed? = Were students able to get something done? |
|
|
Research question
The generic question is simple: Is the service delivering the promised outcomes for students?
Depending on priorities, faculty and staff may wish to apply a more specific question to their service or learning center. For instance, a center or service might want to explore whether a change in procedure was improving the experience for specific student populations, whether a new online service was delivering effective service, or whether students were largely satisfied with the help they received. A service or center might also want to look at its work in the context of larger institutional goals (see “A word on interdisciplinary alignment” below).
Alignment with institutional outcomes
Learning centers and student services are not required, in Program Review, to discuss participation in any institutional assessment.
However, like all College constituencies, centers and services support institutional learning outcomes (ILOs), both directly and indirectly.
The first ILO, especially, addresses skills and abilities directly fostered by engagement with centers and services:
|
Independent learning and development - The ability of students to develop, evaluate, and pursue personal, academic, and/or career goals. Students will be able to:
|
The language of this outcome can provide a framework for center and service assessment, or offer an opportunity for assessment activities that involve collaboration with disciplines and services.
Methods of assessment
Learning centers and student services typically use:
- Student surveys (e.g., after usage of a center or service)
- Pre- and/or post-quizzes (e.g., for workshops)
- Embedded assignments (e.g., for instructional courses)
- Best practices
- Other as determined
(back to top)
WHAT ABOUT INTERDISCIPLINARY ASSESSMENTS?
Faculty in the classroom and student services faculty and staff in learning centers and offices rarely have a chance to look at student learning from perspectives outside their own. Institutional assessment activities can provide a chance to think outside the classroom.
Interdisciplinary assessment matters. Student learning is enhanced when knowledge, skills, and abilities presented in the classroom are reinforced or otherwise referenced across the curriculum. To this end, the more faculty have an opportunity to collaborate, cross-reference, or otherwise make connections between disciplines (and between disciplines and services), the better we will support our students towards their educational goals. Additionally, many aspects of student learning touch on all disciplines (for instance, critical thinking or reading).
Faculty and staff have a number of ways to participate in interdisciplinary assessment activities:
Participate in institutional outcomes flex day workshops: Every year, institutional outcomes are assessed by a team of faculty participating in flex-day workshops.
Organize an interdisciplinary class or activity: Faculty can create an activity or shared project that aligns with a shared institutional learning outcome. Some ideas include:
- inviting guest speakers from another discipline or services;
- interdisciplinary norming of student work;
- teaming up with a colleague from another discipline to share an assignment, develop a course theme, team-teach a concept, etc.;
- creating an interdisciplinary student activity.
Interdisciplinary learning activities like these do indeed constitute institutional assessment activities, provided they:
- address a shared institutional learning outcome (e.g., effective communication, critical thinking, etc.)
- involve an attempt to improve student learning in that outcome
- involve documented discussions and/or activities between faculty.
Research question: Interdisciplinary activities can focus on a generic assessment question, such as: “Are students learning to communicate effectively?”
However, faculty can and should create their own priorities. Some examples of more specific assessment questions might be:
- “How can we better support student reading?”
- “How can we get students to think critically about numbers in different contexts?”
- “If non-science majors can make a connection between good citizenship and scientific awareness, will this help them apply what they learn in science class in their daily lives?”
- “How can I make ethical reasoning figure more explicitly in my curriculum, and will this make a difference to students?”
- “Are the students in my English comp classes really using their skills in other essay-writing classes? Where are they falling short?”
