Handbook

Outcomes should help to define your program and how it differs from the others on campus. The statements should indicate the skills and experiences that your students are expected to attain before graduation. So, before you get started, take a moment to consider what makes your program, office or unit distinctive.

• CoC outcomes worksheet (docx)
• OHC Alignment worksheet (xls)
• Guide to creating Student Learning Outcomes
• Guide to Alignment with LEP and OHC

“Evidence includes qualitative as well as quantitative information.” (Barbara Walvoord, Assessment Clear and Simple)

Now that you have defined your student learning outcomes, you are ready to move on to assessment measures—how will your students demonstrate their knowledge, understanding and skills for you?

• CoC outcome measures worksheet (xls)
• Program outcome measures worksheet (in development)
• Guide to creating your program matrix and timeline
• Guide to assessment measures

When is the best time to collect the data? Timing is critical. Consider the rhythm of the academic year. Generally speaking we recommend collecting evidence at the end of each semester. Someone/a group of hearty individuals is going to have to parse the results and report those to the department.

How can the data be evaluated? You should agree to some level of attainment. What is the minimum performance that a student can achieve on the assessment measure that would not warrant some kind of intervention. For example, a specific percentile score on a major field exam or rubric, percentage of students completing an internship or admitted to grad school, narrative description of student performance, etc.

NOVA has a library of articles on analyzing data including statistical refreshers. 

Quantitative data

Assessment data measured numerically (counts, scores, percentages, etc.) are most often summarized using simple charts, graphs, tables, and descriptive statistics- mean, median, mode, standard deviation, percentage, etc. Deciding on which quantitative analysis method is best depends on (a) the specific assessment method (b) the type of data collected (nominal, ordinal interval, or ratio data) and (c) the audience receiving and using the results.

No one analysis method is best,but means (averages) and percentages are used most frequently.

Qualitative data

SLOs assessed using qualitative methods focus on words and descriptions and produce verbal or narrative data. These types of data are collected through focus groups, interviews, opened-ended questionnaires, and other less structured methodologies.

Often qualitative data is used to articulate common themes that are not drawn out via quantitative methods. They can be the clue that leads to a more focused collection of data on a given topic.

“It is not enough to gather data about student strengths and weaknesses; you need information and hypotheses about the causes of student weaknesses.” (Barbara Walvoord, Assessment Clear and Simple)

Now you have gathered information on how well students perform on the student learning outcomes for your program. Does this performance meet the expectations of program faculty? If so, create a banner and spread the word: Our students learn! If not, you and your colleagues need to identify and implement strategies for change. The point of assessment is to improve student learning, so:

• Use the data to develop plausible hypotheses about why the students did not perform as expected on this SLO. For example: Is the SLO poorly conceptualized or measured? Do the courses in which this SLO is addressed place enough emphasis on the SLO to bring about the desired level of learning? Are the pedagogical methods being used well suited for this SLO and student population? Are students entering the program with insufficient preparation? Do characteristics of the student population and the institutional context create an especially challenging learning environment for this SLO or this program?
• Remind yourselves that you want all of your students to succeed. Consider your outcome statement, your measure(s), your curriculum, advising, and co-curricular programs. What would make students more successful in your program? What would make your assessment planning more successful?
• Test out your hypotheses by changing your curriculum, instruction, or service.
• Gather additional data on student learning to see if the changes you implemented worked.

By doing all of the above, you will have officially “closed the loop”. Take a moment to rest on your laurels, shake hands with colleagues, and enjoy a brownie… but not too long! Assessment is ongoing and cyclical. You are either about to implement some changes that will need to be assessed, or you are about to test the next SLO in your list. So, Happy Assessing!

• Making changes based on evidence

Transparency is the new norm in assessment. It is not enough to use the results of assessment to make changes, you should provide assessment information in an easily digestible format for students, parents, prospectives and other external constituents.

Consider using your assessment results as advertising for your program or providing University Assessment with information that can be shared on our website.

Additional information can be found on the National Institute for Learning Outcomes Assessment (NILOA) website.

Annual Reports

Per the University Assessment Policy, annual reports are to be completed and submitted through the assessment management system by October 31 annually.

Your report should include information about:

• the measures that were used,
• the findings--including how they were discussed and participation of colleagues in the process,
• and proposed changes—including person(s) responsible and a timeframe for completion.

Subsequent reports should indicate how the changes fared and whether they produced the desired results.