Cultural Resource Management MA Timeline
SCHEDULE
The schedule for implementing the assessment plan is noted in the sections below describing the short and long term plans. To summarize, the student learning outcomes for each course will be assessed within the first four years of the program beginning with the fall 2007 semester. The core courses will be assessed multiple times from fall 2007 through the fall 2011 semesters to refine learning outcomes and course goals. The long term program goals will be assessed after the first student cohort has graduated and has entered the workforce. As such the long term assessment will not begin until late 2009 or 2010.
I. Short-term assessment (0-4 years)
A. External: Advisory board
1. review CRM program performance
2. advise on job-specific competencies
3. provide advice for curriculum
4. aid in recruiting students
5. support and encourage internship and culminating project opportunities
B. Internal: Student and Faculty Assessment
1. Student assessment
a. course questionnaires
b. end of year questionnaires
c. internship questionnaires
2. Faculty assessment
a. course questionnaire review
b. end of year questionnaire review
c. student performance review
II. Long-term assessment (4+ years)
A. External: Advisory board
B. Internal
1. Student assessment
a. academic experience (includes I.B.1.a.-c.)
b. professional experience questionnaire
2. Faculty assessment
a. overall student academic experience (includes I.B.2.a.-c.)
b. employer experience questionnaire
c. CRM program structure
3. Professional performance
Short-term Assessment
Two components of the short-term assessment, the advisory board and course questionnaires, will provide us with relatively rapid feedback that will allow us to evaluate how closely we are meeting our learning outcomes and mission statement. The advisory board will play an important role in assessing the overall design of the graduate program from the perspective of non-academic professional CRM archaeologists. Immediate feedback from the advisory board will help to determine if we are heading in the right direction. Course questionnaires will allow the program faculty to monitor how well we are achieving our learning outcomes in the classroom. Because the graduate courses will be offered for the first time this coming year, these questionnaires are crucial to identifying strengths and weaknesses of course content in terms of achieving our guiding mission. The following sections describe the short-term assessment components in more detail.
Advisory Board
Input from an independent advisory board will provide an external assessment of the CRM program. Because we currently lack graduate students, the Advisory Board (hereafter “Board”) will initially be the most important component of short-term assessment. The Board will serve to provide an active link between the professional CRM community and the CRM program. Board membership will represent state, tribal, federal, and private agencies and firms in order to provide a cross-section of the profession. The scope of the Board is advisory only. The CRM program faculty will be solely responsible for interpreting and implementing Board recommendations into curriculum or program requirements. Although the immediate feedback of the Board will help with short-term assessment, continued dialogue between the Board and program faculty will be maintained as an integral component of assessment.
The goals of the Board are to:
1) review CRM program performance in terms of facilities, technology, and overall student
preparedness and professionalism
2) advise on job-specific competencies and performance levels for graduate students
3) provide advice for CRM program curriculum
4) aid in recruiting students into the CRM program
5) support and encourage internship and thesis research opportunities for graduate students as
appropriate
Board members include:
1. Elisse Aune: Mille Lacs Band of Ojibwe, Tribal Historic Preservation Office, Review and
Compliance Coordinator
2. David Cooper: National Park Service, Grand Portage National Monument, Chief of Resource
Management
3. Pat Emerson: Minnesota Historical Society, Head of Archaeology
4. Amanda Gronhovd: Council of Minnesota Archaeologists Representative
5. Jim Jones: Minnesota Indian Affairs Council, Cultural resource Specialist
6. Bruce Koenen: Minnesota Assistant State Archaeologist
7. Dale Maul: Bolton & Menk, Inc., Transportation Planner/Environmental
Specialist/Archaeologist
8. Patrick McLoughlin: Natural Resource Conservation Service, St, Paul, MN, Cultural
Resource Specialist
9. Susan Mulholland: Duluth Archaeology Center, L.L.C., Principal Investigator
10. Darlene St Clair: Lower Sioux Dakota, Tribal Historic Preservation Committee
Representative
11. Mark Muñiz: SCSU CRM Archaeology M.S. Program, Director (ex officio)
Student Assessment
Students will provide an internal assessment of the CRM program by answering course questionnaires designed to identify how well learning outcomes are being met. The responses will be anonymous and returned to the appropriate faculty member after grades are submitted. The faculty member may then use the assessment results to improve the course. In addition to the course questionnaires, students will also answer a questionnaire at the end of the academic year that addresses how well the overall program is achieving its mission. Student responses to these questionnaires will also be anonymous and the qualitative and quantitative results will be tallied and analyzed by the program director. The results will be distributed to the program faculty and discussed at the end of the academic year.
Students will also assess their individual internship experiences. Internship contracts will be developed to insure that the desired learning outcomes are included in the educational experience and the advisor, student, and internship host will maintain communication throughout the internship. The student will complete an assessment questionnaire at the end of the internship that addresses the overall effectiveness of the experience in terms of meeting contractual obligations and how well learning outcomes were achieved. This assessment technique will be valuable in the short term for monitoring how well specific internships serve to provide an educational and professional experience and in the long term by helping us design effective internship experiences from the beginning.
Faculty Assessment
The Anthropology faculty will provide an internal assessment of the program by evaluating how well students understand course material and how well they are completing the overall graduate program requirements. Course material will focus on the major and minor learning outcomes identified in the learning outcome matrix and rubrics will be designed that provide a standardized way of evaluating student performance for written papers and presentations. Exams will include questions that specifically address the learning outcomes outlined above and answers to these “assessment questions” can be evaluated as a gauge of how well students are learning the key points of each course.
As mentioned above, faculty will also use the feedback from assessment questionnaires distributed at the end of each course and the academic year to analyze how well we are achieving our learning outcomes and overall mission. Based on these analyses, appropriate changes may be made as necessary. Faculty will also assess the progress each student is making towards the completion of the degree in a faculty meeting held for this purpose at the end of the academic year. The faculty will compose a letter to each student that acknowledges progress, identifies deficiencies, and offers guidance and encouragement to complete any outstanding requirements. By tracking the progress of each student and each cohort, we will eventually collect long-term data that will be used to assess the effectiveness of how we have designed the overall program requirements in relation to the mission and learning outcomes.
Long-term Assessment
The long-term assessment strategy does not really differ from the short-term strategy except for the addition of two important components and the accumulation of long-term data to better analyze trends. The two new components are separate questionnaires answered by the graduated student and the employer after the student has been hired. After the program has graduated two cohorts of students, we will have preliminary data to evaluate basic trends in how well the CRM program is achieving its learning outcomes and meeting its mission. The following sections describe the components of the long-term assessment.
Advisory Board
As discussed above, the Advisory Board will continue to meet and make suggestions for improving the CRM program and keeping the program responsive to needs in the professional world. The suggestions and dialogue provided by the Board will significantly contribute to the long-term external assessment of the CRM program.
Student Assessment
Long-term student assessment will be comprised of course, academic year, and internship questionnaires compiled over several years. In addition, six months after being hired graduates will be asked to fill out a questionnaire that assesses how well the CRM program prepared them for work as a professional archaeologist. A similar questionnaire will be sent to the employer as well. We will track the type of work setting (e.g., federal, state, tribal, museum, or private) but the specific employer and graduate will remain anonymous on the questionnaire.
Faculty Assessment
The faculty will continue to use questionnaires for assessing courses and the overall program as described above. Long-term data will be evaluated for trends and patterns. The program director will also compile assessment data that will give an indication of how well the structure of the CRM program is accomplishing the overall mission. The program structure will be assessed with the following data.
1. Student enrollment rates compared with withdrawal rates (i.e., attrition)
- will assess the overall difficulty of the program and results can be use to increase retention after patterns and sources of withdrawal have been identified
2. Percent of students completing the program:
a. Full time students (total time for degree completion)
b. Full time students / 2 years (ratio of full time students completing the degree in two years)
c. Part time students (total time for degree completion)
- a and b will assess if an average full time student can complete the program in two years as originally intended when the program was designed
- c will assess how long part time students require to complete the program and the results will be used to advise part time students and to better anticipate how the program may meet their needs
3. Percent of students completing Plan A, B, and C culminating projects and how long, on
average, it takes for each plan
- faculty will use results to advise students selecting each plan and to better anticipate student needs
4. Tracking student class enrollment and credit hours
- will be used to determine which elective courses are being selected and how frequently students are enrolling for additional, non-essential courses beyond the program degree requirements
- faculty will use results to better advise students
Finally, as a means of assessing how well the overall program accomplished its mission, a questionnaire will be distributed to employers six months after hiring our graduates. To a large extent, this questionnaire will provide the ultimate assessment of how well the CRM program prepared students for the demands of the CRM industry.
Professional Performance
Assessing our students’ professional performance after graduation will provide a long-term evaluation of how well our program prepared the graduate for real world situations and requirements—essentially it will provide a grand overview of how well we are accomplishing our mission. Data will be gathered from several sources (including the internship) and analyzed by the program director. The professional performance assessment data include:
1. Success of student internship
- determined by combined feedback from the internship host and the student
- results used to better design future internships
2. Percent of students hired by internship host after graduation
- results used to assess overall internship component of the program
3. Frequency of graduate’s hired within six months after graduation
- determined by student questionnaire and tracked internally
- results used to monitor job placement
4. Satisfaction of employer with graduate at six months after hire date
- determined by questionnaire
- results used to assess how well the program prepares graduates for real world working conditions
