Beginning in Fall 2023, end-of-semester course evaluation surveys will feature a new question set. The new survey was designed to update the previous questionnaire in use since 1987 with particular attention paid to reflect changes in pedagogy and to mitigating potential biases. Over the course of two years, the MCEF Student Evaluation of Teaching Committee carried out the charge of aligning the evaluation with current best practices in course assessments and ensuring diversity, equity, and inclusion considerations are represented in the survey. After working with partners in CNDLS and OADS, graduate and undergraduate students, and consultants, the SET Committee recommended a revised questionnaire which Main Campus Executive Faculty approved in April 2023.
What’s changing in the new survey version?
Several new elements appear in the new survey. A new section, ‘Course Environment‘, has been added to collect feedback on:
- Whether the course environment was welcoming and inclusive for students
- The opportunities provided for discussion with professors and classmates
- Whether all students are provided opportunities to contribute equally
- The diversity of student opinions, thoughts, voices, and perspectives encouraged and/or incorporated throughout the course
Matrix-style question sets will make use of a consistent main prompt and probe with sub-questions on related themes. An additional question appraising overall evaluation of the instructor extends the current assessment of overall evaluation of the course. ‘Overall’ questions will now lead at the beginning of the survey after this investigation revealed the higher response rates which result from that question order. The questionnaire will no longer gauge if the course ‘challenged’ students, incorporating feedback from student focus groups.
How will this impact reporting results and merit review?
Since the questions are changing, evaluation results will not be able to be displayed in one contiguous table with previous evaluation scores. Future requests for historical course evaluation reports will be partitioned in multiple tables based on survey version.
Reports from fall 2023 onwards will be able to be combined in one table and reports prior to fall 2023 will be combined in a separate table. OADS is working with merit review committees and Department Chairs to communicate these changes and provide new report formats, given the changing questionnaire.
Is the revised questionnaire the only survey version I can use?
The revised question set will be applied to the ‘Standard In-Person‘ survey version. All course evaluations are conducted online through Blue’s portal, while ‘in-person’ refers to the corresponding course modality. The ‘Standard Online’ survey version will remain unchanged and may be used for synchronous or asynchronous courses with online components.
This new questionnaire will fully replace the previous 1987 survey version as Georgetown’s primary course evaluation instrument. Up to five additional customized questions may be added by instructors (Blue’s Question Personalization), but the core question set will come from the newly approved questionnaire for in-person courses.
What were the full findings of the MCEF SET Committee’s investigation?
MCEF’s Student Evaluation of Teaching Committee performed a literature review, consulted with internal and external survey experts, ran a pilot test of the new question set in fall 2022, and performed focus groups with graduate and undergraduate students to inform final recommendations.
The new instrument is a valid evaluation instrument whose questions all load onto the theme ‘Teacher Effectiveness’ in a principal component analysis (PCA). Additional themes statistically validated during PCA include ‘Course Content’ and ‘Course Environment/ Inclusive Practices’, meaning the new survey captures distinct key components in teaching.
The final report is available for faculty and department administration; please email email@example.com for a copy.
Who do I contact if I have questions?
If you have any other questions, please reach out to firstname.lastname@example.org.