Evaluation Pilot FAQ

What is the pilot course evaluation? Why should I participate?

In 2020, the Main Campus Executive Faculty charged a task force with revising the standard course evaluation used at Georgetown University. An emphasis was placed on minimizing bias and updating the questions that have been in place for over 30 years. Using primary literature, external course evaluation experts, seeking input from faculty in different disciplines, and using small student focus groups, a new set of questions was developed and approved by the MCEF in June 2021. The goal is to replace the standard set of questions with this updated version for fall 2023.

This fall, CNDLS, in collaboration with OADS and MCEF, is conducting a pilot of the new evaluation questions using a Qualtrics survey. Versions of the new evaluation questions will alter the placement of the “overall evaluation” questions (evaluation of instructor/course) and may contain a newly implemented DEI statement that precedes the survey. Faculty are encouraged to volunteer their courses to ensure a large enough sample size is achieved for post-survey analysis. It is also an opportunity for faculty to gain experience with the new set of questions before they are officially launched. Please note that the pilot survey will be in addition to, and not take the place of the standard course evaluation that is launched in the Blue Course Evaluation platform. The results of this pilot will not impact merit, promotion, or retention decisions.

Why are the course evaluations being updated?

The goals for this update are to align the evaluation with best practices in course assessments, reflect changes in teaching practices, and to mitigate bias.

Will results of the survey responses be shared with participating faculty?

Faculty participating in the pilot will receive de-identified results from their course(s) only if the course achieves a 30% response rate or higher. Please note that these responses will be for the faculty only; it is not intended to take the place of the standard evaluation at this time.

What courses is the pilot intended for?

The new course evaluations were designed to replace the standard set of evaluation questions that is currently used on the Main Campus. Thus, the pilot is intended for these courses.

When will the pilot be active? Will it correspond to the window of the standard course evaluations run through the Blue platform?

The pilot is intended to run concurrently with the standard evaluations that will be open from November 30 to December 9, 2022. The pilot survey will be delivered through Qualtrics.

How can we ensure that students will fill out both the pilot form and the standard evaluation?

No change will be made for participating courses with regards to asking all students in the course to fill out the standard course evaluations. However, in addition, students’ participation in the pilot will be optional but highly encouraged. CNDLS is working on a list of “talking points” that will be distributed to all participating faculty. This will provide guidelines to faculty for how to introduce the course evaluation pilot emphasizing the importance of this process.

How will the pilot be analyzed?

Students receiving the piloted course evaluation will receive one of several versions. This includes a randomized placement of the “overall evaluation” questions as well as the inclusion of a bias statement. Exploratory analysis will be conducted with additional analyses dependent on the total number of faculty participants and the student response rate. All participating faculty will be invited to meet with an analyst to review the old and new evaluations and to provide feedback on the future rollout of the course evaluations. Please note that the results from the standard evaluation will not be directly compared to the results of the pilot.

Will student feedback be requested?

If time permits, a subset of participating students will be invited to participate in several optional focus groups. This will provide us with a chance to gather student feedback and to learn about their interpretation of the bias statement and the various ordering options under consideration.