In this post, I report on our first senior exit interviews. This spring also marked our second year of participation in the CRA Data Buddies survey and Whitman’s second year of major program assessments.

Senior exit interviews are a continual assessment technique I learned from my colleagues at Grinnell College. As Henry Walker likes to say, the one thing that successful programs share in common is that they are never satisfied, but always looking for ways to improve. While exit interview findings might not be immediately acted upon, I can think of several times when they provided the seed of an idea for further investigation and consideration. It was also a graduating senior who gave me the insight that faculty never get good at teaching courses outside their research areas that are offered too infrequently, an insight that became important to the design of Whitman’s major curriculum.

I also found that exit interviews provide an opportunity for one last substantive conversation as adviser and advisee. The interview questions prompt the graduate to reflect on their experiences as a whole and with a bit of distance. It’s also a natural time to congratulate students on their upcoming graduation and check in on plans for life beyond graduation.

With these benefits in mind, it was not difficult to persuade John and Andy to conduct senior exit interviews in our program. Our biggest hesitations were practical: We considered whether John and I could interview all fifteen seniors in a short span of time just before graduation. We needed to decide on questions in a hurry. And we also wondered whether we’d get meaningful feedback from this first cohort of students, for whom the major curriculum was being built just in time.

We decided that scheduling fifteen 30-60 minute interviews during finals week would not be too difficult, especially with three extra days between the senior grade deadline and Commencement weekend, and with Andy around those three days to interview several of his advisees.

We decided to adopt Grinnell’s exit interview questions with minimal adaptations:

1. (a) Looking back on your experiences in the major, what did you like best?
   (b) What did you like least?
2. (a) What did you gain from the major?
   (b) What do you wish you had gained from the major?
3. (a) What courses in the major do you regard as essential to your education in computer science?  Why?
   (b) Were there any courses in the major that you took but wish you had not taken?  If so, which ones and why?
   (c) Were there any courses in computer science or in mathematics that you wish that you had taken but didn’t?  Which ones and why?
   (d) Were there courses outside the major that you regard as essential to your education in computer science?  If so, which ones and why?
4. What suggestions do you have for improving or changing our program for majors?
5. What else would you like me to know about your experience in the CS major or at Whitman?

At Grinnell, we often added a sixth question on a timely topic. This year, we decided to let the main questions stand alone, but next year we may explicitly ask about department culture.

Last and most importantly, we decided we would regret not eliciting these reflections from our first full graduating class. Given we want to conduct exit interviews at some point, there is no time like the present.

Our worry that we might not get meaningful feedback from this first class was unfounded. Yes, some seniors said they regretted not taking courses in the proper sequence – but that is something we had already decided not to permit going forward.

During our annual summer department retreat, we collated our advisees’ responses and discussed the findings. What did we learn?

• Students appreciated departmental community, pair programming, team projects, positive classroom and lab dynamics, opportunities to mentor other students, and our attention to gender in many of those things. We are clearly doing something right. At the same time, we were prompted to discuss rebalancing the weighting of group and individual work in several courses, something we had each considered individually but had not yet discussed together.
• Not surprisingly, the most common “dislike” was limited course selection. We had already talked with colleagues at neighboring Walla Walla University about supporting students in taking advantage of our institutions’ cross-enrollment agreement. We already encourage our advisees to consider broader course offerings as a perk of study abroad. It’s something I tell prospective students about when they ask me to compare studying CS at Whitman to study at a large university such as UW. And it certainly will be a factor in our next proposal for a new tenure line. (Two students made a point of telling us we need more professors. Yes, we’re working on that.)
• Specific courses seniors wished they had been able to take included Web Development, Computer Networking, Computer Security, and an interview prep or problem solving seminar. I’m qualified to teach the first two, if I had room in my schedule. We discussed emphasizing the connections to security in our existing upper-level systems programming course (CS 310). And a problem solving seminar might be an easy overload for one of us, someday, or a student-led activity, for example by members of the new student ACM chapter.
• Every core course (beyond CS 167) was cited by at least one senior as “essential to [their] education in computer science.” Most core courses were cited by 6-8 of the 15 seniors, and most electives were cited at least once as well. The most common response to the question, “Were there any courses in the major that you took but wish you had not taken?” was “No.” Again, a sign we’re doing something right.
• Several seniors spoke of gaining not just general CS knowledge, but specific perspectives on problem solving. Others talked about gaining teamwork and communication skills or thinking critically about computing. This made us feel really good.
• A few seniors wished they had gained more depth of expertise in a particular programming language. That’s a deliberate tradeoff we made in designing our curriculum, but it’s also something we can pay more attention to in how we advise students for whom it’s a goal.
• A few seniors wished they had been challenged more to think about technology from a humanist perspective. This is something we are thinking about, too. We look forward to future discussions with colleagues beyond CS.
• Last but not least, a few seniors wished there were more natural light in our upstairs lab. We can’t do anything about that, since the lab has no interior windows, but we can think about making space for some plants as we consider possible revisions to the classroom.

We did not learn much about advising from this first round of exit interviews. However, conceptualizing the exit interview as the coda of the advising relationship makes me think a bit more about an advising curriculum or major handbook to provide further structure for the 2-3 years during which we advise each CS major. It also makes me think this should be a topic for a future exit interview question.

Data Buddies is a multi-institutional survey administered by the Computing Research Association‘s Center for Evaluating the Research Pipeline, addresses students’ past experiences in computing, knowledge and confidence, sense of belonging, intrinsic interest, and perceptions of department climate. While CERP’s primary goal for the survey is to conduct large-scale research studies, CERP incentivizes institutional participation by providing results to the participating institutions for purposes of program assessment.

With 25 intermediate- and upper-level students taking the survey, few results were significantly different from our overall cohort of similar institutions. The statements where responses were significantly different:

• “My department cares about its students” (mean 4.52 for our institution vs. 4.19 for similar institutions);
• “People in the department often attribute my success to special treatment or luck, rather than my competence (1.72 vs 2.10);
• “My ideas or opinions are minimized or ignored” (1.64 vs 1.99).