Interview with Kelly Rivers

Professor Kelly Rivers

Women@SCS: The first is a very broad question, so answer it however you like. Could you tell us a little bit about your background and journey I guess before teaching at CMU?

Professor Rivers: I guess starting from undergrad might be a good place. I’ve been at CMU as a bit of a lifer. I’ve been here as an undergrad, and a PhD student, and now as a faculty member, but I came to CMU originally as a math major, and I ended up taking an intro CS course my first semester, and just fell in love with it, and ended up TA-ing for that course for the rest of my time as an undergrad, and that was sort of my path into computer science, was teaching and being involved in it that way. I ended up double majoring in math and computer science as a result, and by the end, I was sure I was more interested in CS than the math side.

I knew by the end of it I really loved teaching and wanted to keep teaching, and so I decided – I was told the way you do that to teach at a college level is to get a PhD. It’s not actually true. You can teach at a college level with a master’s, but you do have more room for advancement with a PhD. I didn’t know that at the time. So I applied to PhD programs, ended up getting into the HCI program here at CMU, and I worked there for a while studying different types of learning science, how people learn, how we can make tools to help them learn, and worked on my thesis in that range of trying to build tools to help people learn, a way to help people learn programming, which was pretty fun.

Women@SCS: That’s so cool.

Professor Rivers: Yeah, it was really interesting. But then immediately after that I still knew I wanted to teach, so I applied for teaching positions and ended up here.

Women@SCS: Nice. What kinds of projects or tools did you work on at that time?

Professor Rivers: Yeah, so when I went into the PhD, what I was interested in was specifically helping with a situation that I saw a lot of office hours, which is that we’d be helping people with office hours. People were swamped. There were a ton of questions. Students had to wait a while to get help. But I could see that students would often have the same question as other students. The same kind of bug would occur across a bunch of different students and I wanted to see if I could build some kind of tool that would help with that situation, to be able to say here’s an error that’s coming up for a lot of different students.

Can we provide a hint to those students ahead of time so we can focus on the students who need more help, and just be able to provide that obvious hint to a bunch of students in the class? What I ended up doing was working on a tool that’s an intelligent tutoring system. This is basically a tool where students are working on a problem in an online interface, and then they’re getting feedback on the go as they’re working on the problem. I worked on a component of this, which would automatically provide hints for them and would tell them what they could do next to get closer to the solution based on the current work they’re doing.

So the novel part of what I worked on there was being able to generate those hints by not having people write them ahead of time, i.e. here's what the hint should look like for each student. But instead by gathering information about what previous students who worked on the same problem had done in order to say basically if somebody else – if a bunch of other people had solved the problem before, you could compare a bunch of their solutions to this current student’s solution to find which one’s closest, and look at the difference between the two to say here are the parts you need to change in order to get closer to a correct solution, and then to be able to transform that difference into a hint message, to say think about changing this part of the code.

Women@SCS: Cool.

Professor Rivers: Yeah, that was basically my thesis to develop that whole tool and study it to see what kind of impact it had on people’s learning.

Women@SCS: What age group did this project target?

Professor Rivers: College. I’ve always been interested in teaching at a college level. So I ended up doing a couple of studies, actually. I actually did one study with 110 where students were doing practice problems before an exam. So we did the study where some of them had hints on some of the problems and they could just ask for the next step.

Women@SCS: That’s super awesome. Did it get used in future semesters?

Professor Rivers: It’s interesting. It’s a thing where my end goal is absolutely to use it in the course. Part of the difficulty, though, with writing research code is that it’s research code, and that means it works in theory, and you can show it has certain impacts. In practice, it’s still kind of buggy and doesn’t always do exactly what you’d want.

Women@SCS: Right.

Professor Rivers: So it’s a thing where I’ve been working on setting up an environment where students will be able to use it for practice problems. I actually did some research withundergrads this summer where we were setting up an online coding environment that students could log into and work on practice problems, and get different types of feedback. But that’s another thing where we have to test it out and see how it works with the students, and make sure it’s robust and will stand up to 100 students using it at the same time. The end goal is to get the hint system integrated into that so that students can work on practice problems and get hints as they’re working in order to get more information about how to solve problems. So we’re working towards it.

Women@SCS: Nice. That’s perfect that it was birthed out of CMU, where there’s already tons of great teaching faculty and a great TA system. That’s so cool to hear.

Do you ever see yourself leaving CMU, or do you feel like you’re pretty married at this point? 

Professor Rivers: It’s interesting because at every point in my career, I see myself leaving CMU, and then it keeps not happening. I never intended to stay for the PhD program. Well I thought of it as an option but I wasn’t specifically intending to. I thought I’d go to whatever PhD program I got into that was best to match my interests, and then it ended up that CMU best matched my interests, because it turns out we have lots of people who do excellent work in learning science and intelligent tutoring. And it was a much better match than any other opportunity I had.

So I figured well, you’re not “supposed to” stay at the same graduate school as your undergraduate school, but I thought well, I’m just going to a different department. It will be fine. I’ll just do that. And then when I graduated from the PhD program, I definitely wasn’t intending to stay here. I thought of it as an option, and I knew I’d be happy here, but I figured there were a lot of other good options as well, and I did get a lot of offers from other places to go, but part of what ended up happening there was that I had a two body problem. I met my husband in the PhD program. He was also applying to jobs, so the constraint of trying to find jobs that we’d both be happy in that were both in the same location narrowed down the field quite a lot.

We ended up having a few good options. CMU was one of them. And CMU ended up putting the best offer together in terms of positions for both of us that we would both be really happy in. Part of what that looks like was I was told they were interested in me being part of developing what the intro curriculum would look like at CMU, and that was really exciting because that’s not a thing I would get or expect to get in a different place, because I didn’t know what the intro would look like at some other university, whereas here I had experience with it. I taught the courses here, so I had a chance to help develop them and help evolve them in that way. I was really excited about that so I decided to come here instead of going somewhere else. I wouldn’t say I’m married to the idea of staying here forever and ever, but as long as it keeps being the best option, I’m really happy to stay here.

Women@SCS: That’s awesome. So when you first started teaching and working with first years, was there anything that was surprising or unexpected?

Professor Rivers: Ah, man. See that’s a little hard because I’ve been working with first year students for so long that I can barely remember what it was like first teaching them. Can I sort of answer a tangential answer to that?

Women@SCS: Yeah, of course.

Professor Rivers: One recent experience I had that was different was I went from teaching first year students who were college students, to teaching first years who were high school students. I worked with a program over the summer, the Summer Academy of Math and Sciences that brings in these high school students to take these different STEM oriented classes, computer programming class, and it was really interesting in that context to work with these high school students, some of them were in the program for the first time, and to see how different it was in terms of students at a different level not having the math skills that CMU students have, for example, and just changing things like how you teach them about variables, how you teach them about these ideas of abstraction. I always assumed people already had knowledge of how algebra works, so that would make it easy to understand certain other concepts.

Women@SCS: Right.

Professor Rivers: And when you’re missing that, it changed a lot of how people learn these concepts. So that was definitely interesting, seeing the change and how prerequisite knowledge affects what they can learn. Apart from that most of who I teach is intro students, novice students, so if anything I think I’d be more surprised to see what it’s like to teach more advanced students, but I haven’t had much experience with that yet.

Women@SCS: Do you think you would ever teach a higher-level class?

Professor Rivers: Potentially. I taught a couple of higher level classes as a PhD student, as a TA teaching people how to make GUIs and how to do usability studies, and the difference there is that in the higher level classes there’s a lot more room for discussion in terms of debating what are the best ways to approach this, more of a design problem, which I think is really different from the intro programming classes I teach, where it’s like the program works or it doesn’t work, and you can debate in terms of what’s the best approach for it, but there’s still a lot of cold hard facts such as this is the syntax you have to use to do this thing. So I think the difference is that in the intro courses, it’s more about, here are the tools you use to do something, here’s how you approach this problem, whereas with the higher level courses it’s more about let’s discuss the possible ways to approach solving something.

Women@SCS: I see. That’s definitely interesting. So circling back to the intelligent tutoring system project – is that a current research project?

Professor Rivers: On and off.

Women@SCS: You’re definitely busy with a lot of stuff.

Professor Rivers: Yeah. That ends up being the main problem is that during the core semesters, I’m way too busy to work on research. Over the summer, I work on it a little bit. It’s a project where I’m very interested in getting it actively used, but because I’m only working on the development side of it over the summer, it’s just in small amounts. But it’s definitely getting there. I’m hoping over my break times during this next year I can get the rest of it set up in this system we have running online now so we can start demoing it with students and see how it works with students in the courses I teach.

Women@SCS: Yeah, that will be super exciting to see.

Professor Rivers: It’s just a slow process.

Women@SCS: Right, that makes sense. During your undergrad or grad experiences here, were there or are there any women here at CMU who you looked up to?

Professor Rivers: That’s a great question. So when I was a grad student, there was another PhD student who had been in the program for a lot longer than I was who I looked up to in that way, Leigh Ann Sudol, now Leigh Ann DeLyser. She was also working in CS education, more on the policy side of things. And I liked her a lot because she’s really confident, and really good at finding opportunities, and finding ways to help other people, and solve problems. Especially as a first year grad student, I was really impressed by her ability to put herself out there and try different things. I think she was a really good mentor for me to have in terms of that, early on. This especially helped because when I was first entering the program as a grad student, I very much identified as someone who was a computer science educator, and that was my field of study, and while there were a lot of other people in the department who worked on education in general or computer science in general, not many of them were working on computer science education. So it was good to know her as another person in that field, someone else doing that same kind of work I was.

Women@SCS: Right. Having that connection is pretty important.

Professor Rivers: Absolutely.

Women@SCS: On a similar note, do you have any advice for women in male dominated fields in general?

Professor Rivers: I would say knowing other people in the field, like knowing mentors and other people in the cohort who are working on stuff is helpful just to feel like you’re not the only person. I know I’ve been very lucky in my career to know other people in my field, to have a lot of friends in my field, and not feel alone. I had one experience in undergrad that did not work as well. I went and took compilers design as an undergraduate at CMU, and I was the only girl in the class, and that was definitely stressful. I would often feel like, if I ask a question it’s going to reflect on all women and not just me, and things like that, and I know how silly that is, but that definitely added another level of stress. But I’m lucky that’s the only circumstance I had where I felt singled out in that way, and CMU is really good in that there’s a fairly large population of women in computer science. So it’s been a rare circumstance, but I think knowing other people in the field definitely helps with that.

Women@SCS: That’s good advice. Another broad question – do you have a favorite and least favorite part of your job?

Professor Rivers: Probably my favorite – I would say there’s two parts that are my favorite for the job. One part – and I think something you’ll hear a lot of people who like teaching would say is their favorite part – is seeing the moment when students get it, like if they’ve been struggling with a concept for a while and then suddenly it clicks, and they understand what’s going on, and they can start doing it. That’s always a wonderful moment because you can see them start to enjoy what they’re doing, and really understand it, and that’s always wonderful.

The other part of it though, would be – because I don’t get to do as much one on one teaching as an instructor of a large course – but I do enjoy doing the large lecture teaching, and getting to talk to a group of 100, 200 students in the lecture room, and teach them a bunch of new material directly, and try to make it into a more interactive experience where they’re asking questions and I’m getting them involved in solving a problem. I really enjoy doing that. That is definitely super fun as well. Least favorite part, there’s a lot of administrative stuff that needs to get done in any job, which can be tedious at times for sure.

So dealing with all of the paperwork, and all of the stuff that needs to happen day-to-day to get stuff done, that can not be too great. I would also say when work piles up and it needs to get done, and that means you have to do several hours of work over the weekend, even if you don’t want to, that is stressful too.

Women@SCS: Right. Do you have any tricks for balancing all the tasks?

Professor Rivers: Good time management and good planning. I always try to keep a to-do list of different things I need to get done so it doesn’t all have to be in my head because I’ll forget. So I’ll keep a list of things that need to get done. I’ll set a list of deadlines for things that need to happen, and try to meet them whenever possible. A thing I started doing last semester which has been pretty helpful is I’ll block off time in my calendar to say like, these hours I’m spending on this task that needs to get done, these hours I’m spending on this other task that needs to get done.

And that’s really good for making sure my time doesn’t get whiled away in little tasks, that the curriculum prep things I need to do get done on time. That’s been really helpful for that. I think just planning out your time is really good. For me, I try to avoid doing any work when I go home. I try to set it up so all my stuff gets done in the office. It doesn’t always work out for the weekends unless I need to do stuff over the weekend anyway, but generally I don’t do any work when I go home, just spend time with my dog and my husband, and not worry about work.

Women@SCS: That’s awesome. I was going to ask about the dog bed.

Professor Rivers: Oh yeah, so my dog Stella comes into the office sometimes. She has many fans among the department.

People love her. She’s a really good dog, she’s a very good girl, and that’s part of it, right, when you have a dog to go home to, then you want to be able to spend time with the dog. You don’t want to worry about work.

Women@SCS: Right, definitely. I’m glad you have that separation.

Professor Rivers: I think it’s a really important thing. I know it’s not something I did when I was an undergrad, but it’s something I adopted when I was a PhD student. I was trying to separate out work space from home space, and it’s very good for stress levels because it means you can build yourself a space where you don’t have to think about work, and don’t have to be stressed by it. You can think of other things instead. I think it’s really important for mental wellbeing to have that separation.

Women@SCS: Definitely. Do you feel like being in the PhD program, or certain aspects of it helped?

Professor Rivers: I do think it helped somewhat partially because of expectations people have. I think one of the difficulties of being an undergrad and having that separation is that a lot of the time people expect to do work at night. You’re sort of on a different schedule where you work at night and sleep in the morning, whereas with the PhD student, you’re generally older, so people are working during the day and go home at night. So it’s easier to make the distinguishing factor there: if I go home, I don’t do work. I would almost say if I were to try to do something like this as an undergrad, separate out the morning as your home time to say I don’t think about work in the morning. I’m just going to read a book, or cook, and not worry about it until I have to go to class, because you’re so much more likely to have to do meetings or activities at night. People don’t always schedule things in the morning.

Women@SCS: Right. People schedule group studies at 10:00, 11:00 pm.

Professor Rivers: But again, depending on each person, it’s what works best for you on an individual schedule. But honestly being able to at least say when I’m in this space I’m not going to think about work, can be super helpful.

I didn’t do that as an undergrad. I definitely did too much work as an undergrad, but as long as you’re not doing it forever.

Women@SCS: Right, that’s true. Are there any particular courses that are super memorable to you when you were an undergrad?

Professor Rivers: They probably all changed from what they are now, but compiler design was one I liked a lot. I didn’t like all of it but when we got to the part at the end where we were doing optimizations, I really enjoyed that and it actually ended up feeding into how my thesis research worked into my PhD. Yeah, so I really enjoyed that specifically. 15251 was one of my most favorite courses. It was really a fun field of problem solving.

Those are probably two of my most favorite courses. I’m forgetting other ones I really enjoyed. Probably those are the ones most memorable for me. Then probably the intro to CS course, but I took that a long time ago. That was a huge part of my undergrad experience.

Women@SCS: Beyond even taking the course originally.

Professor Rivers: Mm-hmm.

Women@SCS: So you TA’d as a grad student. Did you also TA as an undergrad?

Professor Rivers: Oh yeah. I took the intro to CS course in my first semester, then in my sophomore year all the way through my senior year, I TA’d for that intro to CS course. It was actually a really interesting process because when I started out as a TA, the course was organized in a way where there were a bunch of small sections that were each taught by a different instructor, and they all did their own thing. So you would have one section teaching Alice, and one section teaching Processing, and one section teaching Java, and they would all do their own thing, but then over time they started combining these sections together to make larger ones, and by the time I was a senior they combined all the sections together so it was one giant intro to programming course that everyone was taking together. 110 and 112 melded into one thing. So it was fun to see the logistical difference in that. When I started out there were four other TAs working with me, but by the end of it I was managing 30, 40 TAs, and that was a cool evolution to see how the management side of teaching worked in terms of changing how things were organized over that course of time.

Women@SCS: Lots of logistics.

Professor Rivers: That’s a part of how I realized I wanted to do this as a job because I still really enjoyed it.

Women@SCS: That’s true. That is good. It’s definitely easier to do all that management type stuff with something you’re personally invested in. Were any other difficult parts of you being a woman in computer science?

Professor Rivers: The one class was a bit difficult when I was the only girl in the room. I think the hard part was that for a large part of my undergrad I identified mainly as a math major or a math and CS major, because my home department was in math. So while I was taking a bunch of CS courses, I wasn’t really part of the CS cohort in the school. And it was also probably math, there weren’t many other women in that department, but that was a whole separate problem. But what was interesting then was that in the PhD program it wasn’t a problem at all, because in HCI it’s nearly 50/50, because you have a lot of people coming from design and psychology backgrounds, then integrating computer science into it, which makes it much more gender-balanced.

So I feel like I forgot some of the difficulties there might have been in undergrad based on that. But again, I’ve also been lucky. I haven’t had people who are biased against me, or having tried to pull me back due to societal expectations or anything. So I’ve been lucky in that I’ve been able to envision a path for myself as a computer scientist, as a mathematician, and people have supported me in that. So I recognize it’s not always been that way for everyone. I want to provide that support to everyone though.

Women@SCS: That’s awesome. Switching gears a little bit, could you expand more on your PhD research project on data-driven tutoring with hint generation?

Professor Rivers: It was first building it, actually getting the tool to work, and then afterwards running studies to see how it affected learning. That was the whole thesis, and it was interesting with the studies because whenever you’re doing any kind of research study, you don’t know how it’s going to turn out, so that means you don’t know if you’re going to find a result or just find that it doesn’t have any impact, which is very stressful when you’re a PhD student and you feel like everything is riding on this one study.

I was measuring how long it took people to submit the problems and how much they learned from it based on their solving these different practice problems, and what was interesting was that we didn’t find a measurable difference in the amount that they learned, but we did find a measurable difference in the time they spent where people who had the hints given to them automatically spent less time and learned the same amount. So that can still be a good finding if you find that people learn the same amount in less time. That’s still really useful. So it’s interesting spending a lot of time doing that kind of study, analyzing the data, trying to see if your tool does anything at all. It’s definitely a stressful research process.

Women@SCS: That’s definitely a core difference between being a pure software engineer versus being a researcher.

Professor Rivers: Absolutely.

Women@SCS: Did you at any point see yourself in industry?

Professor Rivers: I thought about it. I thought about going into academia as more of a research track, then thought about going into industry. I actually tried industry once. I did a software engineering internship at Google just to see what it was like, and it was cool, it was a great company to work at, but I missed teaching at the end of it, and I figured if I missed teaching after three months, then it was something I should be doing more as my main job, so I decided to go back to it after that.

Women@SCS: That’s so cool. It’s great to see you were able to U-turn pretty easily.

Professor Rivers: And it was a thing where I was definitely exploring different options during the PhD, but I was coming back to doing teaching track, as the main thing I was interested in doing, the main course I was interested in taking, and that worked out. But there were other PhD students who went into industry, other PhD students who went into research as a tenure-track professor. So there’s a lot of different tracks you can take with that background. I knew plenty of people who went straight from undergrad to industry as well. It’s certainly a popular track. Everyone can find out what they’re most interested in.

Women@SCS: That’s reassuring to hear.

Professor Rivers: One thing that helps is you don’t have to decide immediately. It’s quite possible to go into industry first, and do that for a few years, and if you’re not enjoying it, you can decide to go to grad school after. That’s something several people in the PhD program had done. They worked a few years in industry and then came to do their PhD afterwards. So you don’t necessarily have to decide right out of undergrad. It’s a hard decision.

Women@SCS: I see. I think a main concern for many facing that decision is from a financial standpoint - it’s much tougher being a grad student versus being in industry.

Professor Rivers: No, that is absolutely true. I will say where you’re doing a PhD makes a big difference. It’s actually quite good to do a PhD at CMU because you’re in Pittsburgh where the cost of living is very low. So a 30k stipend as a PhD student actually goes quite a long way in Pittsburgh, right? You’re not going to be living any kind of fancy life, but you can pay the rent, and make payments on student loans, and eat, and get by. I have heard it’s much harder if you’re in somewhere like Boston or San Francisco where cost of living is much higher just because it doesn’t stretch as far. But it is a huge difference going from a grad student stipend to being on an actual working person stipend. That is a big difference, and that is another good reason to consider doing industry first and then going back to grad school later on is you can build up savings with an industry stipend, and then have that to fall back on during grad school.

Women@SCS: That’s great advice, thank you. I think that’s it from me. Any other general advice, last thoughts, anything you want to add on?

Professor Rivers: I would say figure out what it is you’re excited about doing, that you enjoy doing, and make that your path in life for what you want to do. It doesn’t necessarily have to be what people expect of you. I know when I was in grad school there was a certain expectation from people who were primary mentors who were mainly tenure track professors that you will also go on and be a tenure track professor, and I talked to several people when I was waffling with the ideas: do I want to do industry, research track, teaching track? I talked to many people who were like, why would you do teaching track? You can be tenure track like we are and focus on research more. I do not at all regret going the path I went down - the teaching route, because it is what I enjoy doing, and as long as I can make a career out of that, there’s no reason not to. So I would say figure out what it is that you want to do that you can make a career out of and go with that, even if it doesn’t fit the usual mold.

Women@SCS: That’s awesome. Were there any other outside pressures that made you feel like you should go some other way, parents, friends?

Professor Rivers: I would say not as much. Friends were definitely open to other ideas as well and thinking of different ideas. I would say in my cohort, actually relatively few of my cohort ended up going into research positions, like in a tenure track way. A lot of them went into industry, or consulting, or different types of things. But I guess two of the people in our cohort out of eight ended up going the traditional tenure track, professor positions. So it is definitely a thing where I think it’s easy for somebody who has a job to say my job is awesome. I enjoy it a lot. You should do my job as well. So it’s important to separate out somebody else’s lived experience from your lived experience and what you enjoy doing. I would happily encouraged many people to consider becoming a teaching track professor, but that’s because it’s worked out for me. That doesn’t mean it works out for every single other person.

Women@SCS: That’s true.

Professor Rivers: It’s definitely good advice to talk to a lot of different people, see what matches up with your own expectations for what you want to do.

Women@SCS: Cool. Thanks so much for doing this with me!

Professor Rivers: No problem. I enjoyed it.