Testing and Flipping

This was our last week before Spring break, which means two things: First, there will be no posting next week (although it might be the perfect time to slip in a non-teaching related entry) and second, this was the week of the midterm.

As I watched my students taking the midterm I was quite literally awestruck. Before me sat a room full of incredibly talented students, and I get the privilege of being part of their educational experience. The only other time I've been struck with a similar feeling was when I used to teach our (then) 300-person introductory computer science course (it's now about 750 students and growing strong thanks to the amazingly talented David Malan).

But this was different. These students have been working incredibly hard for the past three weeks getting their operating systems to fork and exec processes. A quick look at the data suggests that a typical student spent almost sixty hours working on the assignment over those three weeks. Yup, that's twenty hours a week for one of four courses. Ouch. I guess that explains the claims of students spending 60 hours weeks on this course -- historically there were always students who didn't do anything the first couple of weeks and then tried to do it all the last week. Things seemed a bit better this year with many people making steady progress over the three weeks.

Anyway, back to the exam. Having loaned my laptop to a student who had somehow missed the discussion about taking the test online, I had nothing to do other than watch them. There they were, all intently focused on the exam, typing, thinking, shuffling through slides. It was an open notes, open book, open course materials (but not open Internet) exam. I appreciated that they asked if they could read the course Q&A site during the exam -- I did draw the line there, even though they all certainly seemed to understand that posting questions was not going to be on the agenda. Towards the beginning of the test, there was much shuffling through materials, but as the test progressed, there was less. My theory has always been that I can make tests open book, etc., so long as I write questions that require synthesizing information so that the materials aren't actually that useful. There were a couple of questions where a glance at a set of slides would prove useful, but in most cases, there simply weren't places they could look for an easy answer, unless they already had a pretty good idea about the question.

I was quite curious to see the results. It's not possible to run a real apples to apples comparison beacuse a) this class is twice as big as it was two years ago, b) it's a different exam, and c) it's a different way of teaching. So, any results are open to interpretation. The results are surprising -- the distribution is almost identical to that two years ago. However, there were no truly bad grades -- last time, 2 of 23 students had grades on the midterm that were cause for alarm; this year, there were no grades that worried me (it appears that grades that worry me are those more than 2 standard deviations below the mean).

I'm pretty sure I cannot conclude anything from this. So, we'll have to wait until the assignments are graded. In this case, the assignment is pretty much identical (I saw pretty much because we asked different code reading questions, but ideally I can dig up the grade breakdown and remove those). Time will tell.

Next: Inadvertent Flip (April 1, 2013)

Half Flip

It's the end of week six -- almost half way through the semester and this week was a perfect way to celebrate that -- half flipped, half not flipped. On Tuesday I gave a fairly conventional lecture. My gut instinct is that the class is a tad more interactive than past years, but I can't attribute too much to that since it's also larger and we're in a different space, and both of those things might contribute.

Thursday was our annual midterm review. In the past I've simply gone over an exam in class or broken the class into fairly large groups and had them make up questions, present the questions, and then answer them. This time I let them submit questions as web work before class, promising that I would, in fact, use some good ones on the exam (having now written the exam, I have selected a few -- I did have a moral dilemma -- one student suggested a nice question, but answered it incorrectly -- I really didn't want to use a student's question and have that student get it wrong ...).

In class, I let them work at tables (groups of 3-5) on sets of problems and then we came together to talk about them as a class. Since many of my exam questions have multiple answers, it provided a good opportunity for both table-wide and class-wide discussion. I guess we'll see how it all works out next week.

I have noticed one other difference between this year and previous years. My staff seems to be thinking a lot more about teaching -- what we teach, how we teach it, etc. It's hard to attribute that to any one thing, because there are so many variables, but I do think that there is something about the setup that gives us more opportunity to think critically about what we're doing. Also, the fact that we have some flexibiity in how we present material means that we can be facile at adapting to the students, what's working and what's not. This week's discussion at staff meeting revolved around design -- how do we teach it? Why don't we teach more of it? What can we do now to do a better job? I have such respect for and appreciation of my teaching fellows who engage wholeheartedly in such discussions -- they are close enough to having learned the material that they remember what is hard and they are enough past it to know what skills matter. We're still in the midst of this discussion -- I'll let you know how it works out!

Next: Flipping and Testing (March 18, 2013)

Unflipped!

We've entered that time of the semester when the students are busily creating user level processes in their very own operating systems. It is both exhilarating and exhausting. Due to that exhausting part, this is the part of the course where I promised not to make them do pre-class work -- there would be no videos to view and no web work (other than quick surveys about how much time they are spending). That means that, for the most part, I have had to revert to a more traditional class structure -- I talk and call on people and we try to collectively learn stuff. It feels dull; it feels unsatisfying; it feels as though students are not engaged. I am looking forward to polling them about this class structure so we can decide how to move forward during the next several assignments.

Since we're not flipped, I have little novel pedagogy to talk about nor do I feel the exhilaration I've felt throughout most of the semester (and any of you who have run into me and dared ask about the course know that I can now babble incessantly about how fantastic flipping is). So, today's entry is about two things: how it feels to be lecturing again and thoughts on what I would do if I had 250 students instead of 50.

The Traditional Lecture

It sure does take less time to prepare for class! Once I finish class notes, I have finished class notes. I don't have to think up pre-class activities; I don't have to think up in-class activities; I do spend a bit more time working through the notes, deciding when I can engage the class and which activities can be converted into class exercises.

As has historically been the case, I find myself interacting with about 20% of the class, instead of all of them. This is rather disappointing. I coax, I tease, I cajole, but some people just don't want to speak up in class. I do find it effective to ask for a raise of hands to vote on whether answers are correct or not and I seem to get better participation than I have in the past, and my sense is that even when I'm trying to extract thoughtful answers from people, I have a slighly larger fraction of the class engaged, but it's nowhere near the 100% that are engaged when the students are completing in-class work.

I also have found it effective to try to convert some of the open-ended questions I typically ask into exercises that the students can complete in groups. Even if it only takes a minute or two, it means that more people are engaged. It is this last tidbit that got me thinking about our next topic ...

Flipping 250 Students

A few of my colleagues have indicated that they have been thinking of how to flip significantly larger courses and it poses an interesting question. What would I do if I were teaching CS51, which is a heavy-duty programming class and has roughly 250 students?

Classroom space is the first obstacle. I love teaching in our new state-of-the-art classroom that has reconfigurable, mobile tables, traveling whiteboards, screens on both ends of the classroom, etc. However, it holds only 54 students, and even if you could build a bigger one, I think it would rapidly become sufficiently cavernous that you'd lose any semblance of a class. So, I would constrain my thinking to what would work for a more traditional lecture hall.

Since the students won't be able to cluster around tables, I'd probably try to use some kind of online shared whiteboard-like tool. That way students could bring laptops and work together in groups of 2-4. I would also want some way to do real-time data collection. I don't think you need any special clickers -- I would see if Google Forms work well enough in real time that people could submit answers and I could see the overall statistical picture of how students are answering.

The physical layout is going to make it difficult for the teaching staff to circulate -- you can get to the edges of rows, but getting into the middle would be tricky. I'm not entire sure how to deal with that -- I suppose the staff could join the online discussion, but it's not the same as interacting with the students directly. This would seem to be one of the biggest obstacles -- I just can't see the teaching staff climbing over students to get into the rows; having the students get out of the rows to talk to the staff doesn't sound better. I'm also not completely sure how the noise level is going to play out in a room with that many students. You really want them to be able to talk to one another, but a room of 50 sounds pretty chatty; I'm not sure how loud a room of 250 will sound.

So, those are my thoughts, but I think that the best thing would be to run a experiment. Perhaps if one of my colleagues invited me to guest lecture in his class, we could prep a single class and give it a try!

Next: Half Flip (March 9, 2013)