Advising Junior Colleagues

About two and a half years ago I wrote a blog post about advising grad students. At long last, here is Part II in my (very slow) advising series. It addresses the question, "What do you tell your Junior Colleagues?"

Let me begin with a disclaimer: I can't guarantee anyone success. When I talk to my junior colleagues, whether at Harvard or elsewhere, I tell them things I really believe, but (unfortunately) I don't make the rules, I don't make decisions, and what I believe may or may not be right for any particular person or institution. So, this is my opinion on what I think one should tell Junior colleagues. But hey, this is my blog, and everything in it is just my opinion.

Let me just start out by putting it bluntly: being a professor is a bit of a crazy profession. It's one that has worked for me and offers many features (perhaps the topic of a later blog), but it's not the kind of job any sane person would make up. You get into this job because you're used to being really good at everything you do. Unfortunately, there are way too many things for a professor to do that you can continue to do all of them at the same quality level that makes you comfortable This is perhaps the greatest challenge that we have to face -- be willing to do things at less than peak performance. I repeat, "You must be willing to do some things at less than your peak performance." This seems totally counterintuitive in a highly competitive job, where you feel that everything you do is highly scrutinized, but if you try to excel in everything simultaneously, you will drive yourself totally crazy.
So, what is an over-achiever to do?

Pick and choose carefully. The key here is that you can't do everything well all the time. Instead, you pick what you are going to do well at any point in time. As an academic, I like to think in terms of semesters. At the beginning of each semester, you have to decide which are the things for which you need to produce A work and for which can you slide through with B work. Each semester, the lists change slightly so that at some point, every functional area gets some A work and some B work and every semester gets some A work and some B work.

Let's make this more concrete.

Let's say that you've just started your first faculty position and you want to make a good impression -- how do you get started? My advice typically goes something like this. No one expects you to crank out much research your first year. If you have some straggling papers from your dissertation or post-doc, that's fantastic -- get those out. But, your real focus in year one is to get your teaching under control and get your research agenda planned. Note that planning a research agenda is different from doing research, but we'll get to that in a minute.

If you are in a kind department, then you can assume that you'll be teaching the same courses for several years. That means that if you invest time this year in developing your courses the way you want to teach them (i.e., you make teaching an "A" priority), then you can leverage that investment over the next several years (expending "B" effort, while ideally obtaining "A" results). So, while many will tell you that any time you invest in teaching is wasted, I disagree. Good teachers get good students. And good students will help you do your research. That said, invest time wisely. Time spent automating grading is time well spent. Time spent developing four thousand step animations is not. (And trust me, it is very easy to fall into this trap. I've done it.) I could go on for quite some time about what is and is not a good use of your time with respect to teaching, but that's another topic. For some detail, I recommend my colleague Harry Lewis's article on the topic or my more recent experience elsewhere in this blog.

And then remember that what works for me may not work for you. The key ideas are:

1. Figure out what works for you.
2. Invest time in things that are reusable.
3. Remember, the quality of your teaching is best measured by how much the students learn, not how entertained they are or how much they think you or your slides rock.
4. True learning takes place when students do things and get feedback -- anything you can do to improve the quality of feedback students get on work increases the chances that students are actually learning something.

Sadly, most students never look at anything other than the final grade on massive assignments. I spend hours grading final papers and both writing and typing up detailed comments. Few students actually read these comments, so perhaps it's not a good use of my time. However, I still do it, hoping that perhaps one of them will actually learn something.

OK, so picking up the main thread here -- invest time in your courses in year one so that in the next two to three years, you can teach the same courses with minimal effort. You hire good teaching assistants, you pull out your notes (and perhaps assignments) from the previous year and you spend almost no time preparing. Sure, in those future years, you're giving a B effort in teaching, but because of the prep you did in year one, you ought to be able to invest B effort and get A- or B+ results.

The second part of year one is planning your research program. That probably means figuring out what you want to do and then writing the proposals to get funding for those projects. This is the time to renew acquaintances with all your friends who are now working in places that fund academic research. See if you can get some supporters lined up for corporate grants -- these are small grants, but require significantly less effort than your classic NSF grant. You'll want to submit one or two of those as well (and I mean one or two, not six or eight), but your main goal is to have a plan for the research you want to do and convince people to give you money to do it. If you have your ideas well in place by the end of the first semester, that puts you in good shape to identify the grad students you want to admit.

Getting a handle on what projects/research you want to undertake also sets you up in semester two and over the summer to write high quality grant proposals. This is the time to squelch your inner procrastinator. Get your proposals written early enough to get feedback. Ask your senior colleagues for critical review -- if you are in a good department, they want you to succeed and will help you submit the best proposal you can. It's also worth teaming up with another colleague or two for some bigger proposals if (and this is a big if) the work necessary on the larger project is compatible with that research agenda you spent so much time crafting. Don't lose sight of that.

So, if all goes well, by the end of year one, you will have lots of reusable materials for your teaching, a research agenda on which you can begin to execute, a couple of proposals in the pipeline, and some incoming grad students whose interests are well-aligned with your research agenda. Begin year two.

Year two is all about getting your research firing on all cylinders. Over the last several years I have become a strong believer in knowing what paper you are writing when you begin a project. If it's real research, then you had better not know the results, but you had better have some hypotheses. Write those down. Really. Honest, write the opening paragraph that motivates what you are doing and why. If you can't write it now, why are you doing the research you're doing? Sure, you don't know how it will turn out, but if you don't have an hypothesis, you don't really know why you're doing what you're doing.

Concurrently with getting that research started, you're also starting to be a graduate advisor. That is a whole other topic, but in the context of this discussion, the message is you have to realize that this is another one of those activities that you need to learn to do. And every student needs a different advisor -- you might be able to play that role for multiple students, but if you walk in believing that you can advise every student identically, you will not be a successful advisor. You need to have some basic advising methodology and a philosophy, but you need to get to know your students, understand their strengths and weaknesses and figure out how to most effectively advise each student. This will take time.

After year two, you should be a on a role of focusing mostly on your research and students, and then periodically having to step back and brush up your teaching (i.e., make it an A task for a semester or two), whether to update a course, update your teaching methodology, teach a new one, develop a new course, translate your course to the web, etc. However, if you use that time effectively, then you can still make research an A priority for many of your semesters. Do not confuse grant writing with research -- grant writing is a necessity (at least in my field). If you are successful, you can secure several year's funding with a semester or two of concentrated effort and then forget about funding for awhile. However, you have to keep track of where you are in the funding cycle and know when it's time to elevate grant writing to an A activity, so you don't suffer from a gap in funding. Personally, this is a huge challenge for me as I'd much rather do research than write grant proposals (I'm kind of guessing I'm not alone here). As a result, I can end up leaving grant writing a low priority for too long -- don't let this happen to you.

Now, some of you are getting ready to scream at me and say, "What about all that other stuff? What about program committees and NSF panels and guest lectures and attending conferences and departmental committees, and undergrad advising and all those other things that our institutions expect us to do????" If your colleagues aren't already ready to fire me for saying that you can sometimes relegate teaching to a B activity, I'm sure some will cringe at the next section.

High order bit: you do not have to do as much of this stuff as you think you do.

There, I said it. Seriously, the closest I've ever come to "yelling" at my Junior colleagues (you know who you are) is when I see they have signed up to do six program committees in one year. Yes, program committees are useful. Yes, they keep you up to date on what people are doing. Yes, they get you known. But six? No! It's not a good use of time.

My advice is no more than two program committees each year. One of those can be a real conference and one can be a workshop. Yup, no more than two. And you don't have to do two every year, but repeat after me, "No more than two." There, wasn't that easy?

Now, how do you pick the two? You want to work your way up to getting on the program committee of the big conference in your area. It needn't happen in years one or two, but by year N-1 (where N is the year you are up for tenure), you want to serve. In the meantime, build up your PC credentials by serving on the slightly less prestigious venues. Pick workshops that are very closely aligned with your current research agenda. It's fine to approach current chairs and let them know that you'd like to serve on a future committee -- they will pass it along to the next chairs (of course, if you already know who the next chairs are and they've not yet put together a committee, you can approach them). You can also ask your senior colleagues/mentors to help you get on the big program committees. However, your best approach is to accept a limited number of invitations and do an outstanding job. Get your reviews in on time, engage in thoughtful discussions, be positive, etc. Trust me, you'll have more invitations than you'll know what to do with. But now you know what to do, say, "I'm really sorry, but I'm completely booked up this year on program committees -- please let next year's chairs know that I'd be interested though."

Note: I do not always heed my own advice, and I almost always regret it.

OK, that's program committees, what about NSF panels? I find that serving on those is extremely helpful right before I am about to submit a proposal, because they remind me how panels work and what's important in a proposal. So, you should do a few of these, but not a lot -- perhaps one every other year? That feels about right to me. And yes, you'll get asked to do more. Fortunately, those invitations are always accompanied by specific dates, so you can use your, "Oh, I'm so sorry, but that day/week/month is simply not possible for me."

How about committees in your own department? This is a bit tricky. If you are in a nice, nurturing department, they won't ask you to do too much, because they don't want to distract you. However, there are some committees that are worth the time -- in my opinion graduate admissions and hiring. The former because it gives you firsthand knowledge of the pool and puts you in good recruiting territory. The latter, because you know who's coming out in the few years after you, who you might want for colleagues, and at whom you should be looking. However, moderation is your friend -- don't do both in one year -- each of these is very labor intensive (admissions a huge burst at one time in the year; search a more steady time sink throughout the year). Just like with your PC invitations, you can be polite and constructive in turning down committee invitations, "Dear Chair You-Who-Controls-My-Destiny: While I would be very excited to serve on the suck-all-my-time-out-of-day committee, I'm afraid that I cannot do it this semester/year. I've already scheduled all my committee time on other activities. I would however be happy to serve next year (and will reserve time now if that's feasible)." What's a chair to do? They can't really dispute what your schedule looks like and you have, in fact, signed up to do it next year. Of course, if it's an activity that you feel you never want to do unless someone is holding a close relative hostage, you'll need to tweak that letter slightly and leave off the part about wanting to do it in the future.

Finally, there are those friendly invitations to "Come give a talk." Giving talks is a good thing; it can be fun; it gets you visibility and gives you a chance to find out what's happening in a department. But, it also takes time and means "yet another trip." So, be strategic. There are times during your pre-tenure years that giving talks is more important: the semester before your department is going to write to every researcher on the planet asking their opinion of you. Yup, the year before tenure and, if you have a formal mid-tenure-track review, the year before that. Even then, you do not need to give 4000 talks. Be strategic. You can probably guess at least half of the people to whom your department is likely to write. Pick the very top departments from that list. Rather than visiting every institution on your list, leverage the most prestigious departments in your field. Be visible -- maybe, just maybe, if you're up for tenure, you give the talk rather than having your student give it. Even if you don't do that, use your conference time wisely: talk to colleagues, let them know what you're up to. Invite them out to lunch/dinner with your students. Finally, rather than traveling to give a talk, you can ask your department to invite them as colloquia speakers. It's funny how many invitations I get to come give talks that are followed up a year later by letter requests. You think it's purely coincidence? I don't.

The other way to make sure people know what you're up to is to convene a mini-workshop on YOUR RESEARCH. You have to get this right, but if you have the funding and can really take advantage of the event and you can persuade people it's worth their time, there is nothing better than inviting 15-20 bigwigs in your field to spend a couple of days at your institution engaged in thoughtful, interesting conversation. If they also happen to come away with the big picture of your research agenda, won't you feel like the star? I believe that it is becoming more common for departments to allocate at least some funds to help you do something like this. It can save you a lot of travel.

Above all, learn to say, "No." You can be polite, but you must be firm. After one particularly brutal year, I adopted the mantra, "It's all my fault." The message to myself was supposed to be that I control my work life and if I'm too busy, it's because I've said yes to too many things. Surprisingly, rather than being a defeatist attitude, it's actually quite empowering. If it's all my fault, then I have the power to change it.

One of the trickiest things for me is saying no to students. I truly love spending time with students -- after all, if I didn't like students, why would I choose to be at a University? When they invite me to a faculty dinner, I'd love to go, but guess what. Dinner time is family time in my life. I already miss way too many dinners when I travel and I frequently have some child-transportation responsibilities in the late afternoon or early evening, so it's quite difficult to really make faculty dinners go. So, what do I do when students ask? Typically I explain that evenings are rough for me, but suggest that we do lunch -- either during the semester, or more frequently during reading period when both our schedules are a bit more flexible. This past year, I had evening duty in Cambridge with one of my kids and that let me attend a faculty dinner or two or use those evenings to take students out. Yes, it cost more than a faculty dinner, but the students are more interested in just hanging out with you and talking than with trying the local 5-star restaurant (should you have one). The hours I've spent over lunch, dinner or coffee with students are among the most fun of the semester -- it doesn't matter if it's in the formal setting of a faculty dinner. In fact, think of the faculty dinners as mechanisms for you to figure out which students might enjoy just chatting and then do that!

OK, those are the things I like to tell my colleagues. Some have found these words useful; I'm sure others will find that they don't work. In any case, you get to make decisions about how you spend your time. If you make those decisions wisely, so that they simultaneously help you be productive and keep you sane, you will either end up tenured or you'll end up not-tenured, but reasonably content with your life, instead of bitter and resentful.

A Note on Powerpoint and Teaching

While in the midst of finishing up an upcoming blog post on mentoring Junior Faculty, I wrote up the following. It seemed like a topic that is more generally interesting and that such detail doesn't belong in that other entry, so here it is all by its lonesome.

I'd like to talk about powerpoint (or any other presentation software), animation and teaching. For years, I avoided teaching with powerpoint, because I like to scribble on slides -- my slides are typically the outline of what I want to talk about and are mostly reminders for *me* so that I remember what I want to tell the students. However, there is information on them that you don't want students to have to frantically scribble down, so you think, "Hey, I can give them copies of the slides, and then they can listen to me instead of scribbling frantically." This is the beginning of the slippery slope. Once you hand out notes, the students believe that they are proxies for lecture, and if they are not, they will complain bitterly. Even though I very clearly explain that the notes are for me and are provided as a service and are not a replacement for lecture, I've been totally reamed for "having four blank bullets on a slide and only filling in three of them during the discussion in class." I kid you not.

Anyway, I finally transitioned to powerpoint when I got a tablet, because then I could have pretty slides and scribble on them. The first year I did this for my OS course, I spent way way way too many hours building meticulous animations of a context switch. You know ... I am 100% convinced that those animations wasted a ton of my time and benefited absolutely no student and probably harmed some. All the literature confirms that students do not learn by passively watching or listening to you -- passively watching you step through the animation, even if you explain it ever-so-eloquently, is absolutely not the same as forcing them to figure out what the steps are. Yup -- it will take a lot longer if you make them figure things out, but you know what -- they'll actually understand it better. (You may argue that with the picture perfect animation they can step through them later and without you there actually understand what's happening. While that may be true, one in twenty students will do it, while pretty much everyone sitting in a classroom has to pay some amount of attention if you're constantly making them explain what is going on.)

I do devote time to trying to make my slides and explanation as clear as possible when I am recording material for students to screen in advance, but I am working hard to avoid spending significant fractions of my waking moments building fancy animations. In my opinion, animation bad; making student think and discuss what things must happen, how they happen, why they happen, etc. is significantly better.

Chasing Mia

This posting was written several weeks ago, but I was in the midst of my flipping sequence and didn't want to break that up, so I delayed posting this one.

As an admitted groupie of the US Women's National soccer team and a Meadowbrook parent, two unrelated postings make me want to write.

As this year's soccer season began for the women's national team, I struggled with the emotional conflict I felt as I prepared to watch Abby Wamback overtake Mia Hamm's goal-scoring record. (I predicted last year it would happen during the Algarve cup and I was wrong, but it will certainly happen this year.) Then, the head of our Middle School wrote a nice piece about why Meadowbrook gives out Spirit Cups for their sports teams.

I have always watched in stunned disbelief as the media and most of our society turn atheletes into role models. Just because someone is good at riding a bicycle (quickly) or throwing a ball into a hoop, why do we immediately assume that they are people our kids should emulate? At the same time, year after year, I look at the women who play for our US team, and I think, "Now there are role models!" These are nearly uniformly amazing women who have dedicated themselves to an underappreciated and underpaying sport. They work hard on the field and off the field, reaching out to fans, working with young people, and dedicating themselves to important causes, without a lot of fanfare, without the trappings of superstardom, and without the negative actions that draw negative publicity. I can hear some of you, "Oh, they are just women soccer players." No, they are world-class professional atheletes. They are among the absolute best in the world at what they do. And they do it in a sport that is important to more of the world than any of our conventional American pasttimes.

At the risk of missing someone deserving, let me run through the list of past and present US Women's National Team players who are extraordinary individuals as well as extraordinary atheletes.

• Joy Fawcett, thank you for embracing your professional atheleticism and motherhood simultaneously, for showing us that the two are not mutually exclusive, that you can bring your kids to work and still perform at the highest levels.
• Michelle Akers, thank you for showing us what it means to give your all -- through illness, injury, and perhaps even beyond the point of what was good for you personally, you gave every last bit of yourself on the field.
• Heather O'Reilly, thank you for showing us what a work ethic is. In every minute of every game that you play, you are 100% engaged on the field, chasing down every last ball or player.
• Amy Rodriguez, thank you for being approachable, for understanding that just talking with and teasing the young people who admire you makes them feel special.
• Carli Lloyd, thank you for helping me never question whether I'd play soccer again after breaking my ankle. As I rehabbed, I watched you rehab. As you returned to the national team, I knew I too would return to my (not quite so competitive) team.
• Shannon Boxx, thank you for demonstrating the importance of having a professional league in this country.
• Cat Whitehill, thank you for seeing into the heart and soul of a recreational soccer team and treating them with the same respect you gave your national teammates.
• Brianna Scurry, thank you for always demanding the best, for handling difficult situations with grace, and of course, for stopping PKs!
• Tiffany Milbrett, thank you for making me proud to wear #16.
• Kristine Lilly, thank you for many things -- for redefining what age means in soccer, for demonstrating what commitment and passion really are, and for showing the world that it's not only OK, but good for women to wield power.

and now, back to where we started:

• Thank you Abby -- for everything -- for the role that's been thrust upon you as the bridge between the past and the future. More than anyone else, you have been the transition between "the 91ers" and the future of US Women's soccer. You have provided leadership through both good and challenging times; you graciously accepted the mentoring of your predecessors and pass that on to your successors. And, thank you for putting Rochester on the map!
• Last, but never least, thank you Mia -- for becoming the face of women's soccer, although it seemed you would have always preferred not to be in the limelight. Women's soccer needed a superstar, a spokeswoman, an idol. You showed us that teamwork was as important as personal performance by wracking up almost as many assists as you did goals (144 to 158). Regardless of whose name appears in what categories in the record books, you will always be Mia -- the first face of women's soccer in this country.

Flipping Over

We are done. The course is over, except for the data analysis, which I expect to entertain me over the summer. This last entry will be a collection of random topics - things that happened over the course of the final exam and grading or observations I've made.

End of year party

Long ago, in 1993, we started the tradition of having an ice cream bash after the take home final. I don't believe we've maintained the tradition, but I decided to resurrect it. In addition, we continued the tradition of class T-shirts. The winning design was:

In addition, since I had artwork for each group, I made that available to students to construct their own team patch, which they could then iron onto their shirts. Groups who didn't have the time to do anything fancy (21 of 22 groups), got the plain hand-drawn black and white animal.

So, on Wednesday afternoon after spending a lot of time on the final, we all gathered in a much too small room to eat Toscanini's ice cream, collect T-shirts, and iron on patches. It was a total blast. Pretty much everyone ironed their patches. The one group who augmented their animal was the marmots, who added color and appropriate attire:

We had a few ironing disasters which prompted a return to QRSTs for a few additional shirts.

Had I been thinking, I would have tried to get a group shot, but I didn't. Next time! One of my students, did take this small group picture:

Students seemed relatively happy and relaxed, although a few still had multiple final projects to complete. Much ice cream was consumed. The pear chardonnay sorbet was not consumed, but I have to say, it was outstanding (it came home and disappeared stunningly quickly). It was a good end to the course!

The final

For the curious, here is the final exam. I thought it was pretty tough; it was certainly time consuming to grade (it took me the better part of a week to grade 45 exams -- I'm guessing over an hour per exam). The students did exceptionally well -- a typical Margo-exam ends up with a median around 67%; this one had a median and mean over 80%!

The first problem was a design exercise and those are notoriously difficult to grade; after two days of grading I wondered why I continued to give these kinds of questions. Then a colleague, Kryzstof Gajos, explained how we graded his final exam, and I am excited to contemplate how I can apply this technique. I've invited him to write a guest entry on the topic, but in the meantime, here is my understanding.

The course he was teaching is called CS 179: Design of Usable Interactive Systems. It is very much a course all about design. So for the final exam, he split the period into two parts. During the first part, students took the (written) exam using a "uniquely" colored pen (provided by Professor Gajos). Then, he collected the pens and the class collectively graded (their own) exams. Thus, in real time, the class discussed the problems, the supposedly right answers, alternative answers, etc. When there was ambiguity, they discussed it. Afterwards, he collected the exams, had the TFs check that the self-grading was accurate and he was done.

This was pure brilliance! I'm not saying that because it reduces grading time (which it does), but because it turns a purely summative evaluation process and makes it partially formative. I try to do this when I grade exams -- I essentially engage in a typewritten discussion about their answers (which is one reason exams take so long to grade). I always have second thoughts about this, because I know that students merely search for the final number and ignore everything I write. Or so I thought (more on this later). The idea of discussing the exam right after they've finished it and being able to address confusion immediately is overwhelmingly appealing to me. It's not entirely clear how to apply this to my exam -- I like pushing students to think about complex issues for which mulling the ideas and questions over is useful (this is why I like the takehome exam format). I don't know how I'd fit both the testing and the grading into a standard 3-hour test slot. If I stick with the 24-hour format, then I'd have to make the exam be, say, a 20 hour exam with a scheduled 2-hour class which sounds like a logistic nightmare. I'm not sure what I'm going to do here, but I see further experimentation in my future.

In addition to a design question, I asked people to read a research paper ( the Barrelfish SOSP paper), and then I asked them questions on it. I'm really looking forward to having some of these students in my graduate OS class, because the high bar they set for a system is quite interesting. Many were accepting of a prototype, but a surprising number took a more business perspective, while perhaps not appropriate for a research paper, indicated to me a much better sense of how the world works than many tech-savvy people are. I found it fascinating.

In another question, I asked them to respond to a (rather poor) technical suggestion, but I asked them to do it in the context of an email to an engineer who worked for them (I put the students in the role of being the project architect). I was quite impressed with the care most students took in crafting a reply that was both technically correct as well as polite.

The post final

Although I write fairly long comments on my students' exams, I know deep inside that they never read them, and yet I continue to do it, because I know that there is the potential for learning there. This year, I was pleasantly surprised -- for the first time in twenty years, I had a couple of people engage with me after the exam to discuss answers. They weren't looking for points but were instead, trying to make sure they understood what a right answer to a few questions would look like. In one case, a student wrote up about 5-6 paragraphs as an answer to one problem to make sure that the student had a complete grasp on the answer. I was (pleasantly) astonished. While I can't make any causal statements between new pedagogy and this behavior, I could certainly believe that because I am much more closely engaged with students, they felt more comfortable doing this. In any case, I was quite pleased.

The end

And so, one woman's adventure with flipping comes to an end. I'm looking forward to crunching data, and I'll certainly post here if/when I find something interesting to say. In the meantime, here are my parting thoughts.

• It's good for an old dog to learn new tricks.
• Flipping lets me spend time with those students for whom the material is challenging.
• Learning takes place by doing, not by listening to me.
• TF engagement is critical.
• It takes a lot of effort to come up with effective in-class work.
• Pre class web forms are AWESOME. They let me engage with students in an entirely different way and to gather lots of interesting data.
• CS161 is even more time intensive than I thought.
• It would be useful to help students learn what it really means to design something.
• Flipping is a great equalizer when students enter with different experience levels or exposure to different topics.
• Fully integrated and coordinated materials take real effort but pay off tremendously.

Flip N-1

The penulatimate week of the semester was a busy one! I have some fun data on how the students view assignments, some experience with the partially-flipped classroom, an interesting discussion of a flipped exam, and finally the totally awesome creativity of my students as expressed via T-Shirt designs.

The Pain/Gain Scale

One of our former PhD students is now on the faculty at Swarthmore and he suggested to me the pain/gain scale as a way to determine if the degree of difficulty in assignments was perceived to be worth it. This seemed like a fine idea, so I polled students on the two major assignments completed so far, asking them to rate on a scale of 1-5 the amount of pain the assignment caused and the amount of gain they derived. The results are fascinating.

Here is the histogram of the responses:

At first I thought that Assignment 3 was really nice correlated, but then I calculated the ratio for each student and graphed that -- it tells a different story:

While most people found the ratio about 1:1, there were some who clearly suffered and others who had a blast. I have not done further analysis, correlating these ratings with how well groups did, but I found it interesting nonetheless.

The Partially Flipped Class

The astute reader will have noticed that while upholding my deal that I will not make students prepare for class while they are also implementing processes, building a VM system, or making a file system recoverable, it's been difficult to maintain the energy and enthusiasm of the flipped classroom. As I entered the homestretch, I decided to try an experiment with what I am calling the partially flipped classroom. I take my lecture notes and I break them into two parts -- a short intro (6-8 slides) and the rest (10-20 slides on technical content). I present the intro and then I dispatch the class for 15-20 minutes deriving from first principles some of the challenging aspects of the particular topic and approaches for addressing those aspects. For example, on Tuesday I was teaching virtualization. Now that they had built major pieces of an operating system and become intimate with other parts of the OS, I let them figure out what the big problems were in building a virtual machine-based environment. The groups each came up with excellent lists, which we then pooled. When this was all done, we'd covered much of the material in the rest of the slide deck. So, I was able to breeze through the rest of the deck relatively quickly and I knew that they had a much more complete picture of the topics, because they had largely derived them themselves. It felt very good. I repeated the process on Thursday with security. I introduce the three A's of security: Authentication, Authorization, and Access Control and then let them play around with identifying what can go wrong if you fail to address each aspect properly and then proposing approaches to solving it. This week, we'll be able to breeze through my slides on the topic relatively quickly, because most of the ideas came up in discussion. That will leave us time to talk about a few real world exploits.

I think this is the approach I want to try for all the material that I had been teaching "the old way." I'm pretty sure that with some careful thought I can make it work and we may be able to maintain a higher level of energy and engagement during the roughest parts of the semester. We shall see!

The Flipped Exam

Thanks to the wonders of Facebook and my friend Jill Levien, I was introduced to the idea of a Flipped Exam. Now, I don't teach game theory and I'm not entirely sure how this would work for my class, but I found it intriguing, so I posted it to our class discussion board with the directions "Read/Discuss." And my fabulous students engaged!

Student1: We should have such a final exam: the entire class: write
as much as possible of an operating system in 24 hours together.
editdelete

Student2: While I do think there is a place for individual examinations,
the current educational system overvalues such tests. My basic
understanding of the "real world" is that most projects of any real
significance are done in teams, be it research, commercial products,
etc. Sure, there have been cases of lone wolfs that create paradigm
shifts or fantastic projects (and this should be encouraged as
well!), but teamwork is ultimately a skill that should be encouraged,
not subdued. (24 hours might be a bit short to get much functionality
done on an OS even with a full class of people, but I certainly
wouldn't mind a collaborative written final).

Student3: Synchronizing >40 people would be so much fun...

Student4: I really like that idea. It will be a fun, collaborative
experience, and we'll end up synthesizing the entire semester
together into one glorious product.

Student5: This idea was the first thing I thought about as well. I
think it would be fun, maybe not necessarily writing a whole new
operating system, but writing another subsystem for OS/161.

TF1: Rob Bowden: Networking!!

Student5: Indeed that would be fun.

Student6: so punny

Student7: You get a synchro bug! And you get a synchro bug! Everybody
gets a synchro bug!

Student8: I find this much more reflective of how the world actually
works and I think it reminds people that the best answer is 99% of
the time not produced in solitude.

I'll second Student2's final suggestion as well.

Student9: If it's a coding assignment, maybe we'd need (want) groups
of a smaller size. 45 people can't all be coding at the same time,
so it seems as if the exercise would likely devolve into "Okay, you
two/three/four code, we'll just watch...and write a DesDoc? And,
uh, run tests!"

What about five partitions of nine, sorted day-of? You've got 24
hours to do what you can on [subsystem], working with some, all,
or none of your tablemates. Coordinate sleep, food, and synchronize
your watches...here's an assignment spec, go!

Right now I think I'm still going with the individual exam, but I'll definitely have to think about the concept of collaborative exams for the future. I am glad the class (or at least a subset of it) thinks that working together would be fun.

T shirt Designs

Even in the midst of a pretty heavy workload, my students know how to have fun. Check out the T-shirt designs on our home page. I'll let you know which design won after the T-shirts have been delivered.

Next: Flipping Over (May 24, 2013)

Flipped Out

It's Friday, April 19 -- my mother's birthday; my colleague's birthday.

I sit at home, working from the relative safety of a town six or seven miles way from the mayhem that erupted in the greater Boston area last night. As is old news by now, within roughly 48 hours of the Marathon Bombings, FBI and police had identified two "persons of interest" who became suspects last night after engaging in a robbery, a shooting, a police chase, and explosives. As of right now (8:13 AM on Friday, 4/19), one of the suspects is dead; the other is the target of an enormous manhunt that has much of the greater Boston area on lockdown. All the local universities are closed; my kids' schools are closed; people in six towns are being told to stay home and leave doors locked. I've certainly never experienced anything like this, and I'm guessing the vast majority of my students, friends, and acquaintances haven't either. So, as I did Monday, I turn to blogging.

The end of the class is in sight! Because many groups took late days on the virtual memory assignment, we pushed the design reviews for assignment 4 (make a file system recoverable by adding journaling) a few days. This gave students a much needed bit of breathing room and as a result they came to class Tuesday pretty energetic.

Rather than continuing with new material, I made Tuesday a flipped class. They had a few short pre-class questions to make sure they'd read the assignment and understood what they needed to do. We then spent class time letting them code read to figure out how the existing file system worked, what kinds of things were going to need to log, and how they might recover those log records. On one hand, we could have done a structured code walkthrough, but I'm pretty convinced that letting them do the walkthrough in small teams results in a better understanding than a guided walkthrough. (Although I'm beginning to wonder if we shouldn't offer code walkthrough sections each week ... that might be a really nice supplement. Or perhaps we could make a series of audios to walk through the code ... oh I'm liking that idea a great deal.)

We spent the first chunk of class time letting students merge new code into their trees. My theory is that this would be a relatively quick process and anyone who ran into trouble would have immediate assistance. I'm not sure that part worked very well. Some students got the merge done quickly, but others spent a significant chunk of time on it, even though they didn't need our help.

They were then supposed to pick a couple of operations and walk through them to figure out what they'd need to log and what recovery would look like. Some groups found this useful; other groups felt lost. Even for those who felt lost, my sense is that it was a decent use of time, as the students really need to understand the process of figuring out how to determine what needs to get logged and what needs to be recovered.

This class was followed up by the peer design review. The room sounded very animated during the review and I heard a lot of good discussions. I think the extra days after A3 really paid off here. We need to work with the timing to make the A3 peer design a more useful endeavor. That's going to require some serious rethinking about what we do in-class versus outside of class.

All in all, this week was a classic flipped classroom and everyone seemed much happier. I also took the opportunity to snap photos of my students with their group placards. For those of you who've never been involved in Harvard's CS161, we ask each group to select an animal as their team name. We interpret the term animal quite broadly and we've had everything from unicorns to meowbears in past years.

This year, I had someone who shall remain nameless draw each team's animals and then we printed them on bright yellow paper, laminated them, and but them in stands to assign groups to tables. With permission from the people photographed, I bring you a subset of the images of 2013 CS161!

The Blobfish

The Anteaters

The Caribouyah

The Centaurs

The Falcons

The Sphinx

The Squids

The Baby Velociraptor (singular)

The Dingos

The Koalas

Next: Flip N-1 (April 30, 2013)

Anxiously Flipped

It is 4:57 on Marathon Monday, and by the time you read this, I'm sure you all know what transpired today in Boston at the finish line of the Boston Marathon. I am grateful to be safely home and am at my keyboard waiting for all my students to check in. As Harvard has classes today, I am optimistic that my students were all in Cambridge, safely out of harm's way. Even so, I'll be happier when I see their messages. If only everyone else would be so lucky.

I figured that blogging might be a distraction from the day's events, but nonetheless, my suspicion is that this will be short and perhaps scattered.

It was a rough week to be a CS161 student. The virtual memory assignment was due on Friday and it is a particularly devilish assignment -- the number of lines of code to write is not huge, but getting the synchronization correct is just downright hard. This sets the students up for a lot of late night debugging, perhaps an all-nighter, and then perhaps a couple of late days. This has all played out reliably over the past many years.

So, this week's status has much less to do with flipping and much more to do with CS161, virtual memory, and why anyone would put themselves through this. For your reading pleasure, I bring to you a Facebook discussion that took place after I posted a note about how guilty I felt coming in at 8:00 AM and finding my students (surprisingly happily) working away, having been up all night. Let me just say that I never looked that happy after an all-nighter.

My original post: There is nothing quite so guilt-inducing as walking into your office at 8:00 am and seeing your students sitting in the lounge working on your problem set after having been working on it all night. Oh wait -- there is -- seeing multiple groups later that day around 4:00 PM -- all of whom have been up all night and are still working. Many of them now have operating systems that mostly support virtual memory, though!

In the following, I have converted names to initials, to provide some shred of anonymity.

• GK: That last bit, "mostly support virtual memory," reminded me of this and this.
• MO: that's guilt-inducing? I thought that's what you live for!
• RH: And yet, it didn't keep you from giving them the assignment in the first place. Hmm! Actually, IIRC, that was the easy assignment. It was the stupid filesystem one that almost killed me. I wonder if these assignments would still seem as hard today as they did 15+ years ago when I did them (am I really that old?).
• GK: Tell them that in 24 hours Andy Sudduth finished his operating system, finished putting together his HeathKit computer (this was in the early-to-mid 1980s), wrote a full screen editor, and then wrote a paper due the next day in his editor on the computer he built running the OS he wrote. Or, you can tell them what I try to tell myself—start sooner. Editorial note: Yes, Andy Sudduth, Harvard legend and Olympic rower is also a graduate of CS161
• Me: We've tried many, many things over the years to attempt to make the problem sets less time consuming, but at the end of the day, learning to build and debug asynchronous, multi-threaded programs is just plain hard. And we continue to get the feedback, even years later, that the time is well spent. I have an awesome class of 45 who, although exhausted, still seem deeply happy with what they are doing.
• GK: And since I mentioned Andy, rest his soul, he also loved that course.
• PA: I have fond memories of programming such things when using an ascii terminal to an under-powered VAX11/750 running BSD4.2 with it crashing every now and again... your students should be happy their development environment is pretty stable!
• RH: @Margo: I agree. It was a good class. It retains, however, the distinction of being the only 40-hour-a-week class I ever took. Second place was about half that. @GK: If those students are anything like me, not starting soon enough wasn't the problem.
• MO: it's no fun without an all-nighter once in a while
• MO: Of course, I never took the course, just had the enjoyment of watching BF and others suffer through it
• MW: Wow, 45? That's awesome.
• PR: It's a public service. Think of of the trouble those students could be getting into if they weren't locked up building an OS.
• NM: I wonder how many techies at the likes of Google, Facebook, etc. would still get a chill down their spine at the mere mention of "assignment 3"?
• DK: I still fondly remember our all-nighters in CS161 (with CY). Professor Hsu said we built the class's most elegant OS (everything was based on a common semaphore implementation). And it always ran for at *least* 30 seconds before crashing.
• CY: I remember the class being a 9 to 5 affair. PM to AM, as that was the only time M, D, and I had in common. And I fear that the elegant semaphores were woefully slow--the guys that used atomic ops to access the priority queue were twice as fast as our version where the semaphores also managed priorities.
• DK: I think that was also the class where we wondered "what happens if you tell the unix shell while (1) fork() ?" and took down the department compute server for the night.
• CY: I think it was only microvax-6.husc.harvard.edu, out of of a fleet of 10 or so microvaxen, and not the file server (thank goodness). The Magican's Apprentice moment was priceless, though, as we tried and failed to kill the parent process and started getting "no more processes" messages from our other shells. The next morning, the sysadmins were more worried that we were doing some important computation than about our abuse of the machine--they just power cycled it and everything returned to normal.
• CY: Actually, the real thing I should say about Prof. Hsu's CS161/261 class is that, of all the hacking classes I took at Harvard, it was by far the most useful in the rest of my career. I think it was the construction of synchronization primitives in a raw environment and figuring out how to debug code built on top of those primitives--there's nothing like it for combining power and danger, and for whatever reason being able to build such things remains a rare skill.
• DK: I think the "whatever reason" is that we have done such a good job of creating layers that shield students from the "raw environment"---students building a web application in Django have no idea how many layers of complexity are hiding between them and the hardware, and wouldn't know what to do down there. The hard question is whether this is a good thing or bad----given that we have already built the layers, do we want to force every student to really understand them, or is it enough for them to use them?
• And to make my colleague MM happy, I won't cut this one out DK: As for most useful classes, I've got to flag Umesh Vazirani's CS124, at least to the extent that "useful" means "determined where I am now".

And to top it all off -- this week's picture, straight from China: Tao Stein's students at midnight working on their ray tracer.

Next: Flipped Out (April 22, 2013)