How to Present your Research: Part 6: Common Presentation Pitfalls to Avoid

In our communication skills group last week, the student presented (for the first time in this group) work-in-progress talks or a 3-5 minute research summary. I try to give everyone detailed feedback, but there are common things that came up, so I figured I'd try to write them down here. This post could really be called "Tips for Presenters."

1. Timing

The assignment was to prepare and deliver 3-5 minute presentation for a generation audience (i.e., not your project team, but a group of smart computer scientists). I then asked each person how long they thought their talk would go before they gave it and then how long they thought it actually went. I timed all the talks.

Unsurprising, practically every presentation went longer than expected. Similarly, everyone knew they'd gone longer than they expected, but they were not super accurate at realizing how long they went. And this is 100% normal on both cases. Our ability to judge how long it is going to take to present material is not particularly good.  The problem is that while a 3 minute talk might run over by a minute and that doesn't seem bad, proportionally, that means your 20 minute talk just became a 26 or 27 minute talk and that's actually a real problem!

Why do we underestimate? (There is likely science on this, but I'm going to draw on my experience here; it would be interesting to compare my experience and assessment with the literature, but I don't have time to do that now, so we'll leave it for a future post.)

I think there are three main reasons: 1) We think more quickly than we speak, so unless we practice the talk out loud, our timing is off, 2) If we practice without an audience, we have no feedback about whether what we're saying is getting through -- when you have an audience, you are getting feedback and that almost always results in your slowing down, 3) We have an inherent fear of running out of things to say, so we over plan.

Just like we read more quickly than we speak, we also think more quickly.  So, if you practice your talk by going through it in your head, your practice will not match your delivery. The solution to this is simple: practice out loud. I hear you, "Oh that's so cringy."  Yes, it really is. Do it anyway.

Similarly, do a real practice talk in front of real people. You clearly are not going to do this for every status report, but when you are presenting for real, do a practice talk. This will be the best way to see how long your talk is going and to check whether you and your material are in sync and conveying the key things you want to convey. And then ask for and gracefully accept constructive feedback. You then get to be judicious in how you incorporate that feedback, but while it's being given, I encourage you to follow the Gotham Booth approach and simply listen, take notes, and ask clarifying questions, e.g., "Was it the graph you found confusing or how I described it?"

Now, how do you conquer that internal dialog that goes something like this, "You know, there is nothing particularly deep here. You're going to be done in two minutes and everyone will think that what you did is simply no big deal." How did I know that's what you were thinking?  Because we all go through that dialog to differing degrees. My advice is to keep the presentation itself streamlined -- explain things clearly and in ways that connect. When you are tempted to off on a digression about some fascinating detail, make a set of backup slides. I don't know how long powerpoint has had "sections," but now that they do, sections are a great way to organize various sets of backup slides. You have them in case someone asks the question (and won't you look so awesomely prescient when you pull them up!), but you can avoid embedding the digressions and extra detail in the talk.  And, the sheer existence of the backup slides should help qualm that annoying little demon who tries to convince you that you have nothing to say.

2. Matching Delivery Rate to Absorption Rate

For longer than I care to admit, I could not exactly figure out why I was uncomfortable with one of my former students who always wrote out their talks completely in their speaker notes. I too sometimes write out what I'm going to say, but I absolutely never actually read what I write while giving the talk, so why did it bother me so much when this student wrote out what they wanted to say? It took me a couple of practice talks to figure it out.

If you read your talk (and aren't excruciatingly careful), you are able to deliver your talk at a rate that is faster than the rate at which your audience can absorb it.  This sounds obvious in retrospect, but only in retrospect. When you are simply reading the words you want to say, you aren't pausing and stopping to allow your audience to keep up with you. When you do not have a written script (and haven't memorized your talk, which is equally problematic), then you tend to pause at the right places, and since you have to think a bit about what you are going to say next, you give your audience time to internalize what you've just said.

Therefore, I advise writing notes/bullets to yourself in your speaker slides (I find numbered points particularly helpful and try to avoid having more than about 3 points I want to make on any slide; a point can require multiple sentences). Think of these are reminders of the topics to cover, but not the exact words, so that you have to engage in a thinking process (not a reading process) when you speak.

If you really really really don't want to have to select words on the fly (and I completely understand many reasons why you might not), then you should 1) repeat the words, "Slow Down" in your brain at the beginning of absolutely every slide and 2) insert <pause> in the right places in your talk,  where you need to stop and give listeners a chance to catch up. I also insert <click> when I want to sequence through animation on a slide.

Pacing your words to keep your listeners engaged is perhaps that most challenging part of giving a talk, and it requires a lot of practice!

3. Buzzword Bingo (this is really about clarity)

There are few things as frustrating as being in a talk and having the speaker toss out terms and acronyms that you don't understand. (My department is terrible at this and when I first joined, I spent every faculty meeting raising my hand and asking what every acronym meant.)

So, nearly all acronyms should be explaind on first use (you can assume everyone knows what a CPU is; you cannot assume that everyone knows what a BANANA -- yes that's an acronym, but not in CS: Build Absolutely Nothing Anywhere Near Anything). In general, avoid inventing acroynms just to make your life easier (I have had students in the past who would introduce 10-20 acronyms in the introduction just to avoid having to type words out later in the paper; this is not a winning strategy for getting papers accepted). I like to think of writing and reading as a give and take process. Every time, you ask your reader to learn something to read the rest of your paper (i.e., what BANANA means), you are taking. Before you take, you'd better give them something pretty darned special to have them be willing to give you the effor to learn your acronym. After being forced to learn about three new acronyms, I am pretty grumpy reading the rest of the paper.

If you find yourself explaining some term and never using it again; don't use it; just explain what you're doing that one place you're tempted to introduce it.

Assume you are talking to a non-expert and define/explain every term that a non-expert is unlikely to know. Ask yourself, "Did I know what this term meant before working on this project?" If the answer is no, then chance are you need to explain it during your talk.  And, the more time you spend explaining terms, the less time you have to discuss things that are truly interesting.

Take Aways

1. Practice your talk out loud.
2. Give a practice talk (and invite a range of people who can give you feedback from different perspectives).
3. Keep the maintalk streamlined and have backup slides (material).
4. Match your delivery tempo to the absorption rate of your audience. This requires careful and thoughful pauses in your speech.
5. Keep acronyms to a minimum.
6. Define all terms specific to your area that are unlikely to be understood by those not in your field.

How to Present Your Research: Part 5: The 3-5 minute (work-in-progress) talk

Recall that in Part 1 of this series, we laid out a collection of different types of talks and then identified what has to appear in each of those talks. We've been working our way through these and today's topic is the laid out in the third column as the "Work in Progress" (WIP) talk. It may also be a talk that goes along with a poster.

The parts that need to appear in a WIP are: The fairy tale (of course), motivation, your work, evaluation, conclusion. Mostly, I like to think of this as a simple expansion of the fairy tale: to a first approximation you are basically going to have approximately one slide for each part of the fairy tale. In all likelihood, you'll have more than four slides, but you probably won't have 10. So think about it as: 1 "once upon a time", 1-2 introduction of the villain, 1-3 introduction of the hero, and 1-2 for the happily ever after which might include results and/or next steps and will certainly include a reminder of why this work is important.

Lest you think that these talks are simple, throw-away talks that don't really require a lot of effort, I want to highlight two instances of such a short talk with enormous consequences: 1) pre-COVID, the papers that were not quite good enough for Oral Presentations (of which there were maybe 15) were selected as Spotlights (around 50 of these, out of the approximately1500 papers accepted) and were given a 5-minute talk slot. 2) The Bell Labs Prize Phase 2 competition requires that teams summarize their work in a slide deck not to exceed 10 slides. Needless to say, a lot of time/effort goes into these short talks.

The Cover Slide

I can see and hear you now: You are rolling your eyes and thinking (or even saying out loud), "You have got to be kidding me -- she wants to tell us what to put on our cover slide?  And the answer is, "No." I believe that you know what to put on your cover slide (the project Title, your and your collaborators' names) and ideally some logo or graphic or picture. However, I am going to talk about what to say when you begin your talk.

What not to say: Please, please, please, do not simply read your slide to the audience. I have sat through so many talks where the session chair says, "Next we'll hear from person, who is going to talk about title." And then person gets up and says, "I'm person and I'm going to talk about title." And, of course, the entire time, I've been staring at a slide that says, "Title" and has an author list with person's name in bold.

I hope by now you realize that if all you do is read your title and name, you are duplicating information and missing a huge chance to get the audience interested in your work.

If, and only if, A) you have not been introduced, B) the audience does not know who you are, and C) you have co-authors, may you introduce yourself.

Then, rather than read the title to the audience, explain why they should care what you are about to tell them for the next five minutes. This might be the 'once upon a time' of your fairy tale; this might be an explanation of your work that avoids buzz words that might appear in your title; this might be a question, i.e., "Have you ever wondered ..." Whatever it is, give the audience something that they will not get from simply reading your slide.

I'm going to pick four random titles from HotOS (the last event I attended) and offer some cover slide material.

Putting out the hardware dumpster fire -- Some of you probably already heard this, but I'd say something like, "Today's hardware is really complex, and for the most part, when we talk about operating system security, we completely ignore this complexity. As a result, I'm going to claim that we aren't really solving the problem that we think we're solving, and I'm going to offer some suggestions for actually solving the hard problem!"

First, Mothy needs to introduction in our field and second, these few sentences are much more descriptive than the title, which I still believe is entirely wonderful.

Degrading Data to Save the Planet -- We will never have less data than we do today! Every day, we produce far more data than we destroy. More importantly, producing the devices on which we store this data (i.e., flash) produces significant carbon emissions. Let me tell you about our new sustainable storage design.

Doesn't that tell you a bit more about what's to come than the (also great) title did?

Towards (Really) Safe and Fast Confidential IO -- Customers do not trust their cloud providers! Confidential computing is an approach to using cloud resources in a way that does not reveal confidential data to the cloud provider. Unfortunately, today's work in confidential computing mostly ignores information that is revealed through IO APIs. Let's talk about how we can improve on this situation.

Once again, I think this tells you more about what's coming than the (perfectly fine and accurate) title.

Automatic kernel offload using BPF -- I explicitly picked this one, because the title tells you exactly what this paper is about. Even so, I think you can say something here that tells your audience more.

The extended Berkeley packet filter (eBPF) has become the de facto approach for running kernel extensions. But how do you decide what should be offloaded? Offloading the wrong thing can produce terrible performance. Let me tell you about a tool we developed to automatically figure out what you should offload to eBPF.

See -- I bet you did not realize that that's what this paper was about.

So, having now written an almost entire blog post on your cover slide, let's move on.

The Once Upon a Time

Others, less well-versed in fairy-tales, will tell you that this is the motivation. They are partially correct -- the motivation is actually the combinination of the once upon a time and the villain. In a short presentation, the once upon a time may also be short -- you just to have place the work you're going to talk about in the right setting. Is this a paper about data centers? containers? isolation? performance? security? privacy? formal verification?

If you are presenting a short talk, chances are that you are one of many talks in a session and there might be little or no relation between the talk before you and yours. That makes it all the more important to help your audience get in the right mind set for your paper.

Consider the Degrading Data paper above. If the paper before you was on disaggregated memory, the audience is primed for talking about in-memory data; if they paper before you was a storage system paper, they are primed for a storage system talk. You want them primed for your talk.

The Villain

Now that you have your audience primed, you really need to motivate them that your work is important. What is the problem you're solving? Do not assume they agree that the problem you are solving is A) important, B) interesting, C) feasible. You need to convince them that they actually care about your problem.

Sometimes the villain takes the form of a story (i.e., "Imagine that you are ..."). Sometimes it could be a performance problem, "Have you ever done X and then had to sit there and wait until Y?" It could be something the user has never wanted to do, "If you are a networking researcher, packet traces are worth their size in gold!" (So, even if the listener is not a networking researcher, they ought to be able to appreciate that someone who is would find packet traces valuable.)

The more concrete you can make the problem the better. Compare, "Keeping data safe is important," to "Customers lost twelve trillion dollars when this very specific bad thing happened."  That ought to make people sit up and take notice. Even if they are not data geeks or security/privacy experts, the thought of losing twelve trillion dollars is likely to seem important.

The Hero

This is where things get hard. I think that with focus and a bit of thinking, you can really nail the context and the problem, but how do you summarize everything you've done in only 2-3 slides?  You don't! Focus on the big picture and maybe (just maybe) one technical detail.

What do we mean by the big picture? This is not necessarily a detailed block diagram of the 100,000 linse of code that you've written to build some system. Instead, what are the three key ideas that you've had to solve this problem?  How do they fit together?

I've spent a fair bit of time developing short 'research vignettes' that let me describe research in myriad different areas to a general audience. Each of these is really just a short talk.  Here are some examples:

For part of a PLDI keynote, I summarized a student's MSc work (tens of thousands of line of code) in four slides: context, picture that shows what the student produced while my voice over explained why we were trying to do it, picture showing the key decomposition approach that let him solve it, and a slide with one key technical approach used to in solving the problem.

At a recent talk at Duke, I summarized another student's MSc work in 6 slides that used the same primary picture on each slide and then added different pictures on the side to illustrate how he solved the key parts of the problem.

It takes awhile to get good at capturing the essence of the work and explaining things at just the right level of detail. I find it helpful to have a specific audience member in mind. This person is most definitely not someone on your project. They are likely not even working in exactly the same area you are. But they are a computer scientist and they are smart. Think about how you might convince them that they should care about what you're doing and then once you've made them care, figure out how to present your work in a way that helps them "get it" even if they cannot appreciate all the nuance and subtlety.

The Happily Ever After

As mentioned above, the happily ever after must include a reason why it was worthwhile to conduct the work. How have you made the world a better place, even if only a tiny bit? In addition, it can also include, next steps, things you learned, suggestions for future projects, etc. If you can leave the audience wanting to work on your problem, they you have succeeded!

How to Present your Research: Part 4: Presenting Results

We've talked a lot about story-telling so far, and I bet you're all wondering, "OK story telling is fine, but I have some actual research results I want to show!  Let's get to that." And indeed, we will. But, guess what -- each result is a mini story.

A result story has three basic parts: 1) Why am I showing these results? 2) How did I obtain these results? 3) Here are the results.  Let's dive into each one.

Why am I showing this to you?

Each experiment you perform or result you show should be either answering a research question or showing evidence to support/refute an hypothesis. I think these two are really the same thing, so for the rest of this post, we'll call them research questions (RQ for short).  Note that it has become a lot more common in recent publications (even in systems and ML) to explicitly state your research questions. I encourage you all to do this before you write any experiment code -- it's great research and mental discipline.

Before you dive into the details of the experiment, explain what RQ you are answering. This could be no more than a sentence, "We want to know if the overhead of OurGreatIdea is acceptable, so we conducted N experiments to determine if we were able to do SomethingAmazing without introducing more than X% overhead."  Wait -- notice how I snuck that goal in the end? I think that 99% of the research out there asks the question, "How much overhead does something introduce?" And then, regardless of what that overhead is, we declare victory!  That's not exactly science. I urge you to think (before you run any experiment), "How much overhead is acceptable?" Once you do that, then you can decide if your overhead is great, good, acceptable, almost there, or terrible.

But, I digress. Sometimes, you simply have to state the RQ. Sometimes, you might want to go into a bit more detail. Some of the greatest insight comes from ablation studies; these answer the question, "Which of the N things I did account for the great results we get?" Let's say that you built a system that introduced three optimizations. People will wonder both A) How much does each thing matter? B) Do the results compound? In the case of such an ablation study, you might want a slide to review the various things that  you did and then explain that you're going to introduce each one by itself or remove each one and leave the others in. Depending on your system, one of those is likely to make more sense. If you can do both, go for it!

A cousin of the ablation study answers the question, "How much time do we spend in each part of our BigComplexSystem?" Usually, you can just explain that with a single sentence; sometimes you'll want an architectural picture to explain it.

Regardless of the complexity of the experiment, please explicitly tell them what question the result you are about to present is going to answer.

How did I do this?

Explain the experiment!  It sounds easy enough, but you'd be amazed how many times I am shown a graph with 0 explanation of: A) the data used, B) the workload run, C) any details about whether I am seeing averages of many runs, a single run, the best of many runs, etc. So, describe your experiment.

In a paper, you will want to give precise details about the machine on which you're running. In a presentation, I think it's fine to categorize the platform, e.g., "We ran this on my laptop." or "We ran this on a huge server." or "We ran this on a typical server with a Big Honkin' GPU."  You can go into more specifics, but I think just putting the detail on the slide is more helpful than saying the particular model of processor you are using (no one can wrap their heads around that verbally; OK maybe someone can, but I do not think most people can).  Typically people want to know how much main memory you had and if you are running a modern machine. If you are doing storage work, then of course, they want to know something about the storage system too.  If you are doing ML, they may care what GPU you are using. Think carefully about what your audience wants to know and the best way to present it (on the slide, but say nothing; on the slide and highligh key features; on the slide and read it).

Describe the workload!  Tell the audience whether this is a standard benchmark in your field or one you made up (and if you made it up, you are going to have to justify why). Is this a microbenchmark or a macrobenchmark?  Is there input data? Where did that data come from? What exactly are you going to show? The average of N runs? One run? One cherry-picked run? A number you made up?  Sometimes, you might even say, "Here is what we expect a good system to do on this benchmark."  Then you can compare that strawman to your actual results. Alternately you might way, "To the best of our knowledge, the best system for this is X and here is how it does on this benchmark." Again, then you can compare your result.

Your goal is that when you finish describing your experiment, the audience is excited to see your results.

My Results

Tell me what I'm looking at! Again, this seems obvious, but I have sat through too many talks where a speaker excitedly tells me how great these results are before I have any clue what I'm looking at. So, breathe ...

Most results (in systems) are either tables or graphs/charts (I'll use the term graph here to describe a visual depiction of data). In either case, describe what the audience is looking at. For example:

Graphs

This graph has throughput on the Y axis and the number of cores on the X axis (if you are using log scale, say that explicitly here, even if it's in the axis labels which it should be). So, big bars (higher numbers) are better. [Please, always add this last part!]  Sometimes I just show the structure of the graph with no data as I explain this.  That allows the audience to understand what they are going to see before they get distracted with the actual data. The error bars show standard deviations, the middle 50th percentile, whatever -- tell me!

Tables

Most of the same rules apply here. Each row represents something and the columns correspond to something else. Tables of numbers are relatively difficult to comprehend, so I encourage highlighting the things you want the audience to take away from the tables -- embolden the best in each row/column; color code good/bad/mediocr results. Basically, help the audience take away the message you want them to take away from your results.

Takeaways

Draw the listener's attention to the things you want them to get from the figure. Do not assume that your results are so obvious that they will get exactly what you want them to! Inevitably, they will draw a wrong conclusion or one different from the one you wanted them to.  Walk them through the results.  I like to compare and contrast with what we might expect. Some examples:

We expected that we would scale well until we were running one thread on each socket, but we found that we actually scaled well until we were running one thread on each core. As we'll see in the next graph, our cache footprint was small enough that the data fit nicely in the per-core caches. [This is a great technique to explain WHY you get the results you do -- show a macrobenchmark result and then follow it up with a microbenchmark result that explains the results you got in the macrobenchmark.]

We had hoped that participants using our tool would complete the task more quickly. While they did produce better solutions (point to the part of the picture that shows that), we noticed that they actually took longer. [When you have a surprising or unexpected result, this usually indicates something interesting, even if it didn't match your intuition. In this case, perhaps it's something like, "Our qualitative survey results suggest that people found our tool so much more enjoyable to use that they were willing to spend more time using it to produce high quality solutions; in contrast, old tool was so painful to use that participants stopped as soon as they produced anything close to correct."]

In my speaker notes, I almost always have a numbered list of 3 +/- 1 key points that I want the audience to take away from any result I present.  I do not write the details of those points in the notes, because I want the timing of my explanation to match the ability of the audience to comprehend what I'm saying. If one writes out the exact explanation, it is almost always the case that that explanation is presented too quickly for the audience to comprehend. If the speaker has to think about it, then their delivery almost always better matches the audience's receptive timing. I think this point is crucial!  When we are nervous, we almost always start to speak more quickly and often become quieter towards the end of our sentences. You want to do everything possible to avoid this, especially when presenting results.

Key Takeaways:

1. Don't forget any of the three parts: question I'm answering, how I am answering it, what I found.
2. Explain the format of the results you are presenting.
3. Use microbenchmarks to explain the results of macrobenchmarks.
4. Highlight surprising results.

How to Present your Research: Part 3: Status Updates

Normally, we think of presentations in terms of formal talks. However, I am willing to bet that we all spend far more time giving information status updates (which are, in fact, presentations) than we do giving formal talks, yet I do not believe I've ever seen any guidance on how to do that effectively.  How strange!

Well, here goes ...

Surprise: I'm going to argue that every status update is, wait for it, a fairy tale! (This is going to be a recurring theme.) Before diving into the details of that fairy tale, let's talk about different categories of status updates.

The 1:1 meeting with your advisor/supervisor/team lead. At some point, you'll find yourself in a 1:1 with someone we'll call "your boss" and you'll want to update them on what you've been doing. The key thing to help organize your thoughts and presentation for this meeting is to ask yourself, "In how much depth does this person understand what I'm doing?" In some cases, the boss will know exactly what you are doing and could be doing the task themself.  In other cases, the boss has a big picture idea of the project, but is not knee deep in the details. Obviously, you're going to have to present your work differently depending on the situation you are in.

The project meeting. This is typically a small meeting with the people directly working on the same project you are working on. In this case, you can assume that the others in the meeting know the project-level fairy tale, so you are going to focus on the smaller fairy tale describing what you are doing.

The large group meeting. This meeting includes people working on projects different from the one on which you are working. Even if you see these folks every week, you probably cannot assume that they remember from week to week exactly what you are doing, so you need to plan accordingly.

Regardless of which of these meetings you are having, you are still going to present your status in the same structure: 1) some context setting, 2) a discussion of the challenge you were addressing this week, 3) where you are in addressing that challenge, and 4) what comes next. (Sound familiar?) It doesn't matter if you're winging this entirely verbally, giving a formal presentation with slides, or communicating via interpretive dance, you pretty much must have all four parts.

So let's focus on some of the finer points in how to think about this.

The 1:1 Meeting

You can pretty much assume that your boss knows the overall project you're working on. If they aren't knee deep in your work, I suggest starting by reminding them of your overall task, e.g., "I'm working on our server performance," and then helping them recall where you were at the end of your last meeting, e.g., "Recall that when last we met (or at the last project update), I had been working on <something> and had gotten to <some particular point>." If they are knee-deep in your project, you can probably skip that. But even if they don't need that context, because, for example, you just met yesterday, I still recommend starting out with what you were working on during the time since the last meeting. For example, "My focus the past day/week/month has been on improving the throughput of our server." If this has been a long-term undertaking, then perhaps there has been progress reported at previous meetings, "And so far, we have 1) added multi-threading, and 2) partitioned the data structures to avoid too much synchronization overhead."

At this point, you've brought your boss up to date on the context in which this period's work happened. This is really important. All too often, it's easy to forget that while every teeny tiny detail is crystal clear in your mind, that is not the case for your boss. If you met two days ago, they may very well be on the same page as you, but a short reminder won't hurt.  If it's been longer, your boss is interacting with way too many people, they have been gone, they have been distracted, or (as is the case with me) they simply have a sieve-like brain, getting them focused on your details will let you make the most of a a meeting.

Now you can shift into "the problem." What specific thing were you trying to accomplish? It may have been an easy week: you had a straight forward implementation challenge; it may have been a difficult week: you were tackling a completely wide open, open-ended task. In either case, the problem/solution part of your meeting is where you can engage others in helping you, so think in advance of any areas in which you might want assistance. In explaining what you were working on during this period, be as specific as possible. You might use data you gathered to describe the problem in more detail. For example, "It was unclear where the next bottleneck was, so I spent some time profiling the server." This is beginning to blur the line between problem and solution, but I think that's actually just fine in this context.

So, the problem is really what you started out the day/week/month intending to do and the solution part consists of all the things you did to address that problem. Again, be specific, show data. If you spent the time reading papers, organize your reading into a structure, so you can say something more meaningful than, "I read a bunch of papers."  Instead, start with why you were reading papers (problem), "I did not feel that I had sufficient background in different ways to improve server performance, so I looked for papers describing similar tuning efforts. I looked in the following conferences for papers, and I found N papers that seemed appropriate."  That is what your problem phase looks like -- in your solution phase, report what you learned. "After reading them, I realize that performance tuning breaks down into three basic classes .... the first two don't apply here, so I skimmed those papers for techniques, but focused primarily on the third class, which was closest to our work. I identified the following three things that might be good next steps."

I picked this example, because I find that many times, students do need to spend time reading, but there is not necessarily a clear goal or perspective from which the reading happens. The more you can focus your reading on trying to fill a specific gap, learn something specific, or build background in a particular area, the more value you will get out of that reading. However, you can apply that same process to your weekly work.

Once you've presented what you've actually been working on you should find yourself in one of two positions. You might know exactly what needs to happen next and you lay that out as your final "Next Up" (happily ever after).  Or, you could use some help.  Then you invite discussion and brainstorming around your problem. Continuing our example, "The profiling didn't seem to indicate any one area where all our time is going, so I'm not really sure what to try next. I was hoping we might brainstorm ideas."

So now you have turned your "status update" with your "boss" into an engaged discussion. I intentionally started with this one, as it's likely to be the most detailed (and perhaps the most stress-inducing for some); all the others are simply subsets of the 1:1 update, so let's move on to them.

The Project Group Meeting

Your assumption here is that everyone knows the project you are working on, but may not know all the intricate details of the specific thing that you are doing. So, your context starts pretty much at sentence number two of the 1:1 context setting part: "Recall that when last we met (or at the last project update), I had been working on <something> and had gotten to <some particular point>." Assuming that these meetings are happening relatively frequently, that's probably all the context you need.  You can move right into problem/solution.

The problem/solution part of your status can be pretty much identical to what you might do in the 1:1 meeting. And it could lead to the, "Here is what I have to do next," or you might say, "I could use some help brainstorming next steps. Perhaps after all the status updates, I could get a few minutes from the team to do this?" If your project meetings are jam-packed, then ask who might be willing to meet at a different time to do this. I fear that students often feel that they must solve every single problem all by themselves, and that's not how research (or development) works.  It's a team sport. It is absolutely OK to ask for help; it just means that when others ask for help, you should try to step up and help. (It's OK to say, "You know I don't have much experience in this area, but I'm happy to do my best to help!") 

The Large Group Meeting

For this one, think of Systopia Standup. Before going into the specifics, I want to take a moment and explain why we have a large-group standup. When I joined UBC, after the students got to know me a bit better, the single most frequent complaint I heard was, "We have no intellectual cohesion. No one knows what anyone else is working on." TLDR: Our large-group standup was invented as a solution to this problem.  But let's ask the question that underlies that complaint: Why is it important to know what others are working on?  For some, the answer may be obvious, but for others it might not be, so I want to be explicit about this.

Why have large group status updates where hardly anyone really understands what I'm doing?

1. The people in the lab form the beginning of your professional social network. You may have absolutely nothing in common with them outside of the lab, but simply by virtue of being members of the same lab, you have a common experience. These people will show up over and over again throughout your professional career, and you never know when you might suddenly find yourself wanting their expertise, assistance, or advice. You don't have to become best friends, but you should take advantage of this opportunity to forge professional relationships with your lab mates. (You never know, 25 years later, you might find that you are their colleague, e.g., Will Evans and I were at UC Berkeley at the same time, and although we were not in the same research group [as I'm sure should be apparent], we knew each other well enough that when I intereviewed here, I felt that I had a colleague here that I knew].
2. Great research rarely happens in silos. Elegant solutions to real problems frequently require expertise in many areas or at least different perspectives from within the same area. If you have some understanding of what your peers are working on, you can figure out who to reach out to when you encounter an obstacle. Help need not only come from your own project participants or only from students with your advisor. Here are some examples of ways in which my students have gained explicit benefit from knowing what's happening around the lab: Ali and I were stumped on a proof for a paper on weighted GOSDT. One of us mentioned it in standup and the next thing you know, Mathias and Ali had come up with a lovely probabilistic bound on the problem we were trying to solve. Zainab is getting direct end-user feedback on her system. It's not a formal user study; it's not even a pilot. It's obvious to us that Paul is the right person to ask about this. (And I know that Nichole has also had similar experiences.)
3. Our field is HUGE. You can't possibly know everything happening in it or even all the techniques people in different areas use. Large group status updates give you a glimpse of what's happening other areas. These are an incredible learning opportunity.
4. Some day you are going to either A) be on the job market or B) be talking to someone who is looking to hire someone. The best thing that labmates can do for one another is promote each other. If you are in category A, would you not love it if your peers introduced you to their former managers who might be looking for someone just like you? Think how impressed your colleague in situation B will be when you say, "Oh, one of my labmates would be perfect for this." If you don't know what others in your lab are doing (and they don't know what you are doing), none of this can happen.
5. And speaking of that job market. I am 100% convinced that one of the reasons I was as successful as I was on the new prof job market (oh so long ago) is that I could actually talk to people in fields other than my own. I knew enough about what folks in my lab were doing and some of the big ideas others in the department were working on that I could ask questions of everyone, from theoretician to computer architect. You want to be that candidate, and you can't be that candidate by staying firmly in a silo and ignoring all that is happening around you.

End Digression

OK, so let's say you now understand why large group status updates are useful. How do you do a good one? Tell a very short fairytale (a nice short story is way better than and endless list of tiny things you accomplished).

Context: Recall that I work on the Awesome Server Project.

Problem: This week I was trying to figure out how to improve throughput on the server. We are handling X requests/per second and the state of the art systems seem to be able to handle 2X requests/second.  We don't necessarily have to match that performance, since we do This Awesome Thing. But a 100% performance penalty will never be considered worthwhile, so we need to get performance to within at least 10% of state of the art.

Solution: This week I tried removing every other line of code. Good news: Performance was great. Bad news: The server no longer works.

Up Next: I think I'm going to be a bit more strategic in what I remove; I'm going to do some dead code elimination to see if I can improve the iCache hit rate, since I took some measurements and I'm getting truly awful results for that.

It's just another fairy-tale, but one focused on your activities for the week, not your overall project, not your dissertation, just the thing that has been keeping you up at night this week.

Wrapping Up

Here is a tiny table to remind you how to think about the four parts of the fairytale in different settings.

	The 1:1	Project Update	Group Update
Context	Remind, Recall, Focus	Recall	Recall (Big Picture)
Problem	Specific (maybe with data)	Specific	Importance
Solution	Specific (include data)	Specific (include data)	Overview
Next Up	Brainstorm or Plan	Specific Plan	High-level Plan

How to Present your Research: Part 2: Visual Materials

How to Present your Research: Part 2: Visual Materials

(I wrote a related post a long time ago about use of visual materials in teaching. It is related to what's discussed here, but is focused specifically on presentations in the classroom. At the same time, I would argue that whenever you are presenting your work, you are, in fact, teaching.)

About a millisecond after you are asked to give a talk, I'd be willing to bet your attention turns to slide preparation. While I too, almost always, use slides when I give a talk, I do want to point out that there are fields where this it not common. There are fields where talks are people (literally) reading their written work out loud and fields where people give talks without any kind of visual aids.  If you're reading this, I am guessing that you might find this anywhere from surprising to shocking.

Once you have gotten over the shock, however, perhaps we should ask ourselves why we use visual aids. I'm completely serious -- why do we feel the need to have slides to accompany our words?  There are a few reasons:

1. To help illustrate complex concepts.
2. To remind yourself what you want to talk about.
3. Because it's expected.
4. To leave behind an artifact that summarizes your work.

I hope you won't be shocked if I opine that 1 and 2 are good reasons, and that 3&4 are not good reasons. (The artifact you leave behind is, in fact, your paper!  Or, today, perhaps you've been asked to produce a short, e.g., 2-minute, video overview.  This is a much better advertisement for your work than a slide deck.)

If you believe that you produce slides purely for #1, let me ask the following: if you are merely trying to illustrate complex concepts, why do you have slides for every part of your talk (including the introduction and thank you)?  Surely those are not complex concepts?  In other areas, you might see talks that are only occasionally punctuated by images or visuals, which really do illustrate a concept -- they show a picture of a novel experimental apparatus or some results or some particularly complicated concept.  However, there are not necessarily slides to accompany every single part of the talk. In our field, that is not the norm.

So, I ask again, what is the purpose of the slides?

I believe that most of the time, whether we admit it or not, slides are mostly to help us remember what we want to say or talk about as well as to remember how we decided to talk about it. Once you embrace this concept, you buy yourself a great deal of freedom in what you place on your slides. Your slides no longer need to parrot your talk. Instead, they can enhance and embellish your talk. Almost ten years ago, I decided to switch from a fairly conventional slide style where words were primary and images secondary to a style with as few words as possible. Two things happened: First, it started taking me MUCH longer to prepare slides for a talk, because it took forever for me to find exactly the right image to convey what I was trying to convey. Second, I began writing much more extensive speaker notes, frequently, because I like strategic animations, and I want to make sure I use them at precisely the right point in the voiceover. (Sadly, I almost never look at my notes while giving a talk; I ad lib almost completely. Thus, I need to go over my slides enough that I know where I'm supposed to click and hope I get it right. This usually works well the first time I give a talk, because everything is fresh in my brain; when I pull out a deck a year later and "tweak" it and then give the talk, I don't do nearly as well.)

Enough rambling about me, what advice do I have about preparing visual materials to accompany a talk?

1. Do keep in mind that the purpose of the visual materials is to accompany the talk, not replace it.

<flame>

One of my greatest frustrations as a teacher is when students believe that the slides are a replacement for lecture. I do not intend them to be. I tell students this. Yet, time and time again, students who cannot be bothered to attend class complain that the slides sometimes are incomplete. That is explicitly by design so that in class we can discuss and probe things!

</flame>

2. If the slides merely repeat what you are going to say, why should people attend your talk?

3. Think hard about how can you use a visual aid to:

A) Burn an image in your audience's brain that will immediately conjure up you and your work (i.e., make your work memorable).

B) Make something abstract concrete

C) Simplify a complex concept

D) Connect multiple concepts 

4. Use slide real estate to communicate with the audience; use notes to communicate with yourself (although it's good of the slides themselves trigger you to remember most of what you want to talk about).

5. Remember that you want the audience, for the most part, listening to you, not frantically trying to read paragraphs of prose on slides.    

How to present your (Systems) Research: Oral Presentations (Part 1)

It has been a long time since I wrote anything for my blog. I have a bunch of half written things, but let me see if I can actually write a complete post!  Well, not a complete post - how about the "first in a series" post?  I'm running a workshop this summer and it's more importanbt that I get little pieces of this out than I finish it all at once, so welcome to:

Part 1: Introduction and The Fairy Tale

It took me a long time as a professor to realize that I spent a great deal of my time working with students teaching them to communicate. Most of our students enter graduate school with excellent technical skills, and most have learned how to teach themselves additional technical skills. However, they have not necessarily been taught A) how to find a good research question worth pursuing or B) how to communicate what they have done effectively. Today's post is about the latter.

I've already written about writing (i.e., how to write a dissertation), so let's talk about how to present orally. There are two different dimensions that we can use to describe presentations: by type and by content. We'll do both. Let's start with the different types of presentation that largely differ in length, which leads to a corresponding difference in the level of detail that can be presented.

1. The Elevator Pitch: this is what you can tell someone in an elevator ride when they ask, "So, what are you working on?" It is also the 1-slide presentation you might give as part of an introduction. In Margo-speak, this is the fairy tale (an idea that will be familiar to anyone who has taken Harvard CS261 or UBC CPSC 508 or to anyone with whom I've written a paper); I'll get to that in just a bit.
2. The Status Update: This is typically presented to someone who knows what you are doing and your goal is to add to their understanding of the project by sharing not simply what you did recently, but what you've learned, how that has advanced the project, and how that leads to next steps. In other words, how did the things you've done in the current time period move the project forward, and what obstacles have you encountered.
3. The Work-In-Progress Talk (WIP): These are perhaps the most difficult. They are approximately 5-minute talks designed to summarize what you are working on now. Many conferences have explicit work-in-progress sessions. However, for years, the majority of NeurIPs presentations were 5-minute Spotlight talks, which were supposed to tell the whole story of a research project. For the most part, you best you can do here is convince someone to read your paper; it's difficult to relay too much technical content in five minutes when you first have to lay out the problem space.
4. The Conference Presentation: This is usually about a 20-minute talk (at least in my field). You get to describe the problem, present your solution, and demonstrate how well your solution works.
5. The Full Talk: This is typically a 45-60 minute talk (although even if you are told 60, I recommend 45 leaving plenty of time for questions). In many ways, this is the easiest talk to give, because you have the luxury of time, and you can tell the whole story.

Let's now disect a talk into its constituent elements (not all of which can be crammed into every talk).

1. The story: I will refer to this as the fairy tale, and I'll talk about that below.
2. An outline: Depending on the length of the talk, you often need to provide some structure, so the listener knows what's coming and how you are going to tell your story.
3. Motivation: What is the problem you are solving and why is it important? You might think it's important, but that doesn't mean your audience will care about it. Your job is to convince the audience that they really care. If you fail to convince them of that, it's going to be quite difficult to keep them engaged, even with the best work.
4. Your work: So, what exactly did you do?
5. The evaluation: How well did your solution work? Ideally you show both the strengths and weaknesses of your approach (although the latter is far too frequently swept under the rug)
6. Related work: How does your work fit into the landscape of existing work?
7. Conclusions: A good conclusion not only repeats what you've said already, but draws some larger takeaways and implications of your work.

If you have read my blog post on writing a dissertation, these parts should all appear familiar.

Now we can examine which elements go into which talks.

 

	Elevator Pitch	Status Update	WIP	Conference	Full Talk
Fairy Tale	✅	✅	✅	✅	✅
Outline	❌	❌	❌	✅	✅
Motivation	✅	✅	✅	✅	✅
Your Work	❌	✅	✅	✅	✅
Evaluation	❌	❌	✅	✅	✅
Related Work	❌	❌	❌	✅	✅
Conclusions	❌	✅	✅	✅	✅

The astute reader will notice that the fairy tale appears in all talks, so let's start there. Humans love narratives. They like stories. They do not like being spoken at -- they want to be engaged and drawn into a compelling narrative. Therefore, if you want to truly engage your audience, you need to tell them a story. I claim that the fairy tale is the ultimate story and is particularly well matched to presenting research.

The Fairy Tale

A fairy tale follows a predictable pattern. It begins with, "Once upon a time." This is context setting. For research, what is the relevant state of the world (technology) from which your work begins? For example, "Computation is moving into the cloud," for work on cloud computing or data center operation. "Eighty-five percent of the world's population own cell phones," for work on mobility or phone apps. "DRAM accounts for 30% of the cost of most data centers today," for work on far memory. "Data centers alone account for 2% of all electricity consumption in the world, and this number is predicted to grow to 8% by 2030." for work on energy efficiency. Notice that this also serves as a teeny bit of motivation - it tells the listener both how to place your work in context but also why your work matters.  Contrast that opening with, "We present a better algorithm for doing X."

Soon after the context setting, a fairy tale introduces us to the villain to be vanquished. This corresponds to the problem we are trying to solve. Try to be precise -- poor performance is a weak villain; making people wait or wasting energy or using resources inefficiently so things are expensive are more compelling than, "We make something X% faster."

And for every good villain, we need a hero -- that's your solution!  How does your solution address the problem you just introduced?  In your initial fairy tale, you need not go into great detail here, just enough to give the listener a flavor of what you've done.

Finally, fairy tales end with a, "happily ever after." What good will come from your work? How does your work make the world better? How does it change that context in which your work took place? Why should the listener care about the work you've done.

So, the minimum story of your work is simply four sentences: the "once upon a time," the villain, the hero, and the "happily ever after."  It's really that simple -- you can write an abstract for pretty much any paper in 4-8 sentences if you can identify these four key points.  Try it!

For a well written paper, you can read the abstract and extract exactly these four elements. For a badly written paper, you will have to wade through at least the entire introduction (and maybe more) before you can extract these four ideas.

So, the key take away for this post is: master the elevator pitch by having a crisp, clear, and easy to understand statement of the four parts of the fairy tale.