Presenting a Technical Talk

There is a lot of literature on public speaking, but most of it is general public speaking advice and is not geared towards presenting technical topics.  Although researchers need to master general presentation skills as well, I found that Joey Asher’s book, Even a Geek Can Speak, has a lot of good tips on how to present complex technical topics to a general audience.  These skills are useful for academics, who often have to present their research to people outside of their immediate area.  For example, although your audience at a conference talk will be domain experts, almost every other situation where you will deliver a talk (department-wide presentations, job talks, presentations to industry) entails presenting to a significant fraction of people who are not well-versed in your topic.

Presenting a good technical talk involves several different aspects.  Preparation requires (1) careful determination of your audience and your purpose for delivering a talk and (2) ample practice.  The talk itself typically involves both (1) presentation of visual aids and (2) delivery of the talk itself.  We’ll explore each of these aspects in depth below.

Determine Your Audience and Purpose

Academia presents many opportunities to present your research in a variety of settings, and these presentations can have many purposes.  Your purpose and audience determines the talk’s content—including how you motivate your work, the level of detail that you choose to include or omit, whether you show graphs or simply summarize results, and so forth.  Therefore, your first goal should be to determine your purpose and your audience.  You must connect with your audience to deliver an effective talk, and the ways that you connect to your audience will differ depending on your purpose and your audience.

Consider the following purposes for giving a talk, and how you might tailor your approach depending on differing purposes (and audiences):

• Encourage other researchers to read your paper(s), or at least appreciate your work. Many published papers ultimately entail a presentation of the paper at a conference.   In these cases, your purpose may be to encourage other researchers in your field to read your paper.  The talk is, in essence, an advertisement for the paper.  Your goal and purpose is to convince other researchers in your area that you have done something intellectually deep and interesting, and possibly to get them to build on your results.  In a conference talk, your foremost aim is to help other researchers understand and appreciate your work, and to encourage them to read the paper for more details.  Distilling complex research contributions into a short conference talk can be extremely challenging; for this reason, these types of talks require significant iteration.
• Find collaborators.  You may wish to present a piece of work to researchers across disciplines to encourage other researchers to work with you.  For example, you may be working on a particular problem and find that you need to enlist the help of a domain expert in another area.  Sometimes, presenting your work to the experts in that area is a way to “cast a net” to find a collaborator who appreciates your problem and shares your passion.  In this type of talk, your goal is to make your content as accessible as possible to researchers in that area.  If you are presenting to researchers in another specific area (e.g., if you are a networking researcher presenting to theoreticians), you will need to tailor your vocabulary to the audience (avoiding domain-specific jargon at all costs), and it can also help to draw analogies to concepts or techniques in the area that your audience is more familiar with (e.g., a networking researcher may be able to abstract the details of underlying protocol implementations and present a problem in terms of a simple graph).
• Obtain feedback.  In the early stages of working on a particular research problem, you may want to give a talk to obtain feedback on your problem formulation, your approach or solution, or some other aspect of your work.  Designing a talk to obtain the feedback that you want requires engaging the audience on your problem so that they understand what you are working on and care enough about the problem to give you feedback on it.  In this type of talk, your audience may typically be a smaller group of trusted researchers, such as other students and faculty members in your research group, or a small community of researchers who may convene for a workshop that is focused on a niche topic.  You may be presenting to other colleagues with whom you may have pre-existing relationships or trust.  You may also deliver these talks in more intimate settings (e.g., research group meetings, small workshops).  These talks need not be completely polished, but they still need to be coherent and clear—if your audience cannot understand your thought process or cannot appreciate the importance of your problem, they may not be motivated to help, or the feedback that they provide may be completely off-target.

The “Message Objective”: Help the talk listeners achieve their goals.  While each of these scenarios has a different audience and requires a different approach to the talk, all of them require “connecting” with the audience.  Connecting with an audience first requires determining why someone is listening to your talk in the first place.  Typically every audience member has a reason or goal for attending the talk.  A listener’s goal may be to learn more about the topic area (at a conference); to assess you, your research, or your teaching capability (at a job talk); to find new collaborators (at a workshop or working meeting); or to give you feedback (at a practice talk).  In each case, you want to help the listener achieve their goals.  Otherwise, they will feel like they are “not getting anything out of your talk” and quickly tune out.  If you can connect your talk with the listener’s goals, you will have an easier time keeping your audience engaged.

Every talk should have a message objective, which has two components: (1) persuading the audience to do something (read your paper, work with you, give you feedback); and (2) explaining what’s in it for them.  For example, your message objective might be “By working with me on this problem, you’ll get to work on a practical problem in an under-explored area.” or “By downloading my software, you’ll be able to solve the software troubleshooting problems that you’ve been having so much frustration with.”  or “By reading my paper, you’ll learn techniques that you could apply to your own research.” or “By giving me a job, you will gain a great colleague.”  There are a variety of message objectives, but every talk should have one.

Practice, Practice, Practice

Never give any talk without some form of practice.  Typically, you should deliver your practice talk to some of your colleagues.  For “lower stakes” presentations (e.g., working meetings), you may be able to get away with a quick run-through by yourself, but as a general rule, you should always practice.  Any talk represents you, your work, your advisor, and your institution, so it is generally a good idea to incorporate as much practice as you can practically achieve.

For “high-stakes” talks (e.g., job talks, conference talks), where you must communicate the value of your work to an audience with a variety of backgrounds, biases, or both, it is critical to hone your message and delivery.  Talks at conferences are, in some sense, “previews” of what your job talk will eventually become, and they contribute to your reputation as a researcher.  People remember clearly delivered talks; on the other hand, if your talk is not clear, the best-case scenario is that they forget the talk (and that you delivered it).

My own rule of thumb is that if I can mentally go through the talk in my head—in terms of knowing slide order, and roughly how I will start each slide—then my delivery will be smooth.  If I can’t mentally “rehearse” my slide order, I know I’m not that well-prepared.

Practice makes more. Practicing is absolutely essential, but practicing alone is not enough—feedback and iteration is essential.  The old adage “practice makes perfect” isn’t quite right.  If  you practice the wrong habits repeatedly, you’ll merely solidify bad habits.  Therefore, it’s best to practice with other listeners in the room, to determine if your talk needs revisions and course corrections.  If possible, try to have people listen to your practice talk who share the background and biases of the audience that you’ll eventually present to.  If you are preparing a department-wide colloquium or job talk, packing the room with only researchers in your area is not likely to yield the best feedback; rather, you want people from outside your area to also listen to the talk, to simulate the environment, questions, and reactions that you will receive at the talk itself.

Practice.  Then, practice some more.  Depending on your purpose and audience, you may need to practice a lot.  You will not get this right on the first “take”, and, just as you would revise a paper, revising a talk to match the audience and your purpose takes iteration.   Just as you would obtain feedback on a paper draft, rewrite it based on feedback, and then read it again, you must re-deliver your talk after implementing feedback.  In doing so, you may find that some suggestions didn’t work that well, or that you need to refine your implementation of certain suggestions.  Also, every revision disrupts your level of comfort with the talk, because content and flow keeps changing.   Practice talks help restore comfort and confidence with revised slides and material.

Don’t let the lack of an audience prevent you from getting feedback.  Although it is always better to practice delivering a talk in person, in front of a live audience, sometimes circumstances don’t allow it.  Scheduling practice talks can be difficult sometimes, particularly with busy, overcommitted faculty members.  You may happen to be busy or traveling around the time of the conference, and schedules may not line up.  In these trying circumstances, though, you can still do better than simply practicing by yourself (which does not give you the same type of “performance” practice) or mailing around copies of slides (which can often be unintelligible—or hard to interpret or react to—without the appropriate “audio track”).  In these circumstances, I recommend Josh Reich’s “Delay Tolerant Talk Preparation“.

Visuals

A significant portion of your talk preparation will include preparing the appropriate visuals.  The point of visuals should be to assist you in your delivery of material, but the focus should remain on you, your speaking, and the work.  Slides should only serve as visuals to help your delivery; they are there to illustrate, clarify, solidify, or corroborate what you are saying.  Anything that distracts from you or what you are saying is not productive.

Keep it simple and spartan.  Some presentations have flashy graphics, animations, colors, and design elements.  Generally speaking, none of these “frills” matter, and they can actually backfire.  I remember attending one particular conference talk that had particularly flashy animations (zooms, flyovers of network graphs, etc.).  After the talk, nobody was talking about the work; rather, people were discussing how flashy the talk was.  I find that black, sans serif text on a white background with minimal animation works well.  Similarly, keep figures and plots as simple as possible while still conveying the point.  People are bad at multitasking; if they are reading your slide (or otherwise trying to digest it), they are almost certainly not listening to what you have to say.  Keep your visuals simple, so that people can return to paying attention to what have to say as quickly as possible.

Don’t use text when an appropriate figure will do.  If a picture is worth a thousand words, then the limited real estate of a slide suggests that you should use figures whenever possible.  That said, the keyword is “appropriate” figure.  Figures should be as simple as possible.  They should omit details of the concept that you are not going to describe, or are not central to the idea you are presenting.  The goal of a figure is illustration, not necessarily accuracy.  For example, when presenting a network architecture, it may be reasonable to leave out certain nodes, edges, or system components that aren’t central to the idea.

Don’t use a graph when an appropriate phrase will do. Sometimes a graph is a great way to illustrate a point, but presenting graphs is not “free”—they require significant audience effort and bandwidth to digest, so you should always ask yourself if a simple summary of the result will do (especially if the audience can find the details and graphs in the paper).  Writing “our system reduced median latency by 20%” might convey a point much more clearly and efficiently than a complicated graph like a cumulative distribution function, particularly if your audience is not used to reading that type of graph.  Remember that for every graph you include, you need to explain what the graph is, what data or experiment was used to present it, what the axes mean, and how to read it.  If I must include a plot, I always put a box on the slide with an arrow pointing to the part of the graph that I want people to look at that conveys the main result.  If the result is a reduction in median latency by 20%, for example, draw an arrow on the slide with a text box that states the main result.  This way, even if people don’t understand how to read the graph, they might still be able to digest the main result.

Beware small fonts and strange colors.  Fonts should always be at least 24 point font.  As far as colors are concerned, beware the projector.  Every projector is different, and certain colors do not show up well.  Also, some audience members may be color blind.  Therefore, take care in choosing colors that project well, and try to design your slides so that understanding them does not depend on the ability to decipher color.  For example, lines on a graph should not only have different colors, but also different markers for the data points on the lines.   White text on black backgrounds is also generally a bad idea: Black is the absence of light, and projectors are designed to project light. Therefore, black is never truly black, and on some projectors, poor contrast can make white text very tough to read.  Also, people are generally used to reading black text on white backgrounds (books, magazines, etc.), and inverting this mode is distracting at best.

Delivery

Practice.  See above.  It’s worth mentioning again.  A smooth delivery depends on lots of practice.  As you gain more comfort with your talk, filler words (“um”, “like”, “you see”, and so forth) will disappear.

Tell a story.  The human mind is very good at processing and remembering stories.  People love stories and are much more likely to stay engaged if you tell a good one.  Stories always have a beginning, middle, and and end.  A good story also typically also has a “dramatic arc”, which may build anticipation, contain surprises in the plot, and otherwise manipulate the reader’s emotions.

• The beginning of your talk is straightforward: it has a short (and easily repeatable) summary of the topic of the talk and your message objective.  (“This talk is about [topic]… By the end of the talk, [message objective]…”).
• The middle of the talk should start with a “hook” that draws the audience in and helps them identify with your topic or problem.  Tell an anecdote.  Relate your work to a current event.  Present a surprising statistic.  Ask a question.  Do something concrete to engage your listeners within the first minute, or you may never get them back.
• At the end of the talk, restate your message objective and leave them with no more than three things that you want them to remember from the talk. If you have one thing you particularly want the audience to remember, tell them.  Also, tell them what you want them to do as a result of listening to the talk.  Do you want them to contact you? Read the paper? Try out your software? Give you feedback?  If you don’t tell people what you want them to do, they won’t do it.  Also, make it easy for them to do it.  When answering questions at the end of the talk, leave your conclusion slide up with the 1–3 takeaway points, and “instructions” for what you want them to do (e.g., if you want them to download code, provide a URL; if you want them to contact you, provide an email address).

Slow down.  Nobody knows your topic as well as you do.   In a technical talk, people are not only listening to what you are saying, they are processing the information as you present it, typically learning it for the first time.  Talk slowly.  The temptation to race is pretty significant with slides because you aren’t bound to discuss everything that’s on a slide, and you may even forget to explain something on a slide if you are going too fast (in fact, I find that using slides in classroom lectures is much less effective than writing on a whiteboard, which forces me to slow down my delivery).

In addition to cognitive reasons for slowing down, acoustics also play a role.  In a large room with your voice being piped through a microphone, echoes and delays are at play, and speaking too quickly can obscure your delivery.  My general rule of thumb is that each slide should take about 1-2 minutes.  If you are progressing through slides more quickly than that, you’re going too fast.  If you feel as though you are speaking too slowly, you are probably going at about the right pace.

Breathe.  Remember to regularly take deep breaths.  This will slow you down, but it will also help you gain some comfort with silence.  It can be terrifying to stand in front of a group without sound coming out of your mouth, so a second of silence can feel like an eternity.  However, people appreciate a slower pace, and unless your silence is erratic or unexpected, it will likely improve your delivery.  Breathing is also a useful tool for eliminating filler words such as “um”.  The next time you are not sure what to say next in a talk, try: (1) pausing; (2) breathing in deeply.  It takes some practice to breathe in rather than using a filler word, but its effects are magical: with no change to actual content (and no additional practice), your delivery sounds pensive and thoughtful, rather than bumbling and unrehearsed.

Use vocal energy.  Volume adds emphasis.  Whispering (assuming you can still be heard) can act like a magnet, drawing people in.  Speed can energize.  A slower pace can inspire wonder or cause people to pay closer attention.  Pauses are useful for emphasis.   Avoid a monotone presentation.  Some parts of a presentation are clearly more important than others—the beginning where you get people interested, the main result, the “sync points” where you draw in audience members whose attention may have drifted—use vocal energy to keep the audience on track.

Get the little stuff out of the way.  Getting comfortable with the room in advance can help you relax, as well, so that you don’t have to worry about the “little things”.  If possible, go stand on the podium before you give your talk so you can get comfortable with “the view” that you will have when giving the talk.  Make sure that your laptop works with the display to avoid last-minute panics.  Always have a last resort: Put a backup copy of your slides on the Web in a commonly readable format (e.g., PDF), so that, if all else fails, you can quickly load them up on someone else’s laptop.

Relax and have fun.  Remember that, if you are presenting your own research, you know your topic better than anyone else in the room.  Relax and stay confident.  Most importantly, have fun.  With the appropriate preparation, giving talks can be quite a lot of fun.  It’s a way to advertise your work, meet new people, educate, and increase the impact of your work.  It can also be a celebration of an excellent piece of work.  Make the most of every opportunity!

Research Patterns

Research creates new ideas, paradigms, and discoveries.  The process of creating new ideas can seem daunting, and somewhat of a mystery.  From some apparently unstructured environment, the mythical process of research will produce new discoveries and knowledge.  Picking a good problem is one important piece of the puzzle; as I have previously discussed, developing taste in research problems and learning how to identify good problems can take many years of experience (indeed, part of the Ph.D. process itself is gaining experience and research taste).

Before we begin our research careers, we experience structured training: we attend classes, read textbooks, and grind through problem sets in areas that might be called “established science”. In contrast, the research process  appears completely unstructured—researchers create new ideas and discoveries, seemingly from nowhere.  Research papers convey the outcomes of research, but they do not shed light on the process of how the problem was identified or how the solution was devised. (In particular, most papers don’t discuss the countless problem formulations or solution attempts that failed before the breakthrough occurred.)  A paper typically only tells a story about the broader context and contributions of the results after the breakthrough has occurred, with the benefit of hindsight and perspective.  Reading a research paper without insight into the process can make research seem even more mystical.  “How did they think of that?”, you might wonder?  In this post, I’ll try to at least partially demystify the process of finding research problems and solving them.

You can apply many common patterns to find and solve research problems.

Research patterns. In fact, the research process is a lot more formulaic than it might appear.  I find that applying particular formulaic approaches tend to work fairly well for identifying (and solving) important problems.  In fact, I would posit that there are research patterns, similar to the software engineering concept of design patterns.  A design pattern is a reusable solution to a commonly occurring problem.  Example design patterns include iterators (for lists) and locks and thread pools (for concurrency).  In research, we also face common problems: selecting a problem, understanding the problem, and planning and executing a solution to a problem.  Fortunately, there are research patterns—reusable solutions to each of these challenges—that can help solve each of these problems.  Below, I will propose some research patterns for each of these challenges that we commonly face in research.

Finding a Problem

How does one find a problem to work on?  There are certainly mainly difficult problems to work on, some of which are worth solving as a researcher and others which are best left to others.  Developing research taste can help us determine which problems are worth solving, but where does one find research problems in the first place?  Below are a few research patterns that I have applied and found useful.

Hop on a trend; look “upstream” for the trends.  Researchers tend to coalesce around certain paradigms as they emerge.  Now, you could of course try to create your own paradigm, but doing so involves effectively staging a revolution that changes the way an entire community thinks (including established researchers who may be slow to adopt radical thinking, yet hold considerable clout; the topic of revolutionary science deserves a post of its own, and I’ll write more about scientific revolutions in a subsequent post).  Short of setting your own trend, you can very easily look around and identify (and even predict!) the community zeitgeist.  Reading conference proceedings to identify the zeitgeist is reasonable, but this might place you as an “also ran” in an area where most of the initial breakthroughs have occurred.  A better place to identify trends, I think, is “upstream” of conferences. Funding agencies often set research agendas, for example, through “calls for proposals”, and the research community chases those opportunities (“following the money”).  So, for example, if you see a funding call for (say) digital health, it’s reasonable to expect that three or four years down the road, a trend or sub-community may emerge around this topic.  Funding agencies even hold workshops to help shape these trends, before calls for proposals are released.  Those workshops often provide significant hints about topics that might appear in a call for proposals—and, hence, what might eventually become a “trend”.  In other words, trends don’t come from thin air—people define and set trends.  If you can’t be a trend-setter yourself (something that’s very tough to do as a Ph.D. student), it’s best to be on the early side of a trend, and looking upstream can be incredibly helpful for doing so.

Develop a secret weapon and look for nails to hit with your hammer.  Become a domain expert at something, or develop knowledge or resources that others don’t have.  By developing some expertise, you become attractive and valuable to many people, and people will begin approaching you with ideas and want to collaborate with you.  People will actually start coming to you with their problems.  The expertise you develop can come in many forms, and it effectively becomes your secret weapon.  You might, for example, develop a software toolkit or system that other people can build on.  By developing a system that others want to use, you place yourself at the epicenter of new ideas that might build on that system.  Or, perhaps you might become an expert on a particular topic or area—a particular set of network protocols, a particular piece of hardware, a particular programming language, or a particular set of statistical methods.  Once you have developed that secret weapon, you can use it to solve problems that others are not able to solve, or to develop solutions that others might not be able to think of.  Once you have developed your secret weapon, you can begin to look for problems where you might be able to apply your unique knowledge, system, dataset, or expertise.

Revisit old problems where assumptions may have changed. Old problems can be a great source of new problems.  Previous solutions to old problems may have assumed certain constraints about processing power, storage, the cost of memory, the set of prevailing applications or protocols, and so forth.  Yet, underlying technologies are continually advancing.  A problem that was difficult to formulate or otherwise intractable five years ago might suddenly become solvable because old assumptions are now invalid, or because the emergence of a new technology (or algorithm) makes previously hard problems suddenly easier to manage.  It is worthwhile to periodically revisit problems that have remained difficult and unsolvable for many years and to consider whether recent developments have made the problems any more tractable.  Lots of areas of research have unsolved problems (not only theory!).  For example, network management problems have remained particularly vexing in computer networking for a long time, but the emergence of recent new technologies has suddenly provided ways to make traction on problems that were previously hard to even formulate.  Your area may have similar developments, and you should continually be on the lookout for those underlying phase shifts.

Look for pain points; eliminate them.  Automate existing solutions.  It is worth looking to industry (and even other researchers) to identify problems that continually recur.  As programmers, if we have to do something more than a few times, we eliminate that pain point by writing a script.  In research, if the same problem continually recurs and is being solved in the same silly, inefficient, or suboptimal way, certainly there must be a better way.  People in industry often face real, important problems, but are may be too busy to step back and take a fundamentally new approach to solving an old, recurring problem.  Fortunately, as a researcher, you have time to consider whether the problem is being solved in the right way, or whether a “pain point” exists that could be eliminated with a fundamentally new approach.  Along these lines, many existing problems remain manual and painful, and automation of an existing solution can often significantly improve the state of the art.  Yet, automation can be incredibly difficult because it often requires complex reasoning, which may be the basis of an important research problem.

Dream, and maintain wish lists. Many research solutions are ultimately about making everyone’s lives better.  Think about the “grand” research problems—curing cancer, putting a man on the moon, and so forth.  These are dreams that seem like science fiction, yet they are achievable.  Each of these dreams involves the many sub-problems, some of which require the creation of new knowledge.  (If they didn’t require new discoveries, the problems would be solved already, because we’d just “turn the crank”.)  A good way to identify research problems is to maintain a wish list of your own, as well as a list of questions that you’d like to know the answer to.  If you don’t know the answers and can’t easily find them, then the answers may not be known. Answering them is then, by definition, research.  These might be large dreams or questions, or they might just be simpler things that bother you and you wish you had a solution for.  Maybe you’re getting too much email spam and want to come up with a better way to solve it.  Maybe you want to know how authoritarian governments block access to certain information or materials.  A good place to start for many research problems is to identify which problems you would like solved that could make your world a better place.

Specialize the general. Applying constraints to a general problem in different ways can sometimes yield a new class of problems.  For example, in networking, “routing” (the process of computing a path between two points) is a classic general problem.  One can create new classes of problems by applying a constraint to the general problem: routing in wireless networks, routing in sensor networks, routing in delay-tolerant networks, and so forth.  Of course, it’s best to find constraints that reflect realistic and important real-world scenarios.

Generalize the specific.  Some problem areas have been subject to solutions that “chip away” at the problem, offering point solutions, rather than trying to tackle the larger problem area.  When reading a collection of papers on a particular problem, it can be helpful to determine whether any particular paper proposes a general approach or algorithm, or whether each merely offers a point solution to some aspect of the problem.  If particular papers only offer point solutions, there may be room for generalization.

Look for linchpins.  When looking for problem areas that are in need of generalization, I think it is helpful to identify if a problem area has a linchpin, whereby a collection of papers each tackle a problem in a specific way, based on the assumption that some underlying problem is already solved.  That keystone might represent a significant research problem, since so many specific solutions rely on a solution existing.  For example, one (open) problem in censorship circumvention is “bootstrapping”: every paper on censorship circumvention that I have read assumes that there is some way of distributing the software and the initial information about how to configure that software to reach an initial set of trusted parties.  All of the existing systems assume a solution to this problem, but no such solution exists.  A solution to the problem could crack open an entire area.  That’s a good problem to work on.  The theoretical computer science and cryptography communities have some famous examples of this (e.g., P vs. NP, discrete log problem, factorization), but I believe most communities have these linchpin, although they may not be quite as commonly accepted or well-known.

Teach.  I have found that there is an extremely tight coupling between teaching and research.  When teaching concepts, I aim not only to teach the concept, but also explain the rationale behind the concept.  For example: How was a particular network protocol designed?  What is the rationale behind a particular design choice in a system?  In seeking to explain certain concepts or theories, sometimes, we can find that things are difficult to explain.  Sometimes, this reflects a gap in our own knowledge or cognition, but sometimes what we are attempting to explain might actually not make sense—the concept, approach, result, or system you are attempting to explain might in fact reflect an incorrect result, a bad design choice, or (perhaps more likely) one that no longer makes sense given today’s constraints, circumstances, or common assumptions.  When you find yourself having trouble explaining the rationale behind a concept, you may have identified a new opportunity or problem area.

Solving the Problem

If solving a problem were easy, someone else would have likely already solved it.  Therefore, identifying a solution to a problem (much like finding the problem itself) often requires a new perspective or way of thinking.  Fortunately, problem solving also has several research patterns that seem to repeatedly arise.  I have commonly used and observed the following approaches to solving research problems.

Consider related problems. Try to restate the problem you are trying to solve in a different way.  Consider different terminologies and representations of the problem that you are trying to solve.  By changing the form of the problem and trying to describe and represent it in different ways, you might find that your problem matches a general problem that is already formalized.  For example, the problem of identifying whether an Internet service provider is intentionally degrading performance might be referred to as a “treatment” of class of customers.  That terminology might make you think about “random treatment”, a process by which biologists can determine whether a particular drug or chemical has any positive (or negative) effect on a group of humans.  Trying to recreate the conditions of random treatment in a network environment might lead you to a statistical approach to solving the problem of identifying ISP service degradation.  This thought process was exactly that of one of my former Ph.D. students, which led to this solution.

Make analogies. Analogies are incredibly powerful.  We use analogies to learn all the time, because we learn a new concept best by relating it to a concept that we already understand.  Similarly, you can solve a hard problem by relating it to a problem that you already know how to solve.  Analogies often create the biggest breakthroughs when they come from outside of your immediate discipline (these are the conceptual blockbusters that other people often aren’t thinking of because they’re typically not looking for solutions outside their immediate discipline).  Computational thinking was one highly publicized example of applying analogies to problem solving—the notion that concepts that we learn in computer science (sorting, queuing, etc.) can help us solve problems outside of the discipline.  But, these analogies can also be applied in reverse.  For example, researchers have applied concepts from epidemiology to understand how computer viruses spread.  These analogies—when applied well—can also often point exactly to a solution, since the solution that applies to the analogous problem can sometimes be translated to the problem you are studying.

Change the problem to one you can solve. Make simplifying assumptions that violate some of the problem constraints, or define some approximation of the ideal solution.  In some sense, this is the dual to the “finding the linchpin” approach to finding a research problem.  Many of the censorship circumvention papers would not have been written if they did not first assume some ability to securely and covertly distribute software and set up an initial configuration.  By making some simplifying assumptions, these papers have made some progress.  And now, these papers have created a linchpin that’s an important area for new unsolved problems.

Just get started, with anything. Just like writing anything down on the page is one way to overcome writer’s block, starting with any solution to a problem (however seemingly bad) can get you started.  The process of iterative refinement is powerful.  In algorithms, propose a simple (sub-optimal) algorithm, and check its correctness.  Refine until you have something that works.   In data analysis, measurement, or modeling, look at simple statistics of a dataset—timeseries, averages, histograms, etc., and begin to look at anomalies.  (I like to say that an anomaly in data analysis is either a bug or a paper!)  In systems, start with a simple, if imperfect, design.  Try to implement it and see where stumbling blocks arise.  Those stumbling blocks may represent the hard research problems, the solutions to which might result in new discoveries.

Consider nature.  When thinking about solutions, ask yourself the questions: How does a human naturally solve this problem?  How does nature solve this problem?  Certain problems in computer security have been solved by considering, for example, human immunology.  Other advances in miniature robotics have come about by studying the behavior of bees.  We estimate our distnace from a lightning strike by watching the lightning and then counting the time it takes for us to hear the thunder; the time elapsed multiplied by the speed of sound gives us the distance.  This same approach has been applied in location systems.   Considering nature’s approach to solving problems is a specific way of applying analogies.

Work backwards from the goal.  In the words of Yogi Berra, “If you don’t know where you’re going, you might not get there.” I find it helps to sometimes have a desired end result in mind, and then to figure out what is needed to get there by breaking the problem of reaching the end state into smaller sub-problems.  I sometimes use a specific version of this approach by asking my students to draw the graphs that they would like to see in their final paper—complete with axis labels and trends.  Given that graph, table, or result, what data do you need to gather to produce the result?  If you can’t get exactly that data, can you approximate it?  Do you need to develop (or apply) any special analysis techniques to produce the result?  Or, suppose you’re aiming to have a working system that performs some task.  What do you want that system to achieve?  Now, what are the building blocks that need to be in place for the system to achieve those goals?  Those building blocks are either solved problems, problems that you need to solve, or problems that you need to dispatch with some simplifying assumptions (see above on simplifying problems).  Working backwards from your goal in this way can often provide a useful roadmap towards the solution.  It can also allow you to solve smaller parts of a larger problem separately (starting with the parts that are most tractable); sometimes, finding a solution to a good sub-problem can turn out to be a significant research contribution by itself.

Think in speech or pictures. We have all had the experience of trying to verbalize a problem to a friend or colleague, only to find ourselves saying “nevermind, I’ve figured it out” before we even finish describing the problem.  Sometimes, the process of thinking in speech can help us arrive at a solution, because verbalizing the problem can help us make the problem concrete and structured in a way that it wasn’t when it existing solely in our heads.  Similarly, drawing a picture can help us think about how a problem is structured and how various sub-problems and components relate to one another.  While the process of communicating a problem in words or pictures can help us arrive at a solution, it also sometimes helps a lot to have someone who is listening and approaches a problem in a different way.  Describing a problem to a colleague from a completely different area might trigger a seemingly naive question that causes you to break out of your way of thinking or discard your current set of operating assumptions—breaking down mental barriers and leading you to the breakthrough.  On that note, it is also useful to relentlessly come at a problem from a variety of different angles.  When applying one set of algorithms doesn’t work, put it aside and try something else.

Teach (and learn).  Just as teaching is useful for identifying problems, it can also help us identify solutions.  If we are teaching “well”, we are constantly trying to keep abreast of new technologies and building blocks for solutions.  By keeping abreast of the latest technologies, we can stay aware of tools that we can apply to problems where we might otherwise remain stuck.  For example, I had been working on a certain network management problem for some time.  In the process of teaching new approaches to network control, I realized that new programming languages that were being developed allowed us to solve some longstanding scalability problems with the solutions that we had come up with.  Technology and knowledge is never in a steady-state; it is continually advancing around us.  Someone is probably working on a technology, concept, or theorem that could help you solve your problem.  Teaching is a good forcing function to continue learning about these advances and increasing the likelihood that you become aware of them.

Relax, and let your subconscious work.  Above all, immerse yourself in the problem, but stay relaxed.  Take breaks.  Generally, creative insights come when we give our minds a break and let them think without structure.  We sometimes create a process for this unstructured thinking, which we call “brainstorming”.  But, probably better than brainstorming is “brain-resting”, where you take a break, go for a walk, go for a run, take a nap, or just go do something else.  Often, the solutions will come when you least expect it.

Why Get a Ph.D.?

Occasionally professors wear suits on the job.

Someone once told me that he got his Ph.D. so that his job would never involve having to wear a suit or being at work by 9 a.m.  The statement is interesting for a couple of reasons.  On the one hand, the statement captures the career autonomy that having a Ph.D. can afford.  On the other, in my experience, the statement appears to be false.  Since finishing my Ph.D., I have sometimes needed to arrive at work well before 9 a.m., and below is a photo of me in a suit as proof that I didn’t manage to avoid that fate, either (in fairness, I don’t have to wear a suit very often, but the need sometimes arises).

What is a Ph.D.?

As new students arrive on campus this week, it’s a good time to discuss the nature of the journey that many Ph.D. students are about to begin.  It is not really possible to describe exactly what the Ph.D. experience will be like, since every Ph.D. program is different and, more importantly, everyone’s Ph.D. experience is quite personal.  Nevertheless, I wanted to write a little bit about what the Ph.D. is, some good reasons to get a Ph.D., and some good reasons not to get a Ph.D.

• A degree signifying expertise (and experience). The Ph.D. acts as a signal to others that you have been through the particular process of writing a dissertation.  The process of creating knowledge and codifying it so that others can learn from your discoveries is something that every Ph.D. graduate goes through.  It signifies that you have acquired a particular skill set, hopefully involving the ability to identify important problems that are worth solving, a particular set problem solving skills, creativity, and the ability to clearly communicate your ideas.  It signifies that you have been through a unique process of creating new knowledge (see below).  If you wonder why certain employers favor Ph.D. graduates even when the job description does not involve research, this is one particular reason why.
• An opportunity for a certain lifestyle. When I was a graduate student, I always thought that I had the best job in the world (the pay notwithstanding).  Perhaps being a postdoc is the best job in the world, because it has all of the perks of being a Ph.D. student, with considerably better pay.  As a graduate student (or postdoc), you have a unique opportunity to work on whatever problems you find interesting.  And, unlike your advisor, you can work on only one or two things at once and dive deep into those problems, hopefully unearthing new discoveries.  The autonomy that a life in research brings and the ability to work at the limits of your own creativity and ambition are rare privileges that many people do not enjoy in life.  Throughout life, you will meet many people who are unhappy in their jobs.  I can honestly say that I have never dreaded a day of work, and even though professors work very hard, you will be hard-pressed to find a professor who does not like his or her job.  I’m not sure I’ve met one.  I believe the reason for this is that we have jobs that never get boring, and if we are bored, we really have only ourselves to blame, since we can work on whatever we choose.  Being a professor carries certain perks over being a postdoc (e.g., more cachet, more autonomy, more pay), but it also brings with it more responsibility, more committees, and more work that can distract from research.  Despite the fact that professors often have three or more jobs (e.g., teaching, research, committees, and other service work), we are generally a very happy bunch.
• A process.  As I mentioned above, the Ph.D. is a process of creating knowledge.  For the first time in your life, you have a chance to work on a problem that nobody has ever worked on before.  The answers are certainly not in a textbook, and the problem itself may not even be written down anywhere.  You are working at the frontier of human knowledge and understanding and trying to bump that frontier just a little bit further out.  The process of creating knowledge can be lonely and frightening, but many people (myself included) find it very exciting.  When you embark on a new project, you often have no idea what you will find, or whether what you are doing will even work.  The only thing you can be reasonably sure of is that you will learn something.  If that sounds exciting to you, you will probably enjoy the Ph.D.  If working outside the lines without someone telling you what to do and without a clear path to a known end goal makes you uncomfortable, you may find that the Ph.D. is not for you.

What can you do with a Ph.D.?

Academia. Although many people might assume that the main career path after gaining a Ph.D. is to become a professor, the truth of the matter is that most Ph.D. graduates do not become professors.  Yet, having a Ph.D. opens a wealth of opportunities.  Having been through the process I described above gives you an important set of skills that not many other people have acquired through such a long process.  Thus, a tenure-track faculty position is really only one possibility for post-graduate employment.  Many other employers are looking for the skill set that a Ph.D. graduate has acquired.  As a Ph.D. graduate, you have the unique opportunity to become research faculty, or faculty at a teaching university or college.  In fact, the main reason I decided to get my Ph.D. was because I deeply admired my undergraduate university professors and I wanted to emulate them—I thought I would like teaching, and I wanted to teach at the university level.  As I went down the path of getting my Ph.D., I discovered that I loved research as much, if not more, than teaching.  (In fact, I believe that research and teaching are extremely closely related to one another.  More on that in a later post.)

Industrial research labs. In addition to ensconcing yourself in the ivory tower, you can get a job at an industrial research lab (e.g., Microsoft Research).  A job at an industrial research lab often affords much of the autonomy of being a researcher in academia, although there are a few important differences.  One of the main differences is that you are not teaching courses on a regular basis.  Some consider this to be a bonus, as teaching is a significant time commitment; some believe that it disrupts their ability to do research.  On the other hand, I love teaching and I believe that the process of teaching (transferring knowledge) has an important feedback loop with research (creating knowledge), and I think I would really miss teaching if I were in a research lab.  Another important difference is that you are mainly working with your peers (other Ph.D. graduates, often within your own discipline), as opposed to working with a group of apprentices (i.e., your own students) and peers from other areas of computer science and even other disciplines.  Finally, another difference is that you typically do not have to apply for funding when you are working in an industrial research lab.  Many people do not like writing proposals and consider this a bonus.  I personally like writing proposals and consider it part of the research process—writing a proposal is the “planning” stage of research, where you outline a multi-year story and try to convince others that what you want to work on is a frontier of knowledge worth pursuing.  I view proposal writing as a useful tool to make sure that I’m not gaining tunnel vision and merely working “paper to paper”.  Others may find this process burdensome and find a job in an industrial lab more appealing.  Still, it is worth remembering that autonomy does not come for free: someone is paying for it.  If you are not writing the proposals to bring in the funding to determine what you are working on, someone else is bringing in that funding and ultimately may have some sway over what you can work on.  Many times, I have seen industrial research labs having to align their research projects with business units; sometimes the research labs are ultimately swallowed by the business units entirely (or worse, eliminated).  This is less of a danger at well-established research labs, but it can happen even there.

Government research labs. In addition to research jobs, another track you can pursue is a job at national government labs, which are like industrial research labs in some sense but receive their funding from the government, rather than the private sector.  Sometimes, government research work ends up being classified and difficult to publish, but sometimes it is not.  And, many people can find it exciting to work on classified projects.  I wouldn’t know much about this, as I don’t have a security clearance, although we have collaborated on one classified project before (which was an interesting process, since we were contributing to a classified project without being privy to the classified information).

A company of your own. In addition, you can start a company. Google began from Stanford’s Digital Library research project, for example.  Of course, Sergey Brin and Larry Page never finished their Ph.D.  You may find that this path turns out to be right for you.  Sometimes people say that you should work on your second-best idea for your Ph.D. and save the best one for your startup.  I think you should always be working on your best idea, for better ones are probably yet to come.  Needless to say, many startup ideas have come out of the Ph.D. process, and many of the skills acquired during the Ph.D. (creativity, working with uncertainty, communicating ideas, etc.) do transfer well to entrepreneurship.

What a Ph.D. is Not

A chance to take more classes. Some people enter the Ph.D. program as a chance to take more classes.  I have heard of cases of students getting bachelor’s degrees in other areas while completing their Ph.D. in computer science.  This is ridiculous; it is not what you come for, and if your advisor finds out you are doing this, he or she will likely not be pleased.  If you want to take more classes and get another degree, you should just go do that.  The research will be a distraction from your main goal.  I tend to encourage my students to complete their coursework as quickly as possible so that they can focus on research.

A “meanwhile” activity. If you enter a Ph.D. program simply because you do not like the “real world” or the thought of having a “real job”, or because you liked school but can’t figure out what you’d like to do with your life, you will almost certainly strongly dislike your Ph.D. experience, and you will probably not succeed.  A Ph.D. is not well defined.  There are no assignments and checklists.  There is no “homework” in the traditional sense.  Although you have an advisor whose job it is to guide you along your path, the path you take in your Ph.D. is ultimately deeply personal, and it is driven by your passions and interests (or lack thereof).  If you do not have a strong passion or the initiative to carry through your Ph.D. to completion, you will find the process to be very painful.  Your success (or failure) ultimately rests on your shoulders.

It is worth mentioning that obtaining the Ph.D. presents a serious financial opportunity cost.  For five (or more) years, you will be making about 20% of what your peers are making in industry out of college.  Furthermore, you will miss out of five years of raises and promotions on top of that higher salary that you could have been already making.  Ultimately, fifteen or twenty years later, you might close this gap, when you become a tenured professor and consulting and startup opportunities present themselves in abundance (should you decide that you want to spend your time on that).  Until then, unless you happen to be one of the fortunate few whose Ph.D. leads to the next breakthrough startup, you should plan on being financially set back from your friends and acquaintances who are in industry.  That is not to say that you will experience those setbacks, but if your goal is to become wealthy quickly, a Ph.D. is probably not the right career path for you.

A Personal Choice

Obtaining a Ph.D. is a deeply personal experience.  Your advisor will make sure that you are making progress (if he or she is doing the job well), but ultimately it is up to you to shape your experience.  To do well in the Ph.D. program, you need the passion for creating new knowledge and the tenacity to push your visions through to conclusions when the going gets tough.  Many people love the Ph.D. process.  For me, it was one of the best experiences in my life, and it has certainly shaped everything I’ve done going forward.  I would recommend it to anyone who shares many of the same attributes and goals that I have.  But, it’s also important to realize that the Ph.D. is not for everyone.  It might even take you some time and experience going through the process to realize that the Ph.D. is not for you; that is also perfectly alright.  There is absolutely no stigma in avoiding further sunk costs; at any time, you should be setting yourself on a path that gives you the best opportunities for success and happiness.  I hope these thoughts have helped you determine whether the Ph.D. is that path for you, but ultimately, it is difficult to know for sure until you have been through (or at least tried) the process yourself.