Cultivating Your Research Taste

Just as each of us develops taste in books, music, art, and food, every researcher ultimately develops a taste for research problems.  Every researcher should spend some time developing “good taste” in research problems.   The world has many challenging problems to work on, and as researchers, we have limited time and bandwidth.  It’s therefore important to develop (good) taste in selecting problems, so that we end up working on the problems that are worth a significant investment of time and energy.    Many research problems will take years to run their course, so it is worth spending some time developing taste in problems.

Cultivating taste requires aggressive sampling, and both good and bad experiences.

Many professions, ranging from designers to architects to programmers to managers, need to develop good taste.  In this post, we’ll focus mainly on developing taste in research problems, although some of these tips for cultivating taste may apply more generally (in fact, some of the pointers below were inspired from an article I recently read about developing taste in design).

Cultivating your own research taste.  We are not born with good (or bad) taste; rather, we develop taste by way of education and exposure to many different opportunities and experiences.  Just as one might cultivate taste in other areas of life, one must cultivate taste in research.  In this post, I’ll offer some tips for cultivating research taste that I’ve found work for me.  Some of these tips I have discovered (and applied) by way of analogy for developing taste in other realms (e.g., music, food).  I’ve spent a fair bit of effort developing taste in music; where applicable, I’ll draw some analogies below.

• Seek out others with good taste.  Perhaps the most important step for developing good taste is to associate and learn from others who have good taste.  These people are generally acknowledged as “having good taste” and are often easy to identify.  Just as you may have friends who you know are well-read about music, wine, or food (you know these people because you’re always asking them for recommendations), the research community has “thought leaders” who are widely acknowledged as having good taste in research problems.  It’s not too hard to figure out who these people are with a little research.  Ask several colleagues who these people are and see whether trends begin to emerge.  Poke around on Google Scholar and see which researchers in your area have highly cited articles on a particular topic.  Intersect these people with those who share your interests, as well.  Once you have identified others with common interests who appear to have good taste in problems, try very hard to associate, exchange ideas with, and work with those people.  Become an apprentice.  As a Ph.D. student, seek these researchers out as possible advisors.  The Ph.D. years are perhaps the most formative for developing research taste, and your taste will likely be shaped heavily by your advisor, so taking the time to find an advisor who will develop your research taste is perhaps one of the most important decisions you will make as a Ph.D. student.
• Read trend-setting conference proceedings, and develop opinions about research problems and trends in your area. The “top tier” conferences in your area are essentially the Architectural Digest, Wine Enthusiast, or Pitchfork for your discipline.  Track your conference proceedings to determine the research areas that the best researchers in your area are working on.  You don’t necessarily have to “jump on the bandwagon” and start working on any of the research areas that are the current hot topics at this year’s conference—just as you might not spend several hundred dollars on the latest wine that’s reviewed or go right out and buy every new music release—but it certainly doesn’t hurt to learn about the latest trends, even if you don’t always resonate with them.  Exposing yourself to the latest trends and developing opinions about them (positive or negative) is an important step in cultivating research taste.  It’s typically not necessary to read the entire conference proceedings to get a feel for what’s going on in an area; simply looking at the names of sessions and groupings of papers can help you quickly identify areas that are receiving a fair amount of attention from the community.
• Sample and experiment with abandon.  Developing taste in any genre involves gaining exposure to many different examples, good and bad.  Just as it’s much easier to appreciate a truly fine wine, dish, or performance after having seen mediocre offerings, since every experience allows us to better articulate what we do or don’t like, sampling a wide variety of research problems (and solutions) is a necessary step for developing taste in research.  In developing taste in music, I find myself reading continually about new artists and albums and listening to new material that pushes boundaries in new ways, and sometimes subjecting myself to music that in the end I might decide I don’t like all that well.  Similarly, in research, we must continually experiment and sample to develop and cultivate our taste.  It can be tempting (and certainly easier) to “turn the crank” on problems that we know how to solve, but ultimately this will result in research that becomes stale and boring.  As researchers, we should be continually learning about new techniques, tools, problems, approaches, and so forth.  We will likely find that some topics, areas, and solutions we encounter are incredibly boring, but those “bad samples” ultimately help us determine what aspects of a research problem or solution we do or don’t like.
• Keep a list of ideas that you like and exchange your favorite ideas with colleagues.  Keeping lists of things you like is good; exchanging them is even better.  Every year, I tabulate a list of my “top 10” music albums for the year; at the end of the year, my friends and I exchange lists.  The list is fine, but making the list is actually more important.  Knowing that I am going to be exchanging a list of music with friends at the end of the year keeps me accountable and ensures that I am always “on the lookout” for new gems.  Similarly, making lists of neat research ideas and regularly exchanging them with colleagues is another way to ensure that you always have a healthy appetite for new, creative ideas.  I don’t exchange lists of research ideas, although perhaps I should; I do, however, regularly exchange papers, articles, and ideas with a small, trusted group of colleagues, often multiple times a week.
• Attend conferences.  Just as a music enthusiast attends live concerts or a book enthusiast might attend a reading, as a researcher you should attend research conferences.  I think it is worthwhile to attend at least one major research conference in your area every year.  Attending a major research conference ensures that you are staying current with research trends (in case you’d like to “watch the movie”—that is, attend the talks—instead of simply poring over the conference proceedings).  Perhaps more important than attending the talks, however, is meeting other researchers.  Conferences are excellent places to seek out others with good taste, and to have conversations with other researchers that can help you develop opinions about various research problems and areas.  These conversations and interactions are all part of the process of developing taste as a researcher.
• Consider the principles and theories that underpin a specific research paper or project.  No research paper (or project) is perfect.  In fact, most papers are flawed, and many are badly flawed.  Yet, papers are often published not for the particular artifact that they produce, but for some underlying concept or idea that they embody or espouse.   When reading research papers, it is far too easy to be dismissive of a paper because of a flaky implementation, a bogus evaluation, or poor exposition.  Try to dig a bit deeper and understand the value that reviewers might have seen in a particular paper.  For example, a paper may develop a new theory or changes our way of thinking.  It might open new avenues for research.  It might be applicable across many disciplines.  It might frame an old problem more clearly.   Look beyond the flaws of a particular instantiation of an idea and consider whether the underlying theories or concepts of a particular approach or solution have value.

Evaluating your experiences and encounters.  Simply having exposure to many research problems and areas is not sufficient to cultivate taste; you need a way to evaluate each new research problem, paper, or area that you encounter.  Just as a connoisseur develops metrics for evaluating a new piece of music, a new book, a bottle of wine, or a restaurant, we need to have yardsticks for evaluating research papers and problems.  When evaluating research that I encounter, I consider the following questions to help further develop my taste.  The answers to these questions are subjective (i.e., reasonable people may disagree about the value of the same piece of work), but they can nonetheless help you articulate why you like (or dislike) a particular research problem or solution.

• Is the problem important?  Of course, “important” is subjective and defined by your own set of values.  I personally like problems of practical importance.  For example, if the solution to a problem would ultimately improve a network’s performance, security, or availability and could affect the lives of a large number of people in a meaningful way (e.g., developing better spam filters, circumventing Internet censorship), then I deem the problem as important.
• What is the “intellectual nugget”?  I like research problems (and solutions) that have a simple, elegant solution that’s intellectually profound and easy to articulate.  That might sound trite, but there are plenty of research papers (and researchers who write them) that involve a hodge-podge of solutions with no crisp intellectual contribution (e.g., “We encountered problem A, so we applied X.  Then, we encountered problem B, so we applied Y.” Repeat ad nauseam.)  In my opinion, many of the best research problems (and solutions) can be succinctly summarized in a single sentence.  I value simplicity.  You may not, just as some designers like ornate designs and others prefer minimalist ones.
• What is the solution or main conclusion?  Is it important?  Although I believe that the size of the problem is at least as important as the goodness of the solution, it is worth considering whether the solution is important, usable, or worthwhile, as well.  Just as “important” is subjective when evaluating problems, the importance of a solution is also subjective—perhaps even moreso than the importance of a problem (which at least might have the benefit of some community consensus if the problem has been studied for long enough).   Determining the importance of a solution is really difficult, even for people with developed taste.  In fact, sometimes developed taste might allow us to overlook a particularly new or innovative solution, just as we can become comfortable with a particular genre of music and fail to recognize a true gem when something disruptive appears.  The establishment is particularly bad at recognizing disruptive breakthroughs because they are used to thinking according to established paradigms.  Our failure to reliably recognize good solutions is perhaps one of the biggest flaws of the peer review system.  (More on reviewing in a later post.)
• Does the content support the conclusion?  Does the content of the work actually support the solution?  Are the methods sound and state-of-the-art, or have they since been obsoleted (an example of this in networked systems is the now-questionable use of simulation, which has been obsoleted in favor of prototyping and, more recently, operational deployment).  What assumptions does the paper make, and if those assumptions change over time, do the conclusions still hold, and are they important?  (For example, one might ask if a system that assumes the deployment of a particular protocol is still important if that protocol is never deployed, or is currently deployed but likely to be replaced.)

You might have a different value structure for what you think is an important research problem or solution and, hence, you might have different taste in problems.  It’s important to articulate what you think is important, and taking some of the steps outlined above will help you refine your answer to these questions.