Tag Archives: Graduate School

What Life is Like in Grad School…

Graduate school is tough. It takes a lot of perseverance and this can be emotionally and mentally draining. While it is important to work hard, it is also important to take care of oneself physically, mentally and emotionally. It is necessary to take vacations, and importantly, to do something else (sports, music, hobbies, socialize, etc.). Otherwise the scenes is this video will become an all to familiar reality…

Although this video is humorous, it does tickle a weird part of the graduate school experience that, unfortunately, too many can relate to!

Goodhart’s Law and Citation Metrics

According to Wikipedia, Goodhart’s law colloquially states that:

“When a measure becomes a target, it ceases to be a good measure.”

It was originally formulated as an economics principle, but has been found to be applicable in a much wider variety of circumstances. Let’s take a look at a few examples to understand what this principle means.

Police departments are often graded using crime statistics. In the US in particular, a combined index of eight categories constitute a “crime index”. In 2014, it was reported in Chicago magazine that the huge crime reduction seen in Chicago was merely due to reclassification of certain crimes. Here is the summary plot they showed:

ChicagoCrime

Image reprinted from Chicago magazine

In effect, some felonies were labeled misdemeanors, etc. The manipulation of the “crime index” corrupted the way the police did their jobs.

Another famous example of Goodhart’s law is Google’s search algorithm, known as PageRank. Crudely, PageRank works in the following way as described by Wikipedia:

“PageRank works by counting the number and quality of links to a page to determine a rough estimate of how important the website is. The underlying assumption is that more important websites are likely to receive more links from other websites.”

Knowing how PageRank works has obviously led to its manipulation. People seeking to have greater visibility and wanting to be ranked higher on Google searches have used several schemes to raise their rating. One of the most popular schemes is to post links of one’s own website in the comments section of high-ranked websites in order to inflate one’s own ranking. You can read a little more about this and other schemes here (pdf!).

With the increased use of citation metrics among the academic community, it should come as no surprise that it also can become corrupted. Increasingly, there are many authors per paper, as groups of authors can all take equal credit for papers when using the h-index as a scale. Many scientists also spend time emailing their colleagues to urge them to cite one of their papers (I only know of this happening anecdotally).

Since the academic example hits home for most of the readers of this blog, let me try to formulate a list of the beneficial and detrimental consequences of bean-counting:

Advantages:

  1. One learns how to write a technical paper early in one’s career.
  2. It can motivate some people to be more efficient with their time.
  3. It provides some sort of metric by which to measure scientific competence (though it can be argued that any currently existing index is wholly inadequate, and will always be inadequate in light of Goodhart’s law!).
  4. Please feel free to share any ideas in the comments section, because I honestly cannot think of any more!

Disadvantages:

  1. It makes researchers focuses on short-term problems instead of long-term moon-shot kinds of problems.
  2. The community loses good scientists because they are deemed as not being productive enough. A handful of the best students I came across in graduate school left physics because they didn’t want to “play the game”.
  3. It rewards those who may be more career-oriented and focus on short-term science, leading to an overpopulation of these kinds of people in the scientific community.
  4. It may lead scientists to cut corners and even go as far as to falsify data. I have addressed some of these concerns before in the context of psychology departments.
  5. It provides an incentive to flood the literature with papers that are of low quality. It is no secret that the number of publications has ballooned in the last couple decades. Though it is hard to quantify quality, I cannot imagine that scientists have just been able to publish more without sacrificing quality in some way.
  6. It takes the focus of scientists’ jobs away from science, and makes scientists concerned with an almost meaningless number.
  7. It leads authors to overstate the importance of their results in effort to publish in higher profile journals.
  8. It does not value potential. Researchers who would have excelled in their latter years, but not their former, are under-valued. Late-bloomers therefore go under-appreciated.

Just by examining my own behavior in reference to the above lists, I can say that my actions have been altered by the existence of citation and publication metrics. Especially towards the end of graduate school, I started pursuing shorter-term problems so that they would result in publications. Obviously, I am not the only one that suffers from this syndrome. The best one can do in this scenario is to work on longer-term problems on the side, while producing a steady stream of papers on shorter-term projects.

In light of the two-slit experiment, it seems ironic that physicists are altering their behavior due to the fact that they are being measured.

Crises in Confidence

While pursuing a PhD in physics, it seems almost inevitable that at some point one will suffer a crisis in confidence. This is usually accompanied by asking oneself some of the following questions, especially if one is intending to go down the academic route:

  1. Am I good enough to be here?
  2. Should I leave with a Masters degree? Have I been in graduate school too long to leave with just a Masters degree?
  3. Should I start developing other skills to make myself a more marketable candidate to pursue other careers?
  4. Do I really like this enough to continue doing this?
  5. Have I made a huge mistake in going to graduate school? My friends who started working right away seem happier.
  6. Is the modern academic climate, where there is pressure to publish, where I want to be?

Obviously, I can’t answer all these questions. Everyone’s answers will be different. The reason I bring these questions up, though, is that they are on everyone’s mind, that is, unless you are going to be the next Feynman. Even the most successful of graduate students will likely go through periods where they are low on confidence.

The only thing that one can do is be honest and work to the best of one’s ability. Despite the immense pressure to publish, I think it is worth pursuing a project that will enable one to say after graduate school, “I accomplished A and I developed skills in B” and not “I published X papers”.

I also think it is worth talking to older graduate students and postdocs about how they combated their periods of low confidence — it may help you get through yours. Talking to one’s advisor about these issues can also help, but be wary that they are sometimes far removed from the graduate school experience.

I think that everyone can and should acknowledge that there is certainly a large element of luck involved in determining one’s scientific path. Sometimes you roll double-sixes and sometimes you roll a two-three combo.

Related: Inna has also written an excellent article about her experience in getting a PhD. You can read it here.

Non-sequitur: I was recently at another beam time run and as with most runs, got little sleep. As fatigue starts to kick in at 4-5 AM, I sometimes (for some bizarre reason) find myself listening to a song on repeat. “Ageless Beauty” happened to be the one this week, a cover of a song originally sung by Canadian group Stars:

Some Questions that Arose During My Graduate Years

Below are questions and concerns that arose during my graduate school career, many of which I still cannot answer. Most of these are specific to an experimentalist, but there are some in there that apply to theorists as well. Here is a list of some of these points:

  1. If I spend my time developing a new instrument and learning how to build a lab, knowing that my publication record will be delayed/suffer as a result, should I take on such a long-term project? Keep in mind that taking on a long-term project will instill me with the necessary skills to construct and design other experiments in the future.
  2. Related to 1: Are experimentalists who prioritize technique development (which may take years) over churning out papers with a few standard experimental techniques necessarily going to suffer as a result? Should the physics community necessarily prioritize one over the other?
  3. How much time should I spend trying to gain a broader knowledge of different subjects in my field and outside my field, knowing that while this may be a fruitful long-term strategy, I am likely to suffer in the short term?
  4. When is a good time to graduate? Is this dictated by the number of papers that I have published or when I feel like I am no longer growing a physicist in my current climate? Should I stay just so that the papers that are in the works get published?
  5. How much time should I devote to theory? Is it worth the time to learn quantum field theoretical methods and the like as an experimentalist? For a theorist: How much time should I spend getting accustomed to the various experimental techniques?
  6. How do I manage the two-body problem in the modern academic climate if my spouse is likely to get a job at an institution where I’m unlikely to receive an offer (or vice versa)?
  7. Is it possible nowadays to make a departure into the industrial sector and make my way back into academia at some point?
  8. If there is a prominent physicist I would like to postdoc with at an institution that does not have the “brand name”/reputation of a premier university, should I still work with him/her? What if the physicist is not particularly well-known, but is prominent in an non-mainstream sub-field?
  9. Are the papers one has published (number, prominence of journals, etc.) the only metric by which to measure the contribution of a graduate student? Have recommendations become more empty?
  10. Should one work in a field that is trendy/popular? Or should one search for physics in places that are less explored, realizing that one will necessarily receive a lower number of citations?

I suspect that many experimentalists (and maybe some theorists) have questions out there that are similar to these. On most of these questions, I think the safest bet is to take the middle path, but one’s natural inclination may be to lean one way or the other. For instance, I lean towards developing new spectroscopic methods (which I find creatively fulfilling), but this can take years, and publication output is not high during such an undertaking. In some sense, it is “safer” career-wise to just perform tried and true experiments on new materials.

I will try in the near future to answer some of these questions, but it should be said that I am far from being an authority of any kind on these topics. These are just some of the questions that inevitably arise in everyone’s mind during their time in graduate school in condensed matter physics, and I thought sharing my thoughts may help foster some conversation.

In some sense, it is a shame that questions like these have to arise in one’s mind at all during graduate school. Ideally, one would like to concentrate on one’s work and scientific output — however, the modern climate is such that these questions become unavoidable especially when one nears the end of a graduate school career.

Please feel free to share more questions/concerns in the comments.