How research works

I was going to write a totally different post today but I wanted to share this instead. Today, I had my absolute very last lecture at this university ever and it so happened to be a revision lecture for General Relativity. Toby, the lecturer, gave us some advice that has really strung a chord with me.

Over the last few months, I have really been struggling with my self confidence when it comes to doing science. I get good results but I work very hard for them and I often feel like there will come a point when I just cannot compensate with working harder anymore. I feel like I'm too stupid to do "real" physics. This might have something to do with the fact that scientific papers and books are full of phrases like "obviously", "a brief calculation shows" or "it can be demonstrated". When I actually try to follow what is going on, the "brief calculation" can be several pages and weeks of pain. "Obviously" is often anything but obvious to me. This might have something to do with people forgetting very quickly all the things they didn't know. To be fair, I'd struggle to write a list of mathematical theorems I knew at high school. They might also be trying to make themselves look smarter. I don't know why but it happens all the time. On top of that, there is this odd mechanism that students who do little to no work but still somehow pass are seen as super intelligent. People like me though, who get very good results but work hard for them, are seen as merely hard-working and that's not a character trait mentioned often in the same sentence as "theoretical physicist". I somehow have this feeling that to be a successful physicist, one has to be super smart and always immediately understand everything, at least everything we are taught at this level.

Even though my project was a success, as far as the time limit allowed, I felt like I had somehow failed because I would often stare at my equations for hours and hours, write pages and pages only to throw it all away in the end. I eventually got where I wanted to be but the process was laborious, even though it said "obviously" in the paper (and to be fair, in hindsight, it was quite trivial).  I somehow felt like I wasn't up to the job.

So much for me and my self confidence issues. This is what Toby said today:

"When I say "trivial", what I really mean is "really obvious in hindsight, after having spent hours, weeks or even months shut in my room with lots of paper, trying and trying and trying until I wanted to bash my head against a wall" Because that's what we really do when we do research in theoretical physics. We bash our head against things a lot, stare at equations and throw bits of paper out of the window. And then we finally get to the end, look at the results and go "Well that was really quite obvious when you think about it this way. Let's go and do it again, except with something harder." It's always like that, it is really just a question of where you are in this process. I have already filed many things under trivial, that you still find incredibly hard, but I still feel like this most days. The only difference is that your problems right now look nasty but generally turn out to be okay. In real research, you will find that things look nasty and turn out to be even worse."

I could have kissed him. This was exactly what I needed to hear. If he, as a senior lecturer well on his way to becoming a professor, can work this way then so can I. Maybe I haven't been doing it wrong all along. Maybe I can become a "real" physicist after all.