After a very long journey, I finally have my first economics working paper out in the world. You can find it on NBER here. This project has had lots of twists and turns and has been in development for a long time, so it’s nice to have a complete version of it out in the world.

Something that this project–along with others–has taught me is the role of flexibility, adaptability, and iteration when doing creative work like research.

When you’re trying to do something new – develop a new idea, create a piece of art or research etc. – there is a way in which it’s impossible to know the best way to do it at the outset.

Even if you have a clear plan or hypothesis at the start, there will inevitably be things you didn’t think of which reveal themselves in the course of doing – new opportunities and pathways to make the project better, more interesting, more rigorous, whatever.

There is a real art to deciding when and whether to pursue such opportunities. On the one hand, from a project management perspective, these kinds of opportunities can be framed as so-called “scope creep”, which is usually understood as a “bad” thing. We are deviating from the initial plans and goals! We ought to stay the course and maximize our efficient progress towards the goal!

Certainly, there is a wisdom to this kind of practicality, and it makes a lot of sense for certain sorts of projects where it is possible to clearly specify exactly what you are trying to do at the outset. For example, if I am working on a contracted engineering project, there might be a clearly-specified deliverable and timeline at the outset. “Success” for the project means exactly: producing that deliverable in the specified timeline.

But with more creative projects, the goals are a bit different. The fundamental goal of any research project should be…to do good research. Good research is novel and interesting and rigorous and compelling (or whatever values you want to substitute here). And so these opportunities to deviate from or revise the initial plan are often far more valuable, because they are chances to move the project into better alignment with this deeper set of more abstract goals.

In this type of work, I think, there is much less wisdom in just “pushing through” and sticking to the original plan. The iteration and revision and learning are not an exception to the ideal project flow – they are the project itself, and exactly what makes the research good. Of course, there is still a balance to be struck. But my point is that the calculation is a bit different when it comes to research & creative work.

A related point is that I’ve found that the personal motivation structure that works best for research needs to be different than for more clearly-scoped projects. If the motivation is about getting to the “end” of the project – e.g. sharing or publishing a draft or whatever – it’s going to be a struggle. First, this “end” may be far away for most of the project, and so it’s easy to get discouraged or overwhelmed if that’s the focus. But more fundamentally, this mindset is miscalibrated in that it can make revisions and iterations feel like a setbacks, when really they are progress.

For me, a better motivation structure is to stay focused more on the process, and just taking the next step. I have found a lot of overlap with the motivation structure of maintenance or repair. For example, the project of “keeping my apartment clean” is one that never reaches an endpoint. Sure, I can clean things up, but things will get dirty again. It’s a perpetual, never-ending task. Likewise for many repair tasks. I can patch a hole in my jeans; but over time it’s bound to reappear (for the same reasons it appeared in the first place). And so the motivation structure that works for these things is not in the “being done”; there is no “being done”. Instead, it’s in the pleasure of attending to things as they are, and making them a bit better in some little way. I take out the trash, or clean part of the bathroom, or do the laundry, or sweep the floors. Each step is small but rewarding; and attending to them over time brings a sense of groundedness for me. Over time, the apartment “feels clean” or attended to, even if there is no point where this task is officially completed.