Joining the Big Data Brain Drain

So long academia and thanks for all the debates!

Today is my last day of a wonderful three years at the University of Texas at Austin. Starting Monday, following Jake Vanderplas' terminology, I join the big data brain drain. I will be joining Continuum Analytics a their Chief Computational Scientist, with a focus of helping them join HPC technologies to the Big Data world. But that is another story.

My story

I have wanted to be a scientist since I was quite young, okay I really wanted to be a jet pilot but my genes killed that one. I remember the day that I decided to become a computational scientist. It was at the Red Raider Minisymposium on Biological Systems. I heard an excellent discussions by James Keener and Charles Taylor on the role of computational mathematics and understanding cardiovascular systems. Additionally I was basically offered a position at the prestigious University of Chicago on the spot. The idea that I could contribute to treating the medical ailment that has claimed the lives of many in my family. To this end, I was elated when center for biomedical computing at Simula started using the software I helped with in graduate school.

After graduate school, I've had the amazing opportunity to collaborate with scientists from a diverse set of applications and backgrounds. I've traveled to four continents to present my work and collaborate. I've released several coding projects that have pushed the boundaries of computational mathematics field in numerics on accelerators, adaptive meshes, and domain specific languages. Some of these discoveries have even pushed into PETSc, the premier HPC platform produced by the Argonne Lab. Additionally, I have had the privileged of helping with the scientific python community taking the role of advocating and organizing the SciPy conference and the NumFOCUS foundation. In the coming months I will be an editor for the Scientific Software Days issue in CiSE magazine. Its been a jam packed three years and there are only a few things I would do differently.

Despite my deep respect for academia, I have come to terms that it is fundamentally not the right fit. I've had on and off moments where I start something in industry and come back, but this time is much different. After finally getting a rather large grant from the US Army, I had thought that I would finally have some time to determine my own direction. Unfortunately, due to no part of mine, the grant was cut in half. My own deliverables were slipping so the rest of the money was on a slippery slope. The government sequestration and shutdown meant that my money would definitely be out before it could be renewed. Just like that, I went from well funded PI to soft money beggar.

This presented with a choice of bouncing around on other PI's grants and hoping to get a more permanent position or moving to industry. I shopped around my CV to see if I had a shot on the academic job market, but was told that I was not competitive by several friendly collaborators. Most hires, even in computer science and applied mathematics, are people who have been out of graduate school for five or more years. On top of this, the general perception was I am "just the coder" for the work. So when my friends at Continuum Analytics gave me a call to do some of the most ambitious work I've seen, I didn't hesitate.

Call for sustaining career paths for scientific coders

Could I have salvaged my academic career? Absolutely. But the events above makes me stop and ask if I wanted to. Having turned down offers that tripled my salary for years, I've stayed with science to make an impact on the wider world. I think academia does make the world a much better place. Unfortunately, I also see my role as the scientific coder being diminished year after year.

I can point to half a dozen folks I know not getting tenure. People in post doctoral roles for a decade. Families living in different countries trying to keep their academic careers alive. And almost no academic I've ever met has a good work life balance. I'm willing to give up a lot to be an academic, but now that I have two children and one on the way, I'm not interested in giving up as much as it takes.

Even with academia being a hard place to make a career, you add to the fray that the field doesn't acknowledge code as a scholarly work and I'm out. At Supercomputing 2013 I saw three different presentations (WSSSPE, Saul Perlmutter's plenary, and a BoF on Sequencing and Accelerators ) all calling for better coders in science. But as pointed out again and again, the motivating factors are not in favor of better code in science. Without a sustainable career path for those who are motivated to be "just the coder", science will continue to not compete.

One of the more persistent complaints I and many of my colleagues have had is being the lone coder in a larger group. One astrophysicist turned startup monkey once told me, "I'm tired of being the only one in my group that knows what an expression tree is." We need more computer science skills for our scientific staff. Unfortunately, computer science departments across the country are trying to move scientific computing out of their department. With only a few scientific computing departments in the entire US, there is a lack of options for scientific computing professionals.

I have volunteered as much time as I can afford to promoting and training folks like me with Software Carpentry and SciPy conferences. The recent news of the Moore Foundation and Sloan Foundation to further the careers of folks like me is a welcome message. With the increase of science relying on advanced technologies and the increased rate of retracted publications due to software errors, academia must adapt. As a person who advocated for better software at an NSF Supercomputing facility, I can only wonder how long until the NSF makes a center for open sustainable software. Money for the hardware is just not enough to advance the cyber-infrastructure of the US.

Future Advocate

While the door to academia never truly closes, I will have an opportunity to work with an amazing team that is using much of the scientific technologies I've grown to love. A scientist at heart, I plan to continue helping with SciPy and NumFOCUS, but as a industrial scientist with a team of very talented coders. My hope is to keep working with folks in academia to take their ideas to new levels of impact on the world they are passionate about helping.

As the great @dabeaz did while I was a student in his class, I give the 21 gun salute to my academic career. So long and thanks for all the great debates.

Photo attribution

By U.S. Navy photo by Photographer’s Mate 2nd Class Denise Martin. [Public domain], via Wikimedia Commons