I can't remember the last time an LLM failed to solve a verifiable problem for me.¹ A verifiable problem is one where the model can get its own feedback: whether the code compiles, whether the tests pass, whether the number goes up, or whether a proof of a very hard problem is valid. One method of post-training LLMs is reinforcement learning with verifiable rewards, which tunes the models to be good at solving these closed-loop problems. These kinds of techniques have led to breakthroughs like solving a Millennium Prize problem by showing that the Navier-Stokes equations can develop a singularity in finite time, backed by an enormous Lean-verified proof.

Continue reading

I want to start writing more about how I'm using LLMs to create more leverage in my software development workflows. LLM coding agents are getting really good, but I think they still require a lot of work to not create slop in brownfield projects (see SlopCodeBench).

My opinion is that if it's important to my job or someone else's job that the answer produced by a tool is correct, then a human needs to fully understand the system they are building and working in. I don't think I'm doing anything particularly novel, but I thought it might be nice to show what I'm doing to create more leverage, go faster, and be more ambitious.

Continue reading

Last year I gave a list of my favorite SciPy 2017 talks. The SciPy 2018 conference took place from July 9 to 15 and the talks and tutorials are now in a YouTube playlist created by Enthought. I have gone through all of this years talks and watched through any that seemed interesting. Read on for my suggestions!

Overall, it felt like there were a lot of machine learning and geoscience/geo-related talks and packages. A lot of the ML talks seemed to be missing practical insight into applying a specific method or seeing how someone solved a problem in a way that generalizes to other problems. The geoscience ones weren't very interesting to me, but might be worth checking out if you're interested.

Continue reading

Recently there has been some buzz around Jupyter Notebooks in science, especially in light of the LIGO team sharing their detection and analysis of gravitational waves in Jupyter Notebooks. Others have claimed that Jupyter Notebooks will render traditional academic journal articles obsolete. The notebook or literate programming format improves on the reproducibility and disseminating of scientific work, however several key factors limit the notebook as a way to communicate science. In the article, I'll touch on these issues and explain why I believe that the current model of sharing computational science is here to stay.

Continue reading

You may have seen my other post on quickly getting papers remotely through your own library with JavaScript. This method works well if your paper is easy to find and you just need to get around the paywall using the access your institution pays for. Sometimes, especially with older papers, you may be able to find a reference, but not a paper. Libraries can offer great utilities to supporting finding original research articles, but they can sometimes be clunky and slow to use. This post will give an example of using JavaScript to file requests on ILLiad, an article requesting service from OCLC used at Penn State, from only a bibtex reference to the article.

Continue reading

If you've done any computational work on a high-performance computing system, odds are you are very familiar with shell scripting and running code in at least 3 different languages that was passed down through the years. You know the theory of how it all works, but can never find the time to weed through, organize and update someone else's FORTRAN code (that was out of date when it was written in 2002) to use the latest techniques. Atomate aims to solve these problems by leveraging the community developed materials science tools and analyses in pymatgen and the workflow management/database creation of FireWorks to document and allow for easy execution of community standardized and easily customizable workflows managed and written using Python.

Continue reading

The full SciPy 2017 conference took place from July 10 to 16. The talks and tutorials are now live in a YouTube playlist created by Enthought. I have watched most of the available talks that seemed interesting from a materials science perspective, specifically talks geared towards building scientific packages and computational tools like pycalphad. Read on to see the four talks I found most interesting

• Keynote - Coding for Science and Innovation

Computing has been driving forward a revolution in how science and technology can solve new problems. Python has grown to be a central player in this game, from computational physics to data science. I would like to explore some lessons learned doing science with Python as well as doing Python libraries for science. What are the ingredients that the scientists need? What technical and project-management choices drove the success of projects I've been involved with? How do these demands and offers shape our ecosystem?

Continue reading

Whether you are at home or traveling to a conference, it can be helpful to have access to all of the literature online that you would have at your institution. This article covers some easy and practical ways to connect to your institution's library easily. Some of these techniques work on any website, and others are for Google Scholar, or with the Alfred app for the Mac. The examples here are specifically for connecting to the Penn State library, but the methods are general enough to work at many or all institutions.

Continue reading

New researchers, graduate and undergraduate students, spend much of their first 6 months to first year in their research group learning the techniques the lab uses to do their science. This is espeically true in computational fields, where there are numerous softwares (open- and closed-source) using in specific domains. Challenges associated with joining computation-focused research groups are exacerbated by the fact that many undergraduate curricula in applied research fields tend to lag behind in the use of software tools. In my experience, relatively few undergraduates in engineering consider themselves to be proficent software developers (CS, CE, etc. aside). Research groups need better ways to teach new scientists software fluency.

Continue reading