Another box in the garage, another notebook.
This one is from 1996, my first year of doctoral study, and it runs to thirty pages of Krylov subspace methods — conjugate gradient, SYMMLQ and MINRES — worked from first principles out of a 1975 paper by Paige and Saunders.
These are methods for solving Ax = b when A is far too large to invert. You don’t solve the system. You build a small subspace, find the best answer available inside it, then grow the subspace and go again. MINRES is the one that comes with a guarantee: the error never gets worse. You can stop whenever you like and know you haven’t gone backwards.
I did it by hand from the paper. No internet. A biro, a lot of evenings, cheap instant coffee, and a serum nicotine level I am not presently proud of.
Claude and I checked it, thirty years late. Two passes — read every line and re-derive it, then implement the recurrences exactly as written and run them against a direct solve.
The kid held up. I’ll keep calling him that — thirty years is enough distance for the third person, and it lets me hand him the credit and keep the mistakes.
All three algorithms land on the exact answer to about fifteen decimal places, on well-behaved systems and on the nasty indefinite ones that make conjugate gradient fall over.
There are four errors in thirty pages, and not one of them is a wrong answer. Every one is a place where I asserted something true without a valid argument for it — a proof by obviously, where the obvious thing happened to be right. That is the better direction to fail in, and roughly what the next three years were for.
Twice he writes something down and admits he hasn’t earned it. I presume this could be sought by a little induction. I recommend verifying this progression formulation before implementing it! Both turn out to be correct. Knowing which of your own steps you haven’t proved is not nothing at twenty-one.
My favourite thing in there isn’t a mistake at all. Paige and Saunders describe a certain 2×2 matrix as a rotation. On page 8 the twenty-one-year-old has stopped, taken its determinant, got −1, and noted that it is in fact a reflection. He was right — and it pays, because reflections are their own inverse, and he leans on exactly that twice in the pages that follow. He didn’t take the paper’s word for it, and got something for the trouble.
The whole thirty pages exist to reach one line. Everything on the right of it is a sine, and no sine is bigger than one, so the residual is a shrinking product and cannot grow. The quantity that makes conjugate gradient blow up — a division by a cosine — cancels in the very last step.
I don’t remember writing the detail. I remember the nights.
The arithmetic is still good, and the notebook is a PDF now.
I’m not posting this out of vanity, and it isn’t a back-in-my-day grumble. There’s a point in it, and some questions.
Thirty years ago the price of a technical PhD was building your own toolbox. Not reading about the tools — deriving them, from the original papers, until you knew why they worked. This notebook and one other, on a Hamiltonian for two molecular fragments, levelled a small rainforest in scratch paper before I wrote out the 90+ pages of neat final versions.
None of it went into the thesis. Not a line. The algebra would have made far better eye candy than most of what did.
So: does anyone still work this way? Was it necessary, or was it ritual? And where does that work go, given it made me competent and made no publication?
I ask in hope rather than complaint. I want the answer to be yes, they still do — because I think that is how you build a scientist. You put in the work and it makes your mind stronger, even when the particular quest has nothing to do with where you end up.
