After attending a Stan workshop given by Charles Margossian at McGill University, Chris Barrington-Leigh wrote:

I just wanted to say that for the first time in three (4!?) years of efforts, I have a way to estimate my model. Your workshop helped me and pushed me to be persistent enough to code up my model. After finally getting it to work late last week in RStan, I’ve even switched over to PyStan now, which means I’m as happy as a clam. Quite giddy, actually!

I can’t believe I don’t even need to code in in all those Jacobians and Hessians I computed analytically last summer…

Thank you! I’m so excited. I have a tonne of reading to do, I suppose, to build up a deeper knowledge of Bayesian approaches, but in the mean time I should have two papers on this stuff in no time, haha!

I’ve been saying to other people that I think this whole phenomenon of top statisticians making this stuff accessible to mortals (and to a wide variety of uses) by putting it all in open source and so quickly from new research to wide availability … is really cool. Like Wikipedia, and like OSS more generally. It’s a bright light.

It’s wonderful to hear this sort of thing. Indeed, sometime in academia less credit is given for (a) teaching, (b) software development, and (c) applied work. So it’s great to hear some appreciation for a combination of these three.

Also, regarding “for the first time in years, I have a way to estimate my model”: this has happened to me too! My colleagues and I have fit models in our applied work (for example, here) that we never would’ve even tried in the pre-Stan era.

What does “estimate my model” mean?

Pretty sure it means estimate the values of the parameters, in this case, sample from the posterior for the parameters.

It can be very difficult to get a good sample from high dimensional parameter spaces. So difficult that people wind up using simpler models that ignore important issues, just because they can get numbers out of it.

Makes sense, just never heard that terminology.

Hope he didnt just constrain the parameter space with arbitrary priors to get convergence. I know thats what I did when first playing with mcmc.

Even more than philosophical bayesian-ness, I think the greatest paradigm shift of Stan, autodiff, and HMC is the reversal of workflow from choosing from a restrictive space of models which fit a tractable asymptotic estimator to building a probabilistic model that reflects understanding first and worrying about estimation properties after.

What’s also important to me in this respect is the ability to understand and explain the model. When fitting, for instance, a (multilevel, ordinal, what have you) regression model with something more advanced than OLS, I’m always putting all my trust into some obscurely named R function and its opaque guts. The only way I can judge its performance os often by inspecting the generated (summoned?) estimates, standard errors, CI’s etc. (Not blaming anybody, this stems from my ignorance of the theory.)

With MCMC, Stan and friends, even though I couldn’t explain how HMC works exactly, I can imagine quite precisely how the estimation works, why it works and how the output (all of which I can inspect in various ways, be it traceplots, histograms, pairs() plots, …) relates to the parameters in the model.

This, to me, makes the model as well as the estimation process (or sampling from the posterior) much more transparent and trustworthy.

This is well put

The credit structure of academia is completely misguided.

But how do we fix it? Needs a revolution. But universities are fairly decentralised. Do none of the academic departments have a critical mass of faculty willing to make large changes to hiring and tenure criteria?

I see nothing other than incremental changes.

Rahul,

There’s no such thing as academic freedom or the sort of independence you’re imagining. The only form of “credit” that matters in the modern research university is funding and funding is in large part predicated on prior funding and publications. The same gatekeepers control grant reviewing and manuscript reviewing and those gatekeepers have no incentive to see that reward structure changed.

The better universities do require good teaching and service but those requirements are just the icing on top of the cake which is continued funding and the publication output required for continued funding.

The objective of a modern university is to fund the administration. Insofar as other things need to be done to get that funding they are done, but only to optimize the administration’s take.

I went to the same workshop and would like to also say how wonderful it was. Charles has such a great teaching style. Highly recommend his classes to anyone who has the chance. Thanks Charles!