Measuring Anything, Before the LLMs

August 14
30 mins

View Transcript

Episode Description

Douglas Hubbard invented Applied Information Economics and wrote How to Measure Anything. In 2016 I asked him how you would measure something everyone agrees is unmeasurable. He asked me how I would measure collaboration, and I gave him the answer everybody gives: count the messages.

Ten years later that is still the answer, it is still wrong, and now it is on a dashboard with the word AI at the top of it.

This episode is an interview from my archive, recut with a new introduction and close. It predates ChatGPT, copilots, and any AI budget anyone had to defend to a board, which is the reason to play it now rather than a caveat about it. Nothing in it needed updating.

In this episode:

* Why there is no such thing as a statistically significant sample size

* Why you have more data than you think and need less than you think

* How to define what you are measuring by the decision it changes, not the thing you can count

* Why refusing to price a human life just means pricing it badly and in secret

* Why you are worse at confidence than you think, and how half a day fixes it

Douglas Hubbard: howtomeasureanything.com

Audio only this week. Full transcript and every surface: https://sigsub.show/episodes/ep-007/

Signals & Subtractions is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit sigsub.substack.com/subscribe
See all episodes