Skip to content

Making Better Decisions Faster with Experimentation

Robert Neal  (Head of Data Science, Experimentation, LaunchDarkly)

Location: Room 150B

Date: Thursday, September 28

Time: 10:15 am - 10:55 am

Pass Type: Digital Pass, All Access Pass, Main Conference

Conference Theme: Data Science

Session Type: Session

Track: Driving Growth

Vault Recording: TBD

Audience Level: Intermediate

Traditionally online experimentation has used statistical significance as the gold standard for shipping changes. But it has turned out to be fool's gold. Using statistical significance to decide to ship changes costs businesses a substantial amount in lost value. Moreover, sometimes teams refrain from running experiments because they don't think they'll be able to reach statistical significance or that the experiment will slow down velocity. This results in poorer measurement and less understanding for the business. In this session we'll talk about a few alternatives to statistical significance and then use concrete examples to show how they allow you to make decisions faster with less data, and add more value to the business. Concepts covered will include Test & Roll, Multi-Armed Bandits, and Bayesian Statistics.


Learn how to use your existing experimentation framework to make better decisions
Learn how to think about risk when making decisions under uncertainty
Be able to communicate the value in making data-driven decisions outside of the confines of statistical significance