Microsoft's Experimentation Platform

Accelerating software innovation through trustworthy experimentation
Home
Contact Us
Job Descriptions
Cool Things about Microsoft
ExP Articles
ExP Talks & Presentations
ExP Tools
What's a HiPPO?
CIKM 2008: Industry Day Invited Talk
 
 
ACM 17th Conference on Information and Knowledge management, Oct 2008
 
The talk contains several new examples from actual experiments at Microsoft.
 
Agenda
 
  • Controlled Experiments in one slide
  • Examples: you’re the decision maker
  • Culture, OEC (Overall Evaluation Criterion)
  • Controlled Experiments: deeper dive

 

Two key messages to remember

  • It is hard to assess the value of ideas. Get the data by experimenting because data trumps intuition
  • OEC: Make sure the org agrees what you are optimizing