“We have an unprecedented opportunity to run A/B tests with online users and innovate more quickly based on actual user response. Microsoft needs to shift the culture from planning the exact features to planning a set of possible features, and letting customers guide us”
-- Ray Ozzie, Chief Software Architect

Do you want to transform the way people build features in websites and online services? Do you want to have impact across multiple products and services such as Live, MSN, and Microsoft.com? Do you want to take an influential role in a strong senior team with huge growth potential? Do you want to be part of the Software+Services revolution? Then join the Microsoft Experimentation Platform team.

The Experimentation Platform enables product teams to radically improve their offerings using controlled experiments with live users. The platform allows testing new ideas quickly using the best-known scientific method for establishing causality between a feature and its effects: randomized experimental design. The methodology is to expose a percentage of users to a new variation of the website or service, measure the effect on the business, and run statistical tests to determine whether the difference in metrics were caused by the change. Experimentation encourages more agile design and release cycles and can transform the way websites and online services are developed. We have been involved in multiple experiments where radical ideas led the introduction of features worth tens to hundreds of millions of dollars.

The platform is to initially enable experimentation everywhere Microsoft has an online presence, whether website or services, then to take experimentation to the developer ecosystem.


As the development manager, you will be responsible for leading and managing the development of a highly-scalable, highly-available platform, comprising of services to conduct experiments, and software to analyze the data, yielding insight through statistical and data mining techniques. Entrepreneurial spirit, ability to attract and mentor top talent, and technical leadership skills are important. You must have very high standards for quality, customer focus, and trustworthy results because experimentation provides the safety net for others to try ideas.
Requirements: