Do you want to transform the way we build online services at Microsoft? Do you want to have impact across multiple products and services such as Live, MSN and Microsoft.com? Do you like building/testing systems that process large streams of data in near real time? Then join the Microsoft Experimentation Platform team.

The Experimentation Platform enables product groups at Microsoft to innovate using controlled experiments with live users. The platform allows teams to test new ideas quickly using the best-known scientific method for establishing causality between a feature and its effects: randomized experimental design. The basic methodology in controlled experiments is to expose a percentage of users to a new treatment, measure the effect on metrics of interest, and run statistical tests to determine whether the differences are statistically significant, thus establishing causality. Additional analysis against the collected data can provide more insights and enable testing other hypotheses. Runtime monitoring, while the experiment is running, enables early shutdown of experiments that are underperforming. The safety provided by this architecture encourages quicker deployments without extensive testing and speeds innovation. Since the platform provides a safety net for other live systems, the platform itself should be highly reliable and scalable. This puts high QA standards on our team.

We are looking for SDETs to build automation that tests both our data collection and analysis subsystems at various levels. The data collection subsystem consists of a highly-scalable, highly-available instrumentation platform built on web services, Cosmos, and SQL Server. Working on this subsystem, you will be testing a pipeline that not only processes data quickly and reliably but is also able to scale up to handle massive websites like MSN.com. You will work with cutting edge technologies including Cosmos and Dryad as you process some of the largest data sets in the world. The analysis subsystem consists of a web application that performs statistical analysis on the data and exposes results to experimenters. Both subsystems make extensive use of web services and are based on .Net and C#.

Our team functions like a startup so you will be joining the test team in doing in whatever it takes - including establishing test infrastructure, rapid automated testing of features/subsystems as required, adapting to changing requirements/processes, writing common reusable test libraries etc. In addition you will also contribute to the design of the product from a QA standpoint. You will be working with very high caliber team in a very flat organization with high visibility. You will be involved in design and coding most of the time, with relatively little documentation.

Requirements for this position are: