Skip to main content
Engineering, Backend

Making Uber’s ExperimentEvaluation Engine 100x Faster

October 3 / Global
Featured image for Making Uber’s ExperimentEvaluation Engine 100x Faster
Image
Figure 1: Parameter representation.
Image
Figure 2: Flipr client parameter evaluation.
Image
Figure 3: Parameter metadata representation.
Image
Figure 4: Experiment and experiment metadata representations.
Image
Figure 5: Flipr client parameter evaluation with experiment override.
Image
Figure 6: Parameter Service Architecture (RPC evaluation).
Image
Figure 7: Initial Architecture: Prefetch parameters mechanism.
Image
Figure 8: Parameter host agent distribution.
Image
Figure 9: Experiment host agent distribution.
Image
Figure 10: Intermediate Architecture: Shadow evaluations in Parameter Service.
Image
Figure 11: Final Architecture: Direct usage of Local Experiment Evaluation in backend microservices.
Image
Figure 12: Shadow evaluation match rate.
Image
Figure 13: Client library version adoption curve
Image
Figure 14: Experiment Evaluation p99 latency.
Image
Figure 15: Local Experiment Evaluation adoption KPIs.
Image
Figure 16: Impact of Local Experiment Evaluation for UberEats store index service.
Akshay Jetli

Akshay Jetli

Akshay was formerly a Sr. Staff Engineer at Uber working on the Experimentation platform and focused on building highly performant, scalable backend microservices and libraries. He is now a Senior Manager leading the Experimentation Engineering team.

Deepak Bobbarjung

Deepak Bobbarjung

Deepak is a Principal Engineer at Uber. He currently works on Uber’s B2B products and platforms with a focus on re-architecting several core entities and systems to unlock both engineering and business efficiency. Previously, as a senior IC of the experimentation platform at Uber, he worked on several aspects of the platform that are used by every team in Uber to conduct effective and accurate A/B experiments. Deepak has expertise building scalable, reliable software systems across a wide array of domains such as Cloud-Agnostic Machine Learning infrastructure, Cloud File storage, Configuration Systems, Virtualization, and Disaster Recovery. He earned his PhD in Computer Science from Purdue University, where his doctoral thesis title was ‘Highly Available Storage Systems’.

Sergey Gitlin

Sergey Gitlin

Sergey is a Senior Manager leading the Experimentation Science team. As part of this role he has led the re-architecture and rebuild of Uber's experimentation platform, and prior to experimentation Sergey worked on subscriptions products at Uber.

Andy Maule

Andy Maule

Andy is a Sr. Staff Engineer at Uber working on Flipr, configuration systems and challenges related to keeping Uber's systems reliable, scalable and productive.

Posted by Akshay Jetli, Deepak Bobbarjung, Sergey Gitlin, Andy Maule