Orbit is a general interface for Bayesian time series modeling. The goal of Orbit development team is to create a tool that is easy to use, flexible, interitible, and high performing (fast computation). Under the hood, Orbit uses the probabilistic programming languages (PPL) including but not limited to Stan and Pyro for posterior approximation (i.e, MCMC sampling, SVI). Below is a quadrant chart to position a few time series related packages in our assessment in terms of flexibility and completeness. Orbit is the only tool that allows for easy model specification and analysis while not limiting itself to a small subset of models. For example Prophet has a complete end to end solution but only has one model type and Pyro has total specification model flexibility but does not give an end to end solution. Thus Orbit bridges the gap between business problems and statistical solutions.

There are many different use cases of time series forecasting at Uber, both strategic ones (long-term) and tactical ones (short-term). This Uber blog post provided an overview of those use cases. Many of them not only require end-to-end forecasting, but also a causal inference structure in order to provide explainability, quantify uncertainty, and perform a what-if scenario analysis. Orbit could improve the quality and efficiency of this process. 

Orbit has a wide range of applications in Uber’s marketing data science team for measurement, planning, and forecasting. Primarily it is used in measuring the performance for various marketing levers at subchannel and daily granularity. 

Orbit enables the easy decomposition of a KPI time series into trend, seasonality, and marketing channels effects. This decomposition enables unbiased forecasting and dynamic insights, including cost curves and ROAS of marketing channels. The forecasting is an important part of planning future marketing budgets and the optimization of spending across different channels and regions. 

Orbit can utilize real world data in multiple formats simultaneously; i.e., incorporating results from previous experimentations along with multiple channels of contemporaneous data. 

What is Orbit?

Orbit stands for Object-ORiented BayesIan Time Series. It is used to conduct time series inferences and forecasting with structural Bayesian time series models for real-world cases and research. Like many other ML use cases, the best model structure is largely dependent on that particular use case and available data. A common approach to determine a well-suited model is to try out different model types, perform backtesting, and diagnose how well they perform. Having one, consistent interface for performing all these tasks largely simplifies our ability to determine the best model for a given use case.

While there are plenty of time series model implementations in the Python ecosystem, Orbit aims to provide a consistent Python interface to simplify Bayesian time-series modeling workflow by linking one command to each step in the following diagram. Check out the How to Use Orbit? section for command examples.

We follow a number of object-oriented design patterns to maintain independence between Estimators and Models. What that means for the end users is that there will be consistency in package usage to accomplish a desired use case. What that means for model developers is that their focus can be on the structure of the model, rather than the Orbit interface or interactions with the underlying PPL.

Estimators are classes that handle interactions with the PPL such as Stan or Pyro. It may seem unnecessary to implement this class if we are using an underlying PPL, however, in addition to the nuances of each PPL, a user must also determine:

We abstract away some of the decisions, and allow a Model to compose a specified Estimator within it. A Model is a concrete implementation of a model structure, and how to perform inference after the parameter estimation. Currently implemented are two main classes of models, named Local Global Trend (LGT) and Damped Local Trend (DLT), which we’ll discuss in further detail below. We found that for our use cases, the current two model implementations performed well relative to other solutions. One important goal of Orbit is to quickly incorporate new models while maintaining a consistent interface. Furthermore, we have modules for backtesting, diagnostics, and visualization that will work out of box with any Orbit model object. To be specific, our package provides:

How to Use Orbit?

Orbit’s APIs are descended to have a similar interface to Scikit-learn (the de-facto standard Python library for general-purpose statistical and machine learning model building). An example workflow using Damped Local Trend (DLT) model in Orbit API is as follows:

For more details, read our quick start guide and documentation page.

After model training, the predictions can be visualized at convenience.

Orbit also provides a rich set of plotting tools to visualize and diagnose the posterior distribution of sampling parameters and the trace of posteriors.

The Orbit package also comes with a general-purpose backtesting utility, to evaluate the time series forecasting performance. Two backtesting schemes are supported: expanding window and rolling window. Figure 2 presented earlier is an animation to visualize the forecasting results by using the expanding window scheme with 6 splits.

A Little Math

Currently, we implemented two major types of Bayesian structural time series models in Orbit:

(With the OO design from Orbit, developers can create their own problem-specific models.)

LGT

LGT model is an additive model with the following forecasting equations:

where ,,, and can be viewed as level, local trend, global trend, seasonality, and error term, respectively.

The update process follows a triple exponential smoothing form:

where , and are the smoothing parameters. They will be automatically estimated during the MCMC sampling process.

Our LGT model is a variant based on the multiplicative model proposed by Smyl in Rlgt. Its advantages over the original Rlgt models are two-fold: first, the computation is more effective with the additive form; second, in Rlgt is parameterized with that introduces dependency in noise generation, while is refined as an independent noise in LGT. With this change, it reduces the computational cost by vectorizing the noise generation process.

One limitation in LGT, however, is that it assumes , . To ensure this condition is satisfied, it requires that , .

DLT

For use cases with , we provide an alternative – Damped Local Trend (DLT) model, which has the following forecasting equations:

Similarly, the update process follows an exponential smoothing form:

We provide flexible choices for the deterministic global trend : the options include flat, linear, log-linear, or logistic. Another important feature of DLT is the introduction of the regression component . This serves the purpose of nowcasting or forecasting when exogenous regressors are known, such as events and holidays.  Without loss of generality, one can assume:

Beyond the normal distribution, there is a rich set of priors for the regression component implemented in the Orbit package, including Lasso, Ridge, and Horseshoe priors. To our knowledge, Orbit is one of the few packages to systematically implement all these priors for structural Bayesian time series models.

Benchmark

In order to evaluate where Orbit stands compared to other time series forecasting packages, we did a comprehensive benchmark study on a rich collection of datasets:

Symmetric Mean Absolute Percentage Error (SMAPE) was used as our performance metric. Following what the competitions suggested, we used 13 forecast horizon and 18 forecast horizon, respectively, for M4 weekly and M3 monthly series with 1 split; for Uber’s datasets, we used 13 forecast horizon, 3 splits with 26 incremental steps for weekly trip series and 28 forecast horizon, 4 splits and 14 incremental steps for daily Eats order series.

We compared the Orbit models, LGT and DLT, to other popular time series models including SARIMA and Facebook Prophet. Both Prophet and Orbit models use Maximum A Posteriori (MAP) estimates and they are configured as similarly as possible in terms of optimization and seasonality settings. It’s worth pointing out that Uber’s data sets cover the Covid-19 lockdown periods across the cities. This was useful for testing how the models would perform with the presence of such extreme events. The benchmark performance table is given below. Orbit models showed competitive results compared to other models, with improved metrics ranging from 12% to 60%.  

Looking forward

We already presented the Orbit-related work at the 40th International Symposium on Forecasting and StanCon 2020, where we received valuable feedback from the conference attendees. Our next major milestone for the Orbit project is to introduce more dedicated Bayesian time series models. As part of this effort, we’ll be focusing on the following priorities:

To learn more about Orbit, be sure to check out repo on GitHub and our white paper on Arxiv. By open-sourcing the package, we are looking forward to more collaborations and adoptions from other Uber Data science teams and the broader data science community.

Acknowledgements

We’d like to thank the following individuals: