Discover how information theory can improve ensemble models for time-series forecasting accuracy.
As we emerge from the societal and economic disruptions of the pandemic, statisticians face significant challenges in accurately forecasting key business variables. The complexities of geo-political events, such as the ongoing war in Ukraine, have only amplified these difficulties. For instance, did rising retail prices in 2022 stem from the war, or was the quantitative easing implemented in 2021 the primary factor? Divergent model predictions complicate our ability to forecast inflation effectively.
The foundation of feedback-mechanisms/">econometrics frequently relies on minimizing distance metrics—like Mean Squared Error (MSE) and Root Mean Squared Error (RMSE)—to determine the accuracy of a model's prediction against actual values within the same domain, be it time series or frequency domain data. These metrics have significantly enhanced forecasting accuracy since the late 1940s, enabling advanced statistical packages to select optimal models, often without requiring the user to assume a specific structure.
However, it's important to recognize the limitations of continuously optimizing traditional metrics. Historically efficient methods offer diminishing returns with each iteration, meaning metrics that measure distance may no longer provide sufficient differentiation among models, which can hinder effective performance ranking.
This article explores two different avenues for addressing these concerns, starting with a contextual analysis of inflationary trends and forecasting accuracy for the Consumer Price Index (CPI).
A deep dive into econometric modeling frequently sees econometricians relying on common variables to construct inflation models. To illustrate, the Bivariate Granger Causal graph comprising four variables provides insightful correlations that enhance our understanding of the economic landscape. Notably, the savings rate can influence both demand and supply sides of the economy, while inflation for producers can feedback into consumer prices. Such insights affirm the importance of selecting appropriate variables when forecasting.
By applying a non-parametric Vector Auto-Regression (VAR) model, it's possible to evaluate the sensitivity of the model to non-causal and randomly induced shocks. A quintessential example resides in the Permanent Income Hypothesis conceived by Milton Friedman in 1957, which posits that a sudden surge in savings (e.g., triggered by COVID-19 stimulus checks) can lead to subsequent demand-pull inflation as consumers inject those savings back into the economy.
While this high-level understanding confirms fundamental economic theories, it falls short of offering the precision required in contemporary econometric demands. Econometricians must reconcile qualitative insights with the need for exact measurements, yet the Federal Reserve's choice to increase interest rates by a consistent 75 basis points each quarter often invites scrutiny. This raises the question: how can multiple models yield accurate, coherent forecasts from the same data?
Issues arise when attempting to effectively differentiate the performance of multiple forecasting models. If traditional accuracy metrics fail to highlight noteworthy differences, how can we potentially weigh the outputs of various models into a cohesive ensemble? This sets the stage for developing an improved weighting framework grounded in measuring performance diversity.
Recent literature has proposed multiple solutions for the ensemble modeling dilemma, each suggesting that viewing the problem through a new lens can yield deeper understanding and adaptive solutions. One such proposition involves forming an ensemble modeling strategy informed by information theory—a complex yet intuitive approach to enhance forecasting accuracy.
Classical forecasting methodologies such as ARIMAX, Exponential Smoothing, Polynomial Regression, and State Space Models operate on the assumption that a time series adheres to a specific structure. In terms of information theory, these models encapsulate three essential components:
A. They observe a source transmitting information. B. Understanding the nature of that information enables future signal prediction. C. Mastery over the source dynamics should theoretically minimize noise in the information transmission process.
To better evaluate differing model performance, we should quantify the information conveyed by each model—essentially recognizing how well each can predict future data points. This quantification introduces the concept of entropy, a vital metric that holds substantial relevance in information theory.
The spectral density, a critical notion in information theory, is defined in the frequency domain and showcases how densely a signal—infused with information—can be distributed across various frequencies. Similar to distance metrics, entropy encompasses properties that present a useful evaluation of model performance separation.
Enter the challenge of incorporating entropy into ensemble modeling. By developing a framework that infers entropy-based ensemble weights, we can enhance our forecast accuracy. This concept, while in need of refinement, strives to deliver an intuitive means of gauging a model's representation of the information-emitting source.
For instance, establishing a minimum entropy threshold of 0.75 during testing reveals insightful results. The out-of-sample entropy of forecasted residuals from the entropy-based ensemble significantly outperforms traditional distance-based counterparts in terms of accuracy, indicating a need for further optimization and exploration.
Comparisons of inflation forecasts generated by both ensemble methodologies demonstrate distinct directions, encouraging a broader understanding of how various models can harmonize to provide more accurate predictions.
Beyond the specific applications for inflation forecasting, the growing complexity of socio-economic environments highlights the pressing need for the forecasting community to rethink established paradigms. Integrating new metrics like entropy into ensemble modeling not only enhances forecast accuracy but reinforces our collective understanding of the dynamics at play.
For practitioners and scholars alike, this forward-thinking approach may offer valuable insights into a broad array of forecasting challenges encountered in the field. As the landscape of data science and econometrics continues to evolve, the encouraging preliminary outcomes of entropy-informed ensemble modeling can spur innovation and lead to more effective forecasting methodologies.
Ultimately, as we recognize the breadth of challenges laid before us, embracing novel perspectives and metrics can guide the way to actionable solutions and increased forecasting efficacy across various domains.
Vedant Bedi is an Analyst at Mastercard, focusing on portfolio development for the North American market. He holds a Bachelor’s degree in Mathematics and Economics from NYU and is deeply interested in the synergies between data science, econometrics, and finance. Vedant is also a member of Phi Beta Kappa, the oldest academic honors society in the United States.