OLS-Regression Forecasting Confidence Intervals Capture Rates: Precision Profiling in the Forecasting Model Selection Process

Forecasting creates projections into an uncertain future. To understand the decision-making implications of the forecast, confidence intervals [CIs] are required. This seems simple enough. However, considering the details of the computations that underlie generating forecasting CIs, one encounters a practical disconnect that can create a decision-making paradox. In the OLS 2-parameter linear regression [OLSR] case, there are two forecasting Models that are often employed: {The Uni-variate Time Series & The Standard two-variable Regression Y:X}. Further, for each of these two Models individually, there are three (1-FPE[α]) %CIs Versions: {Excel, Fixed Effects & Random Effects} each of which is oriented around the same OLSR-forecast value. Given this component configuration, a paradox emerges because each of the forecasting models, {TS or Y:X}, individually produces a forecast with a markedly difference precision profile over the three CI-Versions. In our experience, this is paradoxical as forecasters assume that as the forecasts are the same in each model-class, their Capture Rate—the percentage of time that the actual future values are IN the CIs—should also be the same. To address this seeming paradox, we develop, detail, and illustrate a two-stage OLSR Decomposition and Screening protocol, termed: the [D&S-Triage] protocol that has the following components: (i) Stage A: decomposition of the Model & Version factor-sets to better understand the implications of their Precision differences, and (ii) Stage B: focusing on inferentially significant forecasting model components, create a multilevel quality-algorithm to identify a forecasting model-set that addresses the Quality of the Capture Rate that are the best in their class.


Introduction: Precision Profiles as an Inferential Screen in the Forecasting Context
To introduce the irresistible -innate and sometime inane‖ human disposition to seek a means of fore-telling occurrences, Lusk (2019a) discusses: previsions/ soothsaying/ foretelling/ conjectures/ visions that date from antiquity-Delphi and the Oracles being the prototypical example. These ethereal methodologies have given way to statistical and mathematical methodologies that seem to have their genesis and growth-to-coalescence in the 19 th century. The luminaires of this wave of change from the Smoke & Mirrors of antiquity to reproducible scientific protocols were: Karl Pearson (1857-1936), Sir Francis Galton (1822-1911 and Sir R.A Fisher (1890 -1962). They effectively developed most of the basics that underlie all of parametric statistical analyses that are integral to most Decision Support Systems [DSS] currently in use including: Deep Blue™ i and the now ubiquitous Watson™ ii .
As wondrous though DSS may be as informing the decision-making process there is a contextual drawback. The -glitch‖, in the DSS-milieu, is a legal one founded in the protection of the related intellectual property rights of the Macro-platforms that are the integrated generation functions of the DSS. Simply, very often the information reported by the DSS to the decision-makers [DM] is in summary form and does not include the underlying statistical context imbedded in the Macro-platforms that were used in its generation. In this sense, often the DM is -flying blind‖ in effecting an action plan as there is no probability or likelihood context, empirical or otherwise, for the information produced by the DSS re: the forecasting problem under scrutiny. The guidance needed is one of the four pillars of the Balanced Scorecard [BSC] iii of Kaplan & Norton (1992); in particular, in this case, it is the BSC: Learning Loop [Innovation and Learning Perspective-The time and commitment to generate next generation inferential insights that is at issue. For example, Kaplan and Norton note: During a year-long research project with 12 companies at the leading edge of performance measurement, we devised a -balanced scorecard‖-a set of measures that gives top managers a fast but comprehensive view of the business. The balanced scorecard includes financial measures that tell the results of actions already taken. And it complements the financial measures with operational measures on customer satisfaction, internal processes, and the organization's innovation and improvement activities-operational measures that are the drivers of future financial performance.
Of the four pillars of the BSC, the one that we have selected as critical in the forecasting context is the Innovation and Learning Perspective. This begs the critical question: How can the organization continue to improve and create value in the forecasting context? This is the BSC -Learning Loop‖ that we will address in this research report. Simply, the learning loop is the essential way that the organization learns and adapts to focus on measures that have the best chance of providing relevant and reliable Intel to move the organization effectively and efficiently into the future.
The simple idea for this BSC-platform is that experimental inferential designs that are used in informing decision-making situations need to be transparent so decision-makers could learn from the past and fine tune decision-making for the future. This is the point of departure of our study. Following, we will: 1. Discuss the importance of creating empirical likelihood information in the forecasting-milieu for selecting a forecasting model. For illustrative guidance, we have adopted a market-trading context from the S&P500.

Detail and illustrate the three standard OLS two-parameter linear Regression [OLSR] 95% Confidence
Interval models often incorporated into many DSSs that produce OLSR Confidence Intervals [CIs]-i.e., a statistical likelihood context for the forecasts; specifically, the Excel & Fixed Effects & Random Effects CI-Versions will be detailed, 3. Present the standard OLSR-Model forms: [Y:X]-where the Y-variate to be forecasted is conditioned by the X-variate, and the Time Series form: [TS] where the Y-variate -speaks for itself and is time-indexed, t, over the Panel‖. Each of these two models generates the three above mentioned 95%CIs, 4. Introduce a Modeling Protocol to aid decision-makers [DM] in selecting the forecasting modeling forms that are the best in the class of the OLSR-models tested. This protocol is called: The OLSR Decomposition and Screening protocol, termed: the [D&S-Triage] protocol. There are two Stages for this protocol. Stage A: The Decomposition of the OLSR-[Model & Version]-component set regarding relative precision, and Stage B for the inferentially significant OLSR-components from Stage A, a multilevel quality-sensitive algorithm is suggested and parameterized to evaluate the Quality of the respective 95%CI Capture Rates, 5. The [D&S-Triage] protocol is illustrated for an accrual of firms from the S&P500; the forecasting context selected is a very challenging forecasting context introduced by Lusk & Halperin (2015) and Lusk (2019b) where forecasts are produced and evaluated after Panel trajectory turning points, and 6. Suggest further research possibilities for the forecasting domain.

Overview
We offer, to motive this research report, that there is a dearth of information on the use of the precision of forecasting CIs as a variable of interest in forming a protocol for selecting particular forecasting models-this is the lacuna to be addressed in our research report. We found this lacuna surprising as more than 30 years ago Makridakis, Hibon, Lusk & Belhadjali (1987) [MHLB] offered a very comprehensive set of empirical profiles from the groundbreaking M-study [Makridakis et al. (1982)] on the nature of the capture rates for more than 30 forecasting models. One would have expected that such comprehensive and detailed information would have spawned interest in using CI-precision as a/the/ [at least one] of the selection criterion of identifying forecasting models that outperform others tested-a best in class-set. However, that does not seem to be the case, as a search using: ProQuest™, ABI-Inform™, and Business Source Complete™ for the search terms: yielded only five references, three of which are certainly interesting as a contextual setting iv for this research report; however, none of these offered detailed precision-based empirical protocols or suggestions to inform the decision-making process regarding using precision in identifying forecasting models that are best in the class tested.

Illustration
At this point, it would be instructive, actually essential, to probe the two-stage modeling form of the [D&S-Triage] to better understand the important implication of precision differences in the forecasting context. The inferential information that is required in the forecasting context is the likelihood of a reasonable range for the forecast. These are the confidence or capture intervals, [hereafter: CIs], for the forecast that offer the gambling odds for the values of the population expectation from the specific -Random-Panel‖ selected. This being the case, if different -standard‖ forecasting models have different CI-versions, then this could create an inferential decision-making conundrum. An illustration will elucidate the nature of the likelihood precision conundrum that is due to the three CI  Gaber & Lusk (2017) and Lusk (2017), considering the assumptions underlying the production of the CIs, the orientation that one usually expects is: Where, LL indicates the Lower Limits of the 95%CIs, UL the respective Upper Limits, and Frc the Forecast Value.
For the CMI dataset the actual values are: The forecast [Frc] for all three models is the same, excepting rounding, as the average of the three CIs indicates: AVERAGE [117.21;165.50 Assume that the DM is using a DSS that accepts a TS-Panel from the CMI and then generates a 95% CI that will/[could be used] in forming a stock trading strategy. However, as is usually the case, the DSS produces these 95%CIs but does not report (i) the statistical context or (ii) assumptions upon which the computations are based and (iii) there is no related Capture Rate information created. Therein lies the morass. Using Schematic B the conundrum is evident. It will be the case that the Capture Rate will be directly related to the Precision. If the decision-maker just uses that Capture Rate to select the forecasting model for future use, the Excel Version most always will be the preferred choice-it is the widest so it captures relatively more of the future values. What is needed is the Quality of the Capture Rate vis-à -vis the relative Precision. Also see: Chen & Chen (2016). Therefore, the judgement of the forecaster re: the forecasting problem at hand considering the Quality of the Capture Rate is needed to provide a logical platform for selecting the forecasting configuration that is the best in the class after considering the Precision of the CIs. Although this sounds logical it is more complicated than it appears. The reason for this is that precision is impacted by: (i) the nature of the Panel, and (ii) the appropriateness of the assumptions underlying the statistical configuration of the CIs that is used in the forecasting problem. In order to make a logical inferential choice, one needs to create an inference profile-set for the various components of the forecasting models under consideration, the basic information of which is: The relative Precision-specifically, the precision benchmarked by the forecast, and The respective Capture Rates scored by the DM for their Quality.
By analyzing the actual/empirical inference profile sets as conditioned by the nature of the components of the forecasting models, the forecaster can arrive at a model configuration that performs best in the class of models tested-i.e., an empirical triage protocol in the inferential set tested. Thus, the focus of this research report is to detail: How Precision & Models & CI-Versions & Capture Quality all need to be evaluated in making an intelligent selection of a best in class forecasting model.

Statistical Caveat
This empirically based protocol-focus is a practical alternative to the following -Theoretically Correct CI-selection Protocol v ‖: The Forecaster should/must ascertain: (i) the likely statistical profile of the selected Panel segment vis-à -vis the population set-this could be parametrically Ergodic or Non-Ergodic in the usual four statistical moments, and, for robustness, one must also consider appropriate non-parametric alternatives, (ii) then, specify how this sampled -Ergodic‖ profile set relates to the statistical inferential assumptions of the generating functions of the CIs underlying each of the six components of the Models & Versions, and (iii) finally, based upon these likelihood assumptions & context fit, select the best, usually likelihood,-in the class-Model&Version&CIs protocol. Based upon our decades of experience in the forecasting milieu, we offer that in the history of humanity to date no forecaster engaged in the practice of forecasting has ever followed such a protocol; simply, forecasters rarely have sufficient time, not to mention the expertise, to effect such analyses, and there are no DSSs, of which we are aware that provide such details. For this reason, we offer that the [D&S-Triage] to be detailed in this research report is a practical alternative to profiling the Assumptions and Panel-Context fit to arrive at the likelihood match for selecting the Model and CI configuration. Following, we will discuss how these various components can be woven into an empirical inferential protocol to select a CI-forecasting set to inform the forecasting-model selection process as required in the BSC re: The evolution of the next generation.

Overview
In this research report, we will assume that the forecasting model class of choice is the OLS two-parameter [α: Intercept and β: Trend] linear Regression model [OLSR]. In this class there are two models that are usually considered the standard fare: Time Series models, where the X-variate is the time index, and the Y:X Model where the X-variate is a conditioning variate for the Y-independent variate. These are noted respectively as: {TS & Y:X}. For each of these two models there are three (1-α)%CIs versions: {The Excel, The Fixed Effects & The Random Effects}. Without loss of generality, we will use the 95% CIs as this is the default in many DSS as well as in the Audit context. We will present these Models and Versions in detail and illustrate them using the following Cummins™, Inc. data Panel Cummins, Inc. [CMI: https://www.cummins.com/ S&P500 TS[t=1]:1June2006]. We will use the first ten Panel values in the OLSR-fitting and for the 95% CIs. Thus, the three S&P500 values in italics: {Y i:11,12,13 } are holdback-prices and may be used to evaluate the Capture Rate of the 95% CIs; the bolded value is the Turning Point [TP] identified using the Lusk & Halperin (2015) Screen. Note: after the TP the trajectory of the CMI-Panel changes dramatically. Thus forecasting after a TP will be a challenging test of forecasting acuity.

Regression Models & 95%CI Versions
Following are the three frequently used CI-variations of the two OLSR models regarding the creation of 95% Confidence Intervals [CIs].

OLSR inference from the Excel Parameter Range Model
The Excel Regression functionality forms a -wide-covering‖ confidence interval. See Lusk (2017) and Gaber & Lusk (2017). These 95%CIs are effectively extreme case CI-scenarios as they are produced from the crisp-end-point parameters of the 95% CI for the intercept and the slope jointly. Here we offer the following notation: Extreme Left Side [Lower-Limit [LL]] 95% Boundaries: is the t-statistic for inference for the two-tailed 95% CI that has df = [N-2]; N is the last time index in the data-stream-in our study 10.

OLSR inference from the Expected Random & Fixed Effects Model
The OLSR assumption, in this case, is that there is a random sampling possibility from a well-defined population of variable data streams of realizations, i.e., a population set of: { ; = + which constitutes a large blocked or stratified group of -like‖ firms or repeated bootstrapped Panel sets of sufficient size. In this case, the forecaster believes that the sample estimate and the related CIs are formed under the expectation, E[ ], i.e., the forecast of the mean of ; this is termed the Random Effects [RE] assumption. Experience suggests that the RE assumption is possible though not likely in many forecasting situations context. In a market-Panel collection such as NAICS or SIC groups, the RE event is not unrealistic. Nevertheless, in the service of completeness, the confidence interval for the client value in the RE context for the TS model is for H1: Where: ̂( = +1) is the value of the fitted regression projected for the next X-index time-point [H1] or + 1 using the parameters produced from the OLSR fit for the N data points; = ,∑ 2 − ,,∑ -2 -/N-; =OLS[N]: 1/2 ; and ̅ is the Mean of the time index for the N data points.
For the Y:X version, H1[ 11 ] is: is the value of the fitted regression projected for the next X-index time-point H1, using the parameters produced from the OLSR fit for the CMI-data points; = ,∑ 2 − ,,∑ -2 -/N-; =OLS[N]: 1/2 ; and ̅̅̅̅̅̅ is the Mean of the CMI-values use to fit the regression; and H1 is the first point after the last data point used in the regression.

OLSR Inference from the Fixed Effect Projection
The assumption is that the object of interest is the j th firm with a single set of data stream realizations: i.e., the set of: Where this j th Firm has longitudinal dummy-variable integrity from all of the other firms in the population and so the projection is not the average of all the firms but only for that j th firm. In this case, and given the usual assumptions rationalizing the OLSR of the time-series, the confidence interval for the extrapolation for the next point in the firm time stream is: For the RE version the only change is in the Penalty Parameter-that is:  (1 + 1/10 + 163.2/700.8)] = 1.2

Point of Information
All of these computations are programmed in a VBA-DSS that creates the required Model&Version information and, thus, information at any horizon. Following, we will detail the multi-stage protocol, [D&S-Triage], that will Decompose and then Screen the accrued forecasting data profiles to generate the next version of the forecasting model that will be inferentially the best in class tested.

Overview
Recall the context for this research report. For the Stage A analysis, we are interested in the nature of the precision differences for the six different test-sets: In this context, so as to form the initial robust-profile for the forecasting model components in the spirit of improving the decision-making forecast model selection process, we explore the inference for the precision differences over the accrual set of information using the standard ANOVA-model and the related non-parametric models found in: SAS™ [JMP™v.13[2017]; we mention the JMP-inference platforms to: (i) note that the inferences to be reported herein are founded on basic or standard assumptions that are detailed in most every introductory statistics text, and (ii) thus, provide a simple replication-link vii . The logic of focusing on inferential precision differences is that if it were to be the case that the various standard comparative Nulls could not be rejected, then detailed triage would be moot. As a final note, we elected to use a forecast Panel of three one-period-ahead forecasts: {H1; H2 & H3}. This is the short-term forecasting horizon Median-profile used in the Collopy & Armstrong (1992) and the Adya & Lusk (2016) studies. This horizon election will likely control the OLS-variance and so will enhance power; it is well discussed in these studies that long-forecasting horizons invite high forecasting error. As noted by Makridakis & Hibon (2000, p. 452): -the accuracy of the various methods depends on the length of the forecasting horizon‖. These are the same reults reoprted by Brodie, Buccellato & Scheffel (2011, p. 131) who report: The main objective of this work is to explore the predictive power of individual firms' turnover and capital expenditure series based on four key business surveys administered by the Office for National Statistics. These are the monthly MPI and MIDSS, the quarterly CAPEX and the annual ABI. The results suggest that the predictive power of the data starts to drop substantially after the end of the second year and the beginning of the third.

Stage A: Details and Principal Inferential Tests Addressing Precision Decomposition
The above CMI-set of detailed information is informative not only as a computational exercise for instructing these forecast models in an academic class setting, but also to enable replication and encourage further studies. For this research report, with the above details as the context, we offer the nature of the inferential tests that we will use to determine if there is inferential differential evidence from these empirical datasets re: Relative Precision. Specifically, for the CMI data for the Y:X Model; Fixed Effects 95%CI Version for the first horizon the values for the Precision are: For the Stage A partition, noted as [PT] the analyst will block the forecasting components for the inferential testing environment. Initially, we will use the One-way parametric ANOVA and the related Non-Parametric alternatives for the following One-Way blocking: Given these One-Way Mean/Median difference test results, we will glean the possible effects of the blocking and thus collect information as to the interaction-effects that are likely to be productive to consider in the full n-Way ANOVA. This test will be used to clarify the inference structure to be used in forming the information set for the Stage B where the Quality protocol will be developed.
PT[4] Using the above datasets, we will compute the n-Way parametric-ANOVA with the selected Combinatoric Interaction.
With the above discussion, and recalling that all of this RPrecision information was produced by a VBA:DSS, available at no cost as a download, where the run time for these datasets is less than 3-seconds, we now present the results of the accrual of the 32 S&P500 Panels which will be the accrual test-set drawn from the firms noted in the Appendix.

Horizons [Times: H(t) =11, H(t) =12 & H(t) =13]
We elected to start where it is most likely the case that there are no inferential differences. Experience suggests that for the short-term projection frame there are rarely overall RPrecision inferentially important differences for the Models&Versions. The reason for this is that the error-penalty component is relatively bounded in the short-term. Where there are likely to be major RPrecision differences are in the long-run-effectively after the 6 th projection frame. The Horizon-ANOVA results are presented in Table 3. There is no evidence that there are RPrecision differences in the short-run forecasting horizons. This means that the n-Way ANOVA may be logically consolidated; this will reduce the effect-set of interactions and thus increase the Power and so make the Quality profile easier to configure. Vetting note: The expected increase in RPrecision as well as its variability over the projection horizons is in evidence although not tested inferentially.

ANOVA [PT [2]] RPrecision Re
Forecasting Models: [TS & Y:X] There is copious experiential and theoretical information that indicates that, in the OLSR context, the Time Series and the conditioning model Y:X are, in practice, almost exclusively used as the basic forecasting models. In Table 4 their RPrecision profiles are presented.  (2017) and Gaber & Lusk (2017) studies give preliminary information that the Excel version is likely to produce relatively wide CIs as the Excel-computation uses the extreme case for both parameters in the computation of the CIs. We shall re-consider the utility of this usual assumption in the summary analysis. Further, mathematically, the RPrecision of the Fixed Effects is always greater than that of the Random Effects. Therefore, in this blocked case, one would expect the following relationships presented in Table 5 over the One-Way ANOVA:Versions. , in these cases there will be a 2-Way layout with interactions in the ANOVA context. In addition, we will use the parametric ANOVA-model, as it is the only standard form that is usually part of most of the statistical platforms. Recall that we used robustness tests in the One-Way context as the Wilcoxon & Kruskal-Wallis [Rank-Sum] test is a part of most of the inferential platforms.
The main effects had overall p-values <0.0001 as expected from the One-Way results presented above for the Least Square Means [LS-Means] as reported by the ANOVA. The six-matched interaction profiles presented in Table 6 and screened using the HSD test-matrixes are most interesting. Implication Although this appears complicated to comprehend in the ANOVA: T-K profile-links, the RPrecisions follows the logic of the LS-mean differences of the six-ANOVA-partitions.
The summary implication of Table 6 is that clearly the Y:X, Excel version exhibits, on average, a CI that is very unlikely to provide useful information; this is consistent with the vetting expectation. However, as indicated by the p-values, the LSM-profile suggests that there may be information in the 2-Way results useful in configuring the Quality Profiling stage. Pedagogical Note: We have found that this set of T-K variables for the 2-Way ANOVA interaction [ Table 6]is understandable to the students in our Accounting Information Systems [AIS] undergraduate course. Consider now the Stage B formulation.

Stage B
The Variable of Interest for the n-Way ANOVA-partition: Capture Rate Screening Protocol. It is standard to post-audit the forecasts that are used in creating decision-making information; this is the learning loop that is recommended by Kaplan & Norton (1992) as noted previously. To underscore the importance of the Balanced Scorecard, we offer a vignette of an interview conducted by Prof. Dr. Kaplan (1993)  Larry D. Brady: Although we are just completing the pilot phase of implementation, I think that the balanced scorecard is likely to become tbe cornerstone of the management system at FMC. It enables us to translate business unit strategies into a measurement system that meshes with our entire system of management. For instance, one manager reported that while his division had measured many operating variables in the past, now, because of the scorecard, it had chosen 12 parameters as the key to its strategy implementation. Seven of these strategic variables were entirely new measurements for the division. The manager interpreted this finding as verifying what many other managers were reporting: the scorecard improved the understanding and consistency of strategy implementation. Another manager reported that, unlike monthly financial statements or even his strategic plan, if a rival were to see his scorecard, he would lose his competitive edge.
This is the essential issue that we have addressed in our research report-to wit: how to effectively and efficiently focus the decision making activity of the organization. As expressed by Mr. Brady, the BSC uses screens intelligently selected to ferret out the essential details that are likely to matter in forming actions plans. This is the intent of the [D&S Triage ] and the result of its execution. By identifying the essential components of Quality forecasting models from the past this information can be used in the BSC: [Learning and Innovation Phase] to guide future actions. Specifically, after the forecasts and the respective 95%CIs are produced, then, when the actual realizations are known, the capture information would be recorded. Overtime, there will be a number of such forecasts realizations and so many Capture Rate Profiles can be produced and analyzed. At that point, the forecaster can consider the various capture profiles relative to the particular parameters of the forecasting protocols employed using the [D&S Triage]. This is the Learning Loop of the BSC that enables DMs to make better selections of forecasting models in the future. Consider now an illustration of the Learning Loop for the accrual datasets that we have created.
As we discussed in the introductory section, the driver of the analysis is the Quality of the Capture Rate vis-à -vis RPrecision. In this case, this ANOVA-interaction analysis also needs to be parametrized re: Quality. We have selected the following Quality profiler to screen the CI-capture event for the particular forecasting protocols. An illustration will elucidate these ideas.

Scored Quality of the Capture Rate
A very simple, intuitive, and relevant method of scoring the capture rate is to use the general IF(AND) Excel-screen configuration as our Quality profiler. Specifically, for each of the THREE Horizons, we used:   The actual values of the Ryder dataset are as noted in the first four columns of Table 7: All of the CIs are the 95%CIs produced using the Excel-TS: Forecasting platform. As noted above, as it is the usual case that the magnitude of all of the Panel variables is different, it is necessary to create a normalization so that all of the forecasts can be compared. The simplest calibration is to use the Forecast value. Therefore, the RPrecision in This profiling would be typical for inspecting the contribution components of the various forecasting protocols. These data-profiles should be aggregated to create the valuable post-forecast audit inferential information that will be used to select the forecasting model in the future. This is, of course, the reason for the follow-up in the post-auditing phase or the -Learning-Loop‖ offered as one of the platforms of the Balanced Score Card.
Given the ANOVA profiles presented and discussed above, in particular the 2-Way ANOVA-effects with Interaction, Table 8 was created using the screening function, Ex[A]. This suggests, strongly, that the Null is not likely to be rejected-and so there is likely no difference in the Capture percentage between the TS-Excel and the TS-FixedEffects trials relative to precision. This is very important as there IS a tendency to reject out-of-hand Excel-versions assuming that their relative precision is excessively large. However, in this case, considering the Quality of the Capture Rate, this is not born out. But rather, either of these models would be reasonable choices if a capture rate of on the order of 25% to 20% were to be acceptable.

Inferential Impact Re: The Screening Protocol
There is a DM-judgement aspect to ferreting out the best in class forecasting-model-set. This is the case, as even for this reduced-set where the horizons were not in the possible selection-mix, there were, nonetheless, 15 pair-wise comparisons [ 2 6 ]. Thus, our common-sense recommendation is to select a few of the Model&Versions that have the highest Quality Capture Rates. This will reduce the inferential set and make the choice even more logical and defensible for the forecasters. In the illustrative case, there were, in fact, the following three Quality  Table 9. ] protocol to identify the best among the various possibilities tested. Sometimes the best is not, in fact, useful given the a priori forecasting requirements. This is, of course, valuable information.
2. There is a strong inferential indication that the 95%CIs of the RE-Model are too narrow to garner any serious interest in the RE as forecasting model in a challenging context, even if the stringent assumptions for its use were to be founded, 3. There is also a strong inferential indication, albeit idiosyncratic, that the Y:X Excel version has a precision profile that renders its use moot for this accrual set; this was also suggested by Gaber & Lusk (2017) in another context, 4. The Lusk (2019b) report was not accurate in characterizing the CCPL as not a useful conditioning Market Navigation Platform-variable. Rather, as an elaboration, it seems that in the TP-context using the CCPL as a conditioning variable did in fact emerge as a possibility not inferentially different from the TS-versions as noted above. However, in a portfolio profile, the return would have to be commensurate with the relatively low capture rate-specifically: the Risk & Return where the Quality Capture Rate is at best 30% [i.e., much less than 50%] and so may not be sufficiently high to justify using this model in the TP-context, and finally 5. It is also recommended, based upon our experience, and consistent with the dynamics of the BSC: Learning Loop, that every six months or so in a normal economy where there are no -catastrophic‖ events such as that created by the Money-Mad-Morons of Lehman Bros™, LLP aided and abetted by an shockingly inept SEC, that the [D&S Triage & Ex[A]] protocol be recalibrated using an up-dated longitudinal dataset starting at the end-point of the previous longitudinal dataset.

Summary
The purpose of this research report was to offer an experientially based protocol on the use of confidence interval precision variations to inform the forecasting decision-making process. Thus, we offer that the [D&S Triage & Ex[A]] protocol is a comparatively reasonable, systematic, and time efficient triage-protocol to arrive at a forecasting model that is the best in class tested. Recall that in the introductory section, we noted that a comprehensive set of searches in the peer review literature found no protocols on calibrating the utility of forecasting CIs relative to the nature of their precision for guidance in selecting a forecasting model. In Stage A-i.e., the decomposition stage, 42 statistical combinations were initially identified blocking on the combinations for the horizons. Such a profile-set would be a very challenging Human Information Processing task for even the most skilled decision-makers. However, with a carefully designed decomposition using the One-way and n-Way ANOVA partitions, it was possible to make sense of the various precision profiles. In the example presented, we first tested the various One-way AVOVA effects; we found that there was no evidence that there were relative precision differences over the horizon-set. Therefore, we eliminated them in the 2-Way ANOVA; this reduced the effect-set from 42 effects tests to the six-effects produced in Table 6 [42/ [Sum[ C i 3 , i=1,2,3]].

Overview of Stage B
Once the important inferential components are identified, then using a holdback Panel, a simple Quality algorithm was parameterized to create a second set of statistical profiles so as to select the CIs and the forecasting model forms that are experientially the best for that forecasting context respecting the Quality of the Capture Rate. In this case tested, the following three viable alternatives: TS[Excel] & TS[FE] & Y:X[FE] were found to be the best in the classes tested.

Outlook
The Balanced Scorecard teaches that in a dynamic context, analysis and innovation are required to promote and evolve operating effectiveness and efficiencies. In this vein, we would hope that more research is produced on the OLS-model variants, such as: Moving Average Models, models in the Exponential Smoothing class, in particular the: ARIMA (0,1,1)/Simple Exponential, ARIMA(0,2,2)/Holt Model [and the Winters variant] or ARIMA(1,1,2)/Damped-Trend, as well as Judgement Models such as Rule Based Forecasting (Collopy & Armstrong (1992), and finally taxonomy models such as offered by: Adya & Lusk (2016) Table A1 indicates the firms randomly selected from the S&P500. There were 16 firms and 32 TP-trials selected. For example, for CMI there were three [3] TPs flagged using the screen in Lusk & Halperin (2015).