The Dashboard Takes the Calendar Test

TL;DR The oil article ended with a habit: ask of any feature-importance claim whether the feature predicts the outcome or merely dates the observation. The obvious follow-up arrived within a day: does this website's own dashboard survive that question? So the same test was run on all 25 features the ML models saw, including everything the dashboard uses. The results: one dashboard indicator failed the same way oil did (BTC Fees, whose pooled 25.9-point edge collapses to 3.2 points once the calendar year is controlled for), seven came out stronger than the pooled numbers suggested (Power Law, 200-Week MA, Mayer, MVRV, NUPL, RSI, Puell all show contrarian signals that survive, and mostly grow, within-year), and the two indicators kept on economic reasoning alone, with zero SHAP support, both passed. Fees stays on the dashboard, and the reasons are stated below along with the specific result that would reverse that decision.

The previous article resolved an anomaly that had been sitting in this website's claims ledger for a month: crude oil ranked absurdly high in the machine-learning feature rankings because its price level worked as a timestamp. The models used it to learn when each training row happened, not what happens next. That article closed by turning the finding into a habit: whenever a chart says "feature X drives bitcoin," ask whether X predicts the outcome or merely dates the observation.

A website that hands out a test like that has to take it. This article runs the same discriminator on all 25 features the models saw, which includes every indicator on the dashboard. It covers three things: how the test works in plain words, what happened to each indicator, and the question the results force into the open. One dashboard indicator failed. Why keep it?

The Test, In Plain Words

Take eleven years of daily data. For any indicator, split all days into three buckets: the days when it sat in the bottom third of its range, the middle third, the top third. Then ask a blunt question per bucket: how often was bitcoin higher 30 days later?

Ask it two ways. First, pooled: all eleven years in one pot. Then within-year: the same comparison run separately inside each calendar year, then averaged. The gap between the two answers is the whole diagnostic. An indicator with a big pooled edge that vanishes within-year was not predicting anything; it was labeling eras. Cheap oil "predicted" rallies only because cheap-oil years were bull years. An indicator whose edge survives inside individual years is telling you something the calendar cannot.

One warning before the results, because it cuts in every direction: pooled numbers can neither convict nor acquit anything on their own. That will matter below, when a pooled number seems to say something alarming about fees. The same trap, both ways. Predicts the outcome, or dates the observation: that is the only question the test answers, and it needs the within-year comparison to answer it.

One Failed, Seven Got Stronger

The failure first. BTC Fees, the dashboard's third-heaviest indicator at 11.2%, has the exact oil signature. Pooled, days with high on-chain fees saw bitcoin rise over the next 30 days 25.9 percentage points more often than days with low fees, a huge edge. Within-year, that collapses to 3.2 points. High-fee eras (2017, 2021) were simply the manias. The fee level dated the row. Its impressive SHAP count (SHAP is an itemized receipt for a model's forecast: it scores how much each input moved the output), top-5 in 7 of 12 model runs, the second-best of any feature, was earned the same way oil's was. That evidence is now worth much less, and the ledger says so.

Now the part that was not expected. For the dashboard's valuation and cycle indicators, the within-year signal is not merely intact; it is stronger than the pooled numbers showed. Power Law Position: a 19.9-point contrarian edge pooled becomes 29.4 points within-year. The 200-Week MA: 29.0 points within-year. MVRV and NUPL: 26.5. RSI: 12.2. Puell: 11.1. (Contrarian means the direction the dashboard grades: expensive-versus-its-history was followed by fewer up-moves, cheap by more.) The strangest line is the Mayer Multiple, which pooled appears to point the wrong way, +6.2, and within-year swings to 28.0 points in the correct contrarian direction. For these indicators, era effects were not creating the signal. They were hiding it.

One more result deserves its own sentence. NUPL and Puell never appeared in a single SHAP top-5, and this website kept them anyway, on economic reasoning, disclosing that as a judgment call. Both just passed the era-controlled test. The indicator with the second-best SHAP count failed it. If a single sweep can invert the ranking that cleanly, feature importance and indicator quality are simply different things, and this website's decision to treat its weights as an editorial mapping (a human wrote the weights down after reading the rankings; the model did not set them) rather than a computed output, made months ago for other reasons, ages well here.

So Why Keep Fees?

The question deserves a direct answer, not a shrug, and the answer has four parts.

First, the alarming-looking evidence against fees is the same illusion as the evidence that was for it. Pooled, days in the fee-spike band the dashboard grades F ("overheating") were followed by a median 90-day gain of 42 percent, which seems to say the grading points backwards. But fee spikes cluster in mid-bull years the way cheap oil clustered in 2015 and 2020; that pooled number dates observations too. Era-controlled, the dashboard's fee measure (fees relative to their own trailing year, which is what the dashboard grades, not the raw level the models saw) shows a spread of minus 3.8 points at 30 days and 0.3 at 90: the dashboard's contrarian direction, and close to zero. The honest reading is not "fees point the wrong way." It is "fees, on this test, at these horizons, carry little signal either way."

Second, no single indicator was ever load-bearing. This website has already published, against its own interests, that the composite's weights add no timing edge over naive alternatives; they are an aggressiveness dial, and the composite was validated by backtest as a whole set, not by any indicator's solo forecasting record. Deleting or reweighting one indicator off a univariate test (one indicator judged alone, outside the team it plays in) would repeat the original sin this pair of articles diagnoses: over-trusting single-feature evidence.

Third, the economic rationale never rested on SHAP. Fees are the one number on the dashboard where users pay real money for block space: demand you cannot fake with wash trading (an exchange trading with itself to fake volume) or paint on a chart. It is the sole member of the dashboard's on-chain-demand cluster, there for coverage of a dimension nothing else measures. That is precisely the basis on which NUPL and Puell were kept, disclosed at the time as a judgment call, and those two just out-tested it. Rationale-kept has a better record on this dashboard than rank-kept.

Fourth, the decision is now falsifiable rather than comfortable. The ledger reclassifies fees from SHAP-backed to rationale-backed, its SHAP count carries the calendar caveat wherever it is cited, and the revisit condition is written down: if an era-controlled test ever shows the fee grading pointing the wrong way, not just near zero, the 11.2% weight gets formally reviewed. What would have been quietly awkward is instead on the record with a tripwire attached.

The scoreboard, then. Two features exposed as calendars, one of them on the dashboard and now relabeled. Seven dashboard indicators with stronger era-controlled support than any pooled backtest ever showed, the first time this pipeline has produced direct quantitative evidence for the contrarian grading rather than merely failing to find evidence against it. Two rationale-kept indicators vindicated. And one habit, now applied at home: predicts the outcome, or dates the observation. The sweep is one command, ml/research/calendar_sweep.py, and it reproduces every number here, including the uncomfortable ones. Especially the uncomfortable ones; those are the reason the comfortable ones can be believed.