The SPEC Test

A mid-sized pension fund evaluates a quantitative manager. The manager presents five years of live track record, a Sharpe ratio above 1.2, and a detailed slide deck explaining their "proprietary machine learning infrastructure." The due diligence team reviews the performance attribution, checks the drawdown profile, and reads the risk disclosures. The manager passes.

Two years later, performance collapses. In the post-mortem, it becomes clear that the model's apparent edge came from a single feature: a crude sentiment signal that happened to work during a specific macro regime. The team never asked how the model was built. They only asked whether it had worked.

This is not an unusual story. Academic research on factor strategies documents it systematically. A landmark study by McLean and Pontiff found that the returns of documented factors decline by an average of 26% after publication, with the steepest drops in strategies most likely to attract institutional capital. The explanation is not just crowding. It is that many strategies, when examined structurally, were never built on stable economic relationships in the first place.

A 2023 analysis of 26 Barra factor models found that the majority could not reliably distinguish their genuine signal from noise in out-of-sample testing when structural assumptions were interrogated rather than performance outcomes. The models looked fine. Their foundations were not.

The problem is not that quantitative managers are dishonest. Most are not. The problem is that the standard framework for evaluating them creates systematic blind spots. Performance-focused due diligence is necessary but insufficient. It does not reveal whether a model's apparent success came from genuine insight, lucky configuration, or a feature that no longer exists in the market.

Existing due diligence frameworks were designed for a different era. They ask excellent questions about operational infrastructure, risk controls, and performance attribution. They do not ask whether the model's structure justifies confidence in its future performance.

The gap exists for understandable reasons. Structural evaluation requires asking questions that feel technical, and allocators often assume they lack the background to interpret the answers. That assumption is wrong. The questions required to assess model structure are not technical. They are logical. They require curiosity and scepticism, not a PhD in machine learning.

The other reason is incentive misalignment. Managers who have built strong-looking track records have no commercial interest in inviting deep structural scrutiny. A performance-focused framework lets both sides maintain a comfortable distance from the harder questions.

The SPEC Test changes that dynamic. It gives allocators a vocabulary and a process for structural evaluation that does not require technical expertise and cannot be deflected by performance data.

The strongest signal is not any single answer. It is the pattern across all four questions.

A manager who gives strong answers on Specification Choices and Configuration Sensitivity but struggles on the Explanation Test may have a well-constructed model that is difficult to communicate. A manager who gives confident answers on all four questions but whose explanations are vague or contradictory across questions is a more serious concern.

The pattern also reveals research culture. Managers who have thought carefully about structural questions tend to answer them readily, with specificity and without defensiveness. Managers who have not tend to redirect to performance data, cite operational complexity as a reason for opacity, or provide answers that are technically accurate but empty of content.

The SPEC Test is not a pass/fail instrument. It is a signal generator. Use it to structure your follow-up questions, to calibrate how much weight to place on track record, and to identify where further investigation is warranted.

Three forces have made structural evaluation more important than at any previous point in the history of quantitative investing.

First, the proliferation of AI claims has created an evaluation crisis. When every manager claims to use machine learning, performance-based differentiation becomes less reliable. The structural questions are what distinguish genuine integration from rebranded factor strategies.

Second, the shift toward total portfolio approaches has increased the cost of structural failures. When quantitative strategies are held alongside private markets, infrastructure, and other illiquid allocations, a structural model failure in the liquid sleeve has amplified consequences for overall portfolio management.

Third, the ADIA Lab and similar sovereign research initiatives have elevated the methodological bar for what constitutes rigorous quantitative research. Academic standards are increasingly influencing what sophisticated institutional allocators expect from managers. The SPEC Test aligns practitioner due diligence with those evolving standards.