Converting Historical Results into Projections: An Outline
Let's get Steps 3 & 4 started cuz they're hard as hell
Previously On “Absolute Unit”
The financial world estimates the “value” of an asset as (i) the projected future cash flows it generates, which are then discounted, to price it relative to the amount, risk, and timing of similar assets' cash flows using (ii) the concept of a “required rate of return.” To (i) project future cash flows the basic steps we agreed on were:
Obtain relevant and reliable accounting records of historical results, which are (1a) denominated in the unit of value that you care about (in our case “marginal goal difference contribution”).
Adjust the historical results for noisy, non-recurring items to create “pro-forma historical results.”
Take those pro-forma historical results and project them forward into the future in some kind of documented, evidenced-based manner based on the information publicly available.
Layer in any proprietary information you may have related to the thing being valued.
Apply individual judgments not already included above.
Where we left off a couple posts ago, we were in the process of mapping this finance process over to player scouting and we had obtained the relevant accounting records for our target player, infused them with goal values, and removed the non-recurring, noisy data records, leaving us with a good historical base, from which to project into the future. So let’s talk about that.
Future Troubles
OK where to begin. The trouble with predicting a player’s future contribution to a team’s goal difference, aside from the fact that the future is uncertain, and that the player is constantly changing, is that he’s a part of a team. Soccer isn’t fantasy sports and it isn’t baseball. The players don’t just go out and “achieve output” independent of the other players around them. They are part of a team, they operate as a team, and one player’s contributions to the team aren’t fully discrete from another’s. They are at least partially contingent on the contributions of another player(s) or coaches or performance analysts or opponents, or chance etc. So if you’re going to project a “player’s contribution” to the team, you also need some understanding or mental model of how the team works, how the team goes about generating the results they generate, and then further, how players contribute to those results, and then how all of the various other inputs, factors, and activities in the broader organization impact these results (also an understanding of the sport). That’s hard as hell, but theoretically it is the job of one or more people in the organization (some combination of a GM, a sporting director, etc) to understand, to model, to drive and lead the department toward better understanding how all of this works, because on a conceptual level if not, what are you doing? (The answer is surely a lot of very practical things I don’t want to think about)..
“Value” according to whom?
I make this point about the trouble with team projections and individual contributions and their circular relationship to further crystalize our thinking around the “type of value determination” that we’re after here. While it is true that this is an exercise in “player valuation,” specifically the question we care about is “what is the value in goal difference of contracting with a player to play at our club?” not some more theoretical or objective “market value of a player” agnostic of our own club’s identity. Recall that we are working towards a team objective of goal difference for the upcoming season and the seasons to come. To be sure, our working theory of player value is that his own independent value, his individual ability to contribute to the team is the primary factor that determines his contribution to the team and not vice versa, but we cannot ignore either direction of this relationship entirely.
Toward Budget Allocation and a Theory of Wage and Transfer Discovery
To peak into future posts real quick, (and we should - we need to keep one eye focused on the end game here and work backwards as we slowly crawl forward one post at a time through the process), after we do these projections and then discount them against relevant benchmarks, we’re going to have to translate the value of a player contract, which we calculate in terms of “expected goal difference contribution to our team” into the value denominated in the appropriate share of our approved budget resources (economic resources, $s, etc). This is the foundation of our theory for how to build a team. To extend it further, theoretically in a platonic “Absolute unit” expanded universe, all clubs are performing a similar exercise of valuing player contracts based on the player’s future projected goal difference contribution to their teams, and so ultimately, the key drivers in the differences in valuation between clubs, which we can connect to the drivers in the ultimate outcomes of transfer sagas (what different teams are willing to offer the player, or what different teams are willing to offer another team to terminate an existing contract with the player) are 1) differences in the conceptual design and specific estimates around team performance models and 2) differences in the team approved budgets, and 3) some other stuff.
Projecting Contribution Rate first, then Opportunity/Quantity
I referenced in an earlier post that there’s this consensus conventional wisdom of looking at player metrics on a per 90 minutes (or per 96 minutes if you’re American Soccer Analysis) basis. Basically, because injuries and coaching decisions are often out of the control of the player (i.e. he can’t force the manager to play him or not play him normally), the level of his total output is also out of his control. Per 90 figures strip out variance in opportunities and give you more signal about a player’s ability, making them the better apples to apples comparison across players than the raw totals themselves.
While the mainstream public soccer analytics consensus has been to look at things on a per 90 basis, the late Garry Gelade made the case that “per 90” metrics suffer more from team effects and other “state effects” than similar metrics scaled to “per 100 passes” which carry more signal about “traits.” His hypothesis was that per 90 metrics carried with them additional variance caused from unequal opportunities amongst players on different teams (i.e. team effects). His conclusion if I can read between the lines some was that player recruitment departments should consider changing the scale of their metrics to “per 100 ___” metrics to more precisely model how a player moving from one team to the next might transition in terms of his contribution.
I’m not entirely sure what the most favored “per 100 _____” scale is by the fanciest analytics professionals but I suspect it has something to do with possessions. As sad as it makes me, we’re at the point in the “Absolute Unit” cycle where I have to just continue laying a framework down and then on the hard stuff punt to the analytics experts who are smarter than me. That said if you’ll indulge me, I still struggle to understand how a “per possession” framework would better adjust for a player’s opportunities than a “per touch” framework, which I think is closer to what Garry laid out with “per 100 passes.” I can’t solve this one, so on with it then. (email me if you know)
For our purposes, we are interested in the target player’s future contribution to our team, and this is will ultimately be a function of both his ability (traits) and his opportunities, and we want to model these two things separately, partly because of the above, and partly to show our work to a decision maker, and partly so that we can make adjustments to the “opportunities” model a the highest level of our fully operational system (more on this much later). I’m going to probably just keep using “per 90” as a placeholder here for whatever more trait-like scaled metric you want to use to estimate a player’s ability separate from his opportunities. But at any rate, we want to project a player’s contribution for at a rate level (e.g. per 90), and then secondly the player’s expected minutes over his contract term, the two inputs together comprising the player’s projected contributions to team expected goal difference.
Layers to Projecting Contribution Rate
If we agree with the assumption that the future contribution of a player’s actions towards our team’s goal difference is contingent on certain team-specific factors, then there are layers that we need to strip down and build up to convert his historical accounting records (which are theoretically impacted by past team effects) into future projections impacted by our own team’s effects, while most importantly projecting forward the player-specific growth (or decline) path into future years. The ordering of this that feels most appropriate to me for a couple reasons is as follows:
I use the terms “unlevered” and “re-levered above as a result of some inspiration from and analogy to another corporate finance concept, which I’ve stashed at the end of this post in an appendix, for fear of scaring people as it has a Greek letter. We will absolutely do some deep dive posts into these layers soon, but the basic premise in the above diagram is that there is theoretically a basic developmental path for a player if you squint and assume he develops in an average environment with average access to manager acumen and training insights and technology and he has average drive to do so and average opportunities to train etc etc. That player development trajectory needs to be estimated first using a systematic and rational methodology (before layering in some specific judgments from your scouts and analysts), and it’s this systematic forecast which is happening in the third column above. By keeping in mind the player’s age and applying some sort of development/growth assumptions to the player’s observed historical contributions, we might project his future contributions, but first we have to strip out the team effects of his current (or prior team(s)) via some sort of mathematical model. Having done that, it makes sense to project his development forward (again, we’re still talking about rates of contribution per minute or per possession or per 90 minutes, not the total minutes we think he will earn or have an opportunity to participate in).
And then from an organizational and communications perspective, I propose that only after having already projected these systematic team agnostic pieces out, should you then layer in your own team-specific information and assumptions, impacting the rate of a player’s projected contributions to the team goal difference. I think this has a conceptual organization to it that’s solid, allowing you to separate out the player on his own and then consider specifically how his abilities and forecasted path interacts with your own team’s model for performance, its needs etc. And it allows us to express this “walk” visually (below) in the case where for instance you feel more confident about one of these components than the other and want that to clearly delineate them when presenting to decision makers.
Late edit (Jan 2014): I’ve just read Michael Caley’s Study on Sub Effects, and it strikes me that this is one of the layers sitting in a player’s historical records (and thus a contribution rate) that you need to unwind before you can move forward with projections. That is to say, a player’s historical contributions may be a function not only of his underlying performance but specifically of the amount of time he’s spent in substitution scenarios compared to starter scenarios: See Caley’s post here:
Projecting Player Minutes (Model of Opportunities)
Here is another example where I feel strongly about the framework but not as strongly about the specific course of action, knowing full well there are footy analytics folks out there that will (or already have) come up with the state-of-the-art way to do this. Basically, while the per 90 (or per 100___) metrics tell us something about a player’s ability, when we allocate budget dollars to a player contract, we’re paying for more than ability. If the player doesn’t ever step on the pitch then he cannot contribute to the team’s overall performance as measured in expected goal difference, and accordingly we should not allocate many budget resources to signing him. Off the top of my head, one way a sporting director might project minutes across its roster is to take some data dump of depth charts and minutes played from several seasons across the league, or multiple leagues (probably more than just one specific team), and specifically model how the distribution of minutes for a team falls over the course of a season. Once you compute some sort of average expectation of the number of minutes your starting forward will see, and the number of minutes your backup forward will see, and so on and so forth (again based on averages across many years of soccer data), you can squint at the roster, and the role you are trying to fill by assigning roster spots (and existing players) to those various minutes buckets, including the target player you’re evaluating. Plug him in somewhere based on feedback from the manager and other analysts and assign an expected number of minutes (and from there touches or possessions if you’re doing that) to connect his rate projection above to an overall expected contribution. Having done so, you might also take into consideration the extent to which the target player is injury prone relative to other players at the position, but there is a trade-off between tackling this problem here in the projections phase, or later in the discounting (risk-adjustment) phase. Pin it.
Forward
While it got a bit cumbersome at times, this was — believe it or not — an overview post showing the template we might use to do the dirty work of taking historical accounting records of a player and projecting them into the future, a very specific future where the player has signed at your club. We’ll probably need to dive in a little deeper to some of the powerpoint slides (ugh) above in the coming weeks.
Appendix: Side Story about Unlevered and Re-levered Beta
At the risk of further complicating the clean analogous mapping between finance and soccer, there is a concept in corporate finance, less around the projections of future cash flows and more around the determination of the appropriate discount rate or required rate of return to apply to a set of cash flows, which is assigned the Greek letter of Beta. A company’s “Beta,” is a measurement of how volatile its returns are relative to the entire market, and is calculated by taking the covariance of a given company’s returns versus the overall market’s returns and dividing it by the variance of the overall market’s returns. A company with a Beta of 1 has returns that are equally volatile as the overall market and directionally correlated with the returns of the overall market while a company with a Beta of 1.5 is 50% more volatile than the overall market. Conversely, a company with a Beta of -1 has returns that are equally volatile as the overall market but inversely correlated with the returns of the overall market. A company with a Beta of 0 has returns that are uncorrelated with the returns of the overall market.
Anyhow, when you’re thinking about buying a company and folding it into your own company’s operations, it’s helpful to understand the Beta of the company; however, there’s a problem because a Company’s returns can differ in volatility relative to the overall market not just because of it’s natural business risks relative to those of the overall market, but because of choices management has made around its capital structure. Generally speaking, companies with more debt are riskier and their returns to shareholders more volatile because companies are legally obligated to pay interest to their creditors and repay debts to creditors as they become due. Conversely, companies are not required to pay out returns to shareholders (shareholders have “residual claims” on a company’s assets). This asymmetry means that in good times, companies with more debt achieve higher returns for their shareholders and in bad times the shareholders get very little. And in even worse times, the company is unable to pay its debts and collapses into bankruptcy where creditors get first shot and recouping some of what is legally owed to them, and equity holders get nothing.
I’m struggling to rein this in — back to Betas. Because of this scary talk about debt, a company’s leverage (it's debt) effects its Beta. But, if you want to know a target company’s Beta, you probably want to know what their overall business risk is, and you probably also want to know what their effective Beta would be once they’re folded into your own operations, but your capital structure probably differs from the capital structure of the target company at present, so it’s not as simple as just keeping that same Beta. You have to unlever the target company’s Beta to strip away the amount of volatility imbedded in the Beta that’s really a result of management’s choices around capital structure (debt). After doing some modest amount of math to strip this away, this now unlevered Beta is supposed to reflect the target company’s true business risks and is sometimes referred to as the “asset Beta.” So you have this measure of “true” business risk volatility, but you’re not done yet. You then want to re-lever the Beta to reflect the impact of your own company’s capital structure, or let’s say if you bought the company by loading it up with a bunch of debt (i.e. a “leveraged buy-out”), you would want to reflect your company’s new debt-heavy target capital structure. Having re-levered the Beta with a new capital structure, you now have a true view into the variability in the target business’ returns relative to the rest of the market. To complete the thought - and this isn’t germane to this episode, but maybe a teaser for later, the reason a buying company is doing this at all is to determine the correct “discount rate” to use to price the projected cash flows of the target company. It will plug this “re-levered Beta into the “Capital Asset Pricing Model,” to estimate the discount rate to use for analyzing the target company, and ultimately determining the correct price for the acquisition. So, yea that’s confusing. I’m borrowing this idea of un-levering and re-levering for the “projection phase” when it’s really used in the “discounting phase” of financial valuation. Everything is connected Cheers.