A projection system for OOTP 26

[perfektootp]

Originally Posted by squirrel

Now done the pitchers - same idea, all pitchers and hitters set to 50 everything, then varied the #1 starting pitcher for Team A, one rating at a time

So, what do the blanks mean exactly?

[hjrrockies]

Originally Posted by squirrel

I've posted below the settings I used in the test league that generates the results above.

Some observations:

- These ought to convert to WAR estimates with a runs-per-win constant of something like 10 (so shortstop IF range 38.3 runs is about 3.83 WAR)

- BUT replacement level and 50-rated everything is not the same - the replacement level SS on my waiver wire in a MLB save is rated something like 55 not 50

- I'm surprised by the range results at 2B (too uniform vs my prior expectations) and RF (too low)

- I'm aware the 1-250 scale is more granular than 20-80 but I prefer the immersion of sticking to 20-80

One note on this: although 50 is, by definition, "league average", playing time is not distributed uniformly across players. The better a player is, the more playing time they get. For example:


- In the 2024 season, 526 players received at least 50 plate appearances. In total, they made 179643 PA, and produced a wRC+ of 101. However, the average player in that sample had a wRC+ of 91, which is 10 points lower. How is this possible? It's because there are lots of 90 wRC+ players who get a little playing time, but there are a handful of superstar players who accrue 700 PA every year.


How does this affect estimating the value of player ratings in OOTP? Well, if you fix every player to average (except for one test player), then the league is actually weaker overall. Basically, an average player (i.e. selected at random from a roster) is a below-average performer (i.e. compared to a randomly-selected plate appearance). So, the estimates we get from a method like this (while useful), are almost certainly overstating the run values compared to practice.

[squirrel]

Originally Posted by mmorgan184

That's 100% the issue. I believe this is built for 20-80.

Yep that's right sorry just getting to this. It's built for 20-80 ratings in increments of 5

Been a bit radio silent here lately as trying to work on some improvements

[squirrel]

Originally Posted by perfektootp

So, what do the blanks mean exactly?

Just means I hadn't tested them - which normally means in my MLB save there were no players like that.

I've been working on further stuff that fills in some of those gaps - I'm still sense-checking it for now

[squirrel]

Originally Posted by hjrrockies

One note on this: although 50 is, by definition, "league average", playing time is not distributed uniformly across players. The better a player is, the more playing time they get. For example:


- In the 2024 season, 526 players received at least 50 plate appearances. In total, they made 179643 PA, and produced a wRC+ of 101. However, the average player in that sample had a wRC+ of 91, which is 10 points lower. How is this possible? It's because there are lots of 90 wRC+ players who get a little playing time, but there are a handful of superstar players who accrue 700 PA every year.


How does this affect estimating the value of player ratings in OOTP? Well, if you fix every player to average (except for one test player), then the league is actually weaker overall. Basically, an average player (i.e. selected at random from a roster) is a below-average performer (i.e. compared to a randomly-selected plate appearance). So, the estimates we get from a method like this (while useful), are almost certainly overstating the run values compared to practice.

Yep I agree + this is a totally fair criticism.

I'm pretty sure a better way to do this would be to create a more convincing 'test league' with players of different standards but calibrated to be more similar to MLB teams.

But for now this is what I have and it seems to be working well enough - I'm currently testing out a refinement vs what I have posted here so far in my own save and so far so good

[squirrel]

I have just posted an updated version to github.

https://github.com/squirrelplays/pistachio

When I get some time I'll explain it in a youtube video. It's a comprehensive rebuild. I've been using it in my own save and enjoying it

[H1LLIER]

Thanks for the time and effort it must've taken to build this. I've had a little play around with it and it looks good. Also thanks for adding L/R splits, it was me that asked under your YouTube video if you planned on including them.

I think there may be an error on fielding calculations though, particularly 2B. I noticed it when sorting by wOBA and seeing Yordan Alvarez's SS WAR was over 3x higher than his 2B WAR. He'd be a terrible defender in either position so I wondered why there was such a difference. I did a bit of digging and I think IF range is being undervalued for 2B.



If I'm interpreting the calculations in config.py correctly it looks like IF range is over twice as valuable to 1B than it is to 2B which can't be right. Even 1B IF arm carries more weight at 75 rating

[squirrel]

Originally Posted by H1LLIER

I think there may be an error on fielding calculations though, particularly 2B. I noticed it when sorting by wOBA and seeing Yordan Alvarez's SS WAR was over 3x higher than his 2B WAR. He'd be a terrible defender in either position so I wondered why there was such a difference. I did a bit of digging and I think IF range is being undervalued for 2B.

//

If I'm interpreting the calculations in config.py correctly it looks like IF range is over twice as valuable to 1B than it is to 2B which can't be right. Even 1B IF arm carries more weight at 75 rating

Hey man, thanks for this. I really appreciate thoughts on this...if you see any more 'this looks wrong' stuff please do send it my way. There are bound to be some bugs in this somewhere.

On 2B range here I think the numbers look OK / as I intended them but I ought to explain why. I'll get a video onto youtube soon to explain what I've done, but the basic idea is:

- for each position I've tried to identify what 'replacement level' fielding skills are - so the replacement level IF range for 2B is assumed to be 55, whereas for SS it is assumed to be 60 (these numbers are based on my MLB save)

- the numbers for 'runs saved per 162' are based on the testing I've done and the way I interpret the results is that for 2B there's a lot of downside to having a low-range guy but not much upside for having a high-range guy

- whereas for SS there is quite a bit of upside vs replacement level for having a high range guy, and actually less downside (which seems a bit odd, but that's what the testing results said)

- That's all based on the testing I've done - I will get organised and release all of the testing data / results so anyone interested can see all of the underlying data behind how the projections are put together

Will get that explanatory video out when I can, all being well over the next few days or so.

As I say please do point anything else out that looks weird, very much appreciate it

Thanks

[squirrel]

Youtube video now posted on the updated projection system, with step-by-step guide on how to set up and a detailed run-through on how the projections are calculated:

https://www.youtube.com/watch?v=LKDsEReFWSY

[squirrel]

If you were using the previous version I recommend upgrading, as I have made a lot of improvements and this is a much better version.

If you were intrigued by it but didn't get it going, now I think is the version to jump in on.

Upgrades vs the previous:
- I've rebuilt the projections ground-up using the OOTP 26 engine and ratings formats
- I've added platooning
- I've added a filtering thing to the html to make it much easier to use for finding players without needing to use a spreadsheet alongside it
- I've made the code way easier to read and amend for people who want to do that

I've also released here the game engine testing data / method that went into this in case anyone wants to get into real detail on it - and you will see I have set out there some of the flaws I identify in my approach - any comments welcome:

https://docs.google.com/spreadsheets...#gid=202842399

If anyone uses this, have fun! And if there are any comments on it please let me know here or in my youtube comments, thanks

[crowhop]

Awesome work. Thanks for making it available!

[Guthrien]

This is a wonderful upgrade. Great video too, I'd love to watch more OOTP videos by you like your FML leagues. There's a dearth of interesting non-live play content now (there are of course many worthwhile play creators). Sgt Mushroom used to do quite a bit but you know what a divisive character he is and he's clearly no longer into it.

Also, per the suggestion in your video, it would be fantastic to see a web front end be created. I am not skilled enough to do so, but I would donate to the cause as I imagine a few others might.


post: I've had the chance to run it, it's a wonderful tool and easy once you figure out VSC. It's a nice upgrade since v1. I still feel like there's something off with a lot of the WAR valuations for different positions, I'm wondering if the most solid information is the OBA ratings and then having a nice easy weight for comparing positions. At the same time, it makes me want to hop in and run my team 100k times to see how much I've benefited from RNG with certain players. A lot of conversation on the forum is a constant battle against a small number bias, and the same thing pulls against me comparing my perception of a player through a year or two vs projections. At the end of the day, it really clears up for me who is a slightly better value in my lineup, staff and especially with Free Agents and that's just a fantastic tool with all the edge cases you end up judging.

[Guthrien]

Has anyone else given it a go? My projections have been give and take, but again the best you can probably hope for is a hierarchy rating.

@squirrel I'm having the oddest issue. Every part seems to function, except the exported html files are empty. They don't appear empty in VSC, the source csv are fine, but whenever you open the .html files there is nothing. The only change I can think of is I installed the new beta patch but that should have no relevancy. No other changes to the configuration. I hate to install the whole thing over but I may have to. I wish I had some sort of error to point to.

[squirrel]

Hmm no idea. You didn't change your scouting director in-game did you? If yes you would need to change the ID to reflect that. I forgot to do that once

[Guthrien]

Originally Posted by squirrel

Hmm no idea. You didn't change your scouting director in-game did you? If yes you would need to change the ID to reflect that. I forgot to do that once

Annnnnd you nailed it. Totally forgot he just retired. I guess you did have some idea, thank you for sharing that.

[Eburaci]

Hi squirrel, thanks again for the great work. One question from me: in your testing, did you find a "DH penalty" where players of the same hitting ratings will perform worse if they are the DH? If so, were you able to estimate the detriment to wOBA?

[squirrel]

Originally Posted by Eburaci

Hi squirrel, thanks again for the great work. One question from me: in your testing, did you find a "DH penalty" where players of the same hitting ratings will perform worse if they are the DH? If so, were you able to estimate the detriment to wOBA?

I did, yeah. In the numbers I came up with it was a 2.3% percent penalty - see cell AC102 in the 'hitters' tab of the google sheet upthread. That's reflected in the DH WAR projection in the code

[jaa36]

In my testing, I found the DH penalty to be about a 2% reduction in all types of hits and walks, and about a 2% increase in strikeouts- which works out to around a 2% change in wOBA as well.

[Guthrien]

Originally Posted by jaa36

In my testing, I found the DH penalty to be about a 2% reduction in all types of hits and walks, and about a 2% increase in strikeouts- which works out to around a 2% change in wOBA as well.

If you both see a consistency to that extent, it's absolutely purposeful... Why? I don't know what the real world difference is, but it's not a reliable 2% devolution.

[jaa36]

well, it's a fairly well-documented real-life effect.

http://blog.philbirnbaum.com/2019/09...h-penalty.html

This article comments on it as well, but points out that it should vary by the "reason" someone is serving as DH- as in, in real life, there's less of a penalty for a "regular" DH than someone who is the DH because they're recovering from an injury, which makes sense.

https://tht.fangraphs.com/re-examini...sive-spectrum/

There's also a well-documented pinch hitter penalty, which I believe is in the game too, but harder to test.

https://blogs.fangraphs.com/updating...ules-of-thumb/