AI Line-up Issues/Realism

[Syd Thrift]

Originally Posted by Trav876

Aren't the player expectations (on their main profile page) based on their ratings and/or stats? Couldn't that be something that wouldn't take a lot of coding? Managers/AI could then put guys into the line-up that somewhat matches their expectations? Quite often, guys will have "middle of the order," or "top of the order.." Even a guy that has "bench player" as expectation, eventually develops a different expectation based on if he's playing a lot and how he's performing. So there's obviously some kind of recognition the game has for "clean-up" type hitters, or "top of the order" type hitters, based on stats and/or ratings...

He's not hitting 4th because he expects to be there, he's hitting 4th because your regular cleanup hitter is a 2nd baseman and he's filling in for him.

[Brad K]

It's realistic to for AI to give up on a player who has terrible performance. It's not realistic for a player to grossly under perform (or over perform) what he ever did real life.

OOTP allows randomness around the rating. The rating is based on real life performance. Real life performance is the result of both skill and randomness. So we have double randomness which is not a good thing.

I've referred before to an instance of Tony Armas hitting .325 one year and .179 the next.

[Trav876]

Originally Posted by Brad K

It's realistic to for AI to give up on a player who has terrible performance. It's not realistic for a player to grossly under perform (or over perform) what he ever did real life.

OOTP allows randomness around the rating. The rating is based on real life performance. Real life performance is the result of both skill and randomness. So we have double randomness which is not a good thing.

I've referred before to an instance of Tony Armas hitting .325 one year and .179 the next.

I think that part is realistic. Many examples of players wavering that much in real life season to season (Cody Bellinger, etc.). But I think maybe what you mean is, compared to real life individual players themselves? And in that case, if that is what you're shooting for, can't you do 1-year re-calc, plus historical transactions, injuries, & line-ups? With no personality settings, no coaches for development, keeping the development engine turned OFF, etc.

[Trav876]

Originally Posted by Syd Thrift

He's not hitting 4th because he expects to be there, he's hitting 4th because your regular cleanup hitter is a 2nd baseman and he's filling in for him.

I know. My point is in response to you saying it would take a lot of coding & you don't think it could be done, to have players hitting where they realistically would be given their stats/ratings even in sub situations. I'm saying, seems like it could be done since we already have an "expectation" in the batting order. Even if they're a bench player, the AI obviously has an idea of stats/ratings and where those types of players hit in the order.

[uruguru]

Originally Posted by Brad K

It's realistic to for AI to give up on a player who has terrible performance. It's not realistic for a player to grossly under perform (or over perform) what he ever did real life.

OOTP allows randomness around the rating. The rating is based on real life performance. Real life performance is the result of both skill and randomness. So we have double randomness which is not a good thing.

I've referred before to an instance of Tony Armas hitting .325 one year and .179 the next.

The way around this is using 5-year averages for batting ratings. That smooths out most of the year-to-year randomness in the real life and creates a more stable baseline for the OOTP randomness to play off of.

This avoids the situation of the OOTP RNG randomly taking a player's good year and pushing it upward and then taking his bad year and pusing it downward.

Instead what you get is the OOTP RNG randomly pushing up some average years and pushing down some average years. Your OOTP variation is closer to real-world variation, but it may not match year to year. And if the OOTP RNG isn't too kind, then maybe someone like Maris doesn't have a freak year where he even gets close to 61.

[Brad K]

Trav876 and uruguru

It's interesting that one of you is suggesting using one year stats to create ratings and one of you is suggesting using five year stats to create ratings. I loaded the old save and first I want to say the high BA for Armas was .320 not .325. I was correct in stating the low year was .179.

What we have here is one year where he hit .320 and the next year where he hit .179. I imported him into the game for each year and the editor calls him a .249 for each year. Changing the data used to create the rating doesn't matter in this case. It's an issue of variations due to randomness in the game exceed the variations due to randomness and skill in real life.

Now it might be that TCR changed the ratings during the seasons but my memory is his ratings didn't change much if at all during the seasons. And I do remember that the AI manager insisted on hitting him cleanup all during the .179 year so that's an indication his ratings remained strong in his terrible year.

[uruguru]

Originally Posted by Brad K

It's an issue of variations due to randomness in the game exceed the variations due to randomness and skill in real life.

By suggesting 5-year averages, I was responding to your concern that you were getting a "double whammy" of RNG.... one in the OOTP ratings based on the randomness of real life stats, and then one based on the randomness of the OOTP results based on the ratings.

If the issue is not the "double whammy", but that there is more variability in OOTP than in the real world, then that's a different concern.

I don't think you can look at one player and make that determination, because I think we can find any wild variation in an OOTP player and see it not represented in the real world. But, at the same time, we have wide variation in the real world that we don't see in OOTP because it doesn't attract our attention while simming (unless it's something iconic like Maris & 61 homers).

There are many players in the real world who had massive variations in successive years that you would not see in OOTP when using 5-year averages.

[Brad K]

The example I used was not the result of a double whammy - sorry about being confusing - but it is a factor. Imagine if this can happen with stable ratings what an happen with variable ones!

Interesting that although using multiple years smooths the data OOTP provides a way to "unsmooth" it a bit by double weighting the current year. I wonder how close the ratings are with 3 year versus 5 year with double weight.

[Syd Thrift]

Originally Posted by uruguru

The way around this is using 5-year averages for batting ratings. That smooths out most of the year-to-year randomness in the real life and creates a more stable baseline for the OOTP randomness to play off of.

This avoids the situation of the OOTP RNG randomly taking a player's good year and pushing it upward and then taking his bad year and pusing it downward.

Instead what you get is the OOTP RNG randomly pushing up some average years and pushing down some average years. Your OOTP variation is closer to real-world variation, but it may not match year to year. And if the OOTP RNG isn't too kind, then maybe someone like Maris doesn't have a freak year where he even gets close to 61.

The issue here too that is for some reason being ignored is that Armas also had a "freakout year". He hit in the .260s and .270s in 80-81, then dipped to .233 and .218 the next 2 years before "recovering" to the .260s for 2 years before falling off a cliff for good. The standard deviation for hits per at-bats over the course of a full season is around 45 points, meaning that everyone with a "natural" ability of .260 will hit between .215 and .305 67% of the time. Does that sound like a big gap? It's a huge gap. Batting average is notoriously clumsy that way. But just going by normal distribution a .179 average the year Armas actually hit .218 isn't even outside one standard deviation, and conversely a .325 the year he hit .281 isn't really either.

And I have to say, guys do this kind of a lot. Gorman Thomas in about the same period of time went from .259 one year (1981) to .157 in another (1985). Felix Fermin hit .315 for the Mariners in the early 90s one year and .195 the next. Like, yes, it's not literally an every day or even an every year occurrence, and we are really, really keen on delivering tidy explanations for whenever this happens in real life, but a. just because we've decided that there's a reason doesn't mean that there actually is one outside of weird randomness, and b. OOTP can only do so much with this kind of stuff.

We've already isolated BABIP, which can go all over the place, from K rate, which is generally pretty stable, for instance. I think if anything OOTP players tend to be less streaky than real-life ones. What is the answer people want to this? I can think of one and it's so terrible that I would stop purchasing new versions of the game if they implemented it (well, I'm currently on the beta team but I would stop doing that as well).

[David Watts]

Originally Posted by Syd Thrift

The issue here too that is for some reason being ignored is that Armas also had a "freakout year". He hit in the .260s and .270s in 80-81, then dipped to .233 and .218 the next 2 years before "recovering" to the .260s for 2 years before falling off a cliff for good. The standard deviation for hits per at-bats over the course of a full season is around 45 points, meaning that everyone with a "natural" ability of .260 will hit between .215 and .305 67% of the time. Does that sound like a big gap? It's a huge gap. Batting average is notoriously clumsy that way. But just going by normal distribution a .179 average the year Armas actually hit .218 isn't even outside one standard deviation, and conversely a .325 the year he hit .281 isn't really either.

And I have to say, guys do this kind of a lot. Gorman Thomas in about the same period of time went from .259 one year (1981) to .157 in another (1985). Felix Fermin hit .315 for the Mariners in the early 90s one year and .195 the next. Like, yes, it's not literally an every day or even an every year occurrence, and we are really, really keen on delivering tidy explanations for whenever this happens in real life, but a. just because we've decided that there's a reason doesn't mean that there actually is one outside of weird randomness, and b. OOTP can only do so much with this kind of stuff.

We've already isolated BABIP, which can go all over the place, from K rate, which is generally pretty stable, for instance. I think if anything OOTP players tend to be less streaky than real-life ones. What is the answer people want to this? I can think of one and it's so terrible that I would stop purchasing new versions of the game if they implemented it (well, I'm currently on the beta team but I would stop doing that as well).

This is why I've always liked 1 year recalc the best. Real life baseball is full of ups and downs. I would rather see a Josh Hamilton hit .360 and win the MVP one season, fall to .290 the next and never come close to sniffing .300 ever again. Bryce Harper roll a .330 season one year, followed by a .249 season the next year. I guess I enjoy it better when R P McMurphy hides his meds under his tongue instead of taking them and going all mellow.......

[jcard]

Originally Posted by Syd Thrift

The standard deviation for hits per at-bats over the course of a full season is around 45 points, meaning that everyone with a "natural" ability of .260 will hit between .215 and .305 67% of the time.

Syd, what is your source for this? I agree that the effect of chance is far greater than what many of us would presume, the estimates for the standard deviation for batting average used and / or found on various mathematical and baseball sites seem to fall between one-half and two-thirds of your cited magnitude.

An alternative look at the issue would be to see if—regardless of the magnitude of deviation OOTP uses in observed batting average, etc.—the observed results have an expected distribution around the predicted value. Sometimes it feels like a flat curve, with, for example, +/- .004 no more likely than +/- .020.

[Syd Thrift]

My bad, it’s about half of what I put up, about 25 points for a full year:

https://fantasy.fangraphs.com/battin...e-waiver-wire/

Still, even 2 standard deviations from the mean happens around 5-8% of the time, which is around once per every 2 teams, and even the same guy having it happen to him twice in a row isn’t insanely long odds: 5% of 5% is still what, one or two guys in a 2 year stretch in a 26 team league (I’m just eyeballing the odds here). If you see it happen, you’re bound to remember it because it’s weird and rare, but it absolutely happens IRL, which was my point.

Honestly I would prefer the 3 or 5 year smoothing and then allowing random chance to create the peaks and valleys if I played much historical. I actually do fictional without ratings being shown because I like not knowing if that guy who hit .290 the last 3 years but suddenly slumped to .220 just had a bad luck year or was never that good in the first place (or had a TCR hit).

[uruguru]

Originally Posted by Syd Thrift

Honestly I would prefer the 3 or 5 year smoothing and then allowing random chance to create the peaks and valleys if I played much historical. I actually do fictional without ratings being shown because I like not knowing if that guy who hit .290 the last 3 years but suddenly slumped to .220 just had a bad luck year or was never that good in the first place (or had a TCR hit).

I am 100% on board with the high innate variability of batting statistics in the real world, so in that regard I try to replicate that with as much smoothing as possible (5-year averages) to get as close to the "true" performance of the player and then let OOTP do the RNG for the ups and downs.

The AI issue for me is that ratings don't exist in the real world.. there is just performance. So when a Tony Armas has an extended drop in performance, the team can't tell if its due to just random variability or reflects a physical performance degradation. At some point they pull the trigger and either a) move him down in the lineup, b) bench him, or c) demote him to the minors, or maybe even d) trade or release him.

When OOTP evaluates players by ratings (like the default setup), then it doesn't realistically react to performance drops and just keeps plugging their .160 hitter in the #3 slot like nothing is out of the ordinary. That's why I don't use ratings for historical sims, but something like 0/60/30/10.

Ratings are sort of a cheat code, imo, and should be hidden or obfuscated.

[Trav876]

So, with all this said, I don't at all have any issue with random variability, etc. That seems pretty realistic, and even if there's anybody who thinks it's not, they can do things about it, like re-calc... turning off the development engine, etc.

I still run into instances where guys are put in weird spots in the order (particularly 3/4/5 spots) not just with fatigue subs- though that does seem to happen more often, but also sometimes in regular line-ups.

I don't believe this has anything to do with whether GMs in my sim are valuing ratings vs. stats, because quite often it will be a guy (like in my above examples) who is not a power hitter via ratings or stats, and still gets put in the middle of the order. Which doesn't match his "expectation" either... so I am not sure why this happens or what can be done about it, other than hopefully OOTP devs can address in this or next version somehow.

It's a pretty important part of a baseball sim game.

[David Watts]

Rather than start a new thread, I figured I would just post this here. I've been playing fictional lately and one thing that has started to drive me nuts is, the AI pinch hitting for a teams best player and in some cases maybe the best hitter in the league to boot. I watch one game per day during the season and I would say I've seen this occur at least 10 times now. I even changed pinch hit for position players to very rarely. Saw it occur again this morning. The Phillies were battling the Dodgers and the game went into extra innings. Top of the 10th the Phillies have runners on 1st and 2nd with 2 outs. Their #3 hitter Townsend is coming to the plate. Dude is hitting .335(2nd best in league) is leading the league with 37 home runs and RBI's with 117. AI yanks him for some cat with around 10 at bats. Phillies fail to score and lose in the bottom of the 10th. I checked and Townsend has a +.300 average against both right and left handed pitchers. A year ago, I saw the NL MVP pinch hit for twice in the late innings. Strange thing about this is, more often than not it's the #3 hitter that this happen to.

[uruguru]

Originally Posted by David Watts

Rather than start a new thread, I figured I would just post this here. I've been playing fictional lately and one thing that has started to drive me nuts is, the AI pinch hitting for a teams best player and in some cases maybe the best hitter in the league to boot.

I've noticed the same thing and yes it is annoying. Here is some head canon I use for the AI to help me out: he was tired. he tweaked a muscle swinging a bat in the warm-up circle. someone stole his bat. he left his uniform at the cleaners. he ran out of gas. he had a flat tire. he didn't have enough money for cab fare. an old friend came in from out of town. there was an earthquake! a terrible flood! locusts! it wasn't the AI's fault!

[OutS|der]

I'd like more control over how the AI sets up lineups as well. Something along the Stats and AI page where we can adjust how the AI determines lineups.

Give us control over if say the fastest player bats first, best hitter 3rd, power 4th and so on.

[Brad K]

Stumbled across this. AI is hitting the pitcher 8th.

[Trav876]

Originally Posted by Brad K

Stumbled across this. AI is hitting the pitcher 8th.

I don't think that was ever done back in that era, however in fairness LaRussa and some other managers in the 90's and on would hit the pitcher 8th when they had a "leadoff type" guy in the 9 spot to turn the order over.

You may have known that already but wanted to post it for anyone else reading.

[Brad K]

LA isn't doing it all the time. I haven't analyzed it to try to see a pattern.

I recall many years ago some announcers discussing an old time manager who would sometimes hit pitchers sixth.

I don't know about LaRussa etc in the 90s. I was busy with family and work that decade and didn't have any time for sports.