Good Performances By Bad Pitchers, Revisited

[holes573]

I recently posted data suggesting that bad starting pitchers may be programmed to have good days too often (https://forums.ootpdevelopments.com/...d.php?t=358515).


Two key questions arose among some of the responders to that post:


1. Does OOTP really program pitchers to have good/bad days?
2. Are there better metrics for measuring the quality of pitchers in my analysis?


Let me address each of these questions with new data I just collected, playing the SF Giants in 1965 (OOTP25). I analyzed every inning a starter pitched against the Giants (and for good measure, I did the same for starts against the NY Mets and Houston Astros).


1. Does OOTP really program pitchers to have good bad days?


This is a challenging question because you need to distinguish between good performances that are just due to luck (chance) from those that are due to a programmed good performance.


The key to distinguishing these is that eventually, luck will often run out after a while, during the game. After a lucky pitcher does well in the first inning (such as allowing 0 ERs), they have to continue to be lucky in the second, third, etc. to have a good performance. Each new inning provides an opportunity for the pitcher to lose their luck and start allowing runs. In contrast, a pitcher programmed to have a good day should keep delivering good innings until they tire.


Among the 418 starter performances in my data, in 73% of the outings, 0 ERs were allowed in the first inning. Separately, I found that, regardless of what happened in the first inning, 81% allowed 0 ERs in the second, 78% allowed 0 ERs in the third, and so on. If good performance is due exclusively to luck (chance), then a start that has 0 ERs in both the first and second innings should happen 59% of the time (73% x 81%). And, allowing 0 ER in all three of the first innings should happen 46% of the time (73% x 81% x 78%). Below on the left are the these percentages for consecutive 0 ER innings, for each set of innings. If luck explains all of the good performances of 0 ERs over these innings, these are the percentages we should find when we actually start counting the times consecutive 0 ER innings happen.


But, that isn't what we found. The percentages on the right are what actually happened - how often our 418 starts had consecutive 0 ERs from the first inning on.


0 ERs for first: Luck Only Actual Data

1 inning ...............73%......... 73%
2 innings .............59%......... 58%
3 innings .............46%......... 46%
4 innings .............35%......... 36%
5 innings .............28%......... 32%
6 innings .............24%......... 28%
7 innings .............20%......... 27%
8 innings .............17%......... 27%


Starting about the 5th inning, the actual frequency of consecutive 0 ER innings starts outpacing the predictions based only on luck. In other words, some pitchers strung together long starts of 0 ERs, well beyond what chance/luck would predict. And, for those of you familiar with the concept, these differences are highly "statistically significant" - we should be 99.8% confident that the actual percentages are reliably higher than the luck only percentages. (BTW, these differences happened when facing a good team, like the Giants, and against a bad team, like the Mets.)


These seem like pretty strong data showing that the occasional good performances by pitchers happen more frequently than they naturally would - thus supporting the conclusion that OOTP has, as part of its complex algorithm, a good day/bad day variable at work.


2. Are there better metrics for measuring the quality of pitchers in my analysis?


Last time, I showed data that bad pitchers had essentially the same frequency of good performances as medium and good quality pitchers.


To do that analysis last time, I divided starters into good, medium and bad quality groups based on their 20-80 rating based on scouting. There were suggestions that better ways to do this would be with their 20-80 ratings when they are 100% accurate, or their Projected WAR ratings on the editor. I did the analysis this time with both of these better ratings systems, and found essentially the same results for both. For simplicity, I've presented below only one of these analyses (the one based on Projected WAR).


Here are the frequencies of starters having a "good day" (defined as having a 2.0 ERA or less for the whole outing, that is, allowing only 1 or 0 ERs for their whole outing).


Bad pitchers (1.8 Proj. WAR or less): 28%
Medium pitchers (1.9 - 3.1 Proj. WAR): 32%
Good pitchers (3.2 Proj. WAR or more): 35%


Like last time, the differences between these three groups are not statistically meaningful, so we should treat them as essentially the same. And, that same conclusion (they are essentially the same) held true when I look those starters facing SF, NY and Hou individually.


Also, like last time, I am surprised that so many Bad Pitchers are having good days. My feeling is that Bad Pitchers should have considerably fewer good days than the better pitchers.




Sorry for the data-heavy post, but the only way I think this can be adequately understood is to go to the numbers.

[twins_34]

Originally Posted by holes573

Also, like last time, I am surprised that so many Bad Pitchers are having good days. My feeling is that Bad Pitchers should have considerably fewer good days than the better pitchers.

What are you calling "bad" pitchers though? If you are only going by overall and potential as your basis, then you might not know how to accurately read pitchers. There are some pitchers that have good stuff and movement but are dragged down because they have 45 control. This might make a pitcher a 45 overall, but in no way does that make them a bad pitchers because of how OOTP 25 heavily weights control.

That's one thing that changed in 25 is how pitchers are rated. I would not call someone with 70 stuff, 55 movement, and 40 control a "bad" pitcher. maybe like an average player.

Can you show us what your examples of bad pitchers are? Would you consider these two pitchers to be bad pitchers?

[holes573]

No, I wouldn't.


A bad pitcher in my analysis is one with poor overall ratings (which supposedly are amalgamations of all of their capabilities). In terms of the overall 20-80 rating, I defined a bad pitcher as someone with an overall rating of 43 or less (many of which had overall ratings in the 30's). In both of your cases, the overall rating (45 and 47) would be in my medium category.


A couple of notes: In my first analysis, I defined these with 100% accurate ratings. And, in my second analysis, I used the Projected WAR estimate for each pitcher (which was shown above).


The reason I used these two ratings is that I was predicting their overall performance, not their walks, strikeouts, etc. So, I chose two predictive measures of overall performance.

[MathBandit]

I think you'll have more success if you both don't consider Overall to determine how good a pitcher is, and don't look at ERs to measure performance. I don't consider either to be particularly meaningful when assessing a pitcher, personally.

[twins_34]

Originally Posted by holes573

No, I wouldn't.


A bad pitcher in my analysis is one with poor overall ratings (which supposedly are amalgamations of all of their capabilities). In terms of the overall 20-80 rating, I defined a bad pitcher as someone with an overall rating of 43 or less (many of which had overall ratings in the 30's). In both of your cases, the overall rating (45 and 47) would be in my medium category.


A couple of notes: In my first analysis, I defined these with 100% accurate ratings. And, in my second analysis, I used the Projected WAR estimate for each pitcher (which was shown above).


The reason I used these two ratings is that I was predicting their overall performance, not their walks, strikeouts, etc. So, I chose two predictive measures of overall performance.

This is what giving screenshot examples of your pitchers would be great. We have no idea what you are considering good, medium, or bad. Cause like I said and math said, overall is just not a great rating for pitchers. You might have some dude with extreme GB that is doing his job keeping the ball in the park and on the ground, but has no stuff or control.

Cause even though this dude is rated a 43, he can be a viable pitcher by getting guys to hit into groundballs and double plays. He really is not going to strike many out and might walk guys, but he is probably going to get out of jams because of being able to keep the ball in the park and on the ground.

There is more to look at with pitchers than overall and potential, so that's why I am saying, give us some examples of the dude you are calling bad because they might not be as bad as you think.

[holes573]

First of all, the OOTP25 manual defines ratings as a measure of "player ability." So, that sounds like a measure of overall quality to me.


Nevertheless, let me give you more detail on the 26 pitchers in this Bad Pitcher category. Here are some average stats on this group:


Average current rating (100% accurate): 39
Average projected WAR: 1.0
Average ERA at end of season: 4.75 (Overall league ave. = 3.80)
Average Win-Loss record at end of season: 4W-8L


And here's a couple typical examples, one a bit above average for the category (40 rating) and one a bit below average for the category (38 rating):



https://postimg.cc/ftKSS24Y
https://postimg.cc/7Cr5s8wg
(Sorry, I'm not sure how to post images here - the "Insert Image" button didn't seem to work.)


I don't think there should be much doubt that the starters in this category should be as lower quality.


But, the big picture is that there are 26 starters in this group. If someone else might categorize a couple of them on the higher end of the Bad category into the Medium category, and a couple in the Medium category get categorized into the Bad group - it won't matter much at all. The Bad group have essentially the same frequency of good days as those in the higher categories.

[darkcloud4579]

I see tons of complaints about this mostly on the subreddit, given the game is a simulation and the data is really diffuse based on the complexities of each person's individualized environment, it just feels like people want the game to essentially have it work that someone under 50 OVR in a 80 OVR league will automatically always be bad, but that's just not realistic. Obviously, there's always someone that's sub-replacement level in a game, but when you consider how many players go underscouted, or not in the right situation, it's not that weird that "bad" pitchers have "good" outcomes sometimes.

OOTP isn't meant to be a predicitive baseball sim that always maps to your imagination, frankly, I think that's what makes it such a good game. It's possible to tweak your player creation and league universes to work more align with your desired dreams, mostly by overclocking development and ratings, but it's a lot less fun and also usually results in hitters being too overpowered relative to pitching.

But ratings are like real life, humans don't always evaluate things right. If you stack your team with all the best players and they don't win, that's because that's what also happens IRL, not because the game is somehow fundamentally broken. But so many people want the game to work that way.

[twins_34]

Originally Posted by holes573

First of all, the OOTP25 manual defines ratings as a measure of "player ability." So, that sounds like a measure of overall quality to me.


Nevertheless, let me give you more detail on the 26 pitchers in this Bad Pitcher category. Here are some average stats on this group:


Average current rating (100% accurate): 39
Average projected WAR: 1.0
Average ERA at end of season: 4.75 (Overall league ave. = 3.80)
Average Win-Loss record at end of season: 4W-8L


And here's a couple typical examples, one a bit above average for the category (40 rating) and one a bit below average for the category (38 rating):



https://postimg.cc/ftKSS24Y
https://postimg.cc/7Cr5s8wg

Like said in other posts, going by overall is not the end all in how to rate a player. Another thing said is that in OOTP 25, control seems to be very heavily weighted when it comes to pitchers so a pitcher with 70 control and very little stuff is going to be rated quite high compared to a pitcher with 70 stuff and 40 control.

Now I have been starting players like some you listed in an online league and they have their good days and bad days. They have their times they win by one run and days they get totally shelled.

Now in 2023, I would not play that 35 year old but in1966 where stamina does not matter as much, sure I would have him as a 4th or 5th. He is a pretty average pitcher, except for that low stamina, but once again in 1966, not as big of a deal.

Originally Posted by holes573

(Sorry, I'm not sure how to post images here - the "Insert Image" button didn't seem to work.)

Make sure your browser is not blocking popups as a popup box comes up to allow you to attach images.

Originally Posted by holes573

I don't think there should be much doubt that the starters in this category should be as lower quality.


But, the big picture is that there are 26 starters in this group. If someone else might categorize a couple of them on the higher end of the Bad category into the Medium category, and a couple in the Medium category get categorized into the Bad group - it won't matter much at all. The Bad group have essentially the same frequency of good days as those in the higher categories.

And I do have to agree with darkcloud that, this game is not a guarantee that because you are going up against a 38 overall pitcher that you are going to shell them. Even some of the worst pitchers in history have won games. That's why it seems OOTP plays more to real life. If this game was predictive to the point you knew what games you were going to win or lose, then what's the point of playing it? This game seems to simulate real life pretty good. But, also with your data points, you have to consider what the type of batters they went up against. Now, if they were all superstar players, then I would consider your data to be concerning, but if the ranges of batters are 38 overall to 70 or so overall, then I would say there is not much of an issue.

[MathBandit]

Originally Posted by holes573

Average current rating (100% accurate): 39
Average projected WAR: 1.0
Average ERA at end of season: 4.75 (Overall league ave. = 3.80)
Average Win-Loss record at end of season: 4W-8L


[...]


I don't think there should be much doubt that the starters in this category should be as lower quality.

For my sake, there is still a fair bit of doubt. Rating, ERA, W/L tell me absolutely nothing about the quality of a pitcher. WAR is better, but on it's own (and especially without an Innings component as well) it doesn't say much.

I will say that the fact these are above-replacement pitchers does tell me that they shouldn't be that bad. These are guys you would expect to see in the rotation of the best teams in baseball, and if moved to the pen would probably be very good relievers in most cases.

[holes573]

Twins_34 and darkcloud4579:


You seem to have missed the point of this analysis. Of course bad pitchers will do well at times - there are lots of factors (including luck) about why they should. I haven't argued that "it's weird" that it happens or that there "should be a guarantee" that a 38 pitcher should get shelled.



Instead, I've argued that luck indeed happens, but that above and beyond that there is a "good day" algorithm at work. And that too is fine (and probably matches real baseball).


Rather, I've argued that the application of a good day to poorer pitchers appears too heavily weighted. The frequency of a less than 2.00 ERA outing, for medium and good pitchers should be more than for bad pitchers.

[Rain King]

I'm honestly not able to follow the methodology completely in the original post here.

However, I do think it needs to be factored in that lower rated pitchers are likely to be pulled earlier, including when pitching well.

What percentage of those 7 and 8 inning games are "bad" pitchers? If those innings are over-balanced with good pitchers aren't the results going to likely be better than expected through randomness(i.e. survivors bias)?

[darkcloud4579]

Originally Posted by holes573

Twins_34 and darkcloud4579:


You seem to have missed the point of this analysis. Of course bad pitchers will do well at times - there are lots of factors (including luck) about why they should. I haven't argued that "it's weird" that it happens or that there "should be a guarantee" that a 38 pitcher should get shelled.



Instead, I've argued that luck indeed happens, but that above and beyond that there is a "good day" algorithm at work. And that too is fine (and probably matches real baseball).


Rather, I've argued that the application of a good day to poorer pitchers appears too heavily weighted. The frequency of a less than 2.00 ERA outing, for medium and good pitchers should be more than for bad pitchers.

I think my only real point is that OP is assuming that people are playing OOTP ceteris paribus and that's just not how most people play. If it would take a 231 game series to determine a statistically significant World Series champion, running an OOTP save without everyone running exactly the same conditions would only really give a semblance of insight into how the game actually works.

If players under say 50 OVR weren't remotely viable in any way, there'd be no point in modeling them, better players should not always perform better because it defies reality. How many "bad" players have went on benders where they were really good, only to never be heard from again?

I'm not disagreeing with your general premise or analysis, I'm saying that this is a philosophical argument about what someone values in a computer baseball simulation and that if OP wants a game that's more predictive and where "good ratings = always good" then they should design that game, because trying to foist that idea onto OOTP even if it's an "accurate" idea, is really about what sort of game the designers are trying to make and what sort of game people wish to play.

I see these arguments in Football Manager a lot too, and it usually boils down to "the game engine hates my favorite player" and "my favorite team isn't doing as well as I liked" or "the game is too easy" and that's just not a real critique on the game, but rather, how you're playing it?

You could design an OOTP universe where bad players are less useful to teams, but asking for massive tweaks to the engine for it to work more like you want it to work....just doesn't seem to make sense to me. It's a difference of opinion, not a flaw in the game engine.

[Rain King]

I think the developers are trying to make a game that realistically represents baseball and the way that baseball players perform/evolve.

I think the OP has posed their question because they also want the players in the game to perform realistically.

There is a lot of analysis, discussion, agreement, and disagreement that can fit within both of those premises.

[OutS|der]

Originally Posted by holes573

First of all, the OOTP25 manual defines ratings as a measure of "player ability." So, that sounds like a measure of overall quality to me.

It's a measure, not the be all, end all definition of overall quality.

Overall quality, not just pitching quality.

Also the difference between 43 and 45 is nothing.

[jcard]

Originally Posted by holes573

Twins_34 and darkcloud4579:


You seem to have missed the point of this analysis. Of course bad pitchers will do well at times - there are lots of factors (including luck) about why they should. I haven't argued that "it's weird" that it happens or that there "should be a guarantee" that a 38 pitcher should get shelled.



Instead, I've argued that luck indeed happens, but that above and beyond that there is a "good day" algorithm at work. And that too is fine (and probably matches real baseball).


Rather, I've argued that the application of a good day to poorer pitchers appears too heavily weighted. The frequency of a less than 2.00 ERA outing, for medium and good pitchers should be more than for bad pitchers.

I admire your patience and restraint; your original post deserves better than it has received here.

[Matt Arnold]

If you're playing with morale and personalities on, then some players might get a bonus for the day for high morale. But those are relatively small factors.

I would say in the OP analysis, they might need to be a little more careful about un-intended relations. For example, the odds of someone throwing a shutout 7th inning after throwing non-shutout innings 1-6 is very low, since if they throw non-shutout innings 1-6, almost certainly they have either been pulled from the game. Similarly, if someone throws a shutout 8th inning, almost certainly that could be because they haven't turned the lineup over as much earlier in the game and gotten fatigued. Never mind some survivorship bias at play as well - if your data had 80% or so of 7th innings with shutouts from starters, that's going to be very biased for the guys who have the talent to make it that far.

As for the point of how many good/bad games a pitcher has, I think you might actually be slightly surprised by actual data in that regard. For example, in 2024, the worst SP by qualified ERA this last year was Corbin, and in 8/32 starts of his this year, he allowed 1 run or less. The 2nd worst, Mikolas, had 6/32. 3rd worst was Canning, who had 9/32 starts with 1 run or less. And all 3 ended up with ERAs north of 5.
Now, granted, the top arms this year seemed to be at 50% or more for 2 ER or less appearances. But we also live in a much more sheltered world than in 1965, where you comparison stands. I'd be willing to bet that the Sale or Burnes of the day instead of getting pulled after 6 innings of 1 run ball might last another couple innings and maybe give up a second run.

How much that would be, it's hard to say. IRL, even Koufax in 1965 had 1 run or less in 19 of his 43 starts. Gibson was 12/38. On the other end, Buhl was the worst qualified SP without a bunch of relief games, and he had 7/32 starts with 1 run or less. Simmons only had 4. Fisher had 6/36. So yeah IRL they seemed to have less for the worst guys. While we do our best, our usage isn't always a perfect match for some of those earlier eras. We certainly might be more aggressive at pinch hitting for a guy before he starts to struggle, or be stricter at pitch counts which they wouldn't have really cared about back then nearly as much.

I forget where I heard it or the exact quote, but I think in a lot of cases, the difference between a bad pitcher and a good pitcher isn't what they do on their best days, but what they do on their average or worst days. Chris Sale wasn't great last year because he necessarily had more great days that other guys. He was great because he just didn't have the bad games that others did. He "only" had 50% of his starts with 1 run or less allowed. But all but 3 of his starts he allowed 2 or less.

[Sweed]

Originally Posted by jcard

I admire your patience and restraint; your original post deserves better than it has received here.

In his first thread he was thanked by more than one poster for testing and submitting data. The OP also commented there was "lots of good discussion" in the thread.

What exactly does he deserve other than the civil discussion he has received?

[holes573]

Originally Posted by Matt Arnold

If you're playing with morale and personalities on, then some players might get a bonus for the day for high morale. But those are relatively small factors.

I would say in the OP analysis, they might need to be a little more careful about un-intended relations. For example, the odds of someone throwing a shutout 7th inning after throwing non-shutout innings 1-6 is very low, since if they throw non-shutout innings 1-6, almost certainly they have either been pulled from the game. Similarly, if someone throws a shutout 8th inning, almost certainly that could be because they haven't turned the lineup over as much earlier in the game and gotten fatigued. Never mind some survivorship bias at play as well - if your data had 80% or so of 7th innings with shutouts from starters, that's going to be very biased for the guys who have the talent to make it that far.

As for the point of how many good/bad games a pitcher has, I think you might actually be slightly surprised by actual data in that regard. For example, in 2024, the worst SP by qualified ERA this last year was Corbin, and in 8/32 starts of his this year, he allowed 1 run or less. The 2nd worst, Mikolas, had 6/32. 3rd worst was Canning, who had 9/32 starts with 1 run or less. And all 3 ended up with ERAs north of 5.
Now, granted, the top arms this year seemed to be at 50% or more for 2 ER or less appearances. But we also live in a much more sheltered world than in 1965, where you comparison stands. I'd be willing to bet that the Sale or Burnes of the day instead of getting pulled after 6 innings of 1 run ball might last another couple innings and maybe give up a second run.

How much that would be, it's hard to say. IRL, even Koufax in 1965 had 1 run or less in 19 of his 43 starts. Gibson was 12/38. On the other end, Buhl was the worst qualified SP without a bunch of relief games, and he had 7/32 starts with 1 run or less. Simmons only had 4. Fisher had 6/36. So yeah IRL they seemed to have less for the worst guys. While we do our best, our usage isn't always a perfect match for some of those earlier eras. We certainly might be more aggressive at pinch hitting for a guy before he starts to struggle, or be stricter at pitch counts which they wouldn't have really cared about back then nearly as much.

I forget where I heard it or the exact quote, but I think in a lot of cases, the difference between a bad pitcher and a good pitcher isn't what they do on their best days, but what they do on their average or worst days. Chris Sale wasn't great last year because he necessarily had more great days that other guys. He was great because he just didn't have the bad games that others did. He "only" had 50% of his starts with 1 run or less allowed. But all but 3 of his starts he allowed 2 or less.

Very interesting data.


My first thought (and this is related to the question by Rain King) is that going deeper in games was very common in the 60's and before. That is reflected in OOTP by longer stamina, and shows up in how long the AI keeps starters in. 73% of the starts in my data went at least 6 innings and 53% went at least 7 innings.



That means the odds of throwing a shutout through 6 or more innings is more common in 1965. And, they happen for Bad Pitchers, too. Of the 114 Bad Pitcher starts, 14 (or 12% of them) resulted in 9 innings of shutout being pitched. So, any bias in later innings for better starters is likely much less than it is today. (Also, BTW, my "Luck Only" expectations are based to only those who make it to later innings, so if there is a bias toward better pitchers it affects those %'s as well, and can't explain why the %'s are higher in the "Actual Data" column.)


Your brief review of actual MLB data seems consistent with my expectation - everyone has "good days" some of the time, and better pitchers may have them somewhat more frequently than poorer pitchers.

[twins_34]

Originally Posted by holes573

Twins_34 and darkcloud4579:


You seem to have missed the point of this analysis. Of course bad pitchers will do well at times - there are lots of factors (including luck) about why they should. I haven't argued that "it's weird" that it happens or that there "should be a guarantee" that a 38 pitcher should get shelled.



Instead, I've argued that luck indeed happens, but that above and beyond that there is a "good day" algorithm at work. And that too is fine (and probably matches real baseball).


Rather, I've argued that the application of a good day to poorer pitchers appears too heavily weighted. The frequency of a less than 2.00 ERA outing, for medium and good pitchers should be more than for bad pitchers.

I am not trying to say you are wrong. Please do not take it that way. I was just trying to point out that, even though a pitcher might be ranked as a 38 overall, it does not make them a "bad" pitcher. I mean, they are not a good pitcher by any means either but, Rain made a great point. How many of them bad pitchers were pulled having a good start to their game, but pulled in the 5th with the lead and good relievers came in to close out the game?

I also don't think there is "good day" or "bad day" algorithm. I think it is all just RNG and the luck of a dice roll. I just had 2 - 70 overall pitchers get shelled by 40 and 50 overall batters two games in a row. It... just seems to happen but I do not think there is some hidden mechanic under the engine... not like them unstoppable drives in Madden where no matter what you try and do, you just can't stop that one drive and they just march down the field and score.

[twins_34]

Originally Posted by Matt Arnold

If you're playing with morale and personalities on, then some players might get a bonus for the day for high morale. But those are relatively small factors.

This is another great point that I total forgot about! Now this story takes place in OOTP 24 but maybe still holds true in 25?

In one of my online leagues, I had a pretty mediocre pitching core. During the off-season, I went out and just hired a bunch of coaches that got along with each other really well, and they became ecstatic, their ratings were not the best but, they were all happy with each other. Then I traded off some disruptive and outspoken players, and that helped boost the morale of my team. My overall teach chemistry was ecstatic, everyone had high morale.

All of a sudden, a team that was a little over 500 with some ok, pitching started to play over their ratings. Yeah I had an ace, but the rest of my pitchers were mid to upper 50 pitchers. Pitchers were very limited in this league for some reason. However, I made one trade at the deadline, and my team, that should have had no reason playing how it did and where it did, won the World Series. I beat teams that had way better records.

But yeah when you said that Matt, that took me back a few years and it is amazing how good cohesion and team morale can change how a team plays and even individual players themselves.