test environment??

[chriskelly]

I want to run a test using 19 where I see how different team focusses (power, on base, defense, SP, bullpen, balanced) impact wins. I was toying with setting up a 10 team fictional league with no minors, no injuries, no development; and altering ratings of each team to suit the above (in other words, on team 1, bump everyones power up 10 pts on an 80 scale, etc). every time I think about it, I worry it's too simplistic an approach. I dont' know how to alter defensive ratings. I don't know how to factor in that certain positions are more crucial for defense than others, etc. Any advice?

[NoOne]

make use of backups so you can re-do a test etc.. rename them, keep as many as common sense dictates. don't re-do work that doesn't need to be redone.

changing distribution can have it's own effect on what you are testing. you don't want to do that.

make a league as close to how you intend to play. this will be important.

replace a team, maybe multiple if it doesn't create too much of a ripple, and make them identical (clone players, coaches, gm, scout, budgets everything) in all ways except for what you want to test and one at a time. if 2 leagues and exclusive schedules, even better. run a few leagues in parallel for all the foci you want to test. keep their schedules exclusive from each other of course.

once you make one league, clone it (use template durign creation... then add it to itself... repeat and you have multple of exactly the same settings etc in the same game world -- rename the leagues so you don't get confused)

if it is a fictional league, i'd also zoom out 30years before doing any of the above... seed players (initial) are not the same as what will be created over time. make a backup after this so you can restart a long-run sim if necessary.

make sure stats stay callibrated.. auto-calcing the LTM in year once should be ~enough, if you zoomed out far enough to rid league of seed players, unless there's something funky with how it works in the long-run. just make sure slash and era doesn't drift too far... if it ebbs and flows, that's perfectly normal... it won't go from ~4.10 20year average to a 4.75 20 year average type thing. figure +/- ~10+% over time of era with a static set of LTM/LT.

leadds to.. make sure to set "historic year", not calendar date, to post 2018. that way no historic stats, financials or settings import from a historic year on accident. you want it to remain the same and closely resembling what you intend to play.

since it's relatively the same settings and such, i'd stick to same schedule length, unless you turn off injuries, then more games an dmore teams is likley going to speed up the process, but then how does scaling affect results? it's best to keep it as similar to how you intend to play -- # of teams as well as games played -- even injuries and aging. no telling what a side effect could cause.

you will need more than one season.. more than just a handful years for something as small as this effect. if you can make age a non-factor and lock development i guess it'd be okay? you could reduce their ages -1 each year and import back into game, but you want to avoid having to do anyhtin by hand if you can. it may be better to do a longer run sim and leave development and injuries as they are... you will need an absolutely overwhelming mass of data. let it run over night, then collect data when you have time the next day, maybe even let it run a few nights, lol.

you can have the stats you want to mine exported yearly automatically.. if you built the spreadsheet correcly, you can jsut plug and play at any time. update links to the exported files and pow, it's done and up to date with the new data.

by keeping one league going, it makes exporting and mining the date much easier to automate with a spreadsheet, if you are familiar with setting them up properly. if you do it right, you can re-run the test, export stats, import tinto spreadsheet and it auto-updates the data and the data mining is complete immediately without any work from you. (hint, everythign shuold be a formula linking back to the imported worksheets.. .have a dedicated 'output' worksheet, leave the imported worksheets unmolested and only link to it - "worksheet1.a1" etc)

anyway... whether 1 team and re-run it or 2 teams at once make sure only 1 change is made between the two of what you can control, then compare the results.

you will need a ton of seasons. to give an idea from mlb - its only about 40% chance the best record is the best team. re-run the same season over and over in ootp and you probably have a +/-10wins at least and that's without injuries on. so, you need a sample to reduce that volatility to the point you can distinguish what a coach focus is causing with any confidence, whic is going to be an incredibly small factor of the total pie -- even more reason for a huge sample size..

now, what are you tracking to deduce the effect? wins? BA? hr? slugging? etc etc. keep it as fundamental and as directly tied to what you are testing. since there's no 1 answer, cross-reference as many related factors as possible. runa regression model? something.

anythign less will have no integrity to the findings.