Generating History HTML Takes Hours?

gordyhulten · 06-05-2009, 09:07 PM

I'm running 100-year sims and trying to generate an HTML history, and it's taking four hours or longer. Why?

Here's an example of the options I have checked. I'm using the default reports, and have selected nothing manually, yet it still take four hours or longer to generate. It only takes my computer four hours to sim 100 years, as a frame of reference.

Am I doing something wrong?

cody8200 · 06-05-2009, 11:17 PM

Nope, nothing wrong. Your computer is literally creating several hundred thousand files that are connected. It takes a long time to do that. Thus the wait. I've sen some leagues take 12 hours. Good to run overnight.

mm7607 · 06-06-2009, 02:10 AM

You have got to be kidding? And to get any type record book we must do this after every season. This is why this game need a built in encyclopedia. Instead of wasting space on the silly news stories, ridiculous ball graphics and useless other stats, a record book could be made to rival others. Even the ancient BBW has a record book to put OOTP to shame.

darkcloud4579 · 06-06-2009, 02:22 AM

Nope, it's always this slow and yes it's ridiculous and absurd.

thomamon · 06-06-2009, 06:59 AM

I have noticed that the generating is about 1/10th the time then it took in 9, so I'm a bit shocked by this! I know the History is a beast, it has everything from every season.

Andreas Raht · 06-06-2009, 07:08 AM

Quote:

Originally Posted by gordyhulten

I'm running 100-year sims and trying to generate an HTML history, and it's taking four hours or longer. Why?

Here's an example of the options I have checked. I'm using the default reports, and have selected nothing manually, yet it still take four hours or longer to generate. It only takes my computer four hours to sim 100 years, as a frame of reference.

Am I doing something wrong?

No, you're not doing something wrong. Please try to disable anti virus software for the OOTP saved_games folder so that file creation works faster in that folder. Make sure the saved_games folder is on a fast drive.

I'm interested in testing this on my computer, could you please upload me the report .dat files and the folder /news/txt? (See our FAQ for FTP upload info) Thank you!

fantom1979 · 06-06-2009, 02:47 PM

Quote:

Originally Posted by cody8200

Nope, nothing wrong. Your computer is literally creating several hundred thousand files that are connected. It takes a long time to do that. Thus the wait. I've sen some leagues take 12 hours. Good to run overnight.

All he has checked is the history reports, which according to the screen shot is 2200 pages. Should this really take hours?

Andreas Raht · 06-06-2009, 03:49 PM

Quote:

Originally Posted by fantom1979

All he has checked is the history reports, which according to the screen shot is 2200 pages. Should this really take hours?

No, it shouldn't, and according to our own tests it doesn't... Please send me those league files for testing.

gordyhulten · 06-06-2009, 03:53 PM

Quote:

Originally Posted by Andreas Raht

No, it shouldn't, and according to our own tests it doesn't... Please send me those league files for testing.

I will. Right now I'm running another 100-year test-sim.

Thanks, Andreas.

gordyhulten · 06-06-2009, 05:01 PM

Andreas,

The .dat files and the /news/txt/ folder is uploading now, zipped in a file with my username. Please let me know if I'm doing something wrong.

Thanks again for looking into this.

Andreas Raht · 06-07-2009, 06:12 AM

Thank you! Downloading now, testing ASAP.

thomamon · 06-07-2009, 09:08 AM

Quote:

Originally Posted by Andreas Raht

Thank you! Downloading now, testing ASAP.

Please let us know your results!

fantom1979 · 06-08-2009, 12:26 PM

Quote:

Originally Posted by thomamon

Please let us know your results!

Its only been a day, but anything on this yet?

Andreas Raht · 06-09-2009, 03:13 AM

Alright, loaded the gordyhulten league, game date is 10-20-2000, gpoing to create reports dialog. Checked is only Republican League history. "Select reports", all reports activated except "include pages for retired players". Report level: online reports. Clicked on "create reports" now...

Andreas Raht · 06-09-2009, 03:17 AM

Report 1100 now... but that doesn't mean much because the "slow" reports have not been created yet.
The problem is the huge amount of players. For some history reports OOTP needs to find out which players ever played for a given league or a team. OOTP needs to scan all the player batting and pitching stats to determine it. That needs a long time. We already added a "cache" which is being built when you create reports and which holds this information for each player so that OOTP has to scan the stats only once. But it still needs some time...

Andreas Raht · 06-09-2009, 03:20 AM

2100 now...

Andreas Raht · 06-09-2009, 03:22 AM

2300...

Andreas Raht · 06-09-2009, 03:25 AM

2500...

BTW this is on a Windows XP system with AMD Athlon 64 X2 Dual, 3 GB RAM, with a lot of other apps running in the background...

Andreas Raht · 06-09-2009, 03:28 AM

2800....

Well what I have is a very fast harddisk drive. And I have disabled MS Index Service for my saved_games folder. And I've also disabled the virus scanner for the saved_games folder. That's really important.

Andreas Raht · 06-09-2009, 03:33 AM

3250, getting slower...

Probably a good idea to open up the task manager and check which processes are using CPU power. On my system it's only 50% OOTP10 and 50% system idle process. It there would be 20% virus scanner or cidaemon or something like that I would need to change some settings for these apps.

06-05-2009, 09:07 PM	#1
gordyhulten All Star Starter Join Date: Jan 2003 Location: Champaign, IL Posts: 1,175	Generating History HTML Takes Hours? I'm running 100-year sims and trying to generate an HTML history, and it's taking four hours or longer. Why? Here's an example of the options I have checked. I'm using the default reports, and have selected nothing manually, yet it still take four hours or longer to generate. It only takes my computer four hours to sim 100 years, as a frame of reference. Am I doing something wrong? __________________ Gordy Hulten Owner / General Manager Red Willow Roadrunners -- Kennel Series Champions: 1951, 1959, 1964, 1965, 1972, 1975, 1980, 1982, 1983 Dog Days Baseball - "The World's Best Online OOTP League" Creator inactive: Republican League - OOTP 2009 Dynasty inactive: Republican League Dynasty - Version 2.0 inactive: Republican League Dynasty

06-06-2009, 02:22 AM	#4
darkcloud4579 Hall Of Famer Join Date: Jun 2003 Posts: 8,689	Nope, it's always this slow and yes it's ridiculous and absurd. __________________ ------ My Mods OOTP Advanced Stats & DFS Suite Managerial Strategy Pack Competitive Balance Tax Calculator Major League Women's Baseball (OOTP24) quickstart Indian Premier League \| 300+ years of baseball quickstart \| Expatriate League quickstart \| Off-Field Injuries Update \| Women's Name File for OOTP \| ---- Dynasty classics: Centurion comes to OOTP5 \| DC Moneyball Dynasty (2004)

06-06-2009, 05:01 PM	#10
gordyhulten All Star Starter Join Date: Jan 2003 Location: Champaign, IL Posts: 1,175	Andreas, The .dat files and the /news/txt/ folder is uploading now, zipped in a file with my username. Please let me know if I'm doing something wrong. Thanks again for looking into this. __________________ Gordy Hulten Owner / General Manager Red Willow Roadrunners -- Kennel Series Champions: 1951, 1959, 1964, 1965, 1972, 1975, 1980, 1982, 1983 Dog Days Baseball - "The World's Best Online OOTP League" Creator inactive: Republican League - OOTP 2009 Dynasty inactive: Republican League Dynasty - Version 2.0 inactive: Republican League Dynasty

06-07-2009, 06:12 AM	#11
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	Thank you! Downloading now, testing ASAP. __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-09-2009, 03:13 AM	#14
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	Alright, loaded the gordyhulten league, game date is 10-20-2000, gpoing to create reports dialog. Checked is only Republican League history. "Select reports", all reports activated except "include pages for retired players". Report level: online reports. Clicked on "create reports" now... __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-05-2009, 11:17 PM	#2
cody8200 Hall Of Famer Join Date: Jan 2003 Location: Indianapolis Posts: 2,436	Nope, nothing wrong. Your computer is literally creating several hundred thousand files that are connected. It takes a long time to do that. Thus the wait. I've sen some leagues take 12 hours. Good to run overnight.

06-06-2009, 02:10 AM	#3
mm7607 All Star Reserve Join Date: Jun 2006 Location: Originally from Fla, now in Ky Posts: 710	You have got to be kidding? And to get any type record book we must do this after every season. This is why this game need a built in encyclopedia. Instead of wasting space on the silly news stories, ridiculous ball graphics and useless other stats, a record book could be made to rival others. Even the ancient BBW has a record book to put OOTP to shame.

06-06-2009, 06:59 AM	#5
thomamon Global Moderator Join Date: Sep 2003 Location: NJ Posts: 5,274	I have noticed that the generating is about 1/10th the time then it took in 9, so I'm a bit shocked by this! I know the History is a beast, it has everything from every season.

06-09-2009, 03:17 AM	#15
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	Report 1100 now... but that doesn't mean much because the "slow" reports have not been created yet. The problem is the huge amount of players. For some history reports OOTP needs to find out which players ever played for a given league or a team. OOTP needs to scan all the player batting and pitching stats to determine it. That needs a long time. We already added a "cache" which is being built when you create reports and which holds this information for each player so that OOTP has to scan the stats only once. But it still needs some time... __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments Last edited by Andreas Raht; 06-09-2009 at 03:20 AM.

06-09-2009, 03:20 AM	#16
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	2100 now... __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-09-2009, 03:22 AM	#17
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	2300... __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-09-2009, 03:25 AM	#18
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	2500... BTW this is on a Windows XP system with AMD Athlon 64 X2 Dual, 3 GB RAM, with a lot of other apps running in the background... __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments Last edited by Andreas Raht; 06-09-2009 at 03:26 AM.

06-09-2009, 03:28 AM	#19
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	2800.... Well what I have is a very fast harddisk drive. And I have disabled MS Index Service for my saved_games folder. And I've also disabled the virus scanner for the saved_games folder. That's really important. __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments Last edited by Andreas Raht; 06-09-2009 at 03:29 AM.

06-09-2009, 03:33 AM	#20
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	3250, getting slower... Probably a good idea to open up the task manager and check which processes are using CPU power. On my system it's only 50% OOTP10 and 50% system idle process. It there would be 20% virus scanner or cidaemon or something like that I would need to change some settings for these apps. __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments Last edited by Andreas Raht; 06-09-2009 at 03:37 AM.