Generating History HTML Takes Hours? - Page 2

Andreas Raht · 06-09-2009, 03:42 AM

4000, still 76%, maybe 3 to 5 reports per second...

Andreas Raht · 06-09-2009, 04:12 AM

6400.... well I guess the problem with your computer might just be the RAM... OOTP 10 uses 660 MB now on my computer, a lot of it used by the cache which actually should speed up the history reports creation. If your RAM is not big enough it will use the swap file and then it will get very, very slow. In that case the cache could slow down the report creation instead of making it faster. And the more stuff I cache, the faster it will be here on my computer (having enough RAM) and at the same time it will get slower on your computer.

Andreas Raht · 06-09-2009, 04:35 AM

8500... still 76%... I'm working and compiling at the same time... well if I would run leagues like this I would create the reports over night

Andreas Raht · 06-09-2009, 05:22 AM

Alright, finished after creating 11.771 reports (reports_set_online_league_files.txt has 11.771 reports listed).
Started at 8:13, now it's 10:22, so it needed ~ 2 hours 9 minutes or 129 minutes. That's 91.2 reports per minute. That's slow. But not as slow as on the OPs computer. I guess it's a problem of not having enough RAM.

Some numbers for the test-league:

Number of leagues: 6
Number of teams: 156
Number of players: 59,723
Min year: 1901
Max year: 2000
Number of combinations league/team/player/year: 1,431,594

gordyhulten · 06-09-2009, 01:47 PM

Quote:

Originally Posted by Andreas Raht

Alright, finished after creating 11.771 reports (reports_set_online_league_files.txt has 11.771 reports listed).
Started at 8:13, now it's 10:22, so it needed ~ 2 hours 9 minutes or 129 minutes. That's 91.2 reports per minute. That's slow. But not as slow as on the OPs computer. I guess it's a problem of not having enough RAM.

Some numbers for the test-league:

Number of leagues: 6
Number of teams: 156
Number of players: 59,723
Min year: 1901
Max year: 2000
Number of combinations league/team/player/year: 1,431,594

I have a fairly robust machine with 3GB of RAM, and I just let OOTP run these by itself with nothing else running.

I've run into the issue on multiple leagues on this machine, so I wonder if there's a software conflict somewhere. Thanks for looking into it, and I'll keep testing.

fantom1979 · 06-09-2009, 03:29 PM

Great report Andreas. 2 1/2 hours for html reports is about the same that I experience on my machine. I would love it if somehow the process could be sped up.

I am a speed simmer, I usually do 1/2 a season to a full season every time I sit down and play. I like to create these reports every year and ftp them up to my website for viewing outside of the game. I also create the sql files, combine them using a program called text magician and input that into my Mysql database. I then have php scripts to find out stats that I cannot easily find ingame (most career awards [mvp, cy young, gold glove, etc], best team record over the last ten years, longest streak since a playoff appearance, best players at a certain age, and other stats like that).

For me to create the html files, ftp them up to my server, create the sql files, combine them to together, and input them into the database, is about an eight hour experience. If anyone has any ideas on how to speed that up, I would love to know. Also, i cannot do it all overnight, since there are multiple steps.

Andreas Raht · 06-11-2009, 06:00 AM

Quote:

Originally Posted by fantom1979

Great report Andreas. 2 1/2 hours for html reports is about the same that I experience on my machine. I would love it if somehow the process could be sped up.

I am a speed simmer, I usually do 1/2 a season to a full season every time I sit down and play. I like to create these reports every year and ftp them up to my website for viewing outside of the game. I also create the sql files, combine them using a program called text magician and input that into my Mysql database. I then have php scripts to find out stats that I cannot easily find ingame (most career awards [mvp, cy young, gold glove, etc], best team record over the last ten years, longest streak since a playoff appearance, best players at a certain age, and other stats like that).

For me to create the html files, ftp them up to my server, create the sql files, combine them to together, and input them into the database, is about an eight hour experience. If anyone has any ideas on how to speed that up, I would love to know. Also, i cannot do it all overnight, since there are multiple steps.

Wow, that's impressive!
We continously try to improve the speed of the game. With more and more features being added to the game, it would actually have to become slower and slower. It doesn't, and that is because we really put a lot of work into it to make it faster. Still, it's "slow" when you have everything turned "On", but who wants to turn all the great features "Off"? There are so many different ways to play this game. For most people the game is fast enough. Whether it needs 1 second to sim a day or 3 seconds doesn't matter much when you sim day by day. But when you run a 100 years simulation it will of course matter. That's why there are some users who don't have any performance issues with OOTP and others who shout that it's totally slow

Well, as mentioned, we're working on it

06-09-2009, 03:42 AM	#21
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	4000, still 76%, maybe 3 to 5 reports per second... __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-09-2009, 04:12 AM	#22
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	6400.... well I guess the problem with your computer might just be the RAM... OOTP 10 uses 660 MB now on my computer, a lot of it used by the cache which actually should speed up the history reports creation. If your RAM is not big enough it will use the swap file and then it will get very, very slow. In that case the cache could slow down the report creation instead of making it faster. And the more stuff I cache, the faster it will be here on my computer (having enough RAM) and at the same time it will get slower on your computer. __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-09-2009, 04:35 AM	#23
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	8500... still 76%... I'm working and compiling at the same time... well if I would run leagues like this I would create the reports over night __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments

06-09-2009, 05:22 AM	#24
Andreas Raht Administrator Join Date: Jun 2002 Location: Hollern/Stade/Germany Posts: 8,992	Alright, finished after creating 11.771 reports (reports_set_online_league_files.txt has 11.771 reports listed). Started at 8:13, now it's 10:22, so it needed ~ 2 hours 9 minutes or 129 minutes. That's 91.2 reports per minute. That's slow. But not as slow as on the OPs computer. I guess it's a problem of not having enough RAM. Some numbers for the test-league: Number of leagues: 6 Number of teams: 156 Number of players: 59,723 Min year: 1901 Max year: 2000 Number of combinations league/team/player/year: 1,431,594 __________________ Andreas Raht andreas@ootpdevelopments.com Out of the Park Developments Last edited by Andreas Raht; 06-09-2009 at 09:36 AM.

06-09-2009, 03:29 PM	#26
fantom1979 Hall Of Famer Join Date: Jul 2002 Location: Detroit, MI Posts: 3,498	Great report Andreas. 2 1/2 hours for html reports is about the same that I experience on my machine. I would love it if somehow the process could be sped up. I am a speed simmer, I usually do 1/2 a season to a full season every time I sit down and play. I like to create these reports every year and ftp them up to my website for viewing outside of the game. I also create the sql files, combine them using a program called text magician and input that into my Mysql database. I then have php scripts to find out stats that I cannot easily find ingame (most career awards [mvp, cy young, gold glove, etc], best team record over the last ten years, longest streak since a playoff appearance, best players at a certain age, and other stats like that). For me to create the html files, ftp them up to my server, create the sql files, combine them to together, and input them into the database, is about an eight hour experience. If anyone has any ideas on how to speed that up, I would love to know. Also, i cannot do it all overnight, since there are multiple steps. __________________ Last edited by fantom1979; 06-09-2009 at 03:32 PM.