Dwarf Fortress Bug Tracker - Dwarf Fortress
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0007821Dwarf FortressAdventure Mode -- Sitespublic2014-08-03 13:272014-08-12 12:37
Reportersmjjames 
Assigned ToToady One 
PrioritynormalSeveritymajorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformWIndows 8OSOS Version
Product Version0.40.05 
Target VersionFixed in Version0.40.09 
Summary0007821: Animal overpopulations multiply/replenish after waiting/sleeping in worldgen fortresses, cause extreme lag
DescriptionWhen I went to a worldgen fortress to see if I could get some steel stuff (the dwarves never seem to have steel for some reason) or at least better iron and hit major lag. I tried to wait until night to have the NPCs sleep, but it seems to only have made it worse.

I suspect maybe pathing lag with cats trying to go through doors.
Steps To Reproduce1. Load up adventure save: http://dffd.wimbli.com/file.php?id=9266 [^]

2. try to move.
Additional Information
TagsNo tags attached.
Relationships
parent of 0007931resolved lethosor Really bad FPS lag in Trade Depot 
related to 0005694new  Memory leak / performance issues with swimming long distances 
child of 0006623confirmed Footkerchief Game freezes after each move 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2014-08-03 13:27smjjamesNew Issue
2014-08-05 04:38MopsyNote Added: 0028371
2014-08-05 04:41MopsyNote Edited: 0028371bug_revision_view_page.php?bugnote_id=0028371#r10783
2014-08-05 04:42MopsyNote Edited: 0028371bug_revision_view_page.php?bugnote_id=0028371#r10784
2014-08-05 04:42MopsyNote Edited: 0028371bug_revision_view_page.php?bugnote_id=0028371#r10785
2014-08-05 04:43MopsyNote Edited: 0028371bug_revision_view_page.php?bugnote_id=0028371#r10786
2014-08-05 04:44MopsyNote Edited: 0028371bug_revision_view_page.php?bugnote_id=0028371#r10788
2014-08-05 04:46MopsyIssue Monitored: Mopsy
2014-08-05 13:03cephaloNote Added: 0028394
2014-08-05 14:26MopsyNote Added: 0028396
2014-08-05 14:38smjjamesNote Added: 0028398
2014-08-10 09:42lethosorStatusnew => assigned
2014-08-10 09:42lethosorAssigned To => lethosor
2014-08-10 09:43lethosorRelationship addedrelated to 0007931
2014-08-10 09:46FootkerchiefRelationship addedrelated to 0006623
2014-08-10 10:17lethosorNote Added: 0028733
2014-08-10 10:19smjjamesNote Added: 0028734
2014-08-10 10:22lethosorNote Edited: 0028733bug_revision_view_page.php?bugnote_id=0028733#r11014
2014-08-10 10:24lethosorNote Added: 0028735
2014-08-10 10:24lethosorStatusassigned => confirmed
2014-08-10 10:25lethosorSummaryExtreme lag in worldgen fortress => Extreme animal overpopulation (inconsistent with legends reports) causes lag in worldgen fortress
2014-08-10 10:28lethosorRelationship replacedchild of 0006623
2014-08-10 10:30lethosorNote Edited: 0028733bug_revision_view_page.php?bugnote_id=0028733#r11015
2014-08-10 10:49lethosorNote Edited: 0028735bug_revision_view_page.php?bugnote_id=0028735#r11017
2014-08-10 10:50lethosorSummaryExtreme animal overpopulation (inconsistent with legends reports) causes lag in worldgen fortress => Animal overpopulations multiply/replenish after waiting/sleeping in worldgen fortresses, cause extreme lag
2014-08-10 10:56lethosorNote Edited: 0028735bug_revision_view_page.php?bugnote_id=0028735#r11018
2014-08-11 16:33smjjamesNote Added: 0028842
2014-08-11 16:43smjjamesNote Edited: 0028842bug_revision_view_page.php?bugnote_id=0028842#r11067
2014-08-11 17:01smjjamesNote Edited: 0028842bug_revision_view_page.php?bugnote_id=0028842#r11072
2014-08-11 17:02smjjamesNote Edited: 0028842bug_revision_view_page.php?bugnote_id=0028842#r11073
2014-08-11 17:05smjjamesNote Edited: 0028842bug_revision_view_page.php?bugnote_id=0028842#r11074
2014-08-12 12:37Toady OneStatusconfirmed => resolved
2014-08-12 12:37Toady OneFixed in Version => Next Version
2014-08-12 12:37Toady OneResolutionopen => fixed
2014-08-12 12:37Toady OneAssigned Tolethosor => Toady One
2014-08-13 13:22FootkerchiefRelationship addedrelated to 0005694
2017-11-26 11:47lethosorRelationship replacedparent of 0007931

Notes
(0028371)
Mopsy   
2014-08-05 04:38   
(edited on: 2014-08-05 04:44)
This is probably related or identical to issue 0007553. I second the guess about cats stuck at doors. In the fortresses I've visited that had this problem, there where 30+ cats piled up in a corridor. I couldn't find anything else that seemed out of the ordinary.
(0028394)
cephalo   
2014-08-05 13:03   
Some of these sites have very large populations. I've been trying to explore a goblin dark tower, and it has taken me an afternoon to approach the center of the site from the edge. There are 20,000 creatures at this site, and that's probably why it's so slow, but if pop numbers make areas inaccessible, maybe 20,000 is way too high.
(0028396)
Mopsy   
2014-08-05 14:26   
It occurred to me that this type of lag may be specific to dwarf fortresses (and dark towers?) because of the vertical shape of those sites. If the game always loads every layer of the current area, all the way from the sky down to the HFS, then all the creatures in the fortress will be loaded at once.

That hypothesis still doesn't explain why the lag gets worse every time the site is unloaded and then reloaded.
(0028398)
smjjames   
2014-08-05 14:38   
Actually, I haven't had that problem with dark towers when the site is unloaded and then reloaded. The main reason for the lag in those sites is because they have populations in the thousands, often exceeding 5k in total, or 10k. They also don't have lots of animals running around to clog up the pathfinding.
(0028733)
lethosor   
2014-08-10 10:17   
(edited on: 2014-08-10 10:30)
A list of creatures at this site (generated by DFHack):
 BIRD_SWALLOW_CAVE_GIANT 1
 BLIND_CAVE_OGRE 1
 GIANT_EARTHWORM 1
 OLM_GIANT 1
 RAT_LARGE 1
 REACHER 1
 VORACIOUS_CAVE_CRAWLER 1
 JABBERER 2
 RAT_GIANT 2
 TOAD_GIANT 2
 TROLL 2
 CROCODILE_CAVE 3
 IMP_FIRE 3
 MOLEMARIAN 3
 CAVE_BLOB 4
 HUMAN 5
 MAGMA_CRAB 5
 POND_GRABBER 5
 TROGLODYTE 6
 FLESH_BALL 18
 CRUNDLE 19
 CREEPING_EYE 32
 DWARF 198
 ALPACA 954
 BIRD_PEAFOWL_BLUE 954
 DONKEY 954
 HORSE 954
 RABBIT 954
 SHEEP 954
 BIRD_TURKEY 1908
 CAT 1908

I did locate a large number of turkeys and cats in a 48x48 block in the caverns (http://i.imgur.com/fJaS1mP.png [^]), centered around cursor coordinates (26, 117, 35) (for comparison, the adventurer starts out at (80, 71, 46)).

Edit: Found an even larger cluster of animals around (27, 13, 34), averaging around 8 per tile in nearly every occupiable tile in that 48x48 block (including trees and across z-levels): http://i.imgur.com/gbS1GgP.png [^]

(Both of these are in the first cavern layer.)
(0028734)
smjjames   
2014-08-10 10:19   
I wonder if that is population duplication because I swear the site pop in LegendsViewer was NOWHERE near that. If it was that high, I would have avoided it alltogther.
(0028735)
lethosor   
2014-08-10 10:24   
(edited on: 2014-08-10 10:56)
Research at http://www.bay12forums.com/smf/index.php?topic=142064.msg5560451#msg5560451 [^] confirms that the animals are the cause of the lag.

Edit: waiting (and presumably sleeping) causes the populations to replenish:
 ALPACA 1362
 BIRD_PEAFOWL_BLUE 1362
 DONKEY 1362
 HORSE 1362
 RABBIT 1362
 SHEEP 1362
 BIRD_TURKEY 2724
 CAT 2724


The re-population is reminiscent of 0006015 and 0005106, but I'm not sure if they're directly related.
(0028842)
smjjames   
2014-08-11 16:33   
(edited on: 2014-08-11 17:05)
Just noting I'm going to test this in a worldgen fort in 40.08 so that we have some datasets of before it got laggy (ie. when I exit fast travel in the site) and data for the newer version. I'll post the saves here and on the thread lethosor posted the link to.

Initial arrival at the fortress: http://dffd.wimbli.com/file.php?id=9360 [^]

Legends Viewer says that the population is:
212 Dwarves
33 Dogs
24 Cavies
24 Cats
20 Guineafowls
15 Ducks
12 Pigs
12 Reindeers
12 Alpacas
10 Humans
10 Blue Peafowls
2 Water Buffaloes

Waited 4 hours at trade depot, http://dffd.wimbli.com/file.php?id=9361 [^] The lag got somewhat worse, not the horrible lag I had with the other one, but it's still unplayable lag.


http://dffd.wimbli.com/file.php?id=9362Waited [^] 4 hours again and the lag is..... actually a bit better than at arrival??? The trade depot also seems to have broken because I can't take stuff from it where I could earlier.