0005622: Crash on large embark - memory related

Notes
(0021370) Logical2u (manager) 2012-03-11 13:26	It sounds like you modified system files while running Dwarf Fortress. That's obviously a bad move, and I'm going to pre-emptively close this since you already mentioned that a fresh Dwarf Fortress install fixed the crashing.

(0021382) Nomlets (reporter) 2012-03-11 18:14	I'm reopening this because; One, Dwarf Fortress still froze, even after a reinstall(twice), And two, During the embark, Windows reported that Dwarf Fortress was not responding and closed the program, something it hasn't done on my PC before. Evidence would point to one of the updates I had downloaded, but I have no idea which one. I also have no idea of how to find out a list of the downloaded updates so I could find out through a mind-numbing check of all the updates/whatever I have downloaded today. I suggest, if possible, checking to make sure Dwarf Fortress doesn't conflict with any Windows XP updates.

(0021389) Footkerchief (manager) 2012-03-11 19:28	One, Dwarf Fortress still froze, even after a reinstall(twice), Froze where? Any freeze/crash that continues after reinstalling the game probably has nothing to do with Windows Update.

(0021392) Nomlets (reporter) 2012-03-11 19:35 edited on: 2012-03-11 19:51	Froze during embark process, when I start the embark with "Play now", or embarking in any other way. Quick Edit: Also, during the embark process when it freezes, my PC didn't make noise. It usually does make noise when playing Dwarf Fortress, doing something that takes up time/whatever, ect. This is suspicious, and makes me think that my computer is refusing to do certain actions relating to the embark process. I'm going to figure out more about this in the morning.

(0021394) Logical2u (manager) 2012-03-11 20:07	A specific error message from Windows might help. Also, if you know your memory usage during the embark process, and could report whether truetype is enabled, those two things could also help (although I doubt either is at play here, they could be contributing). If a fresh install of dwarf fortress (including a new save/world, just in case the save/world was corrupted in some way) is still crashing, you may also want to rollback the Windows updates and then install them again, then try again (this time without dwarf fortress running).

(0021407) Nomlets (reporter) 2012-03-12 08:00 edited on: 2012-03-12 08:11	Okay, I'll give you the particular message. Also, I can run taskmanager during the embark, and see what's going on. I'll also use Lazy Newb Pack to check whether TrueType is enabled. Also, I have no idea if I can rollback updates... This is a pretty old computer. Hmm, started and it worked fine. TrueType was enabled, priority was set at realtime... Test two: (This may be a particular biome, as I keep trying to embark on mountains with woods next to them. Either that or Phoebus, which is what texture pack I normally use.) Same as the above settings, only on a mountain with woods. Game froze. It seems to only be mountain/wood biomes that have more then 50% of the map being mountains. I'm just going to stay away from mountains until this is fixed.

(0021409) Logical2u (manager) 2012-03-12 08:43	And what was the memory usage when it started freezing? Mountain biomes have a lot of extra z-levels which get loaded into memory when you embark.

(0021410) Footkerchief (manager) 2012-03-12 09:20	What size areas are you embarking on? The default is something like 3x3.

(0021411) Nomlets (reporter) 2012-03-12 09:49	@logical, I'd have to check again for that. Hold on... 50-60% CPU usage during embark. @Footker, Whole map. However, i've managed to embark on the whole map of a local area with mountain biomes before, so this isn't the problem, if that's what you're wondering.

(0021413) Logical2u (manager) 2012-03-12 10:07	I'd still like the specific Windows error message that pops up when it crashes. That's CPU usage, I'm interested in your memory usage. If you're embarking on an entire world map tile, though, I would not be surprised if you were hitting the 2gb memory limit.

(0021415) Footkerchief (manager) 2012-03-12 10:08 edited on: 2012-03-12 10:12	@Footker, Whole map. However, i've managed to embark on the whole map of a local area with mountain biomes before, so this isn't the problem, if that's what you're wondering. It is almost certainly the problem. See 0000136. Logical2u asked about the memory usage, not the CPU usage, and that is the crucial question here. Some whole-map embarks will hit the 2 GB limit, others won't. It's extremely dependent on local geography. Mountain biomes in particular exacerbate the problem, because twice as many Z-levels requires twice as much memory. Also, if you generated this world in a version before 0.34.03, you may be experiencing 0005077/0002713.

(0021418) Nomlets (reporter) 2012-03-12 10:15 edited on: 2012-03-12 10:31	That isn't the problem. Like I said, i've managed to embark on the whole map, on something similar to what i've tried to embark on now, so that isn't the problem. Edit: Let me check, but I doubt that's the problem either, because this has been on several different mountain/woodland areas. 0.34.02, is what it seems to be. Also, I generated several different worlds. Installed SDL DF 0.34.05. Embarked on wood/mountain biomes together. Froze.

(0021419) Footkerchief (manager) 2012-03-12 10:31 edited on: 2012-03-12 10:33	What happens to memory usage during embark? Click this link and read the report: 0000136. Whole-map embarks will often cause exactly the behavior you're describing. edit: you don't seem interested in answering questions. Do not reopen this report unless you have verified that your memory usage does not approach 2 GB during embark.

(0021420) Nomlets (reporter) 2012-03-12 10:53 edited on: 2012-03-12 10:59	Reopened again because: One, Mem usage happened to be around 22,000K, which I think is less than 2G, Edit: Checked again and saw that it was constantly going down from a maximum of 500K to 22K. And two: I checked the problem in 000136, and ran a series of tests. 16x16 froze, and when I opened up Taskmanager to check, said that two instances of DF were running. 10x10 froze. 9x9 froze. 8x8 froze. 7x7 embark worked, but lagged horribly when unpaused. In 000136, there were claims that 10x10 froze, but 9x9 worked. Specific Windows Error: "Dwarf Fortress.exe has encountered a problem and needs to close. We are sorry for the inconvenience. If you were lost in the middle of something, the information you were working on might be lost." More Info: Appname: dwarf fortress.exe AppVer: 0.0.0.0 ModName: msvcr100.dll ModVer: 10.0.30319.1 Offset: 0008d635 Tech Info: Windows refuses to copy, arggh. Gonna make that a seperate post perhaps.

(0021423) Nomlets (reporter) 2012-03-12 11:09	Just waiting for it to cook... Error Report Contents: Exception information code: 0x40000015 Flags: 0x00000001 Record: 0x0000000000000000000 Adress: 0x00000000078b2d635 System Information: Module 1 Dwarf Fortress.exe Image Base: 0x00400000 Image Size: 0x0000000 Checksum: 0x008a28w6 Time Stamp: 0x4f55f23e Version Information Signature: 000000 StrucVer: 0000000 Filever: (0.0:0.0) ProdVer: (0.0:0.0) FlagMask: 00000000 Flags: 00000000 OS: 00000000 FileType: 000000 SubType: 0000000 FileDate: 0000000:00000000

(0021424) Footkerchief (manager) 2012-03-12 11:26 edited on: 2012-03-12 11:47	One, Mem usage happened to be around 22,000K, which I think is less than 2G, Edit: Checked again and saw that it was constantly going down from a maximum of 500K to 22K. I'm going to hazard a guess that you aren't looking at the memory until it's already crashed. You need to watch memory usage during the entire embark process. 10x10 froze. 9x9 froze. 8x8 froze. 7x7 embark worked, but lagged horribly when unpaused. In 000136, there were claims that 10x10 froze, but 9x9 worked. As said above: some whole-map embarks will hit the 2 GB limit, others won't. It's extremely dependent on local geography. If smaller areas work where larger areas don't, you're getting 0000136, the memory crash. The workaround is to embark on smaller areas.

(0021429) Logical2u (manager) 2012-03-12 13:07	I have heavily edited the report based on the most recent data. Based on your Windows error report, I strongly suspect you are encountering an out-of-memory error on your attempts to embark. This is corroborated by your testing of smaller embark areas. The horrible lag you described is probably just a symptom of the large embark area taking its toll on both your processor and memory. I'm going to leave this open for a few moments, assuming you return and can confirm the memory usage of DF during a crash - a procedure something like this might work - http://www.techrepublic.com/article/solutionbase-monitor-windows-xps-memory-usage-with-a-customized-version-of-performance-monitor/5263541 [^] [^] The solution to your problem is to embark on a smaller area. If there is a small area that consistently crashes the with the same error, then I suggest you make a new report. If a large area keeps crashing with the same error, then that is another example of this problem here, and also part of 0000136 as Footkerchief has already noted.

Issue History
Date Modified	Username	Field	Change
2012-03-11 12:53	Nomlets	New Issue
2012-03-11 13:26	Logical2u	Note Added: 0021370
2012-03-11 13:26	Logical2u	Status	new => resolved
2012-03-11 13:26	Logical2u	Resolution	open => won't fix
2012-03-11 13:26	Logical2u	Assigned To	=> Logical2u
2012-03-11 18:14	Nomlets	Note Added: 0021382
2012-03-11 18:14	Nomlets	Status	resolved => needs feedback
2012-03-11 18:14	Nomlets	Resolution	won't fix => reopened
2012-03-11 19:28	Footkerchief	Note Added: 0021389
2012-03-11 19:28	Footkerchief	Tag Attached: AWAITING UPDATE
2012-03-11 19:35	Nomlets	Note Added: 0021392
2012-03-11 19:35	Nomlets	Status	needs feedback => assigned
2012-03-11 19:37	Nomlets	Note Edited: 0021392	View Revisions
2012-03-11 19:51	Nomlets	Note Edited: 0021392	View Revisions
2012-03-11 20:07	Logical2u	Note Added: 0021394
2012-03-11 20:07	Logical2u	Assigned To	Logical2u =>
2012-03-11 20:07	Logical2u	Resolution	reopened => open
2012-03-11 21:23	Logical2u	Status	assigned => new
2012-03-12 08:00	Nomlets	Note Added: 0021407
2012-03-12 08:11	Nomlets	Note Edited: 0021407	View Revisions
2012-03-12 08:43	Logical2u	Note Added: 0021409
2012-03-12 09:20	Footkerchief	Note Added: 0021410
2012-03-12 09:49	Nomlets	Note Added: 0021411
2012-03-12 10:07	Logical2u	Note Added: 0021413
2012-03-12 10:08	Footkerchief	Note Added: 0021415
2012-03-12 10:08	Footkerchief	Note Edited: 0021415	View Revisions
2012-03-12 10:08	Footkerchief	Relationship added	duplicate of 0000136
2012-03-12 10:12	Footkerchief	Note Edited: 0021415	View Revisions
2012-03-12 10:15	Nomlets	Note Added: 0021418
2012-03-12 10:18	Nomlets	Note Edited: 0021418	View Revisions
2012-03-12 10:21	Nomlets	Note Edited: 0021418	View Revisions
2012-03-12 10:31	Nomlets	Note Edited: 0021418	View Revisions
2012-03-12 10:31	Footkerchief	Note Added: 0021419
2012-03-12 10:32	Footkerchief	Note Edited: 0021419	View Revisions
2012-03-12 10:32	Footkerchief	Status	new => resolved
2012-03-12 10:32	Footkerchief	Resolution	open => duplicate
2012-03-12 10:32	Footkerchief	Assigned To	=> Footkerchief
2012-03-12 10:33	Footkerchief	Note Edited: 0021419	View Revisions
2012-03-12 10:33	Footkerchief	Note Edited: 0021419	View Revisions
2012-03-12 10:33	Footkerchief	Tag Detached: AWAITING UPDATE
2012-03-12 10:53	Nomlets	Note Added: 0021420
2012-03-12 10:53	Nomlets	Status	resolved => needs feedback
2012-03-12 10:53	Nomlets	Resolution	duplicate => reopened
2012-03-12 10:59	Nomlets	Note Edited: 0021420	View Revisions
2012-03-12 11:09	Nomlets	Note Added: 0021423
2012-03-12 11:09	Nomlets	Status	needs feedback => assigned
2012-03-12 11:26	Footkerchief	Note Added: 0021424
2012-03-12 11:26	Footkerchief	Note Edited: 0021424	View Revisions
2012-03-12 11:47	Footkerchief	Note Edited: 0021424	View Revisions
2012-03-12 12:43	Footkerchief	Issue Monitored: Logical2u
2012-03-12 13:07	Logical2u	Note Added: 0021429
2012-03-12 13:07	Logical2u	Assigned To	Footkerchief =>
2012-03-12 13:07	Logical2u	Priority	urgent => high
2012-03-12 13:07	Logical2u	Severity	block => crash
2012-03-12 13:07	Logical2u	Reproducibility	have not tried => random
2012-03-12 13:07	Logical2u	Status	assigned => new
2012-03-12 13:07	Logical2u	Summary	Windows Update message on embark caused file corruption => Crash on large embark - memory related
2012-03-12 17:11	Logical2u	Status	new => resolved
2012-03-12 17:11	Logical2u	Resolution	reopened => duplicate
2012-03-12 17:11	Logical2u	Assigned To	=> Logical2u