0000136: When embarking on large area, DF hits 2GB memory limit and crashes

(0000194)
Kurik (reporter)
2010-04-02 09:10

I am getting this error whenever I try to embark with an embark area larger than 10x10 squares selected on the embark screen.

(0000203)
Nimrod (reporter)
2010-04-02 09:40

i tried 13x13 every single time.
checking something smaller now
brb

(0000207)
Nimrod (reporter)
2010-04-02 09:46

CONFIRMED!

It does not crash with an embark area of 9x9.

(0000320)
jbrown6982 (reporter)
2010-04-02 16:15

I confirm as well. 9x9 is the largest embark area I can select without a CTD. Major for me as I like huge embark areas. :|

(0000333)
DoctorZuber (reporter)
2010-04-02 16:43
edited on: 2010-04-02 16:54

okay, I genned up a quick pocket region simply to test this.

9x9 works
2x10 works
2x16 works

10x10 works
16x16 crashes. eventually, hung for a good couple minutes before throwing a C++ exception at me and finally closing.

5x16 works
6x16 works
7x16 works...

16x2 works

My theory at this exact moment leads me to believe that this is a hardware limitation on your end. Not specifically a bug with the embark size. Insufficient memory most likely. The fact that I'm running this test on a pocket region probably does lower the memory requirements.

(0000405)
bbgun06 (reporter)
2010-04-02 20:44

I'm not sure this can be blamed entirely on insufficient memory. I have 6GB of ram, but I still can not embark on anything larger than 10x10. While the game is running in a 9x9 map, it uses about 1.8GB of memory, with about 28% free.
I am running Windows 7 64 bit.

(0000407)
DoctorZuber (reporter)
2010-04-02 20:49
edited on: 2010-04-02 20:49

not sure what exactly to blame it on, but if I'm able to embark on 10x10 and you are not.... it does sort of imply some sort of hardware issue.

Still, you might try again on a pocket region and see if it lets you embark on a larger area that way.

(0000438)
Nimrod (reporter)
2010-04-02 23:29

I run DF on a Quadcore Intel i7 920 with 6GB RAM
OS is Windows Vista Ultimate 64bit

the hardware theory cant be right as i see it

maybe it got something to do with memory allocation in a 64bit OS?
What are your specs DoctorZuber

(0000452)
Halconnen (reporter)
2010-04-03 00:55

@bbgun06 above.
DF is a 32-Bit application. It -cannot- use more than 2GB at once. So at 1.8GB it's already running near the limit. Hit 2GB and it dies horribly. Increasing from 9x9 to 10x10 increases the amount of embark tiles from 81 to 100, an increase of almost 25%.

I guess that that's a hard limit at the moment.

(0000465)
Nimrod (reporter)
2010-04-03 02:52

Okay, seems this crash is nailed down.

I did some testing and you are right Halconnen.
As soon as DF reaches 2Gig Memory usage it dies witha runtime error.

I set GRAPHICS:NO and did a 13x13 build while watching the memory usage - it went up to 1.8 Gig on embark and worked perefectly.
Same with GRAPHICS:YES hits 2 Gig after a few seconds and dies.

So: its a memory usage problem with graphics enabled

(0000475)
Kurik (reporter)
2010-04-03 04:46

I've had this issue occur even with GRAPHICS:NO. AFAIK the game may allocate you a different number of Z-levels depending on the location you're embarking on, which of course affects memory usage, so that's another factor that can affect whether you get this crash.

(0000627)
bicker x 2 (reporter)
2010-04-03 12:51

I found this issue to ocour with or without graphics being on as well

(0002349)
bicker x 2 (reporter)
2010-04-09 10:08

This seems to again ocour with vertion 02

(0002351)
SirPenguin (reporter)
2010-04-09 10:20

This isn't so much a crash as it is your system not having enough memory. What I mean by that is that I doubt this will ever be fixed - the fix is that your embark size is directly proportionate to your memory size. Too much and it crashes.

(0002384)
DoctorZuber (reporter)
2010-04-09 12:32

Intel(R) Core(TM)2 Duo CPU T7100 @ 1.8 GHz 1.0 GHz
2 GB Ram
32-Bit Operating System
Vista

- I did my testing on a pocket region with various embark sizes.

- you never specified what size region you did your tests on
- larger regions very probably require more memory.

(0002578)
PetePetePete (reporter)
2010-04-10 08:53

Confirming the 2GB Limit: just wanted to embark on a 16x16 (before checking here) and it crashed the moment DF hit 2GB RAM Usage.

Worked in earlier versions, so I blame it on a) the increased amount of z-levels and features or b) an unwanted memory leak somewhere down the line.

(0002580)
DoctorZuber (reporter)
2010-04-10 09:00
edited on: 2010-04-10 09:00

on a personal note, I think you guys are at somewhat insane trying to embark on such a large area, I've never wanted or needed larger than 5x5.

However, I do agree that it shouldn't simply crash on you. If you've got the hardware to run it, go for it, have fun.

(0002913)
Greyhawk (reporter)
2010-04-11 17:30

32-bit operating systems has 2 GB limit for accessed memory.

64-bit operating systems doesn't have that limit, but 32-bit applications (like Dwarf Fortress) are still limited by the 2 GB limit (even though you could have 3gb memory and 3gb virtual memory).

Two possibilities to go beyond 2GB:

- Release a 64-bit version separately.
- Run special 32-bit processes that hold parts of the data.

In the mean time, the game should stop at 95% of total memory, 95% of available memory or 95% of 2GB (whichever comes first) to prevent the locking up before the crash.

(0003034)
James.Denholm (reporter)
2010-04-12 06:44

Of course, 32-bit processors can't run 64-bit applications, so to release a separate 64-bit program could cause confusion and issues for 32-bit users.

But, of course, it would be brilliant for 64-bit users.

(0003365)
jgoodwin (reporter)
2010-04-13 19:32

With Visual Studio, if you specify /LARGEADDRESSAWARE on the link command line you can get 3 to 3.5 GB of address space on 32-bit windows.

See http://msdn.microsoft.com/en-us/library/wz223b1z%28v=VS.80%29.aspx [^]

(This page is for VS 2005, but the switch also works for VS 2008 and VS 2010)

(0003369)
Arkaaito (reporter)
2010-04-13 19:51

Are there any workarounds for this at the moment that would allow embarking on 16x16? I've tried turning graphics off but I still hit the memory threshold.

(0003596)
Logical2u (manager)
2010-04-14 19:50

I have only one question - why do you want to embark on a 16x16 area?

That being said, no - since DF is compiled as an x86 or 32 bit program, it has the maximum memory usage limit of 2gb as mentioned above, which caps your playing area to one in which that memory limit is not achieved.

Turn graphics off - turn off temperature, economy, invaders, set a low population cap - disable all cavern layers/find an embark site with very few underground layers - find a very flat embark site - find an embark site without running water.

Those tricks should lower your memory consumption.

(0003623)
Langdon (reporter)
2010-04-14 22:32

Isn't this the same as 0000002? (which Toady is fixing in .04?)

(0003626)
Gauphastus (reporter)
2010-04-14 23:03
edited on: 2010-04-14 23:04

I got this problem when I embarked on a 4x14 or so site.
I was trying to build a bridge between two continents, something I've really been dying to do for awhile now.
Apparently that's a bit too big.

Oh well.

EDIT: But you know, the game locks up like clockwork after only a few minutes of me not doing anything really.
I uh, have a save here if it's ever requested.
I'll start up a new fort on a different save.

(0003743)
Footkerchief (manager)
2010-04-15 12:34
edited on: 2010-04-15 12:35

@Langdon they're both related to excess memory usage, but I doubt the fix for 0000002 was one that allowed the program to break the 2 GB limit. More likely he optimized the site finder to use less memory. This one will probably require restricting the size of the embark area -- I don't see any way around that.

(0005034)
Firehound (reporter)
2010-04-25 14:10

@Jodoodwin: Reminds me of having to download a new executable for ETW when it came out, and that the only difference was something like that /LARGEADDRESSAWARE so it wouldn't crash and freeze mid-game for no reason.

(0009087)
hyndis (reporter)
2010-06-25 09:28

Sadly, ETW is still, even after being released over a year ago, far buggier and far more prone to crashing than Dwarf Fortress.

Chalk one up for Toady! He beats a multimillion dollar studio and dev team of dozens when it comes to squashing bugs.

(0009377)
smjjames (reporter)
2010-06-30 09:36

Huh? How did this turn into something about ETW?

Anyways, for .08, I don't crash when I do a large area, but it does take a while for it to go through the embarkation screens and it is pretty laggy when I get to the spot sometimes. DF did hang a few times for one particular spot on the map I'm using when I was looking at a 16X13 area, but I eventually got through and I think that spot was just extrenely memory taxing for various reasons.

(0010669)
smjjames (reporter)
2010-07-19 13:38
edited on: 2010-07-26 20:51

Sorry for double post, but:

I'm getting this with the maximum possible on 31.10 and getting the runtime error as well. When I have the taskmanager open, I often see the load going past 50% and nearly maxing out the CPU.

I don't know if it was the exact error, but its definetly a runtime error.

Sometimes it will pull through with a very large area (on the order of more than 10x10), sometimes not. It seems to help when the embark area is totally flat. It handles 16x5 or 16x6 (and vice versa) just fine however.

Edit: Just a little update, it seems to handle large embark zones better in .12, haven't tried maximum possible yet though.

(0013960)
Footkerchief (manager)
2010-11-17 10:10

Reminder sent to: Nimrod

Does this still occur in 31.18?

(0014619)
SirPenguin (reporter)
2010-12-17 22:05

I can confirm this issue still occurs on my system with 31.18

(0017968)
Egodeus (reporter)
2011-06-10 07:21

I can confirm this in .25. I'm running a 64-bit windows 7 with a quad-core and 8 gigs of ram, but as DF hits the 2 gb line it crashes with the following error:

Problem Event Name: APPCRASH
  Application Name: Dwarf Fortress.exe
  Application Version: 0.0.0.0
  Application Timestamp: 4d90764f
  Fault Module Name: MSVCR100.dll
  Fault Module Version: 10.0.30319.1
  Fault Module Timestamp: 4ba1dbbe
  Exception Code: 40000015
  Exception Offset: 0008d635
  OS Version: 6.1.7600.2.0.0.256.1
  Locale ID: 1035
  Additional Information 1: 53ab
  Additional Information 2: 53ab78575e4e4ce741bf82bee235390b
  Additional Information 3: 3b1c
  Additional Information 4: 3b1c7dab8afb70c763830f7ffc3fdc79

Tried it once more to be sure but then df didn't even give me an error. Just crashed.

Making a separate 64-bit version would be sweet and I'd gladly pay good money for it, but I can imagine that the shuffle wouldn't be too straightforward for Toady. Still some sort of fail-switch to shut the game down gently if the memory allocation limit is reached would be a nice addition, especially if it came with a warning screen that "By the bye, you're trying to embark on a too large area as I cannot reserve enough memory for it."

(0021678)
Halconnen (reporter)
2012-03-23 09:03

To give this a little bump for testing:

I currently do not have a system available to properly test this on myself (my main computer is borked, going to take a while to replace), but for users running DF on 64-Bit windows and 4GB or more of RAM:

Have you tried setting the LAA Flag on the Dwarf Fortress exe to see if it still behaves and -does- allocate more than 2GB of RAM?

Toady clearly didn't set it, but also didn't do anything to keep the game from hard crashing when it reaches that limit, so it might do fine unless he uses the spare bit in the pointers as a flag.

Should make no difference on a 32-Bit system since it'd take registry changes for windows to actually react to that, but for 64-Bit windows it should then proceed to allow DF up to 4GB of memory. Which is twice what it gets now.

May not work, may work. It certainly worked for Skyrim before Bethesda went and set the flag themselves.

A program like the CFF Explorer, or the Skyrim/Fallout 3 LAA tools or whatever should be able to do the job.

(0022073)
Echo51 (reporter)
2012-04-04 06:00

Using this LAA tool i was able to get DF to hog up all availble ram(~3gb) and it should be able to use even more, but i didn't have anymore

It seems for 64bit users that just setting the LAA flag would work wonders for huge embarks.

proof pic: http://i.imgur.com/r6UNK.png [^]

http://www.techpowerup.com/forums/showthread.php?t=112556 [^]

(0023514)
MightyRavenDark (reporter)
2012-09-01 08:18

I can confirm this as well for version 0.34.11, but I don't understand it since I used to embark on 16x16 about a year ago on a machine with only 2GB of RAM and much inferior specs overall. Using the LAA on a 32 bit machine allows the process to run fully but the game is beyond laggy, with massive lag spikes every 10 seconds while the game is unpaused.

(0023515)
Kipi (reporter)
2012-09-01 11:12

MightRavenDark:

It totally depends on the land features. I think you had very few z-levels in total when you made that 16*16 embark.

What version were you using when you made that embark?

(0023517)
MightyRavenDark (reporter)
2012-09-01 15:34

It was probably version 0.31.25

(0023747)
Rayanth (reporter)
2012-11-26 23:22

Just to add an update, in 34.11, with no mods and no changes to any settings from download state, this limitation still exists. anything over 10x10 cannot be embarked upon, as soon as it hits about 2 gigs of ram it just crashes.

running the LAA tool to force Large Address Aware allows a 16x16 embark of 198 z-levels (that's a total of almost 117 million tiles) and upon finally drawing the map after embark was taking up 3.6 gigs of RAM. This means I wouldn't be able to actually *play* for very long before hitting the LAA limit of 4 gigs.

System specs are not at play here (quadcore i7, 24 gigs of ram, win 7 x64) , it's the simple fact that 32 bit programs can only use a maximum of 2 gigs of ram (without being linked to LargeAddressAware, or on 32bit systems) or 4 gigs of ram on 64 bit systems with Large Address Aware linked.

With all modern processors capable of running 64 bit or 32 bit mode, and a vast majority of pre-packaged systems going 64 bit, I would like to re-request that future versions be compiled for both x86 and x64. For those of us insane enough to have a Megaproject in mind that requires a full 16x16 embark, it would certainly be nice to be able to play the game to the full capabilities that it proclaims to have. (Why let us *choose* 16x16 if it's physically impossible?) =)

(0023757)
Rayanth (reporter)
2012-12-01 23:20

I did quite a bit more playing around. it is plausible to get a larger embark in smaller memory if you decrease the z-levels.

DF 34.11 with MayDay and dfhack, Large custom world, cut the cave levels down to 1, and embarked on a 16x16 area where a volcano meets the beginning of a stream. there's lots of cliff, but total z-levels is 110, and total RAM usage at embark is 1.8 gigs. This is within the realm of plausibility. If one were to embark on a flatter area, with no cave levels, it could very well be playable at 16x16 without even LAA. Further z-level reduction could be had from other world gen options.

For those questioning 'why 16x16' - it's the only way to guarantee, 100%, a certain 'feature'. And imagine the megaprojects!

(0025909)
SirPenguin (reporter)
2014-07-11 19:42

Not surprising, but behavior still exists in 40.0x

(0026618)
Talvieno (reporter)
2014-07-16 05:28

There's a tool out there that fixes it. Can't recall what it is off the top of my head, though.

(0029016)
ptb_ptb (reporter)
2014-08-14 06:27

LAA Large Address Aware
http://www.techpowerup.com/forums/threads/large-address-aware.112556/ [^]

(0029109)
handofcode (reporter)
2014-08-15 14:00

Fair warning: If you do this you will get a LOT of pathing bugs if you embark on large sites. The most common sign is that when you try to build something in clearly pathable areas it will say "no access to materials". Other things include dwarves being stuck in one spot and slowly starving to death.

This can be resolved by saving and reloading.

I haven't reported this particular issue as a bug since the exe is being modified but I'm pretty sure it might pop up once Toady starts the 64-bit conversion so... conflicted.

(0036086)
Loci (manager)
2016-12-15 14:46

v0.43.05: The 64-bit version removes the 2GB addressing limit; the limitations of the legacy 32-bit version are unlikely to change. It would, however, be nice if the game could proactively reject problematic embarks without crashing.

Any tangential bugs with the 64-bit version (pathing, etc.) should be reported as separate issues after confirming them against an official 64-bit release.

Issue History
Date Modified	Username	Field	Change
2010-04-02 08:17	Nimrod	New Issue
2010-04-02 09:10	Kurik	Note Added: 0000194
2010-04-02 09:40	Nimrod	Note Added: 0000203
2010-04-02 09:46	Nimrod	Note Added: 0000207
2010-04-02 09:47	Nimrod	Tag Attached: embark
2010-04-02 09:47	Nimrod	Tag Attached: crash
2010-04-02 16:15	jbrown6982	Note Added: 0000320
2010-04-02 16:15	jbrown6982	Issue Monitored: jbrown6982
2010-04-02 16:43	DoctorZuber	Note Added: 0000333
2010-04-02 16:45	DoctorZuber	Note Edited: 0000333	View Revisions
2010-04-02 16:47	DoctorZuber	Note Edited: 0000333	View Revisions
2010-04-02 16:48	DoctorZuber	Note Edited: 0000333	View Revisions
2010-04-02 16:52	DoctorZuber	Note Edited: 0000333	View Revisions
2010-04-02 16:54	DoctorZuber	Note Edited: 0000333	View Revisions
2010-04-02 20:44	bbgun06	Note Added: 0000405
2010-04-02 20:44	bbgun06	Issue Monitored: bbgun06
2010-04-02 20:49	DoctorZuber	Note Added: 0000407
2010-04-02 20:49	DoctorZuber	Note Edited: 0000407	View Revisions
2010-04-02 23:29	Nimrod	Note Added: 0000438
2010-04-03 00:55	Halconnen	Note Added: 0000452
2010-04-03 02:52	Nimrod	Note Added: 0000465
2010-04-03 04:37	Kurik	Issue Monitored: Kurik
2010-04-03 04:46	Kurik	Note Added: 0000475
2010-04-03 11:56	Footkerchief	Relationship added	has duplicate 0000188
2010-04-03 12:01	Aquillion	Tag Attached: modding
2010-04-03 12:51	bicker x 2	Note Added: 0000627
2010-04-07 19:29	Footkerchief	Summary	CTD runtime error on hitting 'embark' in Fortress mod => Crash when embarking on large area
2010-04-07 19:30	Footkerchief	Relationship added	has duplicate 0000758
2010-04-08 14:15	Footkerchief	Relationship added	has duplicate 0000822
2010-04-09 10:08	bicker x 2	Note Added: 0002349
2010-04-09 10:14	bicker x 2	Issue Monitored: bicker x 2
2010-04-09 10:20	SirPenguin	Note Added: 0002351
2010-04-09 12:32	DoctorZuber	Note Added: 0002384
2010-04-10 03:06	bicker x 2	Note Added: 0002545
2010-04-10 08:53	PetePetePete	Note Added: 0002578
2010-04-10 08:55	bicker x 2	Note Deleted: 0002545
2010-04-10 09:00	DoctorZuber	Note Added: 0002580
2010-04-10 09:00	DoctorZuber	Note Edited: 0002580	View Revisions
2010-04-11 17:30	Greyhawk	Note Added: 0002913
2010-04-12 06:44	James.Denholm	Note Added: 0003034
2010-04-12 23:40	Footkerchief	Relationship added	has duplicate 0001065
2010-04-13 19:32	jgoodwin	Note Added: 0003365
2010-04-13 19:51	Arkaaito	Note Added: 0003369
2010-04-13 20:01	Arkaaito	Tag Attached: memory
2010-04-13 20:15	Arkaaito	Issue Monitored: Arkaaito
2010-04-14 19:15	Clothar	Issue Monitored: Clothar
2010-04-14 19:50	Logical2u	Note Added: 0003596
2010-04-14 22:32	Langdon	Note Added: 0003623
2010-04-14 23:03	Gauphastus	Note Added: 0003626
2010-04-14 23:04	Gauphastus	Note Edited: 0003626	View Revisions
2010-04-15 08:57	Footkerchief	Relationship added	has duplicate 0001194
2010-04-15 12:34	Footkerchief	Note Added: 0003743
2010-04-15 12:35	Footkerchief	Note Edited: 0003743	View Revisions
2010-04-15 17:15	jbrown6982	Tag Attached: RESOLVED
2010-04-15 17:50	Stregone	Issue Monitored: Stregone
2010-04-15 23:53	jgoodwin	Issue Monitored: jgoodwin
2010-04-25 14:10	Firehound	Note Added: 0005034
2010-04-26 17:26	Footkerchief	Summary	Crash when embarking on large area => Crash when embarking on large area, possibly dependent on graphics
2010-04-26 17:26	Footkerchief	Tag Detached: RESOLVED
2010-04-28 14:05	Footkerchief	Category	General => Technical
2010-04-29 17:50	Footkerchief	Relationship added	has duplicate 0001648
2010-05-12 16:58	cloth4r	Issue Monitored: cloth4r
2010-06-22 18:13	Footkerchief	Sticky Issue	No => Yes
2010-06-22 18:13	Footkerchief	Summary	Crash when embarking on large area, possibly dependent on graphics => Crash when embarking on large area
2010-06-22 18:13	Footkerchief	Relationship added	has duplicate 0002436
2010-06-25 09:28	hyndis	Note Added: 0009087
2010-06-29 07:38	Footkerchief	Category	Technical => Technical -- General
2010-06-30 09:36	smjjames	Note Added: 0009377
2010-07-16 04:08	Logical2u	Relationship added	parent of 0002715
2010-07-19 13:38	smjjames	Note Added: 0010669
2010-07-19 13:50	smjjames	Note Edited: 0010669	View Revisions
2010-07-26 20:51	smjjames	Note Edited: 0010669	View Revisions
2010-11-17 10:10	Footkerchief	Note Added: 0013960
2010-12-13 14:00	Footkerchief	Relationship added	has duplicate 0003814
2010-12-17 22:05	SirPenguin	Note Added: 0014619
2011-03-21 06:19	Logical2u	Relationship added	has duplicate 0004270
2011-06-10 07:21	Egodeus	Note Added: 0017968
2011-08-16 13:00	Footkerchief	Summary	Crash when embarking on large area => When embarking on large area, DF hits 2GB memory limit and crashes
2012-02-19 08:19	Footkerchief	Relationship added	parent of 0004833
2012-02-21 06:49	Footkerchief	Relationship added	has duplicate 0005284
2012-02-21 06:51	Footkerchief	Relationship replaced	related to 0002715
2012-02-21 06:52	Footkerchief	Relationship replaced	related to 0004833
2012-03-07 08:51	Buglist	Issue Monitored: Buglist
2012-03-12 10:08	Footkerchief	Relationship added	has duplicate 0005622
2012-03-12 10:10	Footkerchief	Relationship added	has duplicate 0003659
2012-03-12 12:19	Footkerchief	Relationship added	related to 0005283
2012-03-12 17:11	Logical2u	Issue Monitored: Logical2u
2012-03-23 09:03	Halconnen	Note Added: 0021678
2012-03-29 10:27	erbridge	Issue Monitored: erbridge
2012-04-04 06:00	Echo51	Note Added: 0022073
2012-09-01 08:18	MightyRavenDark	Note Added: 0023514
2012-09-01 11:12	Kipi	Note Added: 0023515
2012-09-01 11:13	Kipi	Issue Monitored: Kipi
2012-09-01 15:34	MightyRavenDark	Note Added: 0023517
2012-11-26 23:22	Rayanth	Note Added: 0023747
2012-11-26 23:27	Rayanth	Issue Monitored: Rayanth
2012-12-01 23:20	Rayanth	Note Added: 0023757
2014-07-11 19:42	SirPenguin	Note Added: 0025909
2014-07-16 05:28	Talvieno	Note Added: 0026618
2014-08-13 14:07	handofcode	Note Added: 0028991
2014-08-13 14:09	handofcode	Note Edited: 0028991	View Revisions
2014-08-13 14:09	handofcode	Note Deleted: 0028991
2014-08-14 06:27	ptb_ptb	Note Added: 0029016
2014-08-15 14:00	handofcode	Note Added: 0029109
2014-09-11 21:11	chuzzum	Issue Monitored: chuzzum
2016-12-15 14:46	Loci	Note Added: 0036086
2018-01-11 21:03	Loci	Sticky Issue	Yes => No
2018-01-11 21:21	BlueManedHawk	Tag Attached: Windows
2018-01-13 10:27	lethosor	Tag Detached: Windows

Notes
(0000194) Kurik (reporter) 2010-04-02 09:10	I am getting this error whenever I try to embark with an embark area larger than 10x10 squares selected on the embark screen.

(0000203) Nimrod (reporter) 2010-04-02 09:40	i tried 13x13 every single time. checking something smaller now brb

(0000207) Nimrod (reporter) 2010-04-02 09:46	CONFIRMED! It does not crash with an embark area of 9x9.

(0000320) jbrown6982 (reporter) 2010-04-02 16:15	I confirm as well. 9x9 is the largest embark area I can select without a CTD. Major for me as I like huge embark areas. :\|

(0000333) DoctorZuber (reporter) 2010-04-02 16:43 edited on: 2010-04-02 16:54	okay, I genned up a quick pocket region simply to test this. 9x9 works 2x10 works 2x16 works 10x10 works 16x16 crashes. eventually, hung for a good couple minutes before throwing a C++ exception at me and finally closing. 5x16 works 6x16 works 7x16 works... 16x2 works My theory at this exact moment leads me to believe that this is a hardware limitation on your end. Not specifically a bug with the embark size. Insufficient memory most likely. The fact that I'm running this test on a pocket region probably does lower the memory requirements.

(0000405) bbgun06 (reporter) 2010-04-02 20:44	I'm not sure this can be blamed entirely on insufficient memory. I have 6GB of ram, but I still can not embark on anything larger than 10x10. While the game is running in a 9x9 map, it uses about 1.8GB of memory, with about 28% free. I am running Windows 7 64 bit.

(0000407) DoctorZuber (reporter) 2010-04-02 20:49 edited on: 2010-04-02 20:49	not sure what exactly to blame it on, but if I'm able to embark on 10x10 and you are not.... it does sort of imply some sort of hardware issue. Still, you might try again on a pocket region and see if it lets you embark on a larger area that way.

(0000438) Nimrod (reporter) 2010-04-02 23:29	I run DF on a Quadcore Intel i7 920 with 6GB RAM OS is Windows Vista Ultimate 64bit the hardware theory cant be right as i see it maybe it got something to do with memory allocation in a 64bit OS? What are your specs DoctorZuber

(0000452) Halconnen (reporter) 2010-04-03 00:55	@bbgun06 above. DF is a 32-Bit application. It -cannot- use more than 2GB at once. So at 1.8GB it's already running near the limit. Hit 2GB and it dies horribly. Increasing from 9x9 to 10x10 increases the amount of embark tiles from 81 to 100, an increase of almost 25%. I guess that that's a hard limit at the moment.

(0000465) Nimrod (reporter) 2010-04-03 02:52	Okay, seems this crash is nailed down. I did some testing and you are right Halconnen. As soon as DF reaches 2Gig Memory usage it dies witha runtime error. I set GRAPHICS:NO and did a 13x13 build while watching the memory usage - it went up to 1.8 Gig on embark and worked perefectly. Same with GRAPHICS:YES hits 2 Gig after a few seconds and dies. So: its a memory usage problem with graphics enabled

(0000475) Kurik (reporter) 2010-04-03 04:46	I've had this issue occur even with GRAPHICS:NO. AFAIK the game may allocate you a different number of Z-levels depending on the location you're embarking on, which of course affects memory usage, so that's another factor that can affect whether you get this crash.

(0000627) bicker x 2 (reporter) 2010-04-03 12:51	I found this issue to ocour with or without graphics being on as well

(0002349) bicker x 2 (reporter) 2010-04-09 10:08	This seems to again ocour with vertion 02

(0002351) SirPenguin (reporter) 2010-04-09 10:20	This isn't so much a crash as it is your system not having enough memory. What I mean by that is that I doubt this will ever be fixed - the fix is that your embark size is directly proportionate to your memory size. Too much and it crashes.

(0002384) DoctorZuber (reporter) 2010-04-09 12:32	Intel(R) Core(TM)2 Duo CPU T7100 @ 1.8 GHz 1.0 GHz 2 GB Ram 32-Bit Operating System Vista - I did my testing on a pocket region with various embark sizes. - you never specified what size region you did your tests on - larger regions very probably require more memory.

(0002578) PetePetePete (reporter) 2010-04-10 08:53	Confirming the 2GB Limit: just wanted to embark on a 16x16 (before checking here) and it crashed the moment DF hit 2GB RAM Usage. Worked in earlier versions, so I blame it on a) the increased amount of z-levels and features or b) an unwanted memory leak somewhere down the line.

(0002580) DoctorZuber (reporter) 2010-04-10 09:00 edited on: 2010-04-10 09:00	on a personal note, I think you guys are at somewhat insane trying to embark on such a large area, I've never wanted or needed larger than 5x5. However, I do agree that it shouldn't simply crash on you. If you've got the hardware to run it, go for it, have fun.

(0002913) Greyhawk (reporter) 2010-04-11 17:30	32-bit operating systems has 2 GB limit for accessed memory. 64-bit operating systems doesn't have that limit, but 32-bit applications (like Dwarf Fortress) are still limited by the 2 GB limit (even though you could have 3gb memory and 3gb virtual memory). Two possibilities to go beyond 2GB: - Release a 64-bit version separately. - Run special 32-bit processes that hold parts of the data. In the mean time, the game should stop at 95% of total memory, 95% of available memory or 95% of 2GB (whichever comes first) to prevent the locking up before the crash.

(0003034) James.Denholm (reporter) 2010-04-12 06:44	Of course, 32-bit processors can't run 64-bit applications, so to release a separate 64-bit program could cause confusion and issues for 32-bit users. But, of course, it would be brilliant for 64-bit users.

(0003365) jgoodwin (reporter) 2010-04-13 19:32	With Visual Studio, if you specify /LARGEADDRESSAWARE on the link command line you can get 3 to 3.5 GB of address space on 32-bit windows. See http://msdn.microsoft.com/en-us/library/wz223b1z%28v=VS.80%29.aspx [^] (This page is for VS 2005, but the switch also works for VS 2008 and VS 2010)

(0003369) Arkaaito (reporter) 2010-04-13 19:51	Are there any workarounds for this at the moment that would allow embarking on 16x16? I've tried turning graphics off but I still hit the memory threshold.

(0003596) Logical2u (manager) 2010-04-14 19:50	I have only one question - why do you want to embark on a 16x16 area? That being said, no - since DF is compiled as an x86 or 32 bit program, it has the maximum memory usage limit of 2gb as mentioned above, which caps your playing area to one in which that memory limit is not achieved. Turn graphics off - turn off temperature, economy, invaders, set a low population cap - disable all cavern layers/find an embark site with very few underground layers - find a very flat embark site - find an embark site without running water. Those tricks should lower your memory consumption.

(0003623) Langdon (reporter) 2010-04-14 22:32	Isn't this the same as 0000002? (which Toady is fixing in .04?)

(0003626) Gauphastus (reporter) 2010-04-14 23:03 edited on: 2010-04-14 23:04	I got this problem when I embarked on a 4x14 or so site. I was trying to build a bridge between two continents, something I've really been dying to do for awhile now. Apparently that's a bit too big. Oh well. EDIT: But you know, the game locks up like clockwork after only a few minutes of me not doing anything really. I uh, have a save here if it's ever requested. I'll start up a new fort on a different save.

(0003743) Footkerchief (manager) 2010-04-15 12:34 edited on: 2010-04-15 12:35	@Langdon they're both related to excess memory usage, but I doubt the fix for 0000002 was one that allowed the program to break the 2 GB limit. More likely he optimized the site finder to use less memory. This one will probably require restricting the size of the embark area -- I don't see any way around that.

(0005034) Firehound (reporter) 2010-04-25 14:10	@Jodoodwin: Reminds me of having to download a new executable for ETW when it came out, and that the only difference was something like that /LARGEADDRESSAWARE so it wouldn't crash and freeze mid-game for no reason.

(0009087) hyndis (reporter) 2010-06-25 09:28	Sadly, ETW is still, even after being released over a year ago, far buggier and far more prone to crashing than Dwarf Fortress. Chalk one up for Toady! He beats a multimillion dollar studio and dev team of dozens when it comes to squashing bugs.

(0009377) smjjames (reporter) 2010-06-30 09:36	Huh? How did this turn into something about ETW? Anyways, for .08, I don't crash when I do a large area, but it does take a while for it to go through the embarkation screens and it is pretty laggy when I get to the spot sometimes. DF did hang a few times for one particular spot on the map I'm using when I was looking at a 16X13 area, but I eventually got through and I think that spot was just extrenely memory taxing for various reasons.

(0010669) smjjames (reporter) 2010-07-19 13:38 edited on: 2010-07-26 20:51	Sorry for double post, but: I'm getting this with the maximum possible on 31.10 and getting the runtime error as well. When I have the taskmanager open, I often see the load going past 50% and nearly maxing out the CPU. I don't know if it was the exact error, but its definetly a runtime error. Sometimes it will pull through with a very large area (on the order of more than 10x10), sometimes not. It seems to help when the embark area is totally flat. It handles 16x5 or 16x6 (and vice versa) just fine however. Edit: Just a little update, it seems to handle large embark zones better in .12, haven't tried maximum possible yet though.

(0013960) Footkerchief (manager) 2010-11-17 10:10	Reminder sent to: Nimrod Does this still occur in 31.18?

(0014619) SirPenguin (reporter) 2010-12-17 22:05	I can confirm this issue still occurs on my system with 31.18

(0017968) Egodeus (reporter) 2011-06-10 07:21	I can confirm this in .25. I'm running a 64-bit windows 7 with a quad-core and 8 gigs of ram, but as DF hits the 2 gb line it crashes with the following error: Problem Event Name: APPCRASH Application Name: Dwarf Fortress.exe Application Version: 0.0.0.0 Application Timestamp: 4d90764f Fault Module Name: MSVCR100.dll Fault Module Version: 10.0.30319.1 Fault Module Timestamp: 4ba1dbbe Exception Code: 40000015 Exception Offset: 0008d635 OS Version: 6.1.7600.2.0.0.256.1 Locale ID: 1035 Additional Information 1: 53ab Additional Information 2: 53ab78575e4e4ce741bf82bee235390b Additional Information 3: 3b1c Additional Information 4: 3b1c7dab8afb70c763830f7ffc3fdc79 Tried it once more to be sure but then df didn't even give me an error. Just crashed. Making a separate 64-bit version would be sweet and I'd gladly pay good money for it, but I can imagine that the shuffle wouldn't be too straightforward for Toady. Still some sort of fail-switch to shut the game down gently if the memory allocation limit is reached would be a nice addition, especially if it came with a warning screen that "By the bye, you're trying to embark on a too large area as I cannot reserve enough memory for it."

(0021678) Halconnen (reporter) 2012-03-23 09:03	To give this a little bump for testing: I currently do not have a system available to properly test this on myself (my main computer is borked, going to take a while to replace), but for users running DF on 64-Bit windows and 4GB or more of RAM: Have you tried setting the LAA Flag on the Dwarf Fortress exe to see if it still behaves and -does- allocate more than 2GB of RAM? Toady clearly didn't set it, but also didn't do anything to keep the game from hard crashing when it reaches that limit, so it might do fine unless he uses the spare bit in the pointers as a flag. Should make no difference on a 32-Bit system since it'd take registry changes for windows to actually react to that, but for 64-Bit windows it should then proceed to allow DF up to 4GB of memory. Which is twice what it gets now. May not work, may work. It certainly worked for Skyrim before Bethesda went and set the flag themselves. A program like the CFF Explorer, or the Skyrim/Fallout 3 LAA tools or whatever should be able to do the job.

(0022073) Echo51 (reporter) 2012-04-04 06:00	Using this LAA tool i was able to get DF to hog up all availble ram(~3gb) and it should be able to use even more, but i didn't have anymore It seems for 64bit users that just setting the LAA flag would work wonders for huge embarks. proof pic: http://i.imgur.com/r6UNK.png [^] http://www.techpowerup.com/forums/showthread.php?t=112556 [^]

(0023514) MightyRavenDark (reporter) 2012-09-01 08:18	I can confirm this as well for version 0.34.11, but I don't understand it since I used to embark on 16x16 about a year ago on a machine with only 2GB of RAM and much inferior specs overall. Using the LAA on a 32 bit machine allows the process to run fully but the game is beyond laggy, with massive lag spikes every 10 seconds while the game is unpaused.

(0023515) Kipi (reporter) 2012-09-01 11:12	MightRavenDark: It totally depends on the land features. I think you had very few z-levels in total when you made that 16*16 embark. What version were you using when you made that embark?

(0023517) MightyRavenDark (reporter) 2012-09-01 15:34	It was probably version 0.31.25

(0023747) Rayanth (reporter) 2012-11-26 23:22	Just to add an update, in 34.11, with no mods and no changes to any settings from download state, this limitation still exists. anything over 10x10 cannot be embarked upon, as soon as it hits about 2 gigs of ram it just crashes. running the LAA tool to force Large Address Aware allows a 16x16 embark of 198 z-levels (that's a total of almost 117 million tiles) and upon finally drawing the map after embark was taking up 3.6 gigs of RAM. This means I wouldn't be able to actually play for very long before hitting the LAA limit of 4 gigs. System specs are not at play here (quadcore i7, 24 gigs of ram, win 7 x64) , it's the simple fact that 32 bit programs can only use a maximum of 2 gigs of ram (without being linked to LargeAddressAware, or on 32bit systems) or 4 gigs of ram on 64 bit systems with Large Address Aware linked. With all modern processors capable of running 64 bit or 32 bit mode, and a vast majority of pre-packaged systems going 64 bit, I would like to re-request that future versions be compiled for both x86 and x64. For those of us insane enough to have a Megaproject in mind that requires a full 16x16 embark, it would certainly be nice to be able to play the game to the full capabilities that it proclaims to have. (Why let us choose 16x16 if it's physically impossible?) =)

(0023757) Rayanth (reporter) 2012-12-01 23:20	I did quite a bit more playing around. it is plausible to get a larger embark in smaller memory if you decrease the z-levels. DF 34.11 with MayDay and dfhack, Large custom world, cut the cave levels down to 1, and embarked on a 16x16 area where a volcano meets the beginning of a stream. there's lots of cliff, but total z-levels is 110, and total RAM usage at embark is 1.8 gigs. This is within the realm of plausibility. If one were to embark on a flatter area, with no cave levels, it could very well be playable at 16x16 without even LAA. Further z-level reduction could be had from other world gen options. For those questioning 'why 16x16' - it's the only way to guarantee, 100%, a certain 'feature'. And imagine the megaprojects!

(0025909) SirPenguin (reporter) 2014-07-11 19:42	Not surprising, but behavior still exists in 40.0x

(0026618) Talvieno (reporter) 2014-07-16 05:28	There's a tool out there that fixes it. Can't recall what it is off the top of my head, though.

(0029016) ptb_ptb (reporter) 2014-08-14 06:27	LAA Large Address Aware http://www.techpowerup.com/forums/threads/large-address-aware.112556/ [^]

(0029109) handofcode (reporter) 2014-08-15 14:00	Fair warning: If you do this you will get a LOT of pathing bugs if you embark on large sites. The most common sign is that when you try to build something in clearly pathable areas it will say "no access to materials". Other things include dwarves being stuck in one spot and slowly starving to death. This can be resolved by saving and reloading. I haven't reported this particular issue as a bug since the exe is being modified but I'm pretty sure it might pop up once Toady starts the 64-bit conversion so... conflicted.

(0036086) Loci (manager) 2016-12-15 14:46	v0.43.05: The 64-bit version removes the 2GB addressing limit; the limitations of the legacy 32-bit version are unlikely to change. It would, however, be nice if the game could proactively reject problematic embarks without crashing. Any tangential bugs with the 64-bit version (pathing, etc.) should be reported as separate issues after confirming them against an official 64-bit release.