0002017: Using COMPRESSED_SAVES causes save corruption on some systems

Issue History
Date Modified	Username	Field	Change
2010-05-21 15:59	Hydrall	New Issue
2010-05-21 16:01	Footkerchief	Note Added: 0007148
2010-05-21 16:03	Footkerchief	Note Edited: 0007148	View Revisions
2010-05-28 03:07	Hydrall	Note Added: 0007415
2010-05-28 03:08	Hydrall	Tag Attached: crash
2010-05-28 03:08	Hydrall	Tag Attached: load
2010-05-28 03:08	Hydrall	Tag Attached: save game
2010-05-28 04:51	Logical2u	Note Added: 0007416
2010-06-01 11:43	Hydrall	Note Added: 0007536
2010-06-01 11:51	Footkerchief	Note Added: 0007538
2010-06-03 12:17	Footkerchief	Note Added: 0007624
2010-06-03 12:17	Footkerchief	Tag Attached: AWAITING UPDATE
2010-06-04 12:39	Hydrall	Note Added: 0007668
2010-06-21 12:18	Footkerchief	Note Added: 0008898
2010-06-21 12:18	Footkerchief	Status	new => resolved
2010-06-21 12:18	Footkerchief	Resolution	open => no change required
2010-06-21 12:18	Footkerchief	Assigned To	=> Footkerchief
2010-06-21 12:19	Footkerchief	Assigned To	Footkerchief =>
2010-06-21 12:19	Footkerchief	Status	resolved => new
2010-06-21 12:21	Footkerchief	Summary	Crashes on startup => Using COMPRESSED_SAVES causes save corruption on 64-bit systems
2010-06-21 12:34	Morbs	Note Added: 0008904
2010-06-21 15:52	Footkerchief	Relationship added	related to 0002212
2010-06-28 06:45	jaked122	Note Added: 0009211
2010-06-28 07:07	Telarin	Note Added: 0009213
2010-06-28 07:23	Footkerchief	Summary	Using COMPRESSED_SAVES causes save corruption on 64-bit systems => Using COMPRESSED_SAVES causes save corruption on some 64-bit systems
2010-06-29 07:38	Footkerchief	Category	Technical => Technical -- General
2010-06-29 20:46	Footkerchief	Category	Technical -- General => Technical -- Saving/Loading
2010-06-29 20:46	Footkerchief	Tag Detached: AWAITING UPDATE
2010-07-13 23:27	Footkerchief	Relationship added	has duplicate 0002610
2010-08-10 06:05	Footkerchief	Relationship added	has duplicate 0002998
2010-11-20 17:39	Logical2u	Relationship added	related to 0003699
2010-11-20 22:03	Logical2u	Relationship replaced	has duplicate 0003699
2011-01-11 14:02	maluraq	Note Added: 0014842
2011-01-24 19:50	zilpin	Note Added: 0014925
2011-02-06 15:16	Dwarfu	Resolution	no change required => open
2011-03-26 23:27	Footkerchief	Relationship added	has duplicate 0004351
2011-04-17 19:28	Logical2u	Relationship added	related to 0004538
2011-10-05 11:26	Dwarfu	Relationship added	related to 0004915
2011-10-05 19:18	Dwarfu	Relationship deleted	related to 0004915
2011-12-11 08:47	Dwarfu	Relationship added	related to 0004915
2011-12-11 08:50	Dwarfu	Relationship replaced	has duplicate 0004538
2011-12-11 09:04	Dwarfu	Relationship added	related to 0004863
2011-12-15 07:26	Dwarfu	Relationship added	has duplicate 0004933
2011-12-15 07:27	Dwarfu	Note Added: 0019178
2012-01-30 11:39	Dwarfu	Relationship added	related to 0005036
2012-01-30 11:57	ninjanomnom	Note Added: 0019435
2012-01-30 11:59	ninjanomnom	Note Edited: 0019435	View Revisions
2012-04-11 11:52	Footkerchief	Relationship added	has duplicate 0005795
2012-09-20 08:06	Footkerchief	Relationship added	has duplicate 0006204
2014-01-20 19:06	Footkerchief	Issue Monitored: stuntaneous
2014-12-30 20:18	Footkerchief	Relationship replaced	has duplicate 0005036
2014-12-30 20:18	Footkerchief	Issue Monitored: ninjanomnom
2015-06-29 12:36	Loci	Note Added: 0032821
2015-06-29 13:33	lethosor	Relationship added	related to 0002587
2015-07-06 17:23	lethosor	Note Edited: 0019178	View Revisions
2015-07-06 17:49	lethosor	Relationship deleted	has duplicate 0005036
2015-07-06 17:54	lethosor	Relationship added	related to 0005036
2015-07-06 18:40	lethosor	Note Added: 0032853
2015-07-06 18:40	lethosor	Status	new => needs feedback
2015-07-06 18:40	lethosor	Summary	Using COMPRESSED_SAVES causes save corruption on some 64-bit systems => Using COMPRESSED_SAVES causes save corruption on some systems
2015-07-06 18:41	lethosor	Note Edited: 0032853	View Revisions
2015-07-08 14:30	Loci	Note Added: 0032864
2015-09-02 10:39	lethosor	Reproducibility	always => random
2015-09-02 10:39	lethosor	Assigned To	=> lethosor
2015-09-02 10:39	lethosor	Status	needs feedback => new
2018-05-08 08:01	lethosor	Assigned To	lethosor =>
2019-10-07 14:14	Loci	Assigned To	=> Loci
2019-10-07 14:14	Loci	Status	new => acknowledged

Notes
(0007148) Footkerchief (manager) 2010-05-21 16:01 edited on: 2010-05-21 16:03	Where's the game installed? If you installed it in Program Files, that causes problems on Windows 7 and Vista. Install it somewhere else instead.

(0007415) Hydrall (reporter) 2010-05-28 03:07	I did not install it in program files, I did it in Documents. And if I can't save it there, where else can it go?

(0007416) Logical2u (manager) 2010-05-28 04:51	Documents should be fine as long as you have administrator write access to the folder. Does your error log have anything useful in it after attempting to load the game?

(0007536) Hydrall (reporter) 2010-06-01 11:43	No, the error log is completely unchanged.

(0007538) Footkerchief (manager) 2010-06-01 11:51	Are you on a 64-bit OS? If so, you should try disabling COMPRESSED_SAVES in init.txt. For some reason, save compression frequently results in corruption on 64-bit systems.

(0007624) Footkerchief (manager) 2010-06-03 12:17	Any updates?

(0007668) Hydrall (reporter) 2010-06-04 12:39	It appears to be working! Thank you very much!

(0008898) Footkerchief (manager) 2010-06-21 12:18	Cool. Wish we knew why save compression causes crashes on 64-bit.

(0008904) Morbs (reporter) 2010-06-21 12:34	I am using Win 7 64 bit. Have had no issues with save compression.

(0009211) jaked122 (reporter) 2010-06-28 06:45	I am using mac os x snow leopard 64bit and no issues with save compression.

(0009213) Telarin (reporter) 2010-06-28 07:07	Win XP Pro, 64-bit. Also no problems with save compression.

(0014842) maluraq (reporter) 2011-01-11 14:02	I've just had the same thing happen myself. I wish I had known in advance to avoid it. Fedora Linux 64-bit here.

(0014925) zilpin (reporter) 2011-01-24 19:50	WORKAROUND: For Windows users. In your INI, disable compressed saves. Then in Explorer right click on the «save» folder. Select «Properties». Click «Advanced». Check «Compress contents to save disk space». This will let Windows compress the save games for you. In addition to the game not corrupting it is faster to save and load than when DF does compression itself. Windows can typical compress my saves to 20% of their uncompressed size with no risk of corruption. For GNU/Linux users who have ReiserFS a similar workaround may work for you.

(0019178) Dwarfu (manager) 2011-12-15 07:27 edited on: 2015-07-06 17:23	Phazorx has a save from 0004933: [file removed]

(0019435) ninjanomnom (reporter) 2012-01-30 11:57 edited on: 2012-01-30 11:59	I might have something like this but i can't even see my save in the loading screen... EDIT oops sorry for necro

(0032821) Loci (manager) 2015-06-29 12:36	This may be related to 0002587. Compressing with x64 may increase the chance of a "non-fatal" i/o exception, which Dwarf Fortress blithely ignores, leading to the corruption. I received this error on a (consistently working) x64 machine when disk space was critically low.

(0032853) lethosor (manager) 2015-07-06 18:40 edited on: 2015-07-06 18:41	There are a few instances of corruption occurring on 32-bit systems (e.g. 0002610:0010218). In addition, none of the saves I could track down in this report actually exhibit corruption caused by compression. In some cases, the data that is actually compressed is corrupted, but it can be decompressed without issues (i.e. it was most likely corrupted during serialization, before it was compressed). Others, like the save from 0002610, are truncated but otherwise valid, which could be due to 0002587. I'm not sure what caused these issues, but I don't see how any of these saves can be linked to faulty compression.

(0032864) Loci (manager) 2015-07-08 14:30	It's unclear what feedback you are hoping for here. The issue remains that enabling compression seems to cause corruption (and, conversely, disabling compression prevents it). Though the compression itself may not be at fault, the corruption is a problem that needs to be tracked and fixed.