creating large archives

Submitted by kbeckers on Mon, 2009-11-09 14:47
in

Hi there,
one of the main reasons for me to purchase betterzip was it's support for spanning archives. I wanted to use it to backup things to DVD. While I managed to test this with an archive spanning to SL DVDs, now I cannot seem to get it to work with more demanding situations. Right now, I'm trying to span ~30GB with ~148000 files across a number of DL DVDs, but whenever I add the folder with the files to backup to the betterzip window, the spinner starts running and nothing happens. If I let it run, it runs for at least a day without change. The transcript drawer is empty.

Now what???
TIA,
Karl.

‹ Resource Fork Files 2.0? ›
Submitted by kbeckers on Mon, 2009-11-09 14:49.

Erm, I meant: "I managed to test this with an archive spanning TWO SL DVDs"

Submitted by Robert on Tue, 2009-11-10 18:54.

When you add files to an archive (window), BetterZip links them to a temporary folder first. If the original files do not reside on your startup disk, the files need to be copied instead. This can take some time, but I guess not a whole day. Was there some output written to the Console.app? Were the files on your startup disk or on a secondary/external/network disk?

Submitted by kbeckers on Tue, 2009-11-10 20:58.

The tip with Console.app was right on:

11/9/09 12:31:31 PM BetterZip[878] *** NSRunLoop ignoring exception 'unknown file type 1180 for file /Users/khb/Temp/192-168-1-5-last-backup/home/khb/.opensync/group1/1/enginepipe' that raised during posting of delayed perform with target 0x295b10 and selector '_addFiles:'

So, the culprit were a couple of named pipes I had among the files to backup.
Adding seems to work without those. Will try spanning next.
The pipes could be treated more gracefully, though.

Thanks,
Karl.

Submitted by Robert on Tue, 2009-11-10 21:20.

Ah, I didn't think of that. I'll address it in one of the next updates. Thanks for letting me know!

Submitted by kbeckers on Wed, 2009-11-11 11:11.

NP.
However, now betterzip crashes when trying to save. I've sent two crash reports (one where I had a couple of dangling symlinks on which chmod choked, and one where I removed those.)


11/11/09 10:03:31 AM BetterZip[9574] BetterZip(9574,0xb057e000) malloc: *** mmap(size=266240) failed (error code=12)
*** error: can't allocate region
*** set a breakpoint in malloc_error_break to debug
11/11/09 10:03:31 AM [0x0-0x124124].com.macitbetter.betterzip[9574] BetterZip(9574,0xb057e000) malloc: *** mmap(size=266240) failed (error code=12)
11/11/09 10:03:31 AM [0x0-0x124124].com.macitbetter.betterzip[9574] *** error: can't allocate region
11/11/09 10:03:31 AM [0x0-0x124124].com.macitbetter.betterzip[9574] *** set a breakpoint in malloc_error_break to debug
11/11/09 10:03:31 AM [0x0-0x124124].com.macitbetter.betterzip[9574] *** set a breakpoint in malloc_error_break to debug
11/11/09 10:04:45 AM ReportCrash[10089] Formulating crash report for process BetterZip[9574]

Is bz trying to allocate the whole 8400 MB chunk in main memory?
Karl.

Edit: But then the 4300 MB chunks for the SL DVDs couldn't have worked, I have 4GB RAM, here.

Submitted by Robert on Wed, 2009-11-11 19:28.

No, the problem here seems to be the icon cache for the file icons. If you throw enough different file types (these days determined by file extensions) at BetterZip, it may lead to this problem, e.g, if you have files named file.0001 ... file.9999.

Submitted by kbeckers on Thu, 2009-11-12 10:52.


$ find home -name '*.*' | awk -F "." '{ print $NF }' | sort -u | wc -l
2241

Any workaround?
Karl.

Submitted by Robert on Fri, 2009-11-13 16:33.

Not at the moment, sorry. I'll add handling of such extreme cases in the next update. You could use the bundled p7zip from the command line. It's path is BetterZip.app/Contents/Resources/7z. If the problem really is BetterZip's icon cache, it is only a GUI issue.

Submitted by kbeckers on Fri, 2009-11-13 21:08.

fair enough.
thanks

Submitted by kurt wiley (not verified) on Tue, 2009-11-17 04:50.

Is there a reason BETTERZIP must copy files to the system disk before it can compress them? For large movies and archives, this can take a long time.

Also, if there are any glitches, BETTERZIP tends to leave copies of the files behind on the master disk.

Submitted by Robert on Tue, 2009-11-17 10:30.

BetterZip works in this way to let you do all operations without actually compressing files or modifying the archive. Only when you save the archive, the real work is done. This is in most cases faster (copying is faster than compressing, linking is even faster) and more secure. If you change your mind or something happens, your original archive is untouched. Now, if BetterZip just remembered references to the original files that the user dragged into an archive, he could believe it was secure to delete the original and empty the trash before saving the archive.

BetterZip keeps track of all the temporary files and folders beyond crashes and moves these left-overs to the trash the next time you start it.

I am aware that something has to be done about files from external disks and I will have a better solution in the next major version.