Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Questions about Cobian Backup 11 (Gravity)
Post Reply
User avatar
markinson
Posts: 9
Joined: 27 Nov 2020, 16:05

Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by markinson »

First of all, congratulations to the developer ;) , because Cobian Backup is an unsurpassed program, which despite the years passed remains among the best - the best in my way of understanding backups - existing available.

Even if my English is really bad, I will try to expose the problem.

My operating system is Windows 10 20H2.

A few days ago, suddenly and without any modification on my part, one, and only one, of the tasks (complete and manually started) of Cobian Backup 11 Gravity no longer starts, it remains stuck at 0% and, in the end, created an (empty) file 1 KB in size. Specifically this is the task for the complete backup of the Mozilla Thunderbird profile (which has a weight of about 13 GB).

All other tasks run smoothly, including another one that handles nearly 19GB of data.

Without success, I tried:
  • to recreate the task;
  • to reset the permissions (as recommended in the FAQ), through "Tools - Reset permissions";
  • to run a chkdsk of the disks, without encountering problems;
  • to start the offending task from the Windows 10 safe mode;
  • to change the backup destination folder;
  • to disable the Volume Shadows Copy;
  • to reinstall the program.
I specify that the backup source folder is "C:\Users\user\AppData\Roaming\Thunderbird", but I don't think the catch comes from here.

I even restored some old images of the operating system, to go back in time, but even in this case that particular task does not want to know to start.

Trying and trying, by pure chance, I changed the type of compression, from "Zip Compression" to "No Compression" or "7zip Compression".
Well, with "No compression" or "7zip compression" the task is executed!
:shock:
It seems to me that it takes longer, however it is started and ended properly.

I investigated further and the problem would appear to come from the "C:\Users\user\AppData\Roaming\Thunderbird\Profiles\profilename\Mail\Feeds\" folder.
Under the "Feeds" folder I find other folders with messages from the RSS feeds inside. Some of these contain thousands of files. One, in particular, over 110 thousand files.
If I exclude these folders (at least those with more than 5 thousand files) using the specific functionality of the program, the task is executed! It takes a long time to start, but it starts up and proceeds as it should.
All this while maintaining the "Zip Compression".

At this point I ask: can the research I have done be considered correct? That is: is it right to conclude that in the presence of folders with thousands and even tens of thousands of files, if you want to use Cobian Backup, it is necessary to use "7zip compression" (or no compression)?

Thank you so much for your attention! :)
"Facesti come quei che va di notte, che porta il lume dietro e sé non giova, ma dopo sé fa le persone dotte"
Dante (Purgatorio, Canto XXII)
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by cobian »

Hello!

Zip compression can always be the source of many problems. In theory it should work if you are using Zip64 on compression (Options-Compression-Zip64 Always). I say in theory because I've seen reports where compressing thousands of files can result in a corrupted archive.

Try changing the Zip64 options to Always and try again to see if forcing this option from the beginning gives a better result.
--
Luis Cobian
Cobian Backup's creator
User avatar
markinson
Posts: 9
Joined: 27 Nov 2020, 16:05

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by markinson »

Dear Luis,
thank you once again for your time! :)

As you suggested, I changed the setting in "Tools - Options - Compression - Use Zip64" from "Automatically" to "Always", but unfortunately the problem with "Zip Compression" was not solved: after more than thirty minutes, the task remains frozen at 0%.

So I investigated further, based on your statement:
I've seen reports where compressing thousands of files can result in a corrupted archive
and I discovered something very interesting.

But first a step back: I couldn't explain why, overnight, that particular task didn't work anymore. This was my gripe.

So I went to recover the backups with "Zip Compression" made in the past months and I checked the number of items contained in the most populous folders of "C:\Users\user\AppData\Roaming\Thunderbird\Profiles\profilesname\Mail\Feeds\".

Well, while in the original folders under "Feeds" I had thousands or tens of thousands of files, in those backed up the files were only a few thousand: I had therefore lost tens of thousands of files! :o

The other day I did the same task but with "7zip Compression" and I checked the contents of the most populous folders under "Feed". In this case all (in some almost all, except a few dozen) elements have been backed up! A folder under "Feed", in particular, should have contained over 118 thousand files, as in the source, and in fact, in the file with "7zip compression" it received exactly the same number of files.

I seem to be able to conclude, if you agree, that:
  • the problem described above (fewer files backed up than those in the source) does not depend on the program used (Cobian Backup or others), but it depends on the compression method (ZIP in this case);
  • to make a backup of folders containing thousands of files it is advisable to use the "7zip Compression" both because otherwise Cobian Backup does not start and because you risk having a backup containing not all the files contained in the source.
Thank you very very much for your very very useful program which, I sincerely hope, will find a future development, because you have been able to condense, better than anyone else, the best features and options for an excellent backup.
Congratulations again! :)
"Facesti come quei che va di notte, che porta il lume dietro e sé non giova, ma dopo sé fa le persone dotte"
Dante (Purgatorio, Canto XXII)
User avatar
markinson
Posts: 9
Joined: 27 Nov 2020, 16:05

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by markinson »

I did some further tests, trying to back up the Thunderbird profile with a program other than Cobian Backup (SyncBackFree specifically), but always using zip compression.
Well, the Thunderbird profile is backed up and the compressed zip file created regularly. So I looked into the most populous folders and found that, with respect to the origin, all files (tens of thousands of files) had been included in the archive.

How can this be explained?
Cobian Backup uses its "own engine" for ZIP compression which, in the presence of thousands of files to work with, cannot guarantee the backup of all of them and, therefore, is it necessary to choose 7zip compression?

Thanks in advance for any possible clarification! :)
"Facesti come quei che va di notte, che porta il lume dietro e sé non giova, ma dopo sé fa le persone dotte"
Dante (Purgatorio, Canto XXII)
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by cobian »

Should be some bug in the zip engine I suppose. :?
--
Luis Cobian
Cobian Backup's creator
User avatar
markinson
Posts: 9
Joined: 27 Nov 2020, 16:05

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by markinson »

Thanks again for your time!
Wow, that's a really pity then. :(

:oops: If you still have the patience to answer me:
- is the ZIP engine managed by the dll "cbEngine.dll"? (I don't think it is "zlib1.dll" because in the portable version of the program it is not present, yet ZIP compression is available);
- if I tried to replace the zip engine (assuming it is the aforementioned dll) with an older version? (I have Cobian Backup in my software warehouse since version 5.2.1.0!) :)

For now I have changed the compression type, from ZIP to 7zip.

P.S.
Although I have subscribed to the thread, I am not receiving notification emails when a new post is published.

Thanks in advance for what you can suggest!
"Facesti come quei che va di notte, che porta il lume dietro e sé non giova, ma dopo sé fa le persone dotte"
Dante (Purgatorio, Canto XXII)
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by cobian »

Yes, its handled by the cbEngine but I'm using an external zip library(ZipForge)

https://www.componentace.com/zip_compon ... pforge.htm
--
Luis Cobian
Cobian Backup's creator
User avatar
markinson
Posts: 9
Joined: 27 Nov 2020, 16:05

Re: Suddenly a task (and only one) gets stuck at 0%: a matter of files?

Post by markinson »

Thank you so much for your kindness and time reserved for me. ;)

Unfortunately, I was unable to put your latest information to good use in any way.

I can confirm that the problem lies in the "Zip compression" backup of folders containing thousands of files (or tens of thousands of files): the process does not start.
No difficulty instead if I use the "7zip compression".

Thanks again!
"Facesti come quei che va di notte, che porta il lume dietro e sé non giova, ma dopo sé fa le persone dotte"
Dante (Purgatorio, Canto XXII)
Post Reply