Wrong name in files/directory list

Questions about Cobian Reflector
dreamx
Posts: 7
Joined: 27 Oct 2021, 21:14

Wrong name in files/directory list

Post by dreamx »

Hello,
There seems to be something off in the files list with large directory trees.
When trying to backup a directory containing Thunderbird profiles this happens: (log follows)

2021-10-27 22:01:41 Backing up the task "Main Data"
2021-10-27 22:01:41 Impersonating the user ".\dreamx"...
2021-10-27 22:01:41 The user .\dreamx is now logged in.
2021-10-27 22:01:41 The user .\dreamx has successfully been impersonated.
2021-10-27 22:01:41 Changing the backup type to full. Reason: first backup of the task.
2021-10-27 22:01:41 Requesting the creation of a Volume Shadow Copy image for the source...
2021-10-27 22:01:44 The Volume Shadow Copy image has been successfully created.
2021-10-27 22:01:44 Starting the copy. The user running the task is: dreamx
2021-10-27 22:01:44 Compressing the sources into a global monolithic archive...
2021-10-27 22:01:44 Adding the directory "E:\Data" to the archive...
ERR 2021-10-27 22:04:26 An error occurred while creating the monolithic archive: File "E:\Data\Thunderbird\Mail\Local Folders\Licenses\" not found
2021-10-27 22:04:26 Checking the archive "C:\Scratch\Temp\Main Data.zip"...
2021-10-27 22:04:26 Clearing the archive attribute of the processed files...
2021-10-27 22:04:49 The monolithic archive "C:\Scratch\Temp\Main Data.zip" has been created. Files in archive: 0. Folders: 13029. Ignored files: 0 Uncompressed size: 24,52 GB
2021-10-27 22:04:49 Copying the file "C:\Scratch\Temp\Main Data.zip" to "\\VAULT\TimeMachine\DX-WS"...
2021-10-27 22:04:49 The file "C:\Scratch\Temp\Main Data.zip" has been successfully copied to "\\VAULT\TimeMachine\DX-WS\Main Data 2021-10-27 22;01;44 (Full).zip".
2021-10-27 22:04:49 * Backup part for task "Main Data" ended. Number of backed up files: 1. Backup part size: 22 bytes. *
2021-10-27 22:04:49 Deleting the temporary archive "C:\Scratch\Temp\Main Data.zip"...
2021-10-27 22:04:49 The temporary archive "C:\Scratch\Temp\Main Data.zip" has been successfully deleted.
2021-10-27 22:04:49 ** Backup for task "Main Data" ended. Number of backed up files: 1. Created folders: 0. Backup size: 22 bytes. **
2021-10-27 22:04:49 *** Number of errors: 1. Time elapsed for this task: 0 hours, 3 minutes, 7 seconds. ***

2021-10-27 22:04:49 Deleting the Volume Shadow Copy image...
2021-10-27 22:04:49 The Volume Shadow Copy image has been successfully deleted.
2021-10-27 22:04:49 The impersonation of .\dreamx has been reverted.
2021-10-27 22:04:49 The user .\dreamx has been logged off.
2021-10-27 22:04:49 The backup has ended. There are errors. Consult the log file.

It seems that Reflector is trying to access the directory "E:\Data\Thunderbird\Mail\Local Folders\Licenses\", however that directory never existed in that location, there is a file "E:\Data\Thunderbird\Mail\Local Folders\Licenses" and a directory "E:\Data\Thunderbird\Mail\Local Folders\Licenses.mozmsgs\" both accessible by the user Reflector impersonates; this happens regardless of the Thunderbird profile being in use or not and there are no orphaned nodes on the filesystem.
Reflector version 0.9.36

Off-topic: Thank you for reviving this project. :D
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Wrong name in files/directory list

Post by cobian »

Thanks. Will check this out.
--
Luis Cobian
Cobian Backup's creator
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Wrong name in files/directory list

Post by cobian »

Tested and it seems to work as expected here. Does the Licenses file has some weird attribute/permissions setting?

Have installed Thunderbird but my directory structure for my profile doesn't contain anything in

C:\Users\cobian\AppData\Roaming\Thunderbird\Profiles\qx9bss3v.default-release\Mail\Local Folders

What is the default directory for the Licenses file?
--
Luis Cobian
Cobian Backup's creator
dreamx
Posts: 7
Joined: 27 Oct 2021, 21:14

Re: Wrong name in files/directory list

Post by dreamx »

In that instance Licenses is just a subdirectory under "Local Folders", in Thunderbird a "subfolder" generates 3 entities: a mailbox file <sdir_name>, an index file <sdir_name>.msf and a message directory <sdir_name>.mozmsgs, none of them have any special permissions or restrictions on the filesystem.

Same thing happens to local git repositories (logs follow)

2021-10-28 13:56:49 Backing up the task "Dev"
2021-10-28 13:56:49 Impersonating the user ".\dreamx"...
2021-10-28 13:56:49 The user .\dreamx is now logged in.
2021-10-28 13:56:49 The user .\dreamx has successfully been impersonated.
2021-10-28 13:56:49 Changing the backup type to full. Reason: first backup of the task.
2021-10-28 13:56:49 Requesting the creation of a Volume Shadow Copy image for the source...
2021-10-28 13:56:51 The Volume Shadow Copy image has been successfully created.
2021-10-28 13:56:51 Starting the copy. The user running the task is: dreamx
2021-10-28 13:56:51 Compressing the sources into a global monolithic archive...
2021-10-28 13:56:51 Adding the directory "E:\Dev" to the archive...
ERR 2021-10-28 13:56:55 An error occurred while creating the monolithic archive: File "E:\Dev\github\freebsd\.git\config\" not found
2021-10-28 13:56:55 Checking the archive "C:\Scratch\Temp\Dev.zip"...
2021-10-28 13:56:55 Clearing the archive attribute of the processed files...
2021-10-28 13:57:06 The monolithic archive "C:\Scratch\Temp\Dev.zip" has been created. Files in archive: 0. Folders: 13637. Ignored files: 0 Uncompressed size: 8,03 GB
2021-10-28 13:57:06 Copying the file "C:\Scratch\Temp\Dev.zip" to "\\VAULT\TimeMachine\DX-WS"...
2021-10-28 13:57:06 The file "C:\Scratch\Temp\Dev.zip" has been successfully copied to "\\VAULT\TimeMachine\DX-WS\Dev 2021-10-28 13;56;51 (Full).zip".
2021-10-28 13:57:06 * Backup part for task "Dev" ended. Number of backed up files: 1. Backup part size: 22 bytes. *
2021-10-28 13:57:06 Deleting the temporary archive "C:\Scratch\Temp\Dev.zip"...
2021-10-28 13:57:06 The temporary archive "C:\Scratch\Temp\Dev.zip" has been successfully deleted.
2021-10-28 13:57:06 ** Backup for task "Dev" ended. Number of backed up files: 1. Created folders: 0. Backup size: 22 bytes. **
2021-10-28 13:57:06 *** Number of errors: 1. Time elapsed for this task: 0 hours, 0 minutes, 16 seconds. ***

2021-10-28 13:57:06 Deleting the Volume Shadow Copy image...
2021-10-28 13:57:06 The Volume Shadow Copy image has been successfully deleted.
2021-10-28 13:57:06 The impersonation of .\dreamx has been reverted.
2021-10-28 13:57:06 The user .\dreamx has been logged off.
2021-10-28 13:57:07 The backup has ended. There are errors. Consult the log file.

and (excluding the previous directory that generated the error)

2021-10-28 14:15:13 Backing up the task "Dev"
2021-10-28 14:15:13 Impersonating the user ".\dreamx"...
2021-10-28 14:15:13 The user .\dreamx is now logged in.
2021-10-28 14:15:13 The user .\dreamx has successfully been impersonated.
2021-10-28 14:15:13 Changing the backup type to full. Reason: first backup of the task.
2021-10-28 14:15:13 Requesting the creation of a Volume Shadow Copy image for the source...
2021-10-28 14:15:15 The Volume Shadow Copy image has been successfully created.
2021-10-28 14:15:15 Starting the copy. The user running the task is: dreamx
2021-10-28 14:15:15 Compressing the sources into a global monolithic archive...
2021-10-28 14:15:15 Adding the directory "E:\Dev" to the archive...
ERR 2021-10-28 14:15:19 An error occurred while creating the monolithic archive: File "E:\Dev\github\jcl\.git\modules\jcl\source\include\jedi\config\" not found
2021-10-28 14:15:19 Checking the archive "C:\Scratch\Temp\Dev.zip"...
2021-10-28 14:15:19 Clearing the archive attribute of the processed files...
2021-10-28 14:15:32 The monolithic archive "C:\Scratch\Temp\Dev.zip" has been created. Files in archive: 0. Folders: 13604. Ignored files: 0 Uncompressed size: 8,03 GB
2021-10-28 14:15:32 Copying the file "C:\Scratch\Temp\Dev.zip" to "\\VAULT\TimeMachine\DX-WS"...
2021-10-28 14:15:32 The file "C:\Scratch\Temp\Dev.zip" has been successfully copied to "\\VAULT\TimeMachine\DX-WS\Dev 2021-10-28 14;15;15 (Full).zip".
2021-10-28 14:15:32 * Backup part for task "Dev" ended. Number of backed up files: 1. Backup part size: 22 bytes. *
2021-10-28 14:15:32 Deleting the temporary archive "C:\Scratch\Temp\Dev.zip"...
2021-10-28 14:15:32 The temporary archive "C:\Scratch\Temp\Dev.zip" has been successfully deleted.
2021-10-28 14:15:32 ** Backup for task "Dev" ended. Number of backed up files: 1. Created folders: 0. Backup size: 22 bytes. **
2021-10-28 14:15:32 *** Number of errors: 1. Time elapsed for this task: 0 hours, 0 minutes, 18 seconds. ***

2021-10-28 14:15:32 Deleting the Volume Shadow Copy image...
2021-10-28 14:15:32 The Volume Shadow Copy image has been successfully deleted.
2021-10-28 14:15:32 The impersonation of .\dreamx has been reverted.
2021-10-28 14:15:32 The user .\dreamx has been logged off.
2021-10-28 14:15:32 The backup has ended. There are errors. Consult the log file.

In this case "config" is a regular file, but the log entry suggests that Reflector is looking for a directory "E:\Dev\github\freebsd\.git\config\" or "E:\Dev\github\jcl\.git\modules\jcl\source\include\jedi\config\"; in all instances the impersonated user has "Full control" on the filesystem.
Manually archiving those locations (winrar and 7zip) or using Backup 11 (Gravity) gets all the files correctly - zero differences between the unpacked content and the original files.

Side-note: the resulting output archive is an empty 22 bytes zip archive in all the tests with this error.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Wrong name in files/directory list

Post by cobian »

I have received some similar reports for .git directories (which I have a lot) but it works without problems for me... :shock:

Will try again with al my git files and see if I can get this error somehow.
--
Luis Cobian
Cobian Backup's creator
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Wrong name in files/directory list

Post by cobian »

Tested now with a .git directory with the same structure as yours:

.git\modules\jcl\source\include\jedi\config

is interpreted as a file and compressed without errors. I wonder why this is failing on some systems but not on others.

Is you .git directory on a NTFS system or EXT3 perhaps?
--
Luis Cobian
Cobian Backup's creator
dreamx
Posts: 7
Joined: 27 Oct 2021, 21:14

Re: Wrong name in files/directory list

Post by dreamx »

All tested directories are on an NTFS filesystem.
I have tested again with the same files in a clean Windows 10 VM, also NTFS, and I've got the same behavior as before - the errors are absolutely identical on .git as well as on the Thunderbird profile files.
I will further test on FAT32 and exFAT which none have alternate streams or extended attributes, maybe GetAttributes() or FileInfo() if you use one of those have inconsistent behavior on NTFS.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Wrong name in files/directory list

Post by cobian »

I'll be glad if you get some more info on this!
--
Luis Cobian
Cobian Backup's creator
dreamx
Posts: 7
Joined: 27 Oct 2021, 21:14

Re: Wrong name in files/directory list

Post by dreamx »

Follow-up with the test results:
Windows 10 21H1 machine - source: local NTFS - destination: samba share
Windows 10 21H2 VM clean install - source: local NTFS/local exFAT - destination: samba share
Windows 7 SP1 VM clean install - source: local NTFS/local exFAT - destination: samba share

The tasks were configured identically with "Incremental backup", user impersonation (Cancel if impersonation fails) and filesystem structure was kept identical on all three systems.
All tests were ran twice on the Windows 10 systems, one time with the file-system monitor active (nod32 AV and Windows Defender respectively) and a second time with the anti-virus disabled, the Windows 7 system had no file-system monitor active.

Compression type: No compression - task finishes successfully on all systems
Compression type: Compress files individually - task finishes successfully on all systems
Compression type: Separated monolithic archives - task fails with identical error: ERR 2021-10-28 13:56:55 An error occurred while creating the monolithic archive: File "E:\Dev\github\freebsd\.git\config\" not found (copy/paste from previous post)
Compression type: Global monolithic archive - task fails with identical error: ERR 2021-10-28 13:56:55 An error occurred while creating the monolithic archive: File "E:\Dev\github\freebsd\.git\config\" not found (copy/paste from previous post)
The errors, when present, were the same for the other .git directories as well as for the specific Thunderbird profile used previously.

An initial conclusion would be that the error has nothing to do with the source filesystem in this case, but with the compression engine when compressing more than one file.
Last edited by dreamx on 28 Oct 2021, 19:05, edited 1 time in total.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Wrong name in files/directory list

Post by cobian »

But the error indicates FILE "E:\Dev\github\freebsd\.git\config\" doesn't exist. Why is it adding a final \ there but not in my computer? Very strange...
--
Luis Cobian
Cobian Backup's creator
Post Reply