Very slow to creating monolithic archive

Questions about Cobian Reflector
User avatar
Paltus
Posts: 29
Joined: 18 Jan 2021, 08:05
Location: Russia

Re: Very slow to creating monolithic archive

Post by Paltus »

Unfortunately, the speed is really low. I compared Reflector and one of the programs (there is a limited free version). Yes, Reflector loses significantly. Moreover, the settings of the two programs were set as identical as possible. If this topic is valid, I will post the log files.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Very slow to creating monolithic archive

Post by cobian »

Still slow with 0.9.15? Should go ~80% faster now.
--
Luis Cobian
Cobian Backup's creator
User avatar
Paltus
Posts: 29
Joined: 18 Jan 2021, 08:05
Location: Russia

Re: Very slow to creating monolithic archive

Post by Paltus »

No, I just saw the new version. I'm going to run tests.

Installed a new version. Compared. To be honest, even without a microscope you can see that the speed has increased. There are practically no differences between the two programs. Bravo!

No compression
Cobian

Code: Select all

 2021-10-08 17:38:20 A backup has started...
    2021-10-08 17:38:20 Calculating the number of files. This could take a while...
    2021-10-08 17:38:20 Backing up the task "New task"
    2021-10-08 17:38:20 Requesting the creation of a Volume Shadow Copy image for the source...
    2021-10-08 17:38:20 The Volume Shadow Copy image has been successfully created.
    2021-10-08 17:38:20 Starting the copy. The user running the task is: СИСТЕМА
    2021-10-08 17:38:20 The destination directory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)" has been successfully created.
    2021-10-08 17:38:20 Backing up the directory "D:\BACKUP\ARCHIV-1" to "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)"...
    2021-10-08 17:38:36 Checking if there are empty subdirectories...
    2021-10-08 17:38:36 The subdirectory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)\HW_BASE\Attaches" has been deleted. Reason: empty subdirectories should be deleted according to the task advanced settings.
    2021-10-08 17:38:36 The subdirectory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)\HW_BASE\CfgReports" has been deleted. Reason: empty subdirectories should be deleted according to the task advanced settings.
    2021-10-08 17:38:36 The subdirectory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)\HW_BASE\Docs" has been deleted. Reason: empty subdirectories should be deleted according to the task advanced settings.
    2021-10-08 17:38:36 The subdirectory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)\HW_BASE\Images\Models" has been deleted. Reason: empty subdirectories should be deleted according to the task advanced settings.
    2021-10-08 17:38:36 The subdirectory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)\HW_BASE\Images\Users" has been deleted. Reason: empty subdirectories should be deleted according to the task advanced settings.
    2021-10-08 17:38:36 The subdirectory "E:\1\D\BACKUP\ARCHIV-1 2021-10-08 17;38;20 (Full)\HW_BASE\NetTemplates" has been deleted. Reason: empty subdirectories should be deleted according to the task advanced settings.
    2021-10-08 17:38:36 Checking of empty subdirectories done.
    2021-10-08 17:38:36 The directory "D:\BACKUP\ARCHIV-1" has been backed up.
    2021-10-08 17:38:36  * Backup part for task "New task" ended. Number of backed up files: 7867. Backup part size: 5,21 GB. *
    2021-10-08 17:38:36  ** Backup for task "New task" ended. Number of backed up files: 7867. Created folders: 13. Backup size: 5,21 GB. **
    2021-10-08 17:38:36 Deleting the Volume Shadow Copy image...
    2021-10-08 17:38:37 The Volume Shadow Copy image has been successfully deleted.
    2021-10-08 17:38:38 The backup has ended without errors.

"Another" program

Code: Select all

08.10.2021  17:37:39  Start of the task.
08.10.2021  17:37:39  
08.10.2021  17:37:39  Scanning the source files ...
08.10.2021  17:37:40      Folder "D:\BACKUP\ARCHIV-1", Size: 5 595 139 494 bytes,  files: 7 867
08.10.2021  17:37:40  Source data: 5,21 GB (5 595 139 494 bytes),  files: 7 867
08.10.2021  17:37:40  Scanning is completed. Duration: 0 sec
08.10.2021  17:37:40  
08.10.2021  17:37:40  Creating backup on demand. FULL backup will be created.
08.10.2021  17:37:40  Backup type: Full backup
08.10.2021  17:37:40  Creating temporary shadow copy of disk D ...
08.10.2021  17:37:41  Temporary shadow copy of disk D, ID={b36a2cb9-1aa0-46e6-a789-bce4b1f4845a},Z
08.10.2021  17:37:41  Operation has been completed successfully. Duration: 1 sec
08.10.2021  17:37:41  (case 1.3)
08.10.2021  17:37:41  
08.10.2021  17:37:41  Creating backup "E:\2\Task---1 -- 2021-10-08 17-37"
08.10.2021  17:37:41  The amount of free disk space on the disk "E:\"  161,65 GB
08.10.2021  17:37:41  Files: 7 867. Number of threads: 1
08.10.2021  17:37:55  Operation has been completed successfully. All files have been copied. Duration: 14 sec
08.10.2021  17:37:55  The amount of free disk space on the disk "E:\"  156,43 GB
08.10.2021  17:37:55  
08.10.2021  17:37:55  Saving results...
08.10.2021  17:37:56  
08.10.2021  17:37:56  Completing the task.
08.10.2021  17:37:56  ---------------------------------------------------------------
08.10.2021  17:37:56  Total duration: 16 sec
08.10.2021  17:37:56  
08.10.2021  17:37:56  Deleting temporary files ...
08.10.2021  17:37:56     Deleted: Temporary shadow copy of disk D, ID={b36a2cb9-1aa0-46e6-a789-bce4b1f4845a}
08.10.2021  17:37:56  Operation has been completed.
User avatar
Paltus
Posts: 29
Joined: 18 Jan 2021, 08:05
Location: Russia

Re: Very slow to creating monolithic archive

Post by Paltus »

I will continue
Monolithic archive without compression

Cobian

Code: Select all

    2021-10-08 17:47:18 A backup has started...
    2021-10-08 17:47:18 Calculating the number of files. This could take a while...
    2021-10-08 17:47:18 Backing up the task "New task"
    2021-10-08 17:47:18 Requesting the creation of a Volume Shadow Copy image for the source...
    2021-10-08 17:47:18 The Volume Shadow Copy image has been successfully created.
    2021-10-08 17:47:18 Starting the copy. The user running the task is: СИСТЕМА
    2021-10-08 17:47:18 Compressing the sources into a global monolithic archive...
    2021-10-08 17:47:18 Monolithic compression doesn't support Volume Shadow Copy in this version...
    2021-10-08 17:47:18 Adding the directory "D:\BACKUP\ARCHIV-1" to the archive...
    2021-10-08 17:47:40 Checking the archive "C:\Windows\TEMP\New task.zip"...
    2021-10-08 17:47:55 Clearing the archive attribute of the processed files...
    2021-10-08 17:47:55 The monolithic archive "C:\Windows\TEMP\New task.zip" has been created. Files in archive: 7867. Folders: 14. Ignored files: 0  Uncompressed size: 5,21 GB
    2021-10-08 17:47:55 The destination directory "E:\2\D\BACKUP\ARCHIV-1" has been successfully created.
    2021-10-08 17:47:55 Copying the file "C:\Windows\TEMP\New task.zip" to "E:\2\D\BACKUP\ARCHIV-1"...
    2021-10-08 17:48:02 The file "C:\Windows\TEMP\New task.zip" has been successfully copied to "E:\2\D\BACKUP\ARCHIV-1\New task 2021-10-08 17;47;18 (Full).zip".
    2021-10-08 17:48:02  * Backup part for task "New task" ended. Number of backed up files: 1. Backup part size: 5,21 GB. *
    2021-10-08 17:48:02  ** Backup for task "New task" ended. Number of backed up files: 7867. Created folders: 0. Backup size: 5,21 GB. **
    2021-10-08 17:48:02 Deleting the temporary archive "C:\Windows\TEMP\New task.zip"...
    2021-10-08 17:48:02 The temporary archive "C:\Windows\TEMP\New task.zip" has been successfully deleted.
    2021-10-08 17:48:02 Deleting the Volume Shadow Copy image...
    2021-10-08 17:48:02 The Volume Shadow Copy image has been successfully deleted.
    2021-10-08 17:48:02 The backup has ended without errors.
"Another" program

Code: Select all

08.10.2021  17:49:16  Start of the task.
08.10.2021  17:49:16  
08.10.2021  17:49:16  Scanning the source files ...
08.10.2021  17:49:17      Folder "D:\BACKUP\ARCHIV-1", Size: 5 595 139 494 bytes,  files: 7 867
08.10.2021  17:49:17  Source data: 5,21 GB (5 595 139 494 bytes),  files: 7 867
08.10.2021  17:49:17  Scanning is completed. Duration: 0 sec
08.10.2021  17:49:17  
08.10.2021  17:49:17  Creating backup on demand. FULL backup will be created.
08.10.2021  17:49:17  Backup type: Full backup
08.10.2021  17:49:17  Creating temporary shadow copy of disk D ...
08.10.2021  17:49:17  Temporary shadow copy of disk D, ID={2a0a3d18-f7f3-45e9-87bb-a0ab0b99561d},Z
08.10.2021  17:49:17  Operation has been completed successfully. Duration: 0 sec
08.10.2021  17:49:17  (case 1.2)
08.10.2021  17:49:17  
08.10.2021  17:49:17  Creating backup "E:\2\Task---1 -- 2021-10-08 17-49.zip"
08.10.2021  17:49:17  The amount of free disk space on the disk "E:\"  145,99 GB
08.10.2021  17:49:17  Preparing...
08.10.2021  17:49:18  Archiving...
08.10.2021  17:49:38  Created "Task---1 -- 2021-10-08 17-49.zip", ZIP size: 5,21 GB,  files: 7 867
08.10.2021  17:49:38  Operation has been completed successfully. Duration: 20 sec
08.10.2021  17:49:38  The amount of free disk space on the disk "E:\"  140,77 GB
08.10.2021  17:49:38  
08.10.2021  17:49:38  Saving results...
08.10.2021  17:49:39  
08.10.2021  17:49:39  Completing the task.
08.10.2021  17:49:39  ---------------------------------------------------------------
08.10.2021  17:49:39  Total duration: 22 sec
08.10.2021  17:49:39  
08.10.2021  17:49:39  Deleting temporary files ...
08.10.2021  17:49:39     Deleted: Temporary shadow copy of disk D, ID={2a0a3d18-f7f3-45e9-87bb-a0ab0b99561d}
08.10.2021  17:49:39  Operation has been completed.
maxdaddy
Posts: 6
Joined: 06 Oct 2021, 16:32

Re: Very slow to creating monolithic archive

Post by maxdaddy »

For me I now get an error because it says Volume Shadow Copy is not supported with monolithic archive at 0.9.15. I turned that option off in the task properties but it's still trying to use it and giving the error.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Very slow to creating monolithic archive

Post by cobian »

What does the error say? Perhaps it's some file that is open by some other application.
--
Luis Cobian
Cobian Backup's creator
maxdaddy
Posts: 6
Joined: 06 Oct 2021, 16:32

Re: Very slow to creating monolithic archive

Post by maxdaddy »

Log:
2021-10-08 15:09:09 A backup has started...
2021-10-08 15:09:09 Calculating the number of files. This could take a while...
2021-10-08 15:09:28 Backing up the task "Projects"
ERR 2021-10-08 15:09:28 Canceling the task because the creation of the Volume Shadow Copy failed.
2021-10-08 15:09:29 The backup has ended. There are errors. Consult the log file.

In the settings for the task, I unchecked the option "Use Volume Shadow Copies" but it looks like it's still trying to use it.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Very slow to creating monolithic archive

Post by cobian »

It seems that you have checked Cancel if Volume Shadow Copy fails on the Settings, but is really Use VSC in the task unchecked?
--
Luis Cobian
Cobian Backup's creator
maxdaddy
Posts: 6
Joined: 06 Oct 2021, 16:32

Re: Very slow to creating monolithic archive

Post by maxdaddy »

Yes, I made sure that "Use VSC" is unchecked in the task. I unchecked the global "Cancel if Volume Shadow Copy" option as well. It now fails saying a particular file is not found, even though the file does exist and I've closed any programs that might have it open. I checked and there is nothing different about this file in terms of permissions. Not sure what's going on.

2021-10-11 08:23:03 A backup has started...
2021-10-11 08:23:03 Calculating the number of files. This could take a while...
2021-10-11 08:23:21 Backing up the task "Projects"
2021-10-11 08:23:21 Starting the copy. The user running the task is: SYSTEM
2021-10-11 08:23:21 Compressing the sources into a global monolithic archive...
2021-10-11 08:23:21 Adding the directory "C:\Projects" to the archive...
ERR 2021-10-11 08:24:30 An error occurred while creating the monolithic archive: File "C:\Projects\AWS\kub-ansible-aws\.git\config\" not found
2021-10-11 08:24:30 Checking the archive "C:\windows\TEMP\Projects.zip"...
2021-10-11 08:24:30 Clearing the archive attribute of the processed files...
2021-10-11 08:24:57 The monolithic archive "C:\windows\TEMP\Projects.zip" has been created. Files in archive: 0. Folders: 16601. Ignored files: 15644 Uncompressed size: 1.99 GB
2021-10-11 08:24:57 Copying the file "C:\windows\TEMP\Projects.zip" to "C:\Temp\Cobian_Reflector_Backup\Projects"...
2021-10-11 08:24:57 The file "C:\windows\TEMP\Projects.zip" has been successfully copied to "C:\Temp\Cobian_Reflector_Backup\Projects\Projects 2021-10-11 08;23;21 (Full).zip".
2021-10-11 08:24:57 * Backup part for task "Projects" ended. Number of backed up files: 1. Backup part size: 46 bytes. *
2021-10-11 08:24:57 ** Backup for task "Projects" ended. Number of backed up files: 1. Created folders: 0. Backup size: 46 bytes. **
2021-10-11 08:24:57 Deleting the temporary archive "C:\windows\TEMP\Projects.zip"...
2021-10-11 08:24:57 The temporary archive "C:\windows\TEMP\Projects.zip" has been successfully deleted.
2021-10-11 08:24:57 Deleting some full backups and associated differential/incremental backups. Reason: maintaining only 1 non-parked full backup sets.
2021-10-11 08:24:57 The file "C:\Temp\Cobian_Reflector_Backup\Projects\Projects 2021-10-11 08;12;15 (Full).zip" has been successfully deleted.
2021-10-11 08:24:57 The backups have been deleted. Actual files may be being deleted on a background thread.
2021-10-11 08:24:58 The backup has ended. There are errors. Consult the log file.
User avatar
cobian
Site Admin
Posts: 4492
Joined: 31 Oct 2020, 01:25
Location: Sweden
Contact:

Re: Very slow to creating monolithic archive

Post by cobian »

Yes, I'm aware of some errors with .git files.

But everything with VSC seems OK here.
--
Luis Cobian
Cobian Backup's creator
Post Reply