large file transfer via FTP fails

flako666 · Post by **flako666** » 24 Jul 2023, 17:30

Hi, I'm using Reflector 2.4 and proftpd 1.3.6 (within OpenMediaVault)
When I transfer 23G files they get truncated or don't upload.
The task I attached has to upload 3 files, and only partially uploads one.
If I use clonezilla to upload the file, it uploads correctly

They are the same files that I talk about (viewtopic.php?f=3&t=1949), but now in Cobian disable the 'passive mode' option and change the proftpd.conf parameters:

Code: Select all

MaxRetrieveFileSize *
MaxStoreFileSize *
TimeoutIdle 600000
TimeoutNoTransfer  65535
TimeoutStalled 65535

But Cobian, it keeps failing..

Can you think of anything else to look at?

The complete log is:

Code: Select all

	2023-07-24 11:29:33 The log files have been successfully deleted.
	2023-07-24 11:29:33 [DEBUG] Deleting the temporary directory that holds the log files: G:\TMP\SYSTEM\57edbbff-c249-4916-9495-516bbe2b40f4...
	2023-07-24 11:29:34 [DEBUG] Getting the backup history for the task "{1F860AFF-11EB-4B23-9A8D-7F5DA63CBB9F}"...

	2023-07-24 11:29:34 [DEBUG] Getting the backup history for the task "e4562482-15d7-4a93-8b2d-1075b70b3b93"...
	2023-07-24 12:05:44 [DEBUG] Getting the backup history for the task "{F53E51B0-BFF9-47AA-9981-2DECFABB4E72}"...
	2023-07-24 12:05:45 [DEBUG] Getting the backup history for the task "e4562482-15d7-4a93-8b2d-1075b70b3b93"...
	2023-07-24 12:05:47 Backing up some selected tasks.
	2023-07-24 12:05:47 [DEBUG] The task DBProduccion-Histo-Mar,Jue,Dom-Remote-SoloFTP (e4562482-15d7-4a93-8b2d-1075b70b3b93) was added to the queue.

	2023-07-24 12:05:49 A backup has started...
	2023-07-24 12:05:49 Calculating the number of files. This could take a while...
	2023-07-24 12:05:49 Preventing the computer to enter sleep mode...

	2023-07-24 12:05:49 Backing up the task "DBProduccion-Histo-Mar,Jue,Dom-Remote-SoloFTP"
	2023-07-24 12:05:49 Applying the parameters to the task "DBProduccion-Histo-Mar,Jue,Dom-Remote-SoloFTP" (if any).
	2023-07-24 12:05:49 Starting the copy. The user running the task is: administrador
	2023-07-24 12:05:49 [DEBUG] Important: The debug information for the ftp or sftp connection will be logged to file "C:\Program Files\Cobian Reflector\Logs\REMOTE DEBUG 2023-07-24 12;05;49.txt" only and won't be shown in the log tab.
	2023-07-24 12:05:49 Connecting to 192.168.0.qqq:21...
	2023-07-24 12:05:49 Connected to 192.168.0.qqq:21.
	2023-07-24 12:05:49 Logged in into the remote server.
	2023-07-24 12:05:49 The remote directory "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549" has been successfully created.
	2023-07-24 12:05:49 Uploading the directory "D:\BACKUP\db-produccion-zip" to "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549"...
ERR 2023-07-24 12:09:30 An error occurred while uploading the file "D:\BACKUP\db-produccion-zip\DBProduccion_MedioDia_2-Comprimir-2023-7-23-Sun_085329 (Full).zip" to "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549": NOOP: data transfer in progress (code=200)
ERR 2023-07-24 12:09:30 An error occurred while uploading the file "D:\BACKUP\db-produccion-zip\DBProduccion_Noche_2-Compimir-2023-7-24-Mon_040236 (Full).zip" to "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549": NOOP: data transfer in progress (code=200)
ERR 2023-07-24 12:09:30 An error occurred while uploading the file "D:\BACKUP\db-produccion-zip\listfiles.md5" to "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549": NOOP: data transfer in progress (code=200)
	2023-07-24 12:09:30 The directory "D:\BACKUP\db-produccion-zip" has been uploaded to "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549".
	2023-07-24 12:09:30 Disconnected from 192.168.0.qqq.
	2023-07-24 12:09:30 [DEBUG] A sub-backup item has been added to the database.
	2023-07-24 12:09:30 [DEBUG] The actual path for the current sub-backup has been added to the database.
	2023-07-24 12:09:30 [DEBUG] A backup item has been added to the database.
	2023-07-24 12:09:31 Deleting some full backups and associated differential/incremental backups. Reason: maintaining only 4 non-parked full backup sets.
	2023-07-24 12:09:31 [DEBUG] The actual paths have been successfully deleted. Actual files may be deleted on a background thread.
	2023-07-24 12:09:31 [DEBUG] The sub-backups have been deleted. Actual files may be being deleted on a background thread.
	2023-07-24 12:09:32 The backups have been deleted. Actual files may be being deleted on a background thread.

	2023-07-24 12:09:32  Backup for the task "DBProduccion-Histo-Mar,Jue,Dom-Remote-SoloFTP" has ended 
	2023-07-24 12:09:32  Processed files: 3. Backed up files: 0. Created folders: 0. Backup size: 0 bytes. 
	2023-07-24 12:09:32  Number of errors: 3. Time elapsed: 0 hours, 3 minutes, 43 seconds. 


	2023-07-24 12:09:32 Mailing the log files...
	2023-07-24 12:09:32 [DEBUG] Creating a temporary directory: "G:\TMP\SYSTEM\cd23311a-6a77-4161-98f6-4f4b61e40898"
	2023-07-24 12:09:32 Flushing the log to file. This can take some seconds. Wait...
	2023-07-24 12:09:32 The computer can now enter sleep mode.

	2023-07-24 12:09:32 The backup has ended. There are errors. Consult the log file.

	2023-07-24 12:09:33 [DEBUG] Getting the backup history for the task "e4562482-15d7-4a93-8b2d-1075b70b3b93"...

REMOTE DEBUG 2023-07-24 12;05;49.txt

	Licence expiry date: 31/12/9999
	Set LocalDirectory to C:\Windows\system32
	Licence expiry date: 31/12/9999
	waitOnShutdownSSL=True
	StrictReturnCodes=True
	Setting socket timeout=600001
	SecureSocket timeout=600001
	SecureSocket timeout=600001
	Command encoding=System.Text.SBCSCodePageEncoding
	Setting socket buffer sizes=-1
	220 ProFTPD Server ready.
	---> FEAT
	211-Features:
	211-CLNT
	211-EPRT
	211-EPSV
	211-HOST
	211-LANG es-AR.UTF-8*
	211-MDTM
	211-MFF modify;UNIX.group;UNIX.mode;
	211-MFMT
	211-MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.groupname*;UNIX.mode*;UNIX.owner*;UNIX.ownername*;
	211-REST STREAM
	211-SITE COPY
	211-SITE MKDIR
	211-SITE RMDIR
	211-SITE SYMLINK
	211-SITE UTIME
	211-SIZE
	211-TVFS
	211-UTF8
	211 End
	---> OPTS utf8 on
	200 UTF8 set to on
	---> USER yyy
	331 Password required for yyy
	---> PASS ********
	230-Welcome user yyy@192.168.0.xxxto 127.0.1.1 FTP server.
	230-The local time is: Mon Jul 24 15:05:48 2023
	230 User yyy logged in
	ChangeWorkingDirectory('.')
	---> CWD .
	250 CWD command successful
	---> PWD
	257 "/" is the current directory
	---> CWD /db-produccion/histo__Mar-Jue-Dom
	250 CWD command successful
	---> CWD /
	250 CWD command successful
	ChangeWorkingDirectory('/db-produccion/histo__Mar-Jue-Dom')
	---> CWD /db-produccion/histo__Mar-Jue-Dom
	250 CWD command successful
	---> PWD
	257 "/db-produccion/histo__Mar-Jue-Dom" is the current directory
	---> CWD /db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549
	550 /db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549: No existe el fichero o el directorio
	Expected reply codes = [200,250] (strict=True)
	---> CWD /db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549
	550 /db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549: No existe el fichero o el directorio
	Expected reply codes = [200,250] (strict=True)
	---> MKD db-produccion-zip-2023-7-24-Mon_120549
	257 "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549" - Directory successfully created
	ChangeWorkingDirectory('/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549')
	---> CWD /db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549
	250 CWD command successful
	---> PWD
	257 "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549" is the current directory
	ChangeWorkingDirectory('/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549')
	---> CWD /db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549
	250 CWD command successful
	---> PWD
	257 "/db-produccion/histo__Mar-Jue-Dom/db-produccion-zip-2023-7-24-Mon_120549" is the current directory
	---> TYPE I
	200 Type set to I
	NewActiveDataSocket(0.0.0.0:0)
	---> EPRT |1|192.168.0.xxx|3304|
	200 EPRT command successful
	SecureSocket timeout=600001
	SecureSocket timeout=600001
	---> STOR DBProduccion_MedioDia_2-Comprimir-2023-7-23-Sun_085329 (Full).zip
	150 Opening BINARY mode data connection for DBProduccion_MedioDia_2-Comprimir-2023-7-23-Sun_085329 (Full).zip
	Awaiting a direct connection on active data-channel.
	Notify: SecureSocket.OnAccept
	Direct connection on active data-channel established.
	SecureSocket timeout=600001
	SecureSocket timeout=600001
	Sending server wakeup message
	---> NOOP
	Sending server wakeup message
	---> NOOP
	Sending server wakeup message
	---> NOOP
	Sending server wakeup message
	---> NOOP
	Sending server wakeup message
	---> NOOP
	Sending server wakeup message
	---> NOOP
	Sending server wakeup message
	---> NOOP
	Closing source stream
	Closing SecureNetworkStream
	Shutdown(Both)
	Closed SecureNetworkStream
	Attempt to close unconnected active data-channel.
	Transferred 23506786346 bytes to remote host
	200 NOOP: data transfer in progress
	Expected reply codes = [225,226,250] (strict=True)
	---> TYPE I
	200 NOOP: data transfer in progress
	NewActiveDataSocket(0.0.0.0:0)
	---> EPRT |1|192.168.0.xxx|3305|
	200 NOOP: data transfer in progress
	SecureSocket timeout=600001
	SecureSocket timeout=600001
	---> STOR DBProduccion_Noche_2-Compimir-2023-7-24-Mon_040236 (Full).zip
	200 NOOP: data transfer in progress
	Expected reply codes = [125,150,151,350] (strict=True)
	Attempt to close unconnected active data-channel.
	Caught exception NOOP: data transfer in progress (code=200)
	Closing source stream
ERR Caught exception
ERR NOOP: data transfer in progress (code=200)
	---> TYPE I
	200 NOOP: data transfer in progress
	NewActiveDataSocket(0.0.0.0:0)
	---> EPRT |1|192.168.0.xxx|3306|
	200 NOOP: data transfer in progress
	SecureSocket timeout=600001
	SecureSocket timeout=600001
	---> STOR listfiles.md5
	200 NOOP: data transfer in progress
	Expected reply codes = [125,150,151,350] (strict=True)
	Attempt to close unconnected active data-channel.
	Caught exception NOOP: data transfer in progress (code=200)
	Closing source stream
ERR Caught exception
ERR NOOP: data transfer in progress (code=200)

Post by **cobian** » 24 Jul 2023, 19:57

Try changing the status of the "Keep alive transfer"

flako666 · Post by **flako666** » 25 Jul 2023, 13:49

cobian wrote: ↑24 Jul 2023, 19:57 Try changing the status of the "Keep alive transfer"

It is active, I attach configuration

Post by **cobian** » 26 Jul 2023, 02:34

Some servers don't like this option. Try unchecking this.

flako666 · Post by **flako666** » 26 Jul 2023, 17:41

cobian wrote: ↑26 Jul 2023, 02:34 Some servers don't like this option. Try unchecking this.

Yes, deactivate "Keep alive transfer" and no cobian does not indicate errors, but leaves the files corrupted.
It's weird, because it copies all the bytes, but the md5sum is different on the destination.

Code: Select all

	 
150 Opening BINARY mode data connection for DBProduccion_MedioDia_2-Comprimir-2023-7-25-Tue_085309 (Full).zip
 Transferred 23382304811 bytes to remote host
 226 Transfer complete
windows size:  21.7 GB (23,382,304,811 bytes)
	
150 Opening BINARY mode data connection for DBProduccion_Noche_2-Compimir-2023-7-26-Wed_040214 (Full).zip
 Transferred 23917715546 bytes to remote host
 226 Transfer complete
windows size: 22.2 GB (23,917,715,546 bytes)

Post by **cobian** » 27 Jul 2023, 01:09

CobianSoft

large file transfer via FTP fails

large file transfer via FTP fails

Re: large file transfer via FTP fails

Re: large file transfer via FTP fails

Re: large file transfer via FTP fails

Re: large file transfer via FTP fails

Re: large file transfer via FTP fails