512-byte network chunks halve throughput on large transfers

[w4zu]

Follow-up to #45, as offered there.

csync2 moves file payloads in 512-byte chunks csync_send_file(),
csync_recv_file() and csync_rs_check() in rsync.c. Over TLS that works out to roughly one record per 512 bytes of file data, which is a lot of overhead for something the TLS layer is going to batch into 16 KiB records anyway.

Bumping the chunk size to 16384, the maximum TLS record payload, takes the same sync from 1938 s to 979 s. Same three nodes, same 31.66 GB / 56525 files pushed to two peers, scheduled jobs and monitoring stopped, and the #45 retry patch applied in both runs so neither of them aborts. Block size is the only thing that differs between the two numbers.

I tried 64 KiB as well, out of curiosity. It gains nothing over 16 KiB , 947 s against 946 s in an earlier round of tests which makes sense, since TLS splits anything larger back into 16 KiB records. So 16384 looks like the natural ceiling rather than an arbitrary pick.

One thing to be aware of: this change needs #45. Without it, any chunk size above the TLS record limit dies on the very first file, because
gnutls_record_send() returns a short count and csync_send_file() treats that as fatal. That is how I found the two were related in the first place.

The csync_rs_patch() local copy fallback keeps its 512-byte buffer on purpose, it never touches the network. And stack use stays modest: the two buffers in csync_rs_check() come to 32 KiB together.

Integrity was checked the same way as in #45, by comparing an md5 digest of the whole tree on all three nodes after each run.

--- a/rsync.c
+++ b/rsync.c
@@ -42,6 +42,8 @@
 #include <w32api/windows.h>
 #endif
 
+#define CSYNC_CHUNK 16384
+
 
 /* This has been taken from rsync:lib/compat.c */
 
@@ -370,7 +372,7 @@
 
 void csync_send_file(FILE *in)
 {
-	char buffer[512];
+	char buffer[CSYNC_CHUNK];
 	int rc, chunk;
 	long size;
 
@@ -381,7 +383,7 @@
 	conn_printf("octet-stream %ld\n", size);
 
 	while ( size > 0 ) {
-		chunk = size > 512 ? 512 : size;
+		chunk = size > CSYNC_CHUNK ? CSYNC_CHUNK : size;
 		rc = fread(buffer, 1, chunk, in);
 
 		if ( rc <= 0 )
@@ -398,7 +400,7 @@
 
 int csync_recv_file(FILE *out)
 {
-	char buffer[512];
+	char buffer[CSYNC_CHUNK];
 	int rc, chunk;
 	long size;
 
@@ -411,7 +413,7 @@
 	csync_debug(3, "Receiving %ld bytes ..\n", size);
 
 	while ( size > 0 ) {
-		chunk = size > 512 ? 512 : size;
+		chunk = size > CSYNC_CHUNK ? CSYNC_CHUNK : size;
 		rc = conn_read(buffer, chunk);
 
 		if ( rc <= 0 )
@@ -495,7 +497,7 @@
 int csync_rs_check(const char *filename, int isreg)
 {
 	FILE *sig_file = 0;
-	char buffer1[512], buffer2[512];
+	char buffer1[CSYNC_CHUNK], buffer2[CSYNC_CHUNK];
 	int rc, chunk, found_diff = 0;
 	int backup_errno;
 	long size;
@@ -534,7 +536,7 @@
 	}
 
 	while (size > 0) {
-		chunk = size > 512 ? 512 : size;
+		chunk = size > CSYNC_CHUNK ? CSYNC_CHUNK : size;
 		rc = conn_read(buffer1, chunk);
 
 		if (rc <= 0)
@@ -568,7 +570,7 @@
 
 	/* drain response */
 	while (size > 0) {
-		chunk = size > 512 ? 512 : size;
+		chunk = size > CSYNC_CHUNK ? CSYNC_CHUNK : size;
 		rc = conn_read(buffer1, chunk);
 		if (rc <= 0)
 			csync_fatal("Read-error while receiving data.\n");

Happy to run it against other shapes of data if that would help ,the numbers above come from a mix of many small files and a handful of 550 MB ones, so a tree of only small files might tell a different story.