Data Transfer with Netcat (NC) and Tar

Idea

A fast and unencrypted data transfer for local network usage. The idea is to send a tar stream over TCP by using the nc tool.

The pipe viewer (pv) is optional but allows tracking the transfer rates and total data transmitted. By comparing with the original data sizes, this can act as a progress indicator.

Local Data Copying with Tar

tar -C SOURCE-DIR -c . | pv | tar -C DEST-DIR -x

Network Data Copying

To copy data as-is, be sure to invoke the tar commands as root such that they can read and write all data including permissions etc.

The --numeric-owner flag is useful to transfer data as-is even on systems with different user and group IDs.

Sender Side

The sender acts as a server on address 192.168.2.1 and port 2244. Change as required.

tar -C SOURCE-DIR --numeric-owner -c . | pv | \
                    nc -q 10 -l 192.168.2.1 2244

Receiver Side

The receiver acts as a client:

nc -w 10 192.168.2.1 2244 | pv | tar -C DEST-DIR --numeric-owner -x

Verifying Integrity

To check if the file contents match on both ends, compute sha256sum and compare it on the target. Note that if the paths are not equal on source and destination, it may be required to edit checksums.sha256 before using it on the receiver side.

Sender Side

find SOURCE-DIR -type f -exec sha256sum {} + | tee checksums.sha256

Receiver Side

sha256sum -c checksums.sha256 | tee checksum-results.txt
grep -v OK checksum-results.txt

Data Copying and Checksum Computation in One

To be tested! Requires shell capable of process substitution.

Sender Side

tar -C SOURCE-DIR -c . | pv | tee >(sha256sum > sha256-sent.txt) \
            >(nc -q 10 -l 192.168.2.1 2244) > /dev/null

Receiver Side

nc -w 10 192.168.2.1 2244 | pv | tee >(sha256sum > sha256-recv.txt) \
            >(tar -C DEST-DIR -x) > /dev/null

Afterwards, compare contents of sha256-sent.txt and sha256-recv.txt. They should be identical!

See Also

nc(1), pv(1), rsync(1), tar(1), tee(1)


Ma_Sys.ma Website 5 (1.0.0) – no Flash, no JavaScript, no Webfont, no Copy Protection, no Mobile First. No bullshit. No GUI needed. Works with any browser.

Created: 2020/10/25 17:27:36 | Revised: 2020/10/25 18:22:37 | Tags: linux, nc, netcat, tar, transfer, kb | Version: 1.0.0 | SRC (Pandoc MD) | GPL

Copyright (c) 2020 Ma_Sys.ma. For further info send an e-mail to Ma_Sys.ma@web.de.

This program is free software: you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program. If not, see <http://www.gnu.org/licenses/>.