What is the best way to transfer big files to RPi?

[kraleksandr]

What is the best way to transfer big files to the RPi?
Now I've test this options:

• transmission (torrent) - up to 5 Mb/s
• Samba from windows PC - 6Mb/s at start, after few seconds avg. spd 1-2Mb/s
• sftp (WinSCP) - less than 1Mb/s
• wget (using HFS as web-server) - up to 4Mb/s, avg - 1.5Mb/s
  

RPi uses wifi connection, at this time avg. spd is around 40 mb/s (up and dl), max is 60 dl and 50 up (iperf).

[BartOtten]

Good question! Hope we can keep this topic clean with only real answers, proven methods and benchmarks.

[f1vefour]

Netcat has zero overhead so is a very good candidate.

You can find example usage here ( http://arkanis.de/weblog/2009-05-22-fast-file-transfer-with-netcat ).

[kraleksandr]

(21st Apr, 2013 10:28 AM)f1vefour Wrote:  Netcat has zero overhead so is a very good candidate.

You can find example usage here ( http://arkanis.de/weblog/2009-05-22-fast-file-transfer-with-netcat ).

How to check speed? Now target file becomes bigger for 300Kb/s

[f1vefour]

(23rd Apr, 2013 02:58 PM)kraleksandr Wrote:  

(21st Apr, 2013 10:28 AM)f1vefour Wrote:  Netcat has zero overhead so is a very good candidate.

You can find example usage here ( http://arkanis.de/weblog/2009-05-22-fast-file-transfer-with-netcat ).

How to check speed? Now target file becomes bigger for 300Kb/s

I would check the speed with the 'time' command.

To calculate speed do this.

Size in Megabyte x 1024 / Transfer time in seconds.

Example 260 Megabyte file in 1 min 12 seconds would be.

260 x 1024 = 266240 / 72 = 3697.66 kilobytes per second transfer speed

[f1vefour]

I just tested SCP from my Android device (Galaxy Nexus) to the Raspberry Pi and I had a sustained transfer rate of 2100 KB/s (260 megabyte file) using an app for the transfer.

You must be having an issue with WinSCP.

[kraleksandr]

(24th Apr, 2013 07:38 AM)f1vefour Wrote:  I just tested SCP from my Android device (Galaxy Nexus) to the Raspberry Pi and I had a sustained transfer rate of 2100 KB/s (260 megabyte file) using an app for the transfer.

You must be having an issue with WinSCP.

Maybe...
When I start copying, speed is up to 2200 Mb/s, after 11 seconds WinSCP becomes "Non Responding" and after ~30 seconds it resumes copying with 300 kb/s speed

[kraleksandr]

BtSync: 2kB/s - 1.7mB/s, avg - 500kB/s

[mk01]

if you can't use nfs, try ftp. just tried now with over sustained 6.3MB/s with 700MB file.

[kraleksandr]

Ok, ftp:
proftp, default conf, root on /media/hdd (external hdd)
up to 4 Mbit/s at start, then avg. 500 kbyte/s or error after 3 seconds
Magic
Looks like I need to buy another wifi-adapter... With MIMO and n-mode

[mk01]

just to be sure, run "iostat 2" during the transfer and check whether your iowait % is not 100% all the time.

writing with dd to the hdd is with no issues?

[kraleksandr]

Ok, when kB_wrtn/s is around 1500, then %iowait is at 70.00
dd from sdcard to hdd bs 512 count 200: iowait 75, kb_wrtn 700, "104857600 bytes (105 MB) copied, 66.848 s, 1.6 MB/s" (dd to file, fs ext4)

[mk01]

this is ok, yes

[Davem]

The results of my tests:

- Setup: Transfer big files from Windows 7 PC (wireless connected to router) to Raspberry pi (wired to router).
- Test file: one 600 MB video file
- Measurement: by NetWorx; average and maximum speeds (measurement done in Windows, manual activated direct after copy command).
- Copy to Raspberry: Internal SD card (ext.4) or External connected USB 2.0 drive (ext.4).

Samba (SMB) - standard Xbian package

1. to USB HDD: av 4.0 MB/s; max 4.2 MB/s
2. to SD: av 4.6 MB/s; max 6.3 MB/s (speed is very diverge during copy, min ca. 3MB/s)
   

NFS - (Using HaneWin NFS server and this method)

1. to USB HDD: av. 2.3 MB/s; max. 2.5 MB/s
2. to SD: av. 2.1 MB/s; max. 3.1 MB/s
   

SFTP - using WinSCP

1. to USB HDD: av 1.6 MB/s; max 2,2 MB/s
2. to SD:          av 1.6 MB/s; max 2,2 MB/s
   

FTP - using wsftpd as server and Filezilla as client; installed by this small guide

1. to USB HDD: av 5.8 MB/s; max 6.5 MB/s
2. to SD:          av 5.2 MB/s; max 6,9 MB/s (speed is very diverge during copy, min ca. 2MB/s)
   

My conclusion: the best way for me to easily copy big files from my PC to my raspberry (normally USB HDD) is by using FTP (replaced Samba for me).
/edit: added FTP measurement