Forum
What is the best way to transfer big files to RPi? - Printable Version

+- Forum (http://forum.xbian.org)
+-- Forum: Software (/forum-6.html)
+--- Forum: Others (/forum-24.html)
+--- Thread: What is the best way to transfer big files to RPi? (/thread-816.html)

What is the best way to transfer big files to RPi? - kraleksandr - 20th Apr, 2013 07:36 PM

What is the best way to transfer big files to the RPi?
Now I've test this options:
  • transmission (torrent) - up to 5 Mb/s
  • Samba from windows PC - 6Mb/s at start, after few seconds avg. spd 1-2Mb/s
  • sftp (WinSCP) - less than 1Mb/s
  • wget (using HFS as web-server) - up to 4Mb/s, avg - 1.5Mb/s

RPi uses wifi connection, at this time avg. spd is around 40 mb/s (up and dl), max is 60 dl and 50 up (iperf).

Re: What is the best way to transfer big files to RPi? - BartOtten - 21st Apr, 2013 08:36 AM

Good question! Hope we can keep this topic clean with only real answers, proven methods and benchmarks.

Re: What is the best way to transfer big files to RPi? - f1vefour - 21st Apr, 2013 10:28 AM

Netcat has zero overhead so is a very good candidate.

You can find example usage here ( http://arkanis.de/weblog/2009-05-22-fast-file-transfer-with-netcat ).

RE: What is the best way to transfer big files to RPi? - kraleksandr - 23rd Apr, 2013 02:58 PM

(21st Apr, 2013 10:28 AM)f1vefour Wrote:  Netcat has zero overhead so is a very good candidate.

You can find example usage here ( http://arkanis.de/weblog/2009-05-22-fast-file-transfer-with-netcat ).

How to check speed? Now target file becomes bigger for 300Kb/s

Re: RE: What is the best way to transfer big files to RPi? - f1vefour - 24th Apr, 2013 05:21 AM

(23rd Apr, 2013 02:58 PM)kraleksandr Wrote:  
(21st Apr, 2013 10:28 AM)f1vefour Wrote:  Netcat has zero overhead so is a very good candidate.

You can find example usage here ( http://arkanis.de/weblog/2009-05-22-fast-file-transfer-with-netcat ).

How to check speed? Now target file becomes bigger for 300Kb/s

I would check the speed with the 'time' command.

To calculate speed do this.

Size in Megabyte x 1024 / Transfer time in seconds.

Example 260 Megabyte file in 1 min 12 seconds would be.

260 x 1024 = 266240 / 72 = 3697.66 kilobytes per second transfer speed

Re: What is the best way to transfer big files to RPi? - f1vefour - 24th Apr, 2013 07:38 AM

I just tested SCP from my Android device (Galaxy Nexus) to the Raspberry Pi and I had a sustained transfer rate of 2100 KB/s (260 megabyte file) using an app for the transfer.

You must be having an issue with WinSCP.

RE: What is the best way to transfer big files to RPi? - kraleksandr - 24th Apr, 2013 04:29 PM

(24th Apr, 2013 07:38 AM)f1vefour Wrote:  I just tested SCP from my Android device (Galaxy Nexus) to the Raspberry Pi and I had a sustained transfer rate of 2100 KB/s (260 megabyte file) using an app for the transfer.

You must be having an issue with WinSCP.

Maybe...
When I start copying, speed is up to 2200 Mb/s, after 11 seconds WinSCP becomes "Non Responding" and after ~30 seconds it resumes copying with 300 kb/s speed

RE: What is the best way to transfer big files to RPi? - kraleksandr - 25th Apr, 2013 02:25 PM

BtSync: 2kB/s - 1.7mB/s, avg - 500kB/s

RE: What is the best way to transfer big files to RPi? - mk01 - 28th Apr, 2013 03:01 AM

if you can't use nfs, try ftp. just tried now with over sustained 6.3MB/s with 700MB file.

RE: What is the best way to transfer big files to RPi? - kraleksandr - 28th Apr, 2013 03:54 AM

Ok, ftp:
proftp, default conf, root on /media/hdd (external hdd)
up to 4 Mbit/s at start, then avg. 500 kbyte/s or error after 3 seconds
Magic Smile
Looks like I need to buy another wifi-adapter... With MIMO and n-mode

RE: What is the best way to transfer big files to RPi? - mk01 - 28th Apr, 2013 04:24 AM

just to be sure, run "iostat 2" during the transfer and check whether your iowait % is not 100% all the time.

writing with dd to the hdd is with no issues?

RE: What is the best way to transfer big files to RPi? - kraleksandr - 28th Apr, 2013 05:25 AM

Ok, when kB_wrtn/s is around 1500, then %iowait is at 70.00
dd from sdcard to hdd bs 512 count 200: iowait 75, kb_wrtn 700, "104857600 bytes (105 MB) copied, 66.848 s, 1.6 MB/s" (dd to file, fs ext4)

RE: What is the best way to transfer big files to RPi? - mk01 - 28th Apr, 2013 05:35 AM

this is ok, yes

RE: What is the best way to transfer big files to RPi? - Davem - 23rd May, 2013 05:47 AM

The results of my tests:

- Setup: Transfer big files from Windows 7 PC (wireless connected to router) to Raspberry pi (wired to router).
- Test file: one 600 MB video file
- Measurement: by NetWorx; average and maximum speeds (measurement done in Windows, manual activated direct after copy command).
- Copy to Raspberry: Internal SD card (ext.4) or External connected USB 2.0 drive (ext.4).

Samba (SMB) - standard Xbian package
  1. to USB HDD: av 4.0 MB/s; max 4.2 MB/s
  2. to SD: av 4.6 MB/s; max 6.3 MB/s (speed is very diverge during copy, min ca. 3MB/s)

NFS - (Using HaneWin NFS server and this method)
  1. to USB HDD: av. 2.3 MB/s; max. 2.5 MB/s
  2. to SD: av. 2.1 MB/s; max. 3.1 MB/s

SFTP - using WinSCP
  1. to USB HDD: av 1.6 MB/s; max 2,2 MB/s
  2. to SD:          av 1.6 MB/s; max 2,2 MB/s

FTP - using wsftpd as server and Filezilla as client; installed by this small guide
  1. to USB HDD: av 5.8 MB/s; max 6.5 MB/s
  2. to SD:          av 5.2 MB/s; max 6,9 MB/s (speed is very diverge during copy, min ca. 2MB/s)

My conclusion: the best way for me to easily copy big files from my PC to my raspberry (normally USB HDD) is by using FTP (replaced Samba for me).
/edit: added FTP measurement