bahattab
24-12-2008, 01:03 AM
File Transfer over SSH using WGET and SCP
If you have a Backup Files on another Server Instead of downloading them on a PC you can transfer them across from server to server. Useful if File Transfers are Large > 1GB
You can either use WGET or SCP
Login to SSH and Type :
# scp username@username.server.com:server1/path/tofile/somefile.tar.gz
OR
wget ftp://username:password@ip:/path/to/file
This will then transfer the file over from one server to your Server
there is a way of transferring over an entire tree,
you have a site on one host which you want to move over to a new server...
Is there any quick way so you don't have to download via FTP and then upload?
yes.. you can do that with scp. like this:
scp -r username@someserver.com:/path/to/the/folder/to/copy
The above line will copy the folder (with all files and sub directories) from remote server to current server.
if you wnat to send something over then use this :
scp -r /local/folder/to/copy username@someserver.com:/folder/you/want/to/copy/to
http://forums.mpadc.com/showthread.php/file-transfer-over-4428.html?
What I do when I upgrade/switch servers(full site transfer, files along with the server logs and database backups... etc.) or simply want to transfer a file from one server to another is to use ssh and a cute little software called "wget".
Login through ssh or telnet to the server with the files you want to transfer.
cd to the directory that contains the file(s)
cd /home/domain.com/htdocs
Once you're inside the directory that contains the file(s) tar them up. This will not only help you with the file size, it will also preserve your file/directory permissions.
tar -cvf filestomove.tar ./
Once that's finished, go to the server that you want to move files to and login through ssh or telnet.
cd to the directory that you want to place the file(s)
cd /home/homepage.com/htdocs
Get the file from the other server using wget
wget -c http://www.domain.com/filestomove.tar
Un-tar the file(s)
tar -xvf filestomove.tar
At this point I would check to make sure the correct owner and group are on your files you just unpacked. You can check to see what they are by typing:
ls -la
If the owner or group is wrong you can type this to fix all the files:
chown -R owner.group ./
owner.group should be replaced with the owner name you want and the group name you want. This will recursively go through all your files and directories and changed them to the correct owner and group.
http://www.sitefuse.com/f16/t22.html
SCP
The scp command allows you to copy files over ssh connections. This is pretty useful if you want to transport files between computers, for example to backup something. The scp command uses the ssh command and they are very much alike. However, there are some important differences.
The scp command can be used in three* ways: to copy from a (remote) server to your computer, to copy from your computer to a (remote) server, and to copy from a (remote) server to another (remote) server. In the third case, the data is transferred directly between the servers; your own computer will only tell the servers what to do. These options are very useful for a lot of things that require files to be transferred, so let's have a look at the syntax of this command:
[rechosen@localhost ~]$ scp examplefile yourusername@yourserver:/home/yourusername/
Looks quite familiar, right? But there are differences. The command above will transfer the file "examplefile" to the directory "/home/yourusername/" at the server "yourserver", trying to get ssh acces with the username "yourusername".
That's quite a lot information, but scp really needs it all. Well, almost all of it. You could leave out the "yourusername@" in front of "yourserver", but only if you want to login on the server with your current username on your own computer.
Let's have a closer look at the end of the command. There's a colon over there, with a directory after it. Just like Linux's normal cp command, scp will need to know both the source file(s) and the target directory (or file). For remote hosts, the file(s)/directory are given to the scp command is this way.
You can also copy a file (or multiple files) from the (remote) server to your own computer. Let's have a look at an example of that:
[rechosen@localhost ~]$ scp yourusername@yourserver:/home/yourusername/examplefile .
Note: The dot at the end means the current local directory. This is a handy trick that can be used about everywhere in Linux. Besides a single dot, you can also type a double dot ( .. ), which is the parent directory of the current directory.
This will copy the file "/home/yourusername/examplefile" to the current directory on your own computer, provided that the username and password are correct and that the file actually exists.
You probably already guessed that the following command copies a file from a (remote) server to another (remote) server:
[rechosen@localhost ~]$ scp yourusername@yourserver:/home/yourusername/examplefile yourusername2@yourserver2:/home/yourusername2/
Please note that, to make the above command work, the servers must be able to reach each other, as the data will be transferred directly between them. If the servers somehow can't reach each other (for example, if port 22 is not open on one of the sides) you won't be able to copy anything. In that case, copy the files to your own computer first, then to the other host. Or make the servers able to reach each other (for example by opening the port).
Well, those are the main uses of scp. We'll now go a bit more in-depth about the differences between ssh and scp.
*: Actually you can also use it just like the normal cp command, withhout any ssh connections in it, but that's quite useless. It requires you to type an extra 's' =).
Specifying a port with scp
The scp command acts a little different when it comes to ports. You'd expect that specifying a port should be done this way:
[rechosen@localhost ~]$ scp -p yourport yourusername@yourserver:/home/yourusername/examplefile .
However, that will not work. You will get an error message like this one:
cp: cannot stat `yourport': No such file or directory
This is caused by the different architecture of scp. It aims to resemble cp, and cp also features the -p option. However, in cp terms it means 'preserve', and it causes the cp command to preserve things like ownership, permissions and creation dates. The scp command can also preserve things like that, and the -p option enables this feature. The port specification should be done with the -P option. Therefore, the following command will work:
[rechosen@localhost ~]$ scp -P yourport
yourusername@yourserver:/home/yourusername/examplefile .
Also note that the -P option must be in front of the (remote) server. The ssh command will still work if you put -p yourport behind the host syntax, but scp won't. Why? Because scp also supports copying between two servers and therefore needs to know which server the -P option applies to.
Another difference between scp and ssh
Unlike ssh, scp cannot be used to run a command on a (remote) server, as it already uses that feature of ssh to start the scp server on the host. The scp command does have an option that accepts a program (the -S option), but this program will then be used instead of ssh to establish the encrypted connection, and it will not be executed on the remote host.
Tips & Tricks with ssh and scp
Quite a handy thing about scp is that it supports asterisks. You can copy all files in a remote directory in a way like this:
[rechosen@localhost ~]$ scp yourusername@yourserver:/home/yourusername/* .
And you can also just copy a whole directory by specifying the -r (recursive) option:
[rechosen@localhost ~]$ scp -r yourusername@yourserver:/home/yourusername/ .
Both of these also work when copying to a (remote) server or copying between a (remote) server and another (remote) server.
The ssh command can come in handy if you don't know the exact location of the file you want to copy with scp. First, ssh to the (remote) server:
[rechosen@localhost ~]$ ssh yourusername@yourserver
Then browse to the right directory with cd. This is essential Linux terminal knowledge, so I won't explain it here. When you're in the right directory, you can get the full path with this command:
[rechosen@localhost ~]$ pwd
Note: pwd is an abbreviation of Print Working Directory, which is a useful way to remember the command.
You can then copy this output, leave the ssh shell by pressing Ctrl + D, and then paste the full directory path in your scp command. This saves a lot of remembering and typing!
You can also limit the bandwidth scp may use when copying. This is very useful if you're wanting to copy a huge amount of data without suffering from slow internet for a long time. Limiting bandwidth is done this way:
scp -l bandwidthlimit yourusername@yourserver:/home/yourusername/* .
The bandwidth is specified in Kbit/sec. What does this mean? Eight bits is one byte. If you want to copy no faster than 10 Kbyte/sec, set the limit to 80. If you want to copy no faster than 80 Kbyte/sec, set the limit to 640. Get it? You should set the limit to eight times the maximum Kbyte/sec you want it to be. I'd recommend to set the -l option with all scp'ing you do on a connection that other people need to use, too. A big amount of copying can virtually block a whole 10 Mbit network if you're using hubs.
Final Words
Well, that was it! I hope you learned a lot. Of course, you can always have a quick look at this tutorial again if you forgot something. Please tell other people who might be interested about this tutorial, you'll help this blog to grow if you do =). Thank you for reading and have a lot of fun with your new knowledge!
http://www.linuxtutorialblog.com/post/ssh-and-scp-howto-tips-tricks
Scp2 (Secure Copy) is used to copy files over the network securely. It uses ssh2 for data transfer: it uses the same authentication and provides the same security as ssh2.
It is probably the simplest way to copy a file into a remote machine. Let's suppose you want to copy the file filename contained in the directory local_dir to your account myname on the directory remote_dir on host host1. Using scp you could enter from the command line:
scp local_dir/filename myname@host1:remote_dir
In such a way the file filename is copied with the same name. Wildcards can be used (read more about those from sshregex man page). The command:
scp local_dir/* myname@host1:remote_dir
copies all files from directory local_dir into the directory remote_dir of host1.
The command:
scp myname@host1:remote_dir/filename .
copies the file filename from remote_dir on host1 to the local directory.
Scp supports many options and allows copies between two remote systems as in the following example:
scp myname@host1:remote_dir/filename myname@host2:another_dir
See its manual page for a complete presentation.
Obviously, using scp, you must know the exact directory tree of the remote machine, so in practice sftp is often preferred.
http://tldp.org/LDP/LGNET/issue64/dellomodarme.html
If you have a Backup Files on another Server Instead of downloading them on a PC you can transfer them across from server to server. Useful if File Transfers are Large > 1GB
You can either use WGET or SCP
Login to SSH and Type :
# scp username@username.server.com:server1/path/tofile/somefile.tar.gz
OR
wget ftp://username:password@ip:/path/to/file
This will then transfer the file over from one server to your Server
there is a way of transferring over an entire tree,
you have a site on one host which you want to move over to a new server...
Is there any quick way so you don't have to download via FTP and then upload?
yes.. you can do that with scp. like this:
scp -r username@someserver.com:/path/to/the/folder/to/copy
The above line will copy the folder (with all files and sub directories) from remote server to current server.
if you wnat to send something over then use this :
scp -r /local/folder/to/copy username@someserver.com:/folder/you/want/to/copy/to
http://forums.mpadc.com/showthread.php/file-transfer-over-4428.html?
What I do when I upgrade/switch servers(full site transfer, files along with the server logs and database backups... etc.) or simply want to transfer a file from one server to another is to use ssh and a cute little software called "wget".
Login through ssh or telnet to the server with the files you want to transfer.
cd to the directory that contains the file(s)
cd /home/domain.com/htdocs
Once you're inside the directory that contains the file(s) tar them up. This will not only help you with the file size, it will also preserve your file/directory permissions.
tar -cvf filestomove.tar ./
Once that's finished, go to the server that you want to move files to and login through ssh or telnet.
cd to the directory that you want to place the file(s)
cd /home/homepage.com/htdocs
Get the file from the other server using wget
wget -c http://www.domain.com/filestomove.tar
Un-tar the file(s)
tar -xvf filestomove.tar
At this point I would check to make sure the correct owner and group are on your files you just unpacked. You can check to see what they are by typing:
ls -la
If the owner or group is wrong you can type this to fix all the files:
chown -R owner.group ./
owner.group should be replaced with the owner name you want and the group name you want. This will recursively go through all your files and directories and changed them to the correct owner and group.
http://www.sitefuse.com/f16/t22.html
SCP
The scp command allows you to copy files over ssh connections. This is pretty useful if you want to transport files between computers, for example to backup something. The scp command uses the ssh command and they are very much alike. However, there are some important differences.
The scp command can be used in three* ways: to copy from a (remote) server to your computer, to copy from your computer to a (remote) server, and to copy from a (remote) server to another (remote) server. In the third case, the data is transferred directly between the servers; your own computer will only tell the servers what to do. These options are very useful for a lot of things that require files to be transferred, so let's have a look at the syntax of this command:
[rechosen@localhost ~]$ scp examplefile yourusername@yourserver:/home/yourusername/
Looks quite familiar, right? But there are differences. The command above will transfer the file "examplefile" to the directory "/home/yourusername/" at the server "yourserver", trying to get ssh acces with the username "yourusername".
That's quite a lot information, but scp really needs it all. Well, almost all of it. You could leave out the "yourusername@" in front of "yourserver", but only if you want to login on the server with your current username on your own computer.
Let's have a closer look at the end of the command. There's a colon over there, with a directory after it. Just like Linux's normal cp command, scp will need to know both the source file(s) and the target directory (or file). For remote hosts, the file(s)/directory are given to the scp command is this way.
You can also copy a file (or multiple files) from the (remote) server to your own computer. Let's have a look at an example of that:
[rechosen@localhost ~]$ scp yourusername@yourserver:/home/yourusername/examplefile .
Note: The dot at the end means the current local directory. This is a handy trick that can be used about everywhere in Linux. Besides a single dot, you can also type a double dot ( .. ), which is the parent directory of the current directory.
This will copy the file "/home/yourusername/examplefile" to the current directory on your own computer, provided that the username and password are correct and that the file actually exists.
You probably already guessed that the following command copies a file from a (remote) server to another (remote) server:
[rechosen@localhost ~]$ scp yourusername@yourserver:/home/yourusername/examplefile yourusername2@yourserver2:/home/yourusername2/
Please note that, to make the above command work, the servers must be able to reach each other, as the data will be transferred directly between them. If the servers somehow can't reach each other (for example, if port 22 is not open on one of the sides) you won't be able to copy anything. In that case, copy the files to your own computer first, then to the other host. Or make the servers able to reach each other (for example by opening the port).
Well, those are the main uses of scp. We'll now go a bit more in-depth about the differences between ssh and scp.
*: Actually you can also use it just like the normal cp command, withhout any ssh connections in it, but that's quite useless. It requires you to type an extra 's' =).
Specifying a port with scp
The scp command acts a little different when it comes to ports. You'd expect that specifying a port should be done this way:
[rechosen@localhost ~]$ scp -p yourport yourusername@yourserver:/home/yourusername/examplefile .
However, that will not work. You will get an error message like this one:
cp: cannot stat `yourport': No such file or directory
This is caused by the different architecture of scp. It aims to resemble cp, and cp also features the -p option. However, in cp terms it means 'preserve', and it causes the cp command to preserve things like ownership, permissions and creation dates. The scp command can also preserve things like that, and the -p option enables this feature. The port specification should be done with the -P option. Therefore, the following command will work:
[rechosen@localhost ~]$ scp -P yourport
yourusername@yourserver:/home/yourusername/examplefile .
Also note that the -P option must be in front of the (remote) server. The ssh command will still work if you put -p yourport behind the host syntax, but scp won't. Why? Because scp also supports copying between two servers and therefore needs to know which server the -P option applies to.
Another difference between scp and ssh
Unlike ssh, scp cannot be used to run a command on a (remote) server, as it already uses that feature of ssh to start the scp server on the host. The scp command does have an option that accepts a program (the -S option), but this program will then be used instead of ssh to establish the encrypted connection, and it will not be executed on the remote host.
Tips & Tricks with ssh and scp
Quite a handy thing about scp is that it supports asterisks. You can copy all files in a remote directory in a way like this:
[rechosen@localhost ~]$ scp yourusername@yourserver:/home/yourusername/* .
And you can also just copy a whole directory by specifying the -r (recursive) option:
[rechosen@localhost ~]$ scp -r yourusername@yourserver:/home/yourusername/ .
Both of these also work when copying to a (remote) server or copying between a (remote) server and another (remote) server.
The ssh command can come in handy if you don't know the exact location of the file you want to copy with scp. First, ssh to the (remote) server:
[rechosen@localhost ~]$ ssh yourusername@yourserver
Then browse to the right directory with cd. This is essential Linux terminal knowledge, so I won't explain it here. When you're in the right directory, you can get the full path with this command:
[rechosen@localhost ~]$ pwd
Note: pwd is an abbreviation of Print Working Directory, which is a useful way to remember the command.
You can then copy this output, leave the ssh shell by pressing Ctrl + D, and then paste the full directory path in your scp command. This saves a lot of remembering and typing!
You can also limit the bandwidth scp may use when copying. This is very useful if you're wanting to copy a huge amount of data without suffering from slow internet for a long time. Limiting bandwidth is done this way:
scp -l bandwidthlimit yourusername@yourserver:/home/yourusername/* .
The bandwidth is specified in Kbit/sec. What does this mean? Eight bits is one byte. If you want to copy no faster than 10 Kbyte/sec, set the limit to 80. If you want to copy no faster than 80 Kbyte/sec, set the limit to 640. Get it? You should set the limit to eight times the maximum Kbyte/sec you want it to be. I'd recommend to set the -l option with all scp'ing you do on a connection that other people need to use, too. A big amount of copying can virtually block a whole 10 Mbit network if you're using hubs.
Final Words
Well, that was it! I hope you learned a lot. Of course, you can always have a quick look at this tutorial again if you forgot something. Please tell other people who might be interested about this tutorial, you'll help this blog to grow if you do =). Thank you for reading and have a lot of fun with your new knowledge!
http://www.linuxtutorialblog.com/post/ssh-and-scp-howto-tips-tricks
Scp2 (Secure Copy) is used to copy files over the network securely. It uses ssh2 for data transfer: it uses the same authentication and provides the same security as ssh2.
It is probably the simplest way to copy a file into a remote machine. Let's suppose you want to copy the file filename contained in the directory local_dir to your account myname on the directory remote_dir on host host1. Using scp you could enter from the command line:
scp local_dir/filename myname@host1:remote_dir
In such a way the file filename is copied with the same name. Wildcards can be used (read more about those from sshregex man page). The command:
scp local_dir/* myname@host1:remote_dir
copies all files from directory local_dir into the directory remote_dir of host1.
The command:
scp myname@host1:remote_dir/filename .
copies the file filename from remote_dir on host1 to the local directory.
Scp supports many options and allows copies between two remote systems as in the following example:
scp myname@host1:remote_dir/filename myname@host2:another_dir
See its manual page for a complete presentation.
Obviously, using scp, you must know the exact directory tree of the remote machine, so in practice sftp is often preferred.
http://tldp.org/LDP/LGNET/issue64/dellomodarme.html