Use the file transfer commands scp or shiftc when the source and destination are located on different hosts—either on two different NAS high-end computing hosts, or on a NAS host and a remote host such as your local desktop system.

The basic syntax is:

copy-command [options]...source...destination

scp Command

The Secure Copy Protocol (scp) command, based on the Secure Shell Protocol (SSH), is a means of securely transferring files between a local and a remote host. Both your authentication information (such as password or passcode) and your data are encrypted.

The most widely used scp program is from OpenSSH.

Where is scp installed at NAS?

A copy of scp from OpenSSH without the patch is available on all front-end systems of NAS compute resources (PFEs, AFEs, CFEs, CGHFEs), Lou, and the secure front-end systems (SFEs).

Do you need it installed on your local host?

If you have a version of SSH installed on your local host, scp is most likely already installed there.

When to use it?

Typically, scp is used to transfer small files within NAS (<< 5 GB) or offsite (<< 1 GB) that take a reasonable amount of time to complete.

Examples

In these examples, "outbound" means the command is initiated on a NAS compute resources front-end host or a Lou front-end host, whether the file is being pushed or pulled. "Inbound" means the command is initiated on your local host.

Note: Omit local_username@ and nas_username@ in the examples below if your local username and NAS username are identical. These examples assume you already know how to log into the NAS enclave.

For outbound transfer:

lou% scp local_username@your_localhost.domain:file1 ./file2

For inbound transfer if SSH passthrough has been set up correctly:

your_localhost% scp file1 nas_username@lou.nas.nasa.gov:file2

For inbound transfer if you have not set up SSH passthrough:

your_localhost% scp -oProxyCommand='ssh nas_username@sfeX.nas.nasa.gov 
ssh-proxy %h' file1 nas_username@lou.nas.nasa.gov:file2

where sfeX is sfe[6-8].

Note: Due to formatting issues, the command line in the above example is shown on two lines. It should be entered as one line.

Performance

Within the NAS secure enclave, depending on source and destination hosts and other factors, the performance range will be 40-100 MB/sec.

If your data will compress well, consider enabling compression by adding -C to your scp command line.

We recommend using OpenSSH 5.0 or a newer version.

Shift Command (shiftc)

Shift (shiftc) is a NAS-developed tool for performing automated local and remote file transfers. Shift utilizes a variety of underlying file transports to achieve maximum performance for files of any size on any file system.

Where is it installed at NAS?

Shift is installed on Lou and on the PFEs.

Do you need it installed on your local host?

For transfers between your local host and NAS systems, you must install the SUP client as discussed in Shift File Transfer Overview.

When to use it?

Shift is the recommended method for transferring files to and from the Lou mass storage system, as it can create tar files as part of the transfer and split a transfer into multiple tar files for oversized directories (larger than 1 TB).

Shift can be used as a drop-in replacement for scp between any enclave systems. For transfers between your local host and NAS systems, the transfer must be initiated from your local host with shiftc invoked via the SUP client (that is, using the command sup shiftc). If an encrypted transfer is required, use the shiftc --secure option.

Example

pfe25% shiftc /nobackupp2/username/filename lou:
your_localhost% sup shiftc pfe:filename .

Performance

Shift uses the highest performing file transport that is available on both sides of the transfer, and is optimal for the sizes of the files being transferred.

For more information, see File Transfer: Overview and Shift File Transfer Overview.