to downloadHere is a general overview of the various file transfer scenarios within the NAS environment, with pointers to related articles.

File Transfers Between NAS Hosts

For basic information about transferring files within the NAS secure enclave, see the following articles:

• Local File Transfer Commands - cp, cxfscp, mcp, shiftc, rsync
• Remote File Transfer Commands - scp, shiftc
• Front-End Usage Guidelines - file transfer between the compute systems or mass storage systems
• Streamlining File Transfers from Compute Nodes to Lou - within a PBS job
• Checking File Integrity - to ensure the integrity of the data before and after the transfer

File Transfers Between a NAS Host and Your Local Host

Transferring files between a NAS host (such as Electra, Aitken, Cabeus, Endeavour, or Lou) and a remote host, such as your local desktop, is more complex. Be aware of the following multiple factors:

Which Commands to Use

Remote file transfer commands such as scp and sup/shiftc are supported on most NAS systems. Depending on the way the transfers are performed, you may need one or both of the client and server software for SCP and/or the SUP/Shift client installed on your local host.

Transfer Rates

File transfer rates with the scp command, especially using SCP from versions of OpenSSH older than 4.7, can be as slow as 2 megabytes per second (MB/sec). For transferring large files over a long distance, consider doing the following:

• Upgrade to the the latest version of OpenSSH.
• Enable compression by adding -C to the scp command line if the data will compress well.
• Use sup/shiftc.

Security Issues

• With SCP, your authentication information (such as password or passcode) and data are encrypted.
• With shiftc without the --secure option, only the authentication information is encrypted, while data is not.
• You can use the SFEs or Secure Unattended Proxy (SUP). SUP is the easiest way to transfer files from and/or to your site if your local system is configured to allow the transfer.

Inbound File Transfers

When a file transfer command is initiated on a remote host, such as your local desktop, the transfer must go through either the SFEs or SUP.

Using the Secure Front Ends

Going through the SFEs requires authentication via RSA SecurID at the time of the transfer. You will be prompted for your RSA SecurID passcode when you issue the file transfer commands (such as scp).

You can use one of the following approaches:

1. One step via SSH Passthrough: Initiate scp from your local system to Pleiades, Endeavour, or Lou if SSH Passthrough has been set up.
2. One step via SSH ProxyCommand: Initiate scp from your local system through an SFE using the ssh ‑oProxyCommand option.

To learn more, see Inbound File Transfers through SFEs Examples.

Using the Secure Unattended Proxy

Going through the SUP does not require RSA SecurID token authentication at the time of transfer. Instead, special "SUP keys" using RSA SecurID authentication must be obtained ahead of time. The SUP keys are good for one week and are used automatically to authenticate your file transfers using scp or shiftc issued on a command line or in a job script.

File transfers going through SUP offers multiple benefits over going through the SFEs:

• SUP allows transfers to be unattended; that is, you do not have to type in your password, passphrase, or passcode when the file transfer command is issued. So, file transfers can be done within a script that can be scheduled to run ahead of time. File transfers through the SFEs cannot be done in a script.
• File transfers through SUP are done in one step, and setting up SSH passthrough is not needed since the SFEs are not involved.

See Using the Secure Unattended Proxy (SUP) and Shift Transfer Tool Overview for more information.

NAS Username and Your Local Username

If your NAS username and local username are different, you may have to add the appropriate username in the scp or sup/shiftc command line.

• If you issue the command on your local host, then username is your NAS username.
• If you issue the command on a NAS host, then username is your local username.

In the examples shown in Outbound File Transfer Examples and Inbound File Transfers through SFEs Examples, you will find the correct syntax for adding the appropriate username in the file transfer commands.

For inbound file transfers, if you have correctly included your NAS username in the ~/.ssh/config file of your local system, you do not have to include your NAS username in the scp or sup/shiftc command.

Improving File Transfer Performance

Some file transfer commands provide options that can be used to improve your transfer rates. For example, enabling compression during file transfers may help in some cases; with the shiftc command, you can use multiple streams instead of a single stream for better performance. Read Tips for File Transfers and Increasing File Transfer Rates for more information.

File transfer performance is also dependent on some system-wide settings. If necessary, ask your local system administrator to look into issues discussed in the following articles:

• TCP Performance Tuning for WAN Transfers
• Optional Advanced Tuning for Linux
• Pittsburgh Supercomputing Center's Enabling High Performance Data Transfers - a properly tuned TCP/IP stack

The NAS Networks team can help you analyze the performance your file transfers. Contact us at support@nas.nasa.gov.