Search:     Advanced search

Increasing File Transfer Rates

Article ID: 292
Last updated: 18 Aug, 2011
Views: 1788
Posted: 18 Aug, 2011
by Beyk D.
Updated: 18 Aug, 2011
by Dunbar J.

One challenge users face is moving large amounts of data efficiently to/from NAS across the network.  Often, minor system, software, or network configuration changes can increase network performance an order of magnitude or more.  This article describes some methods for increasing data transfer performance.

If you are experiencing slow transfer rates, try these quick tips:

  • Transfer using the bridge nodes (bridge1, bridge2) instead of the Pleiades front-end systems (PFEs). The bridge nodes have much more memory, along with 10-Gigabit Ethernet interfaces to accommodate many large transfers. The PFEs often become oversubscribed and cause slowness.
  • If using the scp command, make sure you are using OpenSSH version 5 or later. Older versions of SSH have a hard limit on transfer rates and are not designed for WAN transfers. You can check your version of SSH by running the command ssh -V.
  • For large files that are a gigabyte or larger, we recommend using BBFTP. This application allows for transferring simultaneous streams of data and doesn't have the overhead of encrypting all the data (authentication is still encrypted).

Online Network Testing Tools

The NAS PerfSONAR Service provides a custom website that that allows you to quickly self-diagnose your remote network connection issues, and reports the maximum bandwidth between sites, as well as any problems in the network path.  Command-line tools are available if your system does not have a web browser.

Test results are also sent to our network experts, who will analyze traffic flows, identify problems, and work to resolve any bottlenecks that limit your network performance, whether the problem is at NAS or a remote site.

One-on-One Help

If you still require assistance in increasing your file transfer rates, please contact the NAS Control Room at support@nas.nasa.gov, and a network expert will work with you or your local administrator one-on-one to identify methods for increasing your rates.

To learn about other network-related support areas. see also, End-to-End Networking Services.

This article was:  
Also read
document TCP Performance Tuning for WAN Transfers
document File Transfers Tips
document NAS Remote Network Diagnostic Tools

Prev   Next
Streamlining File Transfers from Pleiades Compute Nodes to Lou     Choosing an MPI Library


Contact Us

General User Assistance

Security

  • Report security issues 24x7x365
  • Toll-free: 1-877-NASA-SEC (1-877-627-2732)
  • E-mail: soc@nasa.gov

User Documentation

High-End Computing Capability (HECC) Project Office

NASA High-End Computing Program

Tell Us About It

Our goal is furnish all the information you need to efficiently and effectively use the HECC resources needed for your NASA computational projects.

We welcome your input on features and topics that you would like to see included on this website.

Please send us email with your wish list and other feedback.