Checking Shift Transfer Status and Restarting Transfers

Article ID: 515

Last updated: 28 Apr, 2022

Shift can provide status on transfers currently in progress, and can report results of transfers that were performed within the past week. You can restart and complete any interrupted transfers or transfers that completed with errors.

Note: Transfers that have been inactive for more than a week cannot be restarted.

Check transfer status from a NAS high-end computing host (for example, pfe21):
```
pfe21% shiftc --status 
```
Check transfer status from remote host (for example, your local system):
```
your_local_system% sup shiftc --status 
```
Restart a transfer that has errors or was interrupted:
```
pfe21% shiftc --restart --id=transfer_id
```

Checking Transfer Status

Show the status of all transfers:

pfe21% shiftc --status
id | state | dirs | files |     file size |  date |    run |    rate
   |       | sums | attrs |      sum size |  time |   left |
---+-------+------+-------+---------------+-------+--------+---------
 1 | done  |  0/0 |   1/1 |     92KB/92KB | 10/03 |     2s |   46KB/s
   |       |  0/0 |   0/0 |     0.0B/0.0B | 17:06 |        |
 2 | done  |  0/0 |   1/1 |     92KB/92KB | 10/03 |     8s | 11.5KB/s
   |       |  0/0 |   1/1 |     0.0B/0.0B | 17:06 |        |
 3 | done  |  1/1 |   2/2 |     99KB/99KB | 10/03 |     1s |   99KB/s
   |       |  4/4 |   0/0 |   198KB/198KB | 17:07 |        |
 4 | error |  1/1 |   1/2 |     92KB/99KB | 10/03 |     3s | 30.7KB/s
   |       |  0/0 |   0/0 |     0.0B/0.0B | 17:08 |        |
 5 | done  |  1/1 | 64/64 | 65.5GB/65.5GB | 10/03 |    29s | 2.26GB/s
   |       |  0/0 |   0/0 |     0.0B/0.0B | 17:09 |        |

Showing Detailed Status

The examples in this section show how to display details about specific transfers that are listed in the output in the previous example.

TIP: Showing the detailed status of a single transfer might result in extremely large output, as a single transfer may contain hundreds, thousands, or even millions of transferred files—and multiple lines may be output for each file. Therefore, if you want to see the detailed status of a very large transfer, redirect the output to a file. For example:


shiftc --status --id=5 > status.id5

You can also search for files matching a given pattern, and show only their status. For example:


shiftc --status --id=1 --search=file2

Show the detailed status of all operations in transfer #2, from your local system:

your_local_system% sup shiftc --status --id=2
state | op     | target              | size |  date |    run |   rate
      | tool   | info                |      |  time |   left |
------+--------+---------------------+------+-------+--------+-------
done  | cp     | lfe2:/u/user1/file1 | 92KB | 10/03 |     5s | 18KB/s
      | rsync  | -                   |      | 17:06 |        |
done  | chattr | lfe2:/u/user1/file1 |    - | 10/03 |     1s |      -
      | sftp   | -                   |      | 17:06 |        |

Show the detailed status of all operations in transfer #3 that involve a filename containing file2:

pfe21% shiftc --status --id=3 --search=file2
state | op    | target               | size |  date |    run |  rate
      | tool  | info                 |      |  time |   left |
------+-------+----------------------+------+-------+--------+------
done  | cp    | /username/dir2/file2 |  7KB | 10/03 |     1s | 7KB/s
      | mcp   | -                    |      | 17:07 |        |
done  | cksum | /username/dir2/file2 |  7KB | 10/03 |     1s | 7KB/s
      | msum  | -                    |      | 17:07 |        |

Show the detailed status of all operations in transfer #4 that have an error state:

pfe21% shiftc --status --id=4 --state=error
state | op    | target                    | size |  date |    run | rate
      | tool  | info                      |      |  time |   left |
------+-------+---------------------------+------+-------+--------+-----
error | cp    | /username/dir2/file2      |  7KB |     - |      - |    -
      | rsync | rsync: send_files         |      |       |        |
      |       | failed to open            |      |       |        |
      |       | "/username/dir2/file2":   |      |       |        |
      |       | Permission denied         |      |       |        |

Showing Transfer History

Show the history of all transfers:

pfe21% shiftc --history
id | origin             | command
---+--------------------+--------------------------------------------------------------------
 1 | pfe21.nas.nasa.gov | shiftc file1 /u/username/dir1
   | [/u/user1]         |  
 2 | pfe21.nas.nasa.gov | shiftc -p file1 lfe2:
   | [/u/user1]         |  
 3 | your_local_system  | sup shiftc -r --no-verify /username/dir1 lfe2:/username/dir2
   | [/Users/user1]     | 
 4 | your_local_system  | sup shiftc -r --secure lfe2:/username/dir2 .
   | [/username]        |  
 5 | pfe21.nas.nasa.gov | shiftc -r --hosts=2 /nobackup/user1/bigdir1 /nobackup/user1/bigdir2
   | [/u/user1]         |

Show the history of all transfers that involve a host or a command containing lfe2 from your local system:

your_local_system% sup shiftc --history --search=lfe2
id | origin             | command
---+--------------------+-------------------------------------
 2 | pfe21.nas.nasa.gov | shiftc -p file1 lfe2:
   | [/u/user1]         |  
 4 | your_local_system  | shiftc -r --secure lfe2:/username/dir2 .
   | [/username]        |