Electra, NASA's first prototype modular supercomputing system, is housed in a module a short distance from the primary NASA Advanced Supercomputing (NAS) facility. Electra is configured as follows:
- 16 Broadwell racks and 8 Skylake E-cells
- 3,384 nodes
- 121,536 cores
- 576 terabytes total memory
- 8.10 petaflops theoretical peak performance
Hostnames
There are 4 individual rack units (IRUs) in each rack. For every two racks, there is a rack leader controlling them. The naming convention for the nodes residing in every two racks uses only odd rack numbers.
- Broadwell
- 18 nodes per IRU, with 144 nodes residing in every two racks (2 racks x 4 IRUs x 18 nodes).
The hostnames for the Broadwell nodes are r[x]i[0-7]n[0-17], where x = odd numbers between 1 and 15.
- Skylake
- 36 nodes per IRU, with 288 nodes residing in every two racks (2 racks x 4 IRUs x 36 nodes).
The hostnames for the Skylake nodes are r[x]i[0-7]n[0-35], where x = odd numbers between 133 and 147.
Processor, Memory and Network Subsystems Statistics
Electra's system architecture is ICE X. The following table provides detailed configuration statistics for the processor, memory, and network subsystems:
| Processor |
|
Broadwell |
Skylake |
| CPU |
14-Core
Xeon E5-2680v4 |
20-Core
Xeon Gold 6148 |
| Newest Instruction Set |
AVX2 |
AVX-512 |
| Hyperthreading |
ON |
ON |
| TurboBoost |
ON |
ON |
| CPU-Clock |
2.4 GHz |
2.4 GHz |
| Maximum Double Precision Floating Point Operations per Cycle per Core |
16 |
32 |
| # of Cores/node |
28 |
40 |
| Total # of Nodes |
1,152 |
2,232 |
| Total # of Cores |
32,256 |
89,280 |
| Total Double Precision TFlops |
1,239 |
6,857 |
| Memory |
| L1 Cache |
Local to each core;
Instruction cache: 32K
Data cache: 32K;
Associativity: 8;
Cache line size: 64B |
Local to each core;
Instruction cache: 32K
Data cache: 32K;
Associativity: 8;
Cache line size: 64B |
| L2 Cache |
256 KB per core;
Associativity: 8;
Cache line size: 64B |
1 MB per core;
Associativity: 16;
Cache line size: 64B |
| L3 Cache |
35 MB shared inclusive by the 14 cores;
Associativity: 20;
Cache line size: 64B |
27.5 MB shared non-inclusive by the 20 cores;
Associativity: full;
Cache line size: 64B |
| TLB |
Local to each core |
|
| Default Page Size |
4 KB |
4 KB |
| Memory/Core |
4.6 GB; DDR4 |
4.8 GB; DDR4 |
| Total Memory/node |
128 GB; |
192 GB |
| Memory Speed and Bandwidth |
2400 MHz;
4 channels;
76.8 GB/sec read/write |
2666 MHz;
6 channels;
128 GB/sec read/write |
| Intersocket Interconnect |
QuickPath Interconnect
4.8 GHz,
9.6 GT/s,
or 38.4 GB/sec |
Ultra Path Interconnect
5.2 GHz,
10.4 GT/s,
or 41.6 GB/sec |
| Inter-node Network |
| IB Device on node |
Dual single-port 4x FDR IB Mezzanine card (2 single-port HCAs); 56 Gbits/s |
Dual single-port 4x EDR IB Mezzanine card (2 single-port HCAs); 100 Gbits/s |
| IB Switches between nodes |
4x FDR;
56 Gbits/s |
4x EDR;
100 Gbits/s |
