To use multiple nodes, you can use TensorFlow's "MultiWorkerMirroredStrategy" variable, which is described in their article Distributed Training with TensorFlow.

To call the strategy in your code, use:

if dist_nodes :
     strategy=tf.distribute.experimental.MultiWorkerMirroredStrategy ( 
             tf.distribute.experimental.CollectiveCommunication.AUTO )
   else :
     strategy = tf.distribute.MirroredStrategy()

The collective operations include the following options:

CollectiveCommunication.RING

Implements ring-based collectives using gRPC as the communication layer.

CollectiveCommunication.NCCL

Uses NVIDIA's NCCL to implement collectives.

CollectiveCommunication.AUTO

Defers the choice to the runtime.

You also need to set up the TF_CONFIG environment variable, which is required by TensorFlow:

if dist_nodes :
    index = int(sys.argv[1])  # $@ has n indices indicating n nodes
    verbose = index < 1       # verbose is True only when index = 0
    nodes=[]
    # node names for each node, append a port # to the names
    for i in range(2,len(sys.argv)):
      nodes.append(sys.argv[i] + ':2001')
    # set TF_CONFIG variable that MultiWorkerMirroredStrategy needs
    os.environ['TF_CONFIG'] = json.dumps({
        'cluster': {
            'worker': nodes
        },
        'task': {'type': 'worker', 'index': index}
    })
    print(os.environ['TF_CONFIG'])
  ##

For more details about TensorFlow programming and an example of how to use the above methodology, please see Multiple CPU Nodes and Training in TensorFlow.