The What-If Tool (WIT) is a helpful visualization tool that aids in analyzing models and datasets. The tool automatically visualizes datasets and allows you to compare different models and see how specific inputs change the output. WIT can visualize both classification and regression models, and gives various visualized statistics like receiver operating characteristic (ROC) curves, confusion matrices, and error calculations, depending on the type of model.

WIT is included in the latest versions of TensorFlow environments installed on NAS systems. To use WIT, you must first set up Jupyter Notebook for your NAS environment.

Note: WIT only works on Broadwell or newer nodes.

Starting a Jupyter Notebook

You can access WIT through an interactive Jupyter notebook. To use WIT, first start a Jupyter notebook by using the steps described in Using Jupyter Notebook for Machine Learning Development on NAS Systems—with one change in Step 4:

• In Step 4, instead of loading a JupyterLab server, load a Jupyter notebook server, as WIT does not work in JupyterLab. The command line becomes:

  (env_name)r313i0n0 % jupyter notebook -–no-browser

The other steps remain the same.

Configuring the Notebook to Use WIT

Once you load a Jupyter notebook in your local browser, you can configure it to use WIT. Complete these steps:

1. In a Python cell of a notebook, load the proper libraries:

   from witwidget.notebook.visualization import WitConfigBuilder
   from witwidget.notebook.visualization import WitWidget

2. Enable your dataset to work properly in the visualizer by converting your data to one of the following data structures:
   • tf.train.Example
   • tf.train.SequenceExample protos
   For more information on these data structures, see this documentation on the TensorFlow website.
3. Create the tool by building a WitConfigBuilder, which specifies the data and model to be used for the visualization. The simplest way to use any model is with a custom prediction function:

   config_builder = WitConfigBuilder(data).set_custom_predict_fn(model)

   If you want to compare two models, you can add:

   config_builder= 
   WitConfigBuilder(data).set_custom_predict_fn(model).set_custom_compare_predict_fn(model2)
   

   where data is a list of tf.train.Example or tf.train.SequenceExample protos, and model is any function used for prediction. The tool directly calls this function for inference with the data provided.
4. Start an interactive window by creating a WitWidget using the following configuration:

   a = WitWidget(config_builder, height=tool_height_in_px)

   where config_builder is the WitConfigBuilder object created above, and height is the height (in pixels) of the interactive window opened in the notebook. We recommend setting the height at 1000, depending on the size of your browser window.

Completing these steps will display the interactive window in the notebook when you run the cell. The window will have three tabs at the top:

• Tab 1: visualizes all the data.
• Tab 2: visualizes the model results.
• Tab 3: visualizes data statistics by features.

Additional Resources

• For more information and in-depth examples, see the WIT website .
• To learn more about what you can do with WIT, review the notebooks provided by the developers.