BrainMiner – View in 3D

More Information

View in 3D

To account for the problems with the 2D approach, we have also developed (in addition to the 2D viewer) a 3D visualization interface that displays correlational data for each ROI along with an MRI volume and a digitized version of the Talairach atlas. Both can be sliced in 3 orthogonal directions and can be overlaid on top of each other. Here is a screenshot of the Graphical User Interface (GUI) of the system, where a basic view with a few ROIs is shown:

Fig. 2: Here we see the Graphical User Interface (GUI) of our newly developed 3D brain visualization software, along with a basic view of a small number of ROIs embedded into a cut-out area of a normalized/standardized MRI brain. For now, all ROIs are spherical in shape. Similar to the 2D viewer, the colors of the ROIs denote the strength of the correlational relationship, on a rainbow scale. The root ROI is colored in yellow. The GUI allows the user to slide the cutting planes up and down and back and forth, to rotate the volume, and to select certain brain surfaces, such as white matter, grey matter, and skull to be semi-transparently superimposed. The correlation thresholds can also be selected, and many more features are available.

The number of ROIs to be displayed, however, can become quite large (about 120-140), which poses challenging problems in the visualization task: In a space too crowded with statistically significant ROIs, it becomes very hard, if not impossible, for the user to tell the 3D positions of the individual ROIs. To overcome these difficulties, a number of techniques were investigated:

  • Superimposing a Talairach atlas slice that can be slid up and down the volume:

Fig 3: A movable sheet that shows the slice of the Talairach atlas at the specified height. ROIs that intersect the sheet are highlighted by a ring.

  • A single light source placed above the volume in a fixed position, providing specular lighting cues for the height and depth of each ROI sphere (this can be seen in Figs. 2 and 3).
  • Enhancing the ROIs by colored halos, where the colors code their height and depth on a rainbow color scheme. The ROIs are connected by iso-lines to the MRI volume cuts which suggests their position in 3D space:

Fig. 4: ROI halos, painted in colors corresponding to ROI height and depth (the rainbow color scheme is used). Dashed iso-height and iso-depth lines emanate from the ROIs and pierce the MRI volume slices at the ROI depth and height.

  • Projecting a colored grid onto the volume cuts, again encoding height and depth on a rainbow map. Colored shadows cast onto the exposed volume slices provide additional cues:

Fig. 5: Iso-lines, with height coded into rainbow colors, are drawn onto two of the three orthogonal MRI volume cuts. The ROI halos are coded in height using the same color scheme. The ROI position with respect to the third MRI cut is suggested by shadows cast by the ROI spheres onto that plane.

  • Projecting the ROIS onto the brain iso-surface, such as white or grey matter, or skin:

Fig. 6: Since the ROIs are mostly located close to the brain surface, i.e. on the brain cortex, one can generate a comprehensive, EEG-like, view by projecting the ROIs onto the cortex surface and paint the projection in the correlation color.

  • Grouping ROI networks into composite polygonal objects, which reduces the object complexity of the scene:

Fig. 7: For now we simply increased the radius of the ROI spheres until they just touched. This approach is rather effective. For the future we plan to estimate the actual hull of a set of ROIs of similar brain function and display this hull as a polymesh.