Data Visualization Across the Extended Reality Francesca Samsel’s focus is on expanding the visual vocabulary for visualization, both physical and digital. At its core, this project focuses on using materials to encode scientific data intuitively. This proposal seeks to lay the design, technological, and experimental foundations for emerging research on data visualization in extended reality that can address such questions. The research plan is organized as three Aims, with the first two focusing on exploring the design space while creating and evaluating lower-level supporting technologies, and the third putting the new tools into practice in a modern team-science context. Each Aim includes two parts, with the first designed to advance or apply the technologies and the second designed to evaluate human performance and potential impact. In summary: Image Description ~ Explore how material could be manipulated along a line, such as a streamline in a fluid flow visualization. Note how variation within a single material might be controlled in ways that can also be associated intuitively with climate data and processes. Project Aims: Aim 1 (Basic Research). Explore the new language of physical data encodings made possible by emerging technologies and characterize perceptual performance relative to virtual counterparts. Key contributions: Part A: A multi-year exploration with professional artists and designers of physical material in modern data visualization along with algorithms for multi-material digital fabrication of data encodings. Part B: A new evaluative methodology and results from comparing low-level human perception of multi-field data encoded physically, virtually, and via a mix of physical and virtual encodings. Aim 2 (Basic Research). Lay the technical foundation for interacting with physical data encodings and characterize performance on data exploration tasks relative to virtual counterparts. Key contributions: Part A: Advances to refine and adapt emerging capacitive and optical technologies for integrating touch input sensing within physical 3D prints. Part B: An evaluation that characterizes the speed and accuracy of common spatial data interaction techniques with data encoded physically, virtually, and via a mix of physical and virtual encodings Aim 3 (Applied Research). With collaborators, apply the new data encoding and interaction techniques to actively studied, multivariate climate data and understand potential impact. Key contributions: Part A: An application of the novel technologies developed to understand multivariate data from supercomputer simulations of forest fires and other scenarios studied by collaborating science teams. Part B: Beyond time and accuracy metrics, a deeper understanding of the potential of interactive data exploration across the Data Visualization in Extended Reality continuum, with an emphasis on understanding interactive, exploratory data analyses conducted in small teams. Each of these examples was handcrafted or arranged by an artist. The wide range of points, lines, and forms demonstrates real, data-driven visualizations could be created with a visual haptic language like this one. One hypothesized benefit is that the contrast between different materials can be used to intuitively encode categorical data. From left to right in the figure: Association: ground, Materials: ceramics, rocks, linoleum, and coffee grounds; Association: ice, Materials: glass and ice; Association: clouds, Materials: pillow stuffing, rice noodles and string; Association: flora, Materials: fimo clay, anise seeds, glass, painter’s tape and a mix of paint, rice and sunflower seeds.
