SciViz: From 3D games to scientific visualization ITP, Tisch School of the Arts, NYU Jean-Marc Gauthier

Jump in as we look at ways to design interactive virtual spaces to help us further understand scientific visualization. This class challenges students to walk a thin line between being faithful to the physical world while expressing themselves using interactive 3D content. Scientific visualization using virtual reality applications took a new turn as interactive 3D brought that reality to an entirely different level. This class explores new possibilities offered by Maya and Virtools for creating and programming various examples of scientific visualization. The favoring of rules, behaviors, physics and artificial intelligence in the structures of video games is examined as a source of inspiration for visualizing scientific content. In addition, the dynamics and forms of real-life evolutionary systems are observed. The class presentations look into projects created for both scientific and creative fields, with the main focus addressing concepts covering medical research and archeology to A-life systems, natural phenomena and bio-mechanisms. Other topics covered in class include: the design of virtual objects with interactive controllable and transformable parameters, virtual simulations of physical spaces using light, sound and physics, and emerging topics in virtual reality including artificial intelligence and autonomous agents. Students prepare weekly assignments and a final project that may take place inside a Cave system where individuals can enter, learn, play and work inside a virtual environment.
No pre-requisite is needed for this class.

Illustrations of topics covered in the class


Creating an immersive environment	Creating a virtual sense of touch

Is this class for beginners or for advanced students? The class is for beginners and for advanced students. The software covered are Maya and Virtools. You can also use a 3D software of your choice. Mike and I spend one hour every week with each student of the class. The software demos presented in class are presented again during weekly one-hour tutorials. The following syllabus with links to tutorials and resources is complemented by a folder with examples available on the 3D server.

Can I take this class after taking "Intro to Interactive 3D"? Yes, this class covers different topics than the Intro 3D class, for example building a CAVE, scientific visualization and telepresence are topics covered only in this class. This class has a more holistic approach of interactivity and behaviors that push you to think about more advanced applications of virtual spaces and to emphasize conceptual design for real world applications.

Can people view online my projects created in class? Yes, the interactive 3D content that you create in Virtools can be viewed online inside a Mac or PC web browser or inside a CAVE, multi-screens installation. You can also export your content to the XBox.

Class 1 : Roadmap

Lecture Convergence of 3D games and Scientific visualization:

Tutorial Presentation of Maya ( MEL programming language ) and Virtools ( Building Blocks, VSL programming language ).

Assignment: Step 1- Reading
I suggest reading the handout from Tufte's "The Visual Display of Quantitative Information", chapter 1. Page 24's case study describing Dr. John Snow's mapping of central London during the 1854 cholera epidemic illustrates a process similar to the one that you will follow for your assignment - in 2006, in Manhattan. The second part of the handout is about Google Earth. I suggest that you try using the software. Look at the interaction between several media displayed on the map: navigation tools, digital images, texts, dynamic interactive graphics.

Step 2- Collecting media
The physical part of the assignment is to collect media: pictures, sound bites, short video clips, during a day of navigation through the city. Since your content will play in real time and online, I suggest that you keep the files at a small size: images in jpg format maximum 500 KB, video clips in avi format 1MB and sound bites in wav or mp3 format 1 MB.

Step 3- Presentation of media on a 2D map
The locations where you collected the media will be carefully referenced on a 2D map. You can pinpoint the media according to street intersection or to mid-block locations. For example Tisch can be described as the intersection of Broadway and Waverly - west side - south corner.
You will present, in class, your 2D map with annotations.

Class 2 : Navigation and loading data

Lecture Maya and Virtools interface and basic tools.

Tutorial Loading, editing and scripting media in Maya and Virtools.

Download tutorial

Assignment Create a 3D map of a section of Manhattan by extrusion of a 2D template.

Class 3 : 3D Toolkit in Maya

Lecture Maya interface and basic tools.

Tutorial Create animated cameras on paths.

<3D interactive map of Mahattan

Download tutorial

Assignment: Create an interactive simulation for the 3D map of Manhattan using several path cameras.

Class 4 : Particle animation

Lecture Creating a nebula, cluster of galaxies and a single galaxy using particle animation in Maya and Virtools. Tutorial Animation of streams of particles.

< interactive galaxy

Creating an interactive galaxy Maya and Virtools

Assignment: You are requested to do at least one Maya and one Virtools tutorial from the pdf. I suggest browsing through this document before choosing a theme that you like.

The purpose of this assignment is to compare how to simulate flows of particles in Maya and Virtools using similar tools and concepts We will discuss how the information is displayed in the case of interactive content created in Virtools and non-interactive content created in Maya. During our next meeting, we will review the pros and cons of using linear versus interactive content. I am looking forward to hearing your feedback about the viewer's experience and the possibility for one type of visualization to replace the other.

Class 5 : Basic nature creation and plants creation tools in Maya

Lecture: Expressions with animation and motion capture.

Animated tree structure tutorial. This tutorial shows how to model and animate, in real time, a tree structure similar to animated structures found in nature.

Tutorial Animating plants and other kinematic systems found in nature. Case study: animated trees. Maya and Virtools.

Assignment: Create an interactive animation of a tree structure.

Class 6 : Basic nature creation and plants creation tools in Virtools

Lecture: From books to dynamic visualization creating an interactive atlas. Chris Bregler gives a presentation about capturing motion.

< dynamic simulation

Cloning, gravity, driving and physics

Assignment: Create an abstract landscape with a terrain, cloned objects or characters, for example trees and dynamic spheres and cubes.

Class 7 : Fishes

Case study: bio-mechanics and fish locomotion

Tutorial Making an animated interactive shark

<interactive shark

Assignment: Create a scene with a moving character, bird, fish or insect on a terrain or in water.

Class 8 : Intro to fluids

Guest speaker: Charlie Paskin talks about the virtual heart project

Assignment Finalize your mid-term assignment.

Class 9 : Waves and ocean

Mide terms assignment presentations

Tutorial oceans and waves in Maya and Virtools. Creating dynamic waves using interactive deformers

<interactive water

Ocean tutorial

Assignment: Create a scene with dynamic water in Maya and in Virtools.

Class 10 : Flocks and crowds

Lecture Instancing, cloning, goal oriented, artificial intelligence. Case study: virtual archeology

< Cloning with friction, gravity and elasticity

Tutorial Model and script a crowd in motion inside the cave.

Tutorial How to create clones in Maya and Virtools.

Assignment: Create a cloning simulation with fixed and moving objects.

Class 11 : Medical visualization

Lecture Challenges of medical visualization: Patrick Kelly talks about the history of 3D interactive navigation in neurosurgery. How to explore a virtual human body. Case study: the Dynamic Virtual Patient

< The Dynamic Virtual Patient

Tutorial How to create a shader, shaders tutorial, how to render the human body using shaders.

How to create a scene in Maya and Virtools using the Z-buffer. Download the Z-buffer tutorial

Assignment: Create a 3D interactive visualization using the Z-buffer.

Class 12: Testing the Cave

Presentation of Final Projects