ITP, Tisch School of the Arts, NYU

 Intro to Interactive 3D and Virtual Spaces.

Instructor: Jean-Marc Gauthier

Tutorials -- updated 02/23/08 - [jean-marc.gauthier@nyu.edu]

Intro to Interactive 3D offers a clear understanding of the steps needed to design 3D interactive content -- beginning with conceptual clay models, storyboards and sketches and evolving to 3D modeling and character animation. Students are invited to explore the unique feeling of being immersed in a virtual environment by creating new types of viewer experiences. Students will create virtual spaces using Maya, Virtools and other media such as video, 3D sculptures, light, photographs, music, text, graphics, 2D animation and interactivity.

 

Creating an immersive experience inside a virtual world – ITP Show 2005

 

The class covers topics including sensors and input devices for real-time animation; shaders and 3D paint; virtual cameras and storytelling (involving the non-linear approach); artificial intelligence and autonomous interactive characters. Students will receive a CDRom with tutorials and handouts on examples for the class. Students with no prior knowledge of 3D and/or programming can use templates (reusable building blocks and behaviors) in order to focus on content rather than coding. Students prepare weekly assignments and a final project.

Additional information about the class: The software covered by the class are Maya and Virtools. You can also use a 3D sofware of your choice. Michael and I spend one hour every week with each student taking the class. The software demos presented in class are presented again during the weekly one-hour-tutorial.The following syllabus with links to tutorials and resources is complemented with a CDRom distributed in class. The CDRom covers tutorials and handouts on examples. Final projects from the class are presented in the ITP winter show inside a special immersive set-up built by the class.

Is there any pre-requisite?  No, and everyone can learn at his or her own pace. This class is for beginner and advanced students.

Can people play online and interact with my projects created in the class?  Yes, the interactive 3D content that you create in Virtools can be viewed online inside a Mac or PC web browser or inside a CAVE, multi-screen installation. You can also export Virtools content to the XBox.


 

Class 1 - Introduction to Virtual Spaces and to Immersive Experiences

Lecture:

Roadmap for ceating interactive 3D content

Reinventing an immersive experience: presentation of interactive projects

Assignment:

Your assignment is to create a videoclip of a walkthrough in the space of your choice. You can choose to shoot inside a house, in the street, for example walking, driving or taking a taxi cab on Broadway, or inside an open space, for example a garden or Central Park. You will shoot as you walk using more than one video camera. You can rig together several cameras back to back or side by side and shoot simultaneously different angles as you walk. You can also use video cameras and mirrors. Edit a short video of the walkthrough using several video clips recorded from several angles composed inside one split screen movie. Bring the videoclip in class in QuickTime movie format. You can work in groups in 2. For references: look at works by Luc Courchesnes on wikipedia and Michael Naimark http://www.naimark.net/projects


Class 2 - Textures: How to apply still images, photographs and live video on 3D objects

Lecture:

Presentation of lights, decals and video textures

Tutorial:

Introduction to sprites -both 2D and 3D- based particle animation and procedural lighting
Download the handout about creating a virtual art gallery

Download the handout about 3D sculpting and 3D painting in Maya (This tutorial is optional)

Assignment:

Create a virtual art gallery with textures in Virtools


Class 3 - Interactive textures: How to paint in 3D, how to control the motion of the hand in a realtime 3D painting

Lecture:

Presentation of artists working with lighting and reflections: Paul Debevec's Fiat Lux

Tutorial:

Creating interactive video textures and combining textures

Download the tutorial about using live video in Virtools

Download movie.cmo , an example of texturing with a video clip. Note that you can play this demo in your web browser and edit this demo in Virtools.

Download the tutorial about interactive textures and setting up a camera in Virtools (This tutorial is optional)

Assignment:

Add interactive textures and Paint Effects objects from Maya to the virtual gallery.


Class 4 - Interactive terrains

Lecture:

Demo of Half Life 2, game by Valve

Dynamic terrains generated with live video and interactive terrains generated from 2D textures

Tutorial:

Download the tutorial about texturing a plane with displacement map in Maya and Virtools..

Assignment:

Create a textured terrain (displacement map) with several cameras. Your terrain will be fully textured with at least two interactive cameras. The cameras will be interactive with proper initial conditions.


Class 5 - Terrains with physics

Lecture:

Introduction to interactive terrains

Tutorial:

Download the Virtools tutorial about setting up physics on a terrain

Download physics.cmo , an example of terrain with physics. Wait 10 seconds for the red balls to appear in the scene. Note that you can load this demo in Virtools.

Assignment:

Create a 3D object moving on a terrain with several attributes and behaviors including physics, driving, cloning and collisions.


Class 6 - Character modeling and animation - part 1

Lecture:

Creating Follow cameras, Tracking cameras, paths, speed controls, proximity sensors and collision detection

Tutorial:

Download the tutorial about creating a human character

Download the tutorial about creating a bird

Assignment:

Model a 3D character using subdivision surfaces.


Class 7 - Character animation - part 2, kinematics

Lecture:

Kinematics and interactive animations

Tutorial:

About animating an interactive character in Maya and virtoools

Character animation in Maya and interactive animation in Virtools

Assignment:

Animate a character in Maya and setup keyboard animation controls in Virtools.


Class 8 - Path Finding: case study a 3D pacman game

Lecture:

Navigation inside a virtual labyrinth: scene architecture and organization of space in virtual worlds

Tutorial:

Interactive virtual cameras

Path guided cameras

Transitional cameras

About setting up a character walking on a terrain in Maya and Virtools

Download the tutorial about setting up a grid for collision detection.

The "Man in a labyrinth" tutorial

Assignment:

Create a small game taking place inside a labyrinth


Class 9 - Designing 3D characters with goals

Lecture:

Maps and images of paths. Creating stories using interactive paths. Animated characters using Go to

and nodal paths.

Tutorial:

Designing and animating a small character with a big mouth

Creating interactive characters using "go to" and Nodal Path

Assignment:

Create several interactive characters and digital puppets using "go to", nodal paths


Class 10 - The Viewer's Experience

Lecture:

New ways to describe the viewer's experience. Examples of projects using Z-buffer and texels.

Tutorial:

How to create a 3D interactive character walking inside a 2D picture
Collision and proximity sensor Download collision.cmo, use the numeric pad to control the character

Assignment:

Email a description of your final project following the list of questions


Class 11 - Using input devices and sensors for VR and games

Lecture:

Design of space, screen and input devices Immersive installations

Strategies for final projects

Storyboards and scrapbooks: (for example) Porco Rosso from Studio Ghibli
Thinking like the player: visualization interface design, navigation and user control

Tutorial:

Hooking up input devices: webcam, Wiimote and Arduino

Assignment:

Fill up the online form about the production of your final project. Create a digital storyboard and a prototype of interface design


Class 12 - 3D sound

Lecture:

3D sound and MIDI

How to set up 2D and 3D sound in your scene

Assignment:

Final project: working prototype of interface design


Class 13 - Final project 60% and class 14 - Final project 90%
Workshop on interactive animation and user control