ITP, Tisch School of the Arts, NYU

3D and the Marriage of the Virtual and the Real

Instructor: Jean-Marc Gauthier

 

Tutorials - updated 01/15/08 - [jean-marc.gauthier@nyu.edu] - URL: www.tinkering.net/vreal

3D and the Marriage of the Virtual and the Real is an introduction to 3D and digital cinematography using Maya, MEL scripting, After Effects, Z-brush and Combustion. This course offers access to the motion-capture lab for live recording of animations and for importing motion into character animation using Maya and MEL scripting. Students create an hybrid of video and 3D animation in order to re-interpret, digitally, motion, colors, textures, camera movements, gestures, sounds and human expressions. Students
create their own stories combining video footage from reality and fantasy virtual worlds. Topics addressed include building sets with digital storyboards, creating animated digital characters from hand-drawn sketches, Machinima movies, organic growth, camera tracking, character animation, lighting and compositing. In addition to the simulation of natural and urban environments, real-time physics, fluid dynamics, sound modeling and procedural textures are covered. This course introduces students to a wide range of techniques and skills used for storytelling, pre-visualization and gaming. Students prepare weekly assignments and a 3D animation for their final project. No pre-requisite is needed for this class. Students can use alternative 3D software tools.

Answers to frequent questions about the class: This class is for beginners and advanced students. The sofware covered are Maya, MEL scripting, ZBrush, After Effects and Combustion. You can use also a 3D sofware of your choice. The software demos presented in class are presented again during the weekly tutorials.


Class 1 - Introduction. From video footage to computer graphics

Lecture:

Roadmap to and from digital cinematography
Scrapbook of pictures about the "digital skin" of the City

Tutorial:

Building a diorama Instructions and templates

Assignment:

Create a diorama of a street corner or of a street intersection on 42 nd St. between 7th av. and Lexington (Manhattan) and
1- Take pictures or draw a NYC street intersection, for example around Times Square: people, cars, street lights, street furniture, signs, street vendors, buildings, sidewalks, textures, smoke, clouds, sky.

2- Print pictures - black and white is fine - and mount them on cards, for example foam board. Print the 2D templates of street intersection available on the 2D template and use the print as a base. Build a diorama the size of a shoe box using paint, collages, paper, cardboard and glue. .
3- Bring the model to class. You can also shoot a one minute videotape inside the diorama with sound or music. No editing required.


Class 2 - Texturing and lighting a 3D scene

Lecture:

Between the real, the virtual, invisible City

Tutorial:

How to create textures in Maya. Creating textures on 3D cubes in Maya, 3D primitives (cubes), extrude tool, split polygons tool, texture mapping, basic lighting and rendering

Assignment:

Create a street corner for your scene in Maya. Model, texture and render one or several architectural details of the street corner.

Digital photograph and ...

... its 3D translation


How to create a photorealistic 3D model of a building


Class 3 - Camera and lighting design. Compositing images with transparencies inside a 3D environment

Lecture:

How to work with video footage inside a 3D scene. This lecture shows you how design cameras and lighting in a scene. We look at various styles of anime. Case Studies: "Ghost in the Shell" "Age of Innoncence", by Mamoru Oshii.

Tutorial:

How to light and animate a scene. Rendering a scene in Maya.

Assignment:

How to create a scene with people using transparencies and to animate lights and cameras. Create night and day lighting environments with people.

Compositing

and rendering

 


Class 4 - Video textures

Lecture:

Introduction to video textures and alpha channel compositing. Creating alpha channels, traveling mattes in After Effects and Maya. Guest speaker Martin Lenclos presenting his photographic and 3D work for NewYork Exit NewYork

Tutorial:

Creating a video texture and a traveling matte in Maya

Assignment:

Create several billboards with video textures inside your street scene. Step 1- Videotape people in front of the green screen, window scenes, and sequences with people in the street. Suggested references for this assignment are Hitchcock Rear Window (the "meet the neighboors" sequence) and Orson Wells Touch of Evil (the intro scene) Step 2- Add billboards with video textures to the buildings and the sidewalks of your Maya scene.


Class 5 - Creating a 3D character

Lecture:

Analysis and examples of character creation

Tutorial:

Step-by-step modelling a of character

Assignment:

Use push-pull and 3D paint

Create a character using virtual clay: subdivision surfaces


Class 6 - Character animation

Lecture:

Survey of origins of character animation from Ray Harryhausen to inverse kinematics

Tutorial:

Character animation setup

Assignment:

Create a short clip presenting your character moving in space in relationship with the video clip of someone in motion. For example your character could walk next to a real person. Record a short video clip - 10 seconds – or take still images-  of a real person in basic motion, side views, for example walking, sitting... These templates can be used as image planes in Maya in order to help you animating in very creative ways.


Class 7 - Motion Capture for storytelling

Lecture:

Motion Capture

Tutorial:

Character animation and motion capture in Maya

Assignment:

Apply motion capture data to your character


Class 8 - Compositing virtual actors inside video footage

Lecture:

Designing a scene with a character

Tutorial:

Compositing character animations and video footage

Assignment:

Animate a 3D character inside video footage, with lighting + sound using Maya and After Effects. Write a small text presenting a story for your final project. Outline spatial elements of the story, environments, the role of lighting, camera frames and movements, music and dialogues. Create a portfolio of previous assignments including street scene, canyon, character animation and compositing over video footage.


Class 9 - Particles, soft bodies and PaintEffects

Lecture:

From hypersurface to liquid -- virtual design using dynamic deformations

Tutorial:

Particles and fields

Rigid and soft bodies

Assignment:

Create a scene with environmental effects using dynamics, collisions, fields, smoke, particles


Class 10 - Sketching a 3D scene

Lecture:

From conceptual design to storytelling. Sketching a panoramic storyboard using modulations of light and color gradients in class.

Clouds

above the ocean

Assignment:

Final project - storyboard conceptual design - Following the exercise covered in class, create a storyboard showing modulations of light and color gradients in your animation

 

Step 1- create freely a gradient strip with mixed media. For example a panoramic image (3500 by 500 pixels) from drawing with pastels, charcoal... or stitching together a collage of digital photographs. Photographs can include windows, doors, prints, wall textures, paint, fabrics, floors, stones, flowers, grass, sand, sky... You can bring the physical collage in class or you can present it digitally. Print as a pdf, select print preview, scale to fit media. View your work by zooming and scrolling the image through the small rectangular window of the pdf.

 

Step 2- Be a photographer inside your panoramic drawing. Create a story by taking picture, framing details of your drawing or collage. Draw the camera frames for each shot. Start the process several times. Present several possibilities of sequences with at least six shots no more than ten per sequence.

 

Render clips with several versions of the soft bodies particle animation that you created last week. Please add lighting, cameras, change materials. Design a scene using particle and rigid/soft body collisions in the context of a chain of actions. See examples of grass pressed by a ball and tree animated with wind on the 3d server > Vreal> class 10


Class 11 - Fluids

Lecture:

Building natural environments

Tutorial:

Clouds and ocean Sculpting a 3D landscape

Assignment:

Create the intro scene. Fill up the online form about the workflow (Form1) and production (Form 2) of your final project


Class 12 - Environmental effects and MEL scripting

Lecture:

Blending together the elements of a scene

Tutorial:

Creating a cloud of particle from a 2D image or a video

Assignment:

Final project: create scenes coming before, after and during the pivotal moment of your animation.


Class 13 and 14 - Presentations of final projects
1 - webpages with visuals of physical models and/or hand drawn sketches presenting your process explaining the process

2 - video clips of final project