the lord of the rings: the visual effects that … · 5 eric saindon cg supervisor eric saindon,...

33
THE LORD OF THE RINGS: THE VISUAL EFFECTS THAT BROUGHT MIDDLE EARTH TO THE SCREEN Course 10 SIGGRAPH 2004 Presenters Matt Aitken Greg Butler Dan Lemmon Eric Saindon Dana Peters Guy Williams Weta Digital, Ltd 9 – 11 Manuka Street Miramar,Wellington 6003 New Zealand All Images courtesy of New Line Productions, Inc ~ All rights reserved

Upload: hatuong

Post on 05-Jul-2018

216 views

Category:

Documents


0 download

TRANSCRIPT

THE LORD OF THE RINGS: THE VISUAL EFFECTS THATBROUGHT MIDDLE EARTH TO THE SCREEN

Course 10

SIGGRAPH 2004

PresentersMatt AitkenGreg Butler

Dan LemmonEric SaindonDana Peters

Guy Williams

Weta Digital, Ltd9 – 11 Manuka Street

Miramar,Wellington 6003New Zealand

All Images courtesy of New Line Productions, Inc ~ All rights reserved

2

Introduction

Weta Digital's work on the "Lord of the Rings" trilogy involved it in all aspects offeature film visual effects, from creature and digital double animation to massivebattle scenes, from the creation of entirely digital environments to the ground-breaking digital performance of Gollum.

This course will present an insight into how each of these aspects of theproduction was achieved and offer detailed breakdowns of techniques developedand procedures used in this milestone production.

3

PRESENTER BIOGRAPHIES

Matt AitkenDigital FX Supervisor

Matt Aitken was born in 1961 and grew up in Wellington, New Zealand. He has aB.Sc. in Mathematics from Victoria University of Wellington and an M.Sc. inComputer Graphics from Middlesex University, London.

Shortly after completing the M.Sc. in 1994 Matt was offered work at PeterJackson's then fledgling digital effects facility, Weta Digital. In those early daysworking at Weta involved all aspects of digital film effects production from filmscanning and recording to animation, compositing and software development.Matt has worked at Weta ever since: supervising computer graphics on many filmprojects including "The Frighteners" "Contact" and the "Lord of the Rings" trilogy.

Matt has had technical papers published by Eurographics and Graphite, and haspresented on Weta's work at many conferences and festivals includingSIGGRAPH in the US, the Australian Effects and Animation Festival, the LondonEffects and Animation Festival and the Imagina Computer Graphics Festival inMonte Carlo. In 2003 he received an inaugural Visual Effects Society award forhis work on "Lord of the Rings".

Matt is currently working as Digital FX Supervisor on Peter Jackson's remake of"King Kong".

4

Greg ButlerCG Supervisor

Greg Butler came to Weta Digital in January of 1999, hired to help create apipeline and infrastructure capable of handling the immense amount of worknecessary to complete the Lord of the Rings trilogy. After a few months, he wasasked to head the creatures department: to hire and guide the team that wouldcreate the muscle model, digital doubles and the system for creature rigging andskinning.Greg went on to become the Sequence Supervisor for the first LOTR trailer, andthen for the “watcher in the water” sequence in the Fellowship of the Ring. Hecontinued to shape the overall Weta Digital pipeline, while lighting and renderingshots, and working closely with many other departments. During the Two TowersGreg continued his focus on creatures, CG Supervising Gollum sequences. ByReturn of the King, Greg was responsible for every shot of Gollum in the film,continuing to improve the look, from re-implementing his hair solution to creatingnew and detailed looks.Greg’s skills as a CG artist are broad, as he came to the computer effectsindustry after getting a Bachelor of Arts in film/video production and theatrelighting design from Hampshire College. Currently he is one of four CGSupervisors at Weta Digital, and has just finished shots on the Return of the KingDVD while also preparing for King Kong.

Dan LemmonCG Supervisor

Dan Lemmon worked for 7 years at Digital Domain as a Technical Director andCG Supervisor before coming to Weta in early 2002. In addition to the Lord of theRings trilogy, he worked on such films as: The Fifth Element, Titanic, Fight Club,How the Grinch Stole Christmas, and A Beautiful Mind, as well as a number ofcommercials. He has worked as a shader writer, FX animator, lighter, andcompositor.

Dan presented in previous Siggraph Courses on Image Based Lighting andRendering with Paul Debevec. He has of BFA in Industrial Design from BrighamYoung University.

5

Eric SaindonCG Supervisor

Eric Saindon, currently a CG Supervisor on "I Robot", was previously CGSupervisor for the "Battle of Pelennor Fields", a major portion of "The Lord of theRings: The Return of the King." Saindon has been at Weta Digital since 1999. During that time he has worked extensively on the creation of Gollum for the Lordof the Rings: The Two Towers, and was the Creatures/Character SetupSupervisor for "The Lord of the Rings: The Fellowship of the Ring."

Saindon came to Weta Digital from Santa Barbara Studios where he was amodeling supervisor and lead technical director on such movies as "Star Trek:Insurrection" and "The Little Vampire".

Dana PetersCreatures Supervisor

Dana Peters is head of the creatures department at Weta Digital, where hebegan working as a creature TD on The Fellowship of the Ring in 2000. By theearly stages of The Two Towers he had taken over the department that wasresponsible for the “skin and bones” of the creatures of Middle Earth, fromHobbits to Mumakil. He’s been in the computer graphics industry for more than10 years, working on projects that include high-end television commercials and3D amusement park rides as well as documentaries and feature films. In hiscurrent position he is responsible for recruiting, training and supervising the teamof artists who rig puppets for animation and provide muscle, skin and clothsimulations. He was integral to designing the creature pipeline and continues tomaintain and expand it.

His past experience includes not just character rigging and setup but alsoanimation, lighting, and motion capture manipulation. He graduated fromMississippi State University with a degree in computer graphics and animation.Dana and the artists in the creature department are now working on I Robot andthe pre-production for King Kong.

6

Guy WilliamsCG Supervisor

Guy Williams is currently a CG Supervisor at Weta Digital. He has more than tenyears of experience in 3D film graphics, with a specialty in photo-real lighting andshading, from look development through to artist setup and supervision. Hecame to Wellington in 1999, and has worked on all three of the Lord of the Ringsfilms. As the head of the shading and texturing department during pre-production, he set up pipelines and hired a staff of shader writers and textureartists. When production began he moved into shot development andsupervision, first on the underground city of Dwarrowdelf. He was responsible forthe look development of Treebeard and the Ents in The Two Towers. During TheReturn of the King he was CG Supervisor for Shelob, developing the pipeline,overseeing the look development and supervising a team of artists. At presenthe is working on the film I Robot.

Guy attended Mississippi State University, intending to study aerospaceengineering, but became interested in CG effects and moved into the computerarts program. Since leaving university to work at Boss Films, he has had broadexperience in both 2D and 3D work, on projects including feature films, television,commercials and ride films

7

COURSE OUTLINE

Gollum

Design ProcessMotion CaptureNew Facial TopologyModeling TechnologyShader DevelopmentReturn Of The King UpdatesLeveraging Technical Knowledge for the Finished Product

Other Creatures

BalrogShelobGwaihir

BREAK

Massive

Pelennor Fields

Digital DoublesMumakilGreat BeastTrolls

Conclusion

8

GOLLUM

Weta Digital started with R & D on a generic human muscle system in 1998.Eventually they built ‘George’, a digital copy of a local personal trainer. ‘Real’muscles were built based on anatomical reference, matching almost everymuscle except those of the face. The system was designed to be built on top of‘bone’ primitives, which were parented under ASF joints.

The plan was to build a master muscle system and then scale to fit any bi-pedalform. Dutifully following real anatomy as well as expecting the system to easilyscale proved to be better conceptually than in practice. The Cave Troll in TheFellowship Of The Ring was the first real proving ground for the system.

The Fellowship Of The Ring was a learning ground. Discovering that lowerresolution skin geometry gave better and faster results, fewer larger muscles withmany inter-connections gave more stable results and that muscle dynamics couldbe used sparingly and should have tight springs were just a few of the things thatwould need addressing as soon as time permitted.

Like many other attempts at simulation, all it took was one missing element totake the system away from reality. Not having inter-muscle collision detection, forinstance, meant that building musculature based on reality was not the totalsolution.

By the time The Return Of The King was turned over, the muscle system,especially as represented in Gollum, was very much a hybrid system. Itcontained Maya skinning, joint driven blendshapes and many non-anatomicallycorrect muscle shapes. The results, however, had become very predictable andrealistic.

9

Design Process

Working from initial sketches from the design team, Weta Workshop sculptorsbegan creating physical maquettes of Gollum for scanning. The digital modelsteam used these maquettes as a basis for a hero NURBS model which was usedas the basis for all Gollum work on The Fellowship Of The Ring. This wassufficient for the requirements of Film 1, but with an increasingly significant roleover the course of the trilogy Gollum needed to be completely re-created. For thesecond installment he was re-built as a subdivision puppet, allowing far moreflexibility throughout the pipeline.

Motion Capture

Actor Andy Serkis performed the role of Gollum and provided artists with detailedreference. Motion capture data provided the movement to re-create thecharacter, and with human facial reference to work with the animation team, ledby Animation Supervisor Randy Cook, could re-create expressions down to thefinest wrinkles. The new Gollum is possibly the most malleable animatedcharacter the industry has seen thus far.

10

New Facial Topology

In order to amplify emotional range wrinkles and pores were painted into theshader for The Two Towers. This allowed his mischievous or angry outbursts toappear or disappear seamlessly, and with far less effort than in the first film. Thespilt between Gollum and Smeagol needed to be subtle enough to blink andmiss, but also be available to Jackson as a directable option. Animator controlledsliders triggered the wrinkle values to appear in his forehead, on the sides of hisnose and around the lips and eyes.

11

Modeling Technology

The success of Gollum in The Two Towers inspired a complete retooling of thecreature pipeline. Subdivision surface geometry became the technology for allmodeling work, allowing much more control over model building for animationand texturing. NURBS models had been perfectly adequate for the first film, butfor The Return Of The King all models were completely rebuilt as subdivisionsurface models, including the digi-doubles.

Shader Development

Skin and Bones

Starting with a skeletal framework Weta Digital took an inside-out approach tocreature rigging. Muscles were attached, built up into a complete form, and thenlayered with skin. For The Return Of The King the muscle system became moredefined, which in turn created better definition in the joint areas such as kneesand elbows. Gollum’s ever increasing emaciation was created by shader artistspainting bruises and cuts onto his hands and feet, followed by a bloody nose anda bump on the head in later scenes.Sub-surface scattering completed the semi-translucent appearance of Gollum’scomplexion, a technique that earned creators Joe Letteri and Ken McGaugh atechnical achievement Oscar from the Academy of Motion Pictures Arts andSciences in 2003.

Hair Dynamics with Maya Cloth

Weta Digital examined several ways that hair dynamics could be simulated. Some of the options included writing a dynamic hair system, writing a jointdynamics system, or repurposing existing tools. Looking at some existing toolsthat were already in Maya, they realized that hair has very similar properties tosmall strips of cloth. Cloth had already been implemented in Maya, and already

12

had a set of tools to manipulate it: a solution already existed.

Using Maya's cloth they built a set of strips. The top of the cloth was constrainedto Gollum’s head, and the strips were given properties to mimic the movement ofhair. Hair curves were then wrapped to the cloth, and rendered.

If only it were that simple. Gollum's hair dynamics were constantly evolvingthroughout the production of The Two Towers to adapt to new shot requirementsand to work around limitations in the cloth solver. Anyone who has worked withcloth knows how difficult it can be to get the exact results you want. It is onething to be able to simulate cloth, but another thing altogether to mold it into theparticular shapes that director Peter Jackson wanted for particular shots. Thesetup was constantly changing to give TDs different levels of control over thesimulation, effectively allowing them to sculpt the hair into the poses they wantedduring or after the cloth solve.

13

Return of the King Updates

Hair dynamics were a constant problem during The Two Towers and it wasdifficult producing the results Weta Digital wanted. They began looking at othercommercially available hair and cloth systems, but there was nothing available atthe time that met the requirements. However, they did find that Syflex couldpotentially solve some of the problems they had with Maya's cloth simulations.

One of Maya Cloth's limitations is that it tangles easily with fast moving objects,and won't untangle. Syflex handles these situations better, and will not tangle aseasily. Using Syflex allowed breaking the solution down into smaller and smallerstrips. The final solution was to have one strip of cloth driving each bundle of hair,giving more independent movement.

One of the requirements of the new hair solution was to be able to handle wind. The simulations on The Two Towers did not really need wind, but for The ReturnOf The King Gollum was standing on cliffs next to live actors whose hair wasblowing wildly. The Syflex wind solution was implemented; allowing the use ofmore cloth strips to give more independent movement.

14

Leveraging Technical Knowledge For The Finished Product

Gollum was unique to Weta Digital and The Lord Of The Rings films, but also tothe industry. To achieve the most realistic performance from a digital charactertook many hands, and more than a handful of departments to create. Down to thedirt under his fingernails, Gollum had to be believable. The audience had to hatehim, suspect him, love him and feel sorry for him without his ‘digital’ personinterrupting the flow of the story.

The trial and error from the early days was grueling. Moving through the forwardprogression and being a step ahead or behind as long as it worked was theeasiest way to for Weta Digital to work. Gollum didn’t need to be anything morethan he was for The Fellowship of the Ring, but implementing the changesbetween films to make sure he could stand up to his next performance sawsignificant changes from liquid to solid, NURBS to subdivision surface geometry,old to new. All of these things increased his flexibility as an ‘actor’, which isexactly what Peter Jackson needed.

Weta’s current development work on muscles will both incorporate more realisticsimulation capability as well as more tools for ‘cheating’ as required. Shotsculpting was a common procedure on creature shots, though less so on TheReturn Of The King. The new muscle and skinning system makes that processeven easier for future use.

15

OTHER CREATURES

One of the things that made the project unique was the number of digitalcreatures that were required to operate at the “hero” level.

Balrog

The digital model for Balrog started life as a series of scannable maquettes,including a head twice the size of the rest of the creature for more detail.Displacement maps were extracted from the scan detail and a NURBS modelwas then rigged using the muscle system. In many ways Balrog followed thetypical Weta creature pipe, but with one exception; fire.

Fire was the distinguishing element. Not so much flames on its back, but a giantcreature swathed in forest fire level flames. The first solution consisted of fullComputational Fluid Dynamics fire. It looked great, but if the Balrog moved toofast it blew out. Also it was incredibly expensive to render. With these limitationsturnaround on Balrog shots was too slow. The result of this was an effect thatwasn’t directable: we needed a fire solution with rapid turnaround for this. Aboveall, everything needed to be directable.

With only a couple of months to go on The Fellowship of the Ring the artistsstarted looking at an alternative method using sprites; an intelligent hack ratherthan a full fire simulation. Using this approach there was now fire and smoke.Sprite sequences selected for ignition flare and decay were now in place, withartists able to trigger these sequences at appropriate times to give life to fire andhigh level of realism.

16

Maya particles were animated with standard particle fields: turbulence, gravity,uniform and particularly the volume axis field. There were between 6000 - 30,000sprite-rendered particles in any given frame.

These particles were rendered in the Maya hardware render buffer for film 1, forfilms 2 and 3 Weta Digital’s code department wrote a software sprite renderer.Other particles were used to deform the plates using optical flow software WetaDigital developed for the Wraithworld sequences.

The particle technique developed for the Balrog fire proved to be an incrediblyuseful tool and the solution was extended out to water and atmospheric effectsfor other shots.

17

Shelob

Shelob was one of the most significant creatures in ‘The Return of the King’. Sheappeared in the second of Tolkien’s books, but for dramatic effect was saved forthe third of Jackson’s film interpretations.

Her original design was based on the tunnel web spider. From there, a complexdesign process involving Jackson, the Weta Workshop design team and WetaDigital saw the character come to life. Concept drawings from Jeremy Bennett &Alan Lee were used as a basis for the sculpting of physical maquettes at WetaWorkshop. Hi-res in-house scanning at Weta Digital of the head, body and legs,followed by testing and re-working the original design eventually led to the finalrendering of the character. Shelob was a great example of how the two facilitiesworked together.

Shading saw variations from a blue (original) to a grey and orange to a red to ablue (final) spider. Some of the effects that were looked into early on made it in,like the beer bottle glow on the legs.

The hardest aspect of the design to lock was the head. In fact we finaled all theshots in which the head was out of frame before there was a head on thecreature.

18

Gwaihir

The Fellowship Of The Ring

The Fellowship Of The Ring didn’t have Gwaihir scripted, so the creature was onthe back burner with plenty of time to get around to creating the models in timefor The Return Of The King. Jackson decided three months before delivery thatthe giant eagle would have to make an appearance. Fast turn-around wasessential.

The shots indicated little detail would be required; nighttime environments, swiftmovements past camera and long shots of the creatures in the distance – theydidn’t have to hold up to much in the grand scheme of things. The fill eagle usedfor The Fellowship Of The Ring was a cheat – the bare minimum to final theshots. There would be time to recreate the creatures in a more sophisticatedmanner later.

For The Return Of The King a much more robust, believable and sophisticatedbird was required. The old model was completely re-built, again moving fromnurbs to a sub-division body geo and rigged with a muscle model. Detailing wasput in place with extracted displacement maps sourced from a combination ofrange data and ZBrush.

Feathers

The new feather code for The Return Of The King differed and improved uponthe first film. A new feather setup was created using layering; differentiatingbetween hero feathers, feather code, and fur feathers. Subsurface andanisotropy shading tricks were used to define the feathers. 147 Hero featherswere created out of nurbs patch triples, and under individual control of theanimator through a sophisticated rig they could manipulate in great detail.

19

The significant feature with Gwaihir is the non-hero feathers. Weta Digitalestablished early on that each feather had to be modeled individually, as paintingthe feathers on to the body gave the appearance of a plucked bird wearing arubber feather suit. Simple feather elements were created out of polys, and10,000 of these were distributed across the body.

It soon became apparent that the key to the body feathers was to avoid theappearance of them interpenetrating. Once the wing flap cycles were animatedthe ripples of interpenetration across the body created waves of colour change asthe darker wing tips poked through the lighter base feathers. The initial solutionwas based on collision detection from proc feather to proc feather. Ifinterpenetration was detected the feather on top was moved out a little.

Unfortunately this tended to have a ripple effect, as the feathers lay over theothers down-wind of them for aerodynamic effect. The resulting look was that thebird looked puffed out and scared, not the majestic saviour of Middle Earth theyrequired. Quite close to delivery Weta Digital came up with an alternativesolution: ignore the interpenetration entirely. They took advantage of thefundamental ordering of the feathers and wrote a shader to render them on top ofeach other, even if they were interpenetrating.

Conceptually this was somewhat like the painters algorithm in that it renderedeach feather in full over the top of those further down the body of the bird. Inpractice it was done with a smart Renderman shader. Pre-calculation of the orderin the bind pose by performing collision, followed by detection to find whatfeathers appeared on the outer layers created flow curves to determine an overallordering. The feathers were generated at render time with a PRman DSO whichgenerated a subdivision patch description of each feather in shared memory.These feathers were generated so they lay coincident with a separate subdivisionskin modeled to define the feather surface. This surface was like a sock aroundthe original surface of the bird and was cleverly rigged not to interpenetrate thehero feathers when they moved.

20

Following this process the space the entire bird filled was voxelised, and in eachvoxel a list of feathers was stored whose bounding boxes intersected that voxel.For each feather a BSP tree was generated which enabled them, given a point Pand a particular feather, to find how close they were to a particular feather.

PRman shadeops were also written which allowed the shader at a point to findthe ordered list of feathers which that point is 'near'. The shadeop used somevery fiddly shared memory code to get access to the BSP trees, subdivisions,and voxel structures that the DSO that produced all the feathers generated.

There was a choice, either generating the feathers as real subdiv patches, shadethose (and the shader on them would use this trick shadeop to tell it what featherought to be visible at that point). The idea with this method was that the shadercould displace the bit of the relevant feather to take into account ruffles.However, for those shots which didn't call for ruffled feathers there was nowanother much faster way of doing it.

In this mode generating real geometry for the feathers was never done. Insteadthe sock geometry was shaded. Because the feathers were supposed to lie onthe sock geo they could do a lookup at the point being shaded, and find the stackof feathers lying over that particular part.

There were a whole host of problems, mainly caused by messing up the wayPRman would do texture derivatives. The stacking changing from micropoly tomicropoly meant having to do the texture lookups within the shadeop too.

The feather solution finally worked so well that it was used it for chain mail onRohan Royal Guard digital doubles. Once again there was the technology tospread into other areas of the film.

21

MASSIVE

Weta Digital’s crowd animation software ‘Massive’ was specifically designed andcontinually developed for The Lord Of The Rings films. Massive uses aspects ofartificial life to procedurally animate individual agents. Each agent has a ‘brain’and a network of fuzzy logic switches giving them a primitive sense of sight andhearing. They can react to their immediate surroundings, and each other.

The ‘brain’ results in complex emergent behaviour. Very quickly there is a sensethat each agent is an individual doing their own thing. This supports a muchhigher level of realism than traditional ‘instance canned motion cycles to particles’type approaches. The human eye is very quick to detect repetition in these.

Massive Pipeline

Creating a new agent starts with motion tree design. This tree maps out theoptions that will be available for an agent to take at any point in time. A typicalcombat agent brain would have access to 200 different actions. Motion trees arethen procedurally broken down into the individual motion clips required for theagent. Most of these will be created using motion capture but the few that can'tbe motion captured will be created using keyframe animation

Terrain adaptation is covered in motion capture by capturing each locomotioncycle three ways: level, max down and max up. Massive will then blend betweenlevels and move up or down to the degree implied by the slope. Secondarydynamics such as hair and cloth are also supported. Lots of effort is put intoadding individual flavour to the different armies. For example the Uruk Hai fight ina medieval two-handed broad sword, lumbering hack and slash style while theelves adopt a more elegant, eastern, martial arts style of sword-fighting. Thesedifferences are reflected all the way back to the motion tree design, and result in

22

stunt performers with different skills and areas of expertise being hired for themotion capture sessions.

Motion captured clips go through motion edit and are then imported into Massiveaccording to the motion tree design. The processing that goes on when the clipsare imported includes converting joint rotation motion to IK on the arms and legs.This enables all the variance scaling, terrain adaptation and motion blending thatMassive will apply to the agents to use the same motion clips.

Agents are assembled for basic geometric components and variance incomponent design and agent scale is assigned. Information is then written outinto a creature file which describes all possible variations of the agent in terms ofgeometry, texture and shaders. Given the current motion of an agent, and therange of available motion clips, which action is actually taken by a given agent.isdetermined by its brain. This decision is based on sensory input and actiontendencies as filtered through a network of fuzzy logic switches.

Over the years that Weta Digital has been running Massive software some prettysmart Massive brains have been developed. Massive supports cutting andpasting of brain sections. There are sections of brain for terrain adaptation,targeting (engaging another agent in combat), sound and vision and proceduralvariation (gait variance for example) that can be used to greatly speed updevelopment of a new brain.

Simulations are run on a shot by shot basis. The Massive TD will typically take ageneric brain and custom build agents for specific actions within specific shots.This gives much more control than trying to build an Orc agent that will doeverything Orcs are required to do in these particular films. Sims are reusedwherever this is appropriate, often the same background action can be sampledfor several different shots. Sims are reworked until the motion is approved by thesupervisors and director. Then a subsequent pass is run to simulate the hair andcloth dynamics on top of the approved simulated motion. With only wanting to dothis once motion can be approved without this detail.

23

The first time the Massive TD sees the sim is once it is rendered. The sims resultin an AMC per agent per shot. A command line tool repartitions this data for therender process: per frame, the AMC data for all agents for that one frame, andthe frames either side of it for motion blur. This is now completely formatted forGrunt, the Massive renderer.

Grunt is a fast poly scan-line/a-buffer renderer optimised to render very largescenes with a minimal memory footprint using the per-frame data describedabove. Grunt lighting takes place in Maya, using particles instanced with proxybillboard geo to keep the scene light. Grunt lights are exported back into Massivealong with the per-frame AMC data, the callsheet describes the composition witheach individual agent, and the same low poly terrain that the sim was run on forshadow casting. Grunt builds the scene frame by frame, agent by agent, loadingeach agent when its bounding box is hit. In this way it can render scenes withmany millions of polygons.

The Fellowship Of The Ring prologue: A baptism in fire

An example of the use of Massive in The Fellowship Of The Ring is the Mexicanwave shot. The foreground is a bluescreen element, with background numbersadded with Massive. The trick here is in synchronising the action of the Massiveagents to the action in the plate to disguise the transition.

24

The Two Towers Helms Deep: Massive gets to stretch its legs

Helms Deep was the first opportunity to see Massive agents in the mid-ground.Eight complex shots were turned over late in the post production stage, but WetaDigital were able to turn these around quickly with a team of only seven in thedepartment as all the setup work had already been done.

How many soldiers did you say?

There were four hundred Massive shots in The Return Of The King, with someshots containing more than 200,000 Massive agents.

Horse and rider agent

For horses with riders Massive used one agent with two brains and two motiontrees. These could operate independently; for example an agent can turn and firean arrow independently of what the horse is doing. The rider brain establisheswhere the horse is at with its run cycle and simulates the effect of the horsemotion onto the rider with squash and stretch transforms in synch with the horsemotion. This independence adds more complexity to the combined agent.

Horses were motion captured providing a library of two to three hundred horseactions which were implemented into Massive to create the huge battle scenesfor The Return Of The King. A huge number of these shots were turned over latein post-production, including the Black Gates sequence.

The Black Gates included for the first time Massive agents on animated terrain.The shots where Orcs fall into the chasm were done with the agents simulated onper frame obj geometry, which required adding new functionality to Massive. TheOrc brains were enhanced so that as the ground they run on starts to fall awaythey at first turn and run up the direction of the slope, then when the ground getsabove a certain angle they start slipping and fall into the chasm. At this point they

25

switch from being animated with motion clips and become animated withsecondary dynamics.

The huge numbers of Orc soldiers seen in Pelennor Fields required the creationof a new type of agent; the multi-body agent. This is a single brain controlling 25soldiers in a 5 x 5 grid. These brains were built in such a way that terrainadaptation was still supported. This approach was used for creating a platoonagent, and a dead bodies agent for littering the field with after the battle.

26

PELENNOR FIELDS

The battle for Pelennor Fields saw us realizing some of the most challengingwork in the whole trilogy.

Some shots in this sequence encompassed all our bag of tricks: hero creatures(Mumakil, horses), digital doubles, integration of bluescreen elements, massivecrowds rendered in a separate render pipeline, complex compositing, all takingplace in an entirely digital environment

In fact many shots in this sequence are entirely digital.

Digital Doubles

In each of the Lord of the Rings movies, there were shots involving the principalcharacters that were not possible to film with flesh-and-blood actors. Nearly everylead character had at least one digital double. With costume changes and variousenhancements made between films, some characters ended up with severaldigital doubles. In addition to the leads' doubles, we also created digital doublesfor the more "active" extras: those that were thrown from horses and off thebacks of giant elephants.

The doubles for the leading actors were by far the most challenging. Not only didthey have to look absolutely real, but they also had to look exactly like the actorthey were meant to replace. By The Return of the King, the audience had alreadyspent many hours looking at the faces of the lead actors, and they would pick upon even minor discrepancies between the actors' and their doubles' faces.

27

Legolas is a good example of how our digital doubles evolved over the course ofthe Trilogy. For Fellowship of the Ring, Legolas was built from photo referenceand head scans of Orlando Bloom. At the time our scanning hardware was state-of-the-art and could achieve tolerances of 1 mm. This was perfectly adequate forFellowship and Two Towers, but by the time we got to the Mumakil sequence inReturn of the King, we realized that we didn't have enough facial detail to achievewhat we were being asked to do.

Legolas was originally built as a NURBS model, but over the course of the filmswas rebuilt into a mostly SubD model, though portions of his clothing remainedNURBS. The Legolas shaders were rewritten for each film, incorporatingadvances in our shader libraries to try to meet the increasingly demanding shotsof each film. As, for example, our Gollum skin shader improved; we folded thesenew functions into our skin shaders for various digital doubles, including Legolas.

While we improved our digital doubles, Peter Jackson handed over increasinglychallenging shots. The success of one small shot in The Two Towers whereLegolas acrobatically mounts a horse spawned an entire sequence in The Returnof the King. In that sequence Legolas climbs onto the back of a Mumak, fights offa bunch of bad guys, and then single-handedly takes the animal to the ground. Innearly every shot in that sequence, there was a hand-off between Orlando Bloomand his digital double - countless opportunities for a discerning audience toscrutinize the synthetic Legolas and compare him to the real thing. Also, thesequence was shot on top of a pile of sandbags on a bluescreen stage. Becausesandbags are considerably less lively than giant elephants, we were charged withcreating new camera moves to give the impression that the Mumak was chargingdown the plain. In some cases only a few frames of the plate was usable - wehad to replace the rest with CG.

28

The Witch King

Though an actor was used to play the role of the Witch King a digital double wasused for many of the shots, especially those shots where the character was ridingthe Fell Beast. Both rider and his ferocious winged Fell Beast were also createdsuing sub-d surface models, with detail displacements painted using Pixologic’sZBrush to remove the smooth plastic appearance that we were seeing in close-ups. ZBrush proved to be very useful for creating surface detail when we didn’thave anything to scan.

29

Mumakil

Mumakil were one of the more challenging creatures to bring to the screen forThe Return Of The King. Not only did Weta Digital have to build a realisticcomputer generated elephant, but also had to have him drive a large structurewhich, in turn, drove the animation of the humans riding inside of this structure. Add on top of this a dynamic simulation for the leather panels and ropes, and wehave one very complex creature.

This affected many aspects of the character, everything from his anatomicaldesign and model to the animation and dynamic simulations. If the Mumakil wereanimated too wildly, the soldiers riding in the baskets would be tossed out. Theanimators had to be constantly aware of the effects their animation would haveon the tower and riders.

The Mumakil's underlying muscle structure was a difficult endeavour in itself. The movement of the skin had to help portray a sense of the creature’senormous mass. Achieving just the right amount of muscle dynamics in just theright places was crucial to selling the giant scale.

30

The structure of the tower was driven by the motion of the Mumakil, but also hadto deform the Mumakil's skin to give a sense of weight. The tower had to bendand flex to accommodate the massive creature moving underneath it, but had toappear rigid enough to realistically support the weight of the structure and itsriders.

To make the challenge even more difficult, we had shots where the towers had tobreak apart as Mumakil fell to the ground or, ultimately, collided with each other,tossing their occupants out amidst shards of the broken tower.

31

Great Beast

Great Beast was a complex hero level creature that was designed for only threeshots. With only one month to put Great Beast on film Weta Digital relied on thenew pipeline and quick solutions. Unsure of what Jackson would require, theGreat Beast needed to hold up as a hero foreground creature if need be.

The creature itself followed the same subdivision surface model pipeline, addinghero features such as hair and a detailed harness to complete the character.Problems with interpenetration were addressed instantly and in the fastestturnaround the company had seen Great Beast was complete.

32

Trolls

Trolls had meager beginnings in Middle Earth in Bailins Tomb, but weredisplayed at their finest in the siege of Minas Tirith in The Return Of The King.

Six different trolls were designed, each completely unique by the end of the thirdinstallment. Each Troll had their own ‘kit’, starting in diapers, and ending in ornatearmour with interchangeable assets that gave them individual identities. With thehair pipe now well established detailing with a ‘coat’ of hair similar to Great Beastwas also implemented to complete the creatures.

33

CONCLUSION

The Seven Year Odyssey Comes to an End

At Weta Digital the work of the artist is paramount, as our work on Lord of theRings demonstrates. While we have an impressive array of technology at ourdisposal, including world leading and academy award winning solutions for digitalcreature rendering and crowd animation, this technology exists only to serve as atoolset for our artists. It is the work of our animators in visualising a whole worldsuch as Middle Earth, placing the audience in the middle of the battle of PelennorFields, and bringing a complex digital character like Gollum to life that weconsider to be Weta Digital's true strength.