Nuke – Week 10 – 3D Matchmove/Camera Projection

In today’s lesson, we’ve been covered about the 3D aspect of Nuke compositing. One of Nuke’s strongest tool is 3D Camera Projection which are very commonly used in

Gizmos Recommended:
– DasGrain by Fabian Holtz

Matchmoving:

The process of matching the camera movement/object movement in 3D space so the software can interpret the shot and match the 3d objects/geometry corresponding to the movement of the real plate.

Lens Distortion

Every lenses have some kind of distortion, and before we can start tracking/matchmove we have to account for this by using LensDistortion node to undistort the image first. This can be done automatically with checkerboard (if provided by production)

Nuke can automatically solve checkerboard footage from the lens

However, if no checkerboard was provided then there are ways to solve it manually, by drawing lines which are supposed to be straight then let Nuke solve the distortion (More lines the better Nuke can solve)

Manually drawing lines to help Nuke solve the lens distortion

Week 10 – Maya – Photorealism

Maya – Shading model with Arnold materials + HDRI

In this lesson, we’ve covered topics about 3 point-lighting (keylight, backlight, and side light), Arnold materials/lights and also rendering passes in Maya (z-depth (for fog or DOF in Nuke), specular (for metals) and others)

We’re provided the models for the robot and tasked to create a photorealistic scene using basic lights and materials in Maya. We’re also introduced to HDRIs (high dynamic range imagery) which are used for lighting/backdrop references

Term 1 – Final Project Ideas/Pitch

For the final assignments of this term. I have gathered some references and ideas for the project.

Futuristic Bangkok

Since I was young, I had always been fascinated by the architecture of buildings and skyscrapers. And would always imagine what the future buildings would look like. So the concept for this project is to try and imagine/recreate what “futuristic Bangkok” would look like but at the same time, stay grounded to reality as possible and keeping the same signature look of what makes it “Bangkok” at first glance.

Since Thailand has been known as a country associated with ancient temples and tropical/natural landmarks. There are many features which can be added to signify the building as more “Bangkok-like”. By attempting to integrate Thai’s ancient relics with modern structures, I think in my opinion, will compliments each other and produce a futuristic yet grounded to heritage style.

V1 – Hero building model in Maya
Initial scene setup in Unreal Engine
Mood & Tone / Layout reference for this project
V2 Textured and Added more details + foliages
V2.5 Camera/layout setup
V3 Lighting improvement + BG buildings + Minor details
Compositing ref. – I would film myself walking in/out of the frame, overlooking the futuristic view. Composite them together with Unreal renders in Nuke. The virtual camera would then pull focus accordingly (render depth pass, use defocus node in Nuke)
V3.5 Environment details improvement + Materials redone
Final Version – Rendered from UE5 in Path Tracing mode
The render options I used to achieve the final results. After multiple trials and experiment, I have found that the following render settings produced the best results while keeping render time reasonable.
I also rendered depth pass to feed into Nuke for defocus
I also separately used a simple 2D monorail train element to composite into the render later, to enhance the futuristic look
Result after compositing
Defocus node test to blur the background and focus on foreground element.
Defocus settings – Anamorphic look
Original Shot Footage
Roto Alpha Channel

Final Composite

VFX Breakdown

Self-Reflection

After I have completed my project, I am both satisfied yet felt that there could have been more to be done. What I liked about the final results is that it achieved my initial goal of portraying a futuristic interpretation of my hometown. I loved the quality of the reflection on glass surfaces, in my opinion it helps add a lot more realism to the shot, as I put a lot of care into iterating what would have been reflected in the building. Also, I feel like the compositing is very coherent and brought the final image together quite well. As I made sure I color corrected every plate to match the plate from real photography. What I didn’t quite feel satisfied with is the lack of intricate details in the scene. If I had more time, I would have added futuristic details like flying cars, neon/hologram billboards etc. And also the fact that the city feels lifeless due to the lack of movement or people in the scene. Also I feel like there aren’t enough details to signify that it is Bangkok, I would have added more cultural details or buildings to help add the essence of my hometown.

Nuke – Week 6 – Tracking/Planar Tracking

In today’s class we have learned about tracking nodes in Nuke, and the different types of tracking. Including:

  • 2D Tracking – Mostly single point (For transform, rotation)
  • 2.5D Tracking (Planar Tracking) – For tracking surfaces or flat objects
  • 3D Tracking (Camera Tracking, Matchmoving) – For tracking full 3D scenes or objects

Nuke – Week 5 – Merging and Color Space Management

In today’s class we have learned some new basic operations in Nuke such as, Merge, Shuffle, Grade, Premult, Read/Write. Which are all important nodes to know in order to understand Nuke

Premultiplying is a concept that we have gone through today about its importance. Any video/image with alpha channel needs this process to appear “transparent”

Most image/video editing softwares that we might have used before (Photoshop, After Effects, Premiere Pro) will do the premultiplying already for us. Whereas in Nuke, this process is done manually with Premult/Unpremult nodes.

Nuke – Week 4 – Rotoscoping

In today’s class we have learned the basics of Nuke’s user interface and have been introduced to the concept of “rotoscoping”.

Rotoscoping is a process of masking then separating the elements (either person or object) in the frame that would need to be manipulated, by manually drawing masks and keyframing the movement of the said object as accurately as possible.

What we also learned to avoid to do is:

  • Avoid doing a single roto node/spline for a complicated subject (humans, animals etc.) Instead, do separate splines for each smaller parts.
  • Lookout for motion blur and fast moving subjects.
  • Avoid drawing/moving roto for every single frame of the shot. Instead, try skipping ahead a few frames to see if it can be done with the least amount of keyframes
  • Avoid drawing too many spline points (Less is better!)
  • DO Try to use tracker to assist in roto when possible (static object with no complicated movements)

Example Project: