Week 1

The goal of this exercise was to get used to the basics of roto, learn the rules of the technique and how it can be implemented into larger shots. Here we learnt that you want to start on the hero frame, and we wanted to roto out one of the pieces of candy.

The hero frame is the frame at which the object can be best visualised, this generally means that the object can be fully seen in the shot and that it is a frame where the object has little to no motion to minimize the motion blur.

In the image above you can see how we use the roto node to mask out the piece of gum, this selection is then manually animated throughout the footage to ensure that it tracks the candy and properly masks it out.

One of the rules for roto is that in a frame you want to do the least amount of work as possible, you want to use the least amount of points as possible, and the same goes with the keyframes for the animation.

In the following image we can see that the Roto node is connected to a Blur node, this is because in the footage the gum isn’t extremely sharp and the edges kind of blur out, so we use this node to replicate the colour diffusion that we can see on the gum.

These next images showcase the way that the footage and imported assets should be set up on the Project Settings of the file. By setting up the project directory correctly we can then delete everything that is included in that directory on each of the imported files, this allows us to move and send the nuke script to other computers where we simply need to change the project directory so that the software can find all of the footage. If this isn’t done and we transfer the file it simply won’t work, this is the same situation as when a software like Maya can’t find an image texture in a material it will change it to a bright pink.

After finishing the gum shot, we moved to this one. Which, is more complicated than the previous shot because it also has rotation.

By implementing the same principles of keeping the minimum number of points, we can create and animate the roto for this scene fairly easily.

We then used a premult node to cut out the black parts of the alpha that the roto created. Using a merge node, we could place the footage of the roof over a light blue background.

On the other hand on the image above, we created this other version of the script where it uses the roto to get a mask for a grade node. Here I changed the colour to a bright green but this same technique can be used to brighten up an object in the footage or properly colour-correct it.

Week 3: This week we focused on more complex roto and looked at a new technique. The previous technique is good and very useful for general objects but what happens when we need to roto out a complex and or organic shape? This is when we use this technique. As you can see the progress of the thumb and hand, this technique is very similar to last week’s one but this time we use multiple simple shapes to create a complex one. This is done with simply one Roto node as we can see above, but we can create different different roto shapes as shown in the highlighted section to the right. On the right, we can also toggle motion blur for the roto shapes and Nuke will create some motion blur based on that shape’s movement and apply it to the alpha.

Finally, we have this shot with a whole human body which moves across the frame, the issue with this shot is that not only is the roto of the person complicated but we have to set up multiple roto’s for different objects in the footage since the person walks behind them.

There is also a new problem that arises with this and it is that we have to somehow keyframe the activation of the roto’s for the objects where the person walks behind since later on they walk in front of the objects and we don’t want them to be cut out again.

This can be achieved by adding a grade node and turn the multiply from 0 to 1 and keyframe that to calculate when the roto will be present or not.

Week 4:

This week we learnt some different techniques regarding keying, we worked on 3 different shots with different goals we had to fix to practice certain techniques.In this first shot, we used a Keyer node with the operation set to luminance key to adjust the luminance of the image and use that to select everything in the image except for the sky. This is because, in a sky swap like this one, there is a lot of detail such as the trees in the background that we need to keep. We combined the key with a garbage matte of the foreground to have a simple flat plate that we can use and by using a premult node we can remove the sky from the original footage and combine the new sky behind it.Above you can see some code, this code we can copy and paste into the Nuke viewport and it will create a node based on this node. This is extremely useful for creating custom nodes that might not be in Nuke but might be useful for certain shots. The specific shot we made was called a colour dilation node and it helped integrate the background of the trees with the new sky image.Here you can see how we used the light wrap to further integrate the forest with the replaced sky. Below is another example of a different sky replacement with a similar node setup.This next shot involves a green screen removal and we will change the background, the method is pretty similar to the one we did before with the sky removal but we need to use some new techniques to help the woman merge with the new background.To do this we used a keylight node, we used two of these nodes, one to remove most of the background, this was a harsher selection and resulted in the loss of detail from areas such as the hair, this is why we added the other keylight which made a softer selection and allowed us to keep such detail. By combining both we could have no green screen and maintain all of the detail.Above you can see that we followed this node path by connecting it to the copy node and adding a despill node. This node is responsible for removing the greenish tint on the actor from having the green screen reflect light.Following this we created a garbage matte to make sure that no part of the greenscreen would remain visible throughout the process, we animated this roto as well to ensure that it didn’t go over the actor at any point in the footage. To finalize the script, we added a lightwrap similar to how we did for the sky replacement and a final premult and merge node.The final shot we worked on was similar to the previous one but had one slight difference: some parts of the cloth the woman was wearing were translucent, meaning they were slightly green but we still wanted to see them in the footage after the process was finished.To do this we used the same node system that we implemented previously but adjusted all of the values carefully to ensure that the cloth would not simply disappear.

Week 5:

This next week we continued working on the shot of the woman by further integrating her into the background, the lighting that was hitting her wasn’t exactly the same as the lighting in the office environment so to change this we added multiple Grade nodes with different functions.

We first adjusted the luma, colour, and saturation to ensure that the actual colours of the actors matched the ones of the actors. To do this we adjusted the blacks and the highlights as well as slightly desaturated the actor. By the end of this process, we were able to add a final grade node with a roto shape masking it. We used this to adjust the exposure of just the left section of the actor creating the illusion that the light from the windows is hitting them.

For the final stage, we simply roto out some of the sections of the windows and floor reflections and connected them with a grade node so that we were capable of adjusting the colour and some other parameters of the glass. Adding the blur node helped recreate the blur from the camera onto the roto shape.

Week 6:

This week we worked on this shot where the goal was to remove a section of the sign in the background, to do this we had to learn about tracking since the sign moves across the screen throughout the footage. We started by adding a tracking node that allowed us to add a track onto the sign and follow its path. Nuke will automatically track the object based on what is inside of the tracker but if it doesn’t track it properly we can also do it manually.
The next step is to create and add a frame hold node that will ensure that the section we roto won’t change due to the movement of the footage. We can use the roto to cut out the section of the sign we want to remove and add a premult node to remove the rest. Once this is done we can drag the tracker node from the top part back into the pipeline under the premult and combine the now moving roto back onto the original footage with a simple merge node. This next shot is the same one we looked at a couple of weeks ago, this time instead of doing a manual roto to follow a section of the building we can track it. The issue with the way we did it earlier is that this footage also has rotation, meaning that one tracker won’t suffice, by adding two we can also calculate the rotation and transfer it to the roto, allowing us to now apply the grade to adjust it to our liking.

Week 7:

This week we had this shot where there was a man that we wanted to remove from the footage. To do this we combined almost all of the techniques we have learnt. The camera wasn’t completely still and it slightly jittered so we placed a tracker on a high contrast point to track that jitter and be able to apply it to the patch later on. We then added a frame hold node and selected a frame where the man was no longer present. Then we created a roto shape to select the piece of the frame we wanted to use to cover the man and used a premult node to remove everything else. Following this we added the tracker so that the patch would move with the footage, but we encountered the same issue we had a couple of weeks ago, later on in the footage the women that we wanted to focus on passed in front of the roto and get half masked out. To fix this we add a grade node and multiply the roto by 1 when we want it to be present and by 0 when we want it to disappear. Finally, we add a denoise node at the beginning of the footage to completely remove all noise, at the end of the script we add a grain node and replicate the original noise, the reason this is done is so that the noise can be the same across the whole footage and to help it integrate seamlessly across the footage without having distinct differences that could reveal what we did.

Week 8:

For this crowd removal, we used the roto paint node to create a patch. The roto paint node works by duplicating other parts of the surface and placing them wherever you place them, acting like a paint tool that paints the same texture present in another location.

Following this, we added a frame hold so that the roto paint would stay the way it was on our hero frame. We then graded it and tracked it to the camera movement with a corner pin node, finally merging it with the original video.

In this shot using the same technique to create a patch with the roto paint node isn’t possible since water constantly moves therefore we can’t use the same technique and we have to think of a new one.

This new technique consists of creating a roto of the original footage and then placing it in another part making sure to cover the part we want. We also grade this roto to make sure it looks good as well as blending the actual roto so that it creates a soft transition between the roto and the original footage.

We continue grabbing different parts of the ocean and placing them over the parts of the footage we want to cover until we reach our final result.

Week 9:

For this VFX shot we wanted to replace the logo on the box. To do this there are various steps, the first is to remove the original logo, and to do this we use the same technique that we used to remove the crowd last week. We create a patch with roto paint, then we grade it and then we track it using the corner pin node.

Following this we use the new logo and we use the corner pin node again to place it on the same location and a transform. Then we simply grade it, add a defocus, and track it onto the box to make sure it follows the same movement as the box.

For this VFX shot it was relatively simple, it was similar to a VFX shot we did a couple of weeks ago with the green screens and background removal. So we keyed out the green screen, then we placed the background behind her and moved back up the pipeline to grade her in a way that matched her onto the background by creating different rotos that influenced different sections of her face and body.

Week 10:

This green screen wasn’t as easy as the normal greenscreens we have done, the reason is that there are a lot of different shades of green, many of the objects that have greenscreen have slightly different tonalities so we used this advanced keying technique.

This technique consists of keying a piece of the greenscreen, close to a specific part of the actress’s body for example the head or the arm. After this we roto it out and combine it with a keynote node so that we can combine multiple different alphas and keys into one that is completely clean and merges with everything properly.

We then follow this by creating a rough roto around the actress and using a premult to cut everything out, basically creating a garbage matte. Then we combine it with our chosen background in this case a bright pink background just to show off the key and it is finished.

There are multiple ways to key a shot like this one though, the previous one was quite slow but accurate which is good in some scenarios. In this case, we are going to try again by using the IBK Colour node, by using this node and placing different values inside the node we are capable of creating a quick selection that keys out all of the green in an accurate way.

Even though this is a really fast way to do it and in this case it was accurate in others it might not be so effective so in many instances, it is useful to combine different keying techniques for different pieces of a shot.

Tim VFX Shot:

Final_Example_vid

Dancing Robot VFX:

 

Bomber Plane VFX: