Tracker

One important task of compositing is motion tracking. Nuke provides the Tracker node for this purpose. The Tracker has been updated for Nuke 7 and is much more powerful and easier to use. I have brought in some footage of a man walking down the street with a moving camera. This will be a great place to try motion tracking. For example, I can motion track this sticker on this post and replace it with something else. Let’s give that a try.

Go back to the first frame, select the Read node and right mouse button click, select Transform > Tracker. Again, this has been re-designed for Nuke 7. Right here, you have a track field. This is where you add tracks. You can have one track or as many tracks as you want. I will click the add track button; the track listed here is track 1. It shows me what I want to track, translate, rotate, and our scale. I am just going to do translate for now, but if you want to, you can turn on rotate and scale also. Associated with that is the anchor box for track 1 in the Viewer. You can click+drag the center of this to move it. I will go ahead and position this over the sticker. The inner box is the pattern box, that’s the pattern you are trying to track over time. I can scale this by click+dragging the ends. The outer box is the search box. This is where the node goes if it has a hard time finding the pattern. Now, with this version, there is a new zoom window right here. One nice feature is you can Shift+click+drag to zoom in or out, so you can see the pixels really closely. Another thing you can do is click+drag with your mouse in this area to position the entire box. I will zoom back out. Now, I have positioned the anchor box for track 1 on the first frame.

I can approach this as a more traditional transform tracking or go straight to my track buttons, press those, and let the program analyze the footage to create a motion path. The other way I could do this is to actually set several keyframes. That’s where you set keyframes over time and let the program analyze between those. This will help make sure the resulting motion path is more accurate. Let’s give that a try. Because I have placed my track 1 box on the first frame already, I get the first keyframe for free - I will see the blue dash right here. Now, I can go to a different keyframe, say frame 195, and re-position the box. As soon as I do that, I get a new keyframe, plus a motion path. Once again, I can use my zoom window to fine tune the position. You will also notice these two boxes up here at the top. These are keyframe patches. It shows you what’s underneath that box for each of those keyframes. You can click on those patches to compare them. The view of the patch is put into the zoom window, so it’s a great way to make sure your positioning is accurate. If you go back and forth, ideally, you should see that pattern stay static and not move around. You can always go ahead and add some more keyframes, for example, I will go to frame 200. This one is tricky because the man’s arm covers up the sticker. I can still position this box as best as I can to make it in the same relative position. As soon as I do that, I get a third keyframe patch. Then I can compare the keyframe positions. I will try to fine tune this last keyframe to make it more accurate. It looks like the relative position of that post and that handrail don’t really change between keyframes. Once I have a few keyframes, I can then analyze. I am not going to use my regular track buttons. Instead, I will use a special button right here, which is called Key Track All, that will track between the keyframes, and take the keyframes into account as it tries to determine the best motion path. Let’s give that a try. It’s going to analyze forward. When it’s finished, it will give me a motion path. Once it’s done, you can play it back and see how well the track follows the pattern.

It’s looking pretty good, however, at the very end, because of the overlap with the arm, it gets confused with 2 frames. Here’s a nice feature of this system. I am going to zoom in here and what I can do is re-position this box on one of the bad frames. For instance, on frame 199, click+drag it and it actually updates the surrounding keyframes. It takes in account the keyframes that are already set and it takes into account the updated position, and then re-calculates a path in that area. In order to improve the motion path at this point, I am going to spend more time adjusting keyframes and, in fact, I can add additional ones if I need to. I also always have the option to go back to any of my track buttons and reanalyze either part of the timeline or the entire timeline, and go forward or backward. It’s going to take a few minutes for me to adjust this so I am going to fast forward.

Alright, so I’m back and I have improved the motion path. So, basically, I spent more time adjusting keyframes for the problem areas near the end of the timeline. If you watch the zoom box, you will see the pattern is more stable now. Now, this is not the only way to see how accurate the path is; there is another new button for Nuke 7, which is right here. It’s the Show Error on Track button. If I click that, it color codes the motion path. The color of each point on the motion path indicates the error value generated by the node and the error represents the confidence with which the node has identified the pattern for that frame, with the patterns established by the keyframes. So, in other words, green is good. Now, in order to demonstrate what it looks like when it’s not so good, I can remove some of these keyframes and recalculate. To delete, just select the patch and press the Delete Key button. Let’s say that I also rushed the placement on my last keyframe, it’s going to recalculate automatically because I moved that box for that frame. So yellow, to orange, to red indicates a rising error value. I want to undo this - Ctrl/Cmd+Z to go back to my previous version of the motion path. I will fast forward.

Now we are back with a good motion path with the five keyframes. Now that we have a good path, we can export the data. You can go to this menu and click the style of data you want to export. Now, if you have four tracks, you can choose the CornerPin. In this case we want to use match-move, so I will set that to Transform (match-move) and then click the create button. That creates a Transform node that is hooked up to the Tracker via an expression. If I open up the Transform node, I will see the translate, rotate, scale, and center is linked via an expression to the Tracker. Now, I can plug something into the Transform to have it follow that motion path. Let me close all these windows.

Now, in order to keep my background, I am going to make a new Merge node: press the M key. I will hook up the B input to the Read1 node - that’s the background - and I will hook up A to the new Transform node, and hook the Viewer into the Merge node. Now, I just need something to match-move. I could bring in a Read file with a bitmap or, in this case, I will just bring in a Constant to test this (Image > Constant). I will change the color of the Constant so it’s more interesting, like this green. I will change the format. Initially the Constant is the same size as the project. I will pick something smaller. If you don’t see a resolution you like, you can always make a brand new one - just click the new button and enter a new name, and a new size. I made one already and it’s called square75, and it’s 75x75 pixels. I will go ahead and hook that up. Now, one issue here is the Constant is placed at the bottom left-hand corner of the composition. You can see that if I go to the first frame of the timeline, right here where 0,0 is, what I can do though is line it up with where the center of the motion path is by adding 1 more Transform node. I will select the Constant and press the T key. I do need to see the motion path, so I will open the Tracker. Now what I can do is grab the transform handle for the Constant, and move it so it lines up with the center of the anchor box. I will zoom in here so I can see this better, and position that until it lines up right there in the center. Now, I can close all of these windows and play it back.

And now that new Constant follows where that sticker was. There are still a few issues, for instance, I would have to rotoscope to get behind this man's arm, but the basic motion tracking is now working. So, there’s a quick introduction to the new Tracker in Nuke.

“Tears of Steel” footage courtesy (CC) Blender Foundation - mango.blender.org

Hello and welcome to this tutorial on the new Tracker in Nuke 7.0. If you have seen Dan’s tutorials on the Tracker, then you will have a good idea of the new interface. I am just going to start off and introduce you to it.

Bring up the Tracker in the same way as we did before, and as you can see already, there are a number of differences. The first thing you will notice is that we have a panel that allows us to store as many tracks as we want - we are no longer limited to the initial four. And also, we have an Export panel that allows us to create CornerPins and Transforms easily from the Tracker. We also have a lot of the parameters and controls moved into the toolbar, and the tracking controls themselves moved away from the panel. This means we can now work full screen for all of our tracking. So, I will go full screen and we will work in this mode. I will go to the first frame and pick some tracking points. To create a tracking point, I can click this icon here, and while it’s active, wherever I click we get a new tracker. I could also hold the Shift key down and click anywhere at any time to add a new tracker.

In our panel, we now have four trackers. If I select one, it selects it in the Viewer, and if I select it in the Viewer, it selects it in the Tracks panel. The tracks selected are the ones that will be tracked when I click on track forward or track backward. You will also notice we now have hotkeys for the main tracking parameters, C and V for going forwards, X and Z for going backwards. We also have a new stop tracking button in the middle (Esc). There is also a hide progress button that stops the progress window from popping up when we are tracking. So I go back to tracking and if I select one track, you will see we have a zoom window that shows us in detail the size of the track. If I press Shift, I can zoom in and out to get even greater detail. Select track 1 and 3 and track forward, stop the track at this point, and use the V key to carry on tracking. The Esc key stops the tracking and C will track forward a frame at a time, and back to the V. Working this way makes it very, very fast to set up tracks to do your tracking. I will just go back to full-screen mode and create a new track. The one I have created is the one I can instantly track, so I will press Z to track backwards. So, we have a very nice and fast workflow.

Now let’s have a look at another tracking issue. So I am back in Nuke 6.3 for the moment, and I have a very old piece of footage that I would like to stabilize. There are a lot of luminance changes and lots of noise, all sorts of problems. If I try and track this with this Tracker, it is an instant failure. So let’s see how Nuke 7.0 gets on. Back in Nuke 7.0, I bring up the new Tracker. I will hide the progress bar for the moment, and I will create a track in the same position as we did in Nuke 6.3. It’s closer, but it’s still not correct. If I go into Settings, there is a nice new feature called adjust for luminance changes, so turn that on and it will start working through the luminance changes for the shot. As we can see, we instantly have a much better track. So, if you are having problems tracking a shot with lots of luminance variations, turn this on and you should be able to get a much better track. I have also found it works on lots of other shots that don’t necessarily have luminance changes, so if you are having trouble it’s worth trying it and seeing if it works.

I will expand the interface because we have some other parameters that have been hidden because of the size of the interface. One of the advantages of this Tracker is that it lets us have as many tracks as we like. A very good technique when you have noisy footage is to put lots of tracks into the same area, and average them all together. Now this can be quite difficult at times because there are a lot of maths and expressions involved, and you may need more than the four tracks then you are offered in Nuke 6.3. Here I have six tracks. I select all, just make sure adjust for luminance changes is still on, and track forward (V).

Now we have this new traffic light system (show error on track), which would normally let us see where the tracks have any problems or not, but when they overlap like this they are not much use. Instead, I will use the Viewer to see if they are any good or not. When I am happy, I can select all of the tracks, and the new average tracks button will combine all of the tracks into one keyframed track. I now have one super track, in this case based on six tracks, but in the past I used sixteen or more trackers to get a good track. This is a great technique when you have noisy shots like this, but also when there is heat haze or any other disturbance that will get in the way of the pattern you are trying to track.

Back to our dancers, I would like to show you a couple of other new features that are in the new Tracker. Under the Settings, we have a new feature called snap to markers. What snap to markers will do is show what Nuke considers to be a good tracking point. You can choose from blobs or corners. Then with a green circle Nuke will show you a good candidate for tracking, and then snaps a tracker to that point.

Now I would like to show you a new feature that allow us to combine tracking and roto together. What I will do is use the Tracker to allow me to roto around this greenscreen. I will create a Roto node and a Tracker node, as these both have to be present for this feature to work. I start creating my shape, and then I select those points, right-click, and select Link to > Tracker 1. I can create new links, and what that has done is in the Tracker itself it’s created four track points, one for each of the points in the roto shape I created and selected. Now, in this shot the bottom of the greenscreen goes out of frame. I am going to deal with this using a neat feature in the Tracker. What I need to do is to be able to turn off tracks 1 and 2 as the greenscreen leaves the bottom of the frame. So now, using the new Enable button I can keyframe when the trackers are on or off, and now they won't bother to track at that point. I will select all four of them and hit Z to track backwards. You will see that when track 1 and 2 hit the bottom of the screen, they will no longer continue tracking, but tracks 3 and 4 will. So just close this track for the moment, and just look at the Roto. You can see now that those points of the roto have been linked. Where the other two trackers stopped, you can see at the bottom of the screen, these two parts of the Roto have also stopped. So I go back into the Tracker, and select track 1 and 2, right-click, and set Interpolation > Before/Linear and now those points carry on based on the extrapolation of the curve, and as you see, my Roto has continued onwards. Now this is a stereo image, but the Tracker has only tracked one eye. When you work in stereo in Nuke 7.0, you have this new stereo offset parameter, and I can split it open and offset the Roto to where it should be in the left view. Now we have both eyes rotoscoped using the Tracker.

This is Howard Jones for The Foundry, goodbye.

Hello, Dan Ring here, and in this tutorial I’m going to show you how to get started with Nuke’s new Tracker. Because a lot of things have changed, we are going to go back to basics on creating new tracks and setting up everything you need to get a good track. We will start by loading up the Tracker node. This is what the new Tracker node looks like. If you are familiar with the old Tracker, you can see all the track controls have been moved up to the toolbar. We have also removed the limit of four tracks and replaced it with a track table, allowing you to create as many tracks as you want.

Now, let’s start creating some tracks. You can do this in a couple of ways. The first is by using this add track button. This places a track in the center of the Viewer. The second way to create a track is to use the add track button up here, which when toggled on, allows us to quickly create tracks anywhere we click the mouse. We can also use the Shift key to toggle on or off this add track button. This is probably the quickest and easiest way to create lots of tracks.

There are a couple of interesting things here in the Viewer when we add tracks: the first thing is this zoom window. This allows us to get a good picture of where the track currently is in the frame, and also to position it. If we hold Shift and drag, we can magnify this zoom window even further and really get a much finer position. The second interesting thing is this keyframe window up here. This gives us a snapshot of all the keyframes for this track. For example, when we created this track, we created a key at frame 196. If we go back a few frames and move the Tracker to create a new keyframe, you can see a new window appears. Or we can create lots more of these. These windows become a lot more important when we look at our keyframe based tracking workflow later on. This will be covered in a later tutorial.

For the moment, let’s simplify things and let’s clear all of these tracks and start from the beginning. To delete all of these tracks, we need to first select them. We can do this by either dragging and selecting them all in the Viewer, or pressing Ctrl/Cmd+A or pressing the select all button here. Then to delete them, we can either go to the Viewer and click the Delete or Backspace key, or we can use the delete tracks button here.

Let’s create our first track. In this example, I would like to track the girl’s nose. I am going to use the zoom window to give us a precise location for it. To do the tracking, we use these controls up in the toolbar here. Each of these allows us to say what frames we would like to track. For example, the buttons with the R in them allow us to specify frame ranges. The buttons with the typical playhead arrows allow us to track to the beginning or the end of the shot. The buttons with the vertical bar and the arrows allow us to track backwards and forwards one frame at a time, and very importantly, the stop button allows us to cancel tracking at any point. Let’s just track to the end of this shot, and if we look in the zoom window, we can see that’s doing a pretty good job. So, congratulations -- we have our first track, and that looks pretty good.

It’s very important to note that most of these buttons are bound to shortcut keys on the keyboard. For example, if we want to track to the start or the end of the shot we can use the Z or V keys, or if we want to track backwards or forwards one frame we can use the X or C keys. Let’s track backwards to the beginning of the shot using the Z key. I’m going to cancel that now using the Esc key; this is the same as pressing stop in the toolbar. This Esc key is going to become very important when you start looking at more advanced tracking shots.

Now, let’s talk a little bit about how pattern tracking actually works. To do this, let’s go back to our original starting frame and clear all of this track data. We can do this by either scrubbing along the timeline, or a handy shortcut is actually clicking on the keyframe window up here. So we created this keyframe when we started the track, and clicking on it allows us to jump back to frame 165. To clear the track animation path, we can then use this clear all button up here. If we zoom in on this track, we can see three important things: the first is the track position, the second is the pattern window, and the third is this outer search window.

The way pattern tracking works is, it grabs this pattern on one frame and then tries to match it within the search window of every other frame. So, you can see if we skip on one frame that our original pattern is contained nicely within the search window here. This is actually really important when we are tracking. One of the main reasons that tracking fails is because the search window is not wide enough to capture the movement between frames. Let me show you an example of this problem in this shot. Let’s pretend that we want to track the toe of this girl, so lets zoom in and create a track. We will resize the pattern window to fit just the toe. If we scrub on one frame, you can see the toe is now completely outside the search window. This means the track has no chance of following it into the next frame, so let’s try that by tracking forwards one frame. You can see the track has completely failed. The key to solving this is to make sure the search window is large enough so that it can contain the toe within the next frame. If we go out here, and make the search window massive like this, we can try to re-track now. You can see now that the toe has been correctly found in the next frame. This is an extreme example to show that high motion needs large search areas.

About 90% of all failed tracks can be generally fixed by increasing the search window size to match the amount of motion of the object and also its direction. The next thing to look at is the effect of the pattern window size on the track, so let’s delete this track and go back to our first track. After the search window, the pattern window size is the next most important thing to set for tracking. The pattern window defines exactly what will be tracked. In this example, when we were tracking the nose, we were actually using pixels from the face and the background to help us track. Having a larger pattern window size increases the stability of the track. The problem then is it might not localize the feature you want to track properly. For example, let’s do this track again and I will cancel it mid way. If we look at the zoom window, we can see the track does not exactly follow the tip of the nose properly. This is expected as we are not exactly tracking the pixels of the nose; we are using those of the face and the background as well. Let’s now go back to our start frame, and adjust our pattern window size to fit just those of the nose, and then track forwards and, again, I will press Esc to cancel. Instantly, if we look at our zoom window, we can see we have a much better lock on the tip of the nose.

To summarize, always make sure the pattern window size is just big enough to cover the object you want, for example the nose. Using a larger pattern window size might give you a more stable result, the track might appear smoother and less jittery, however, it might not localize the point that you actually want to track well enough. Most importantly, if you find your track is failing for some reason, check to make sure your search window size is large enough to capture the object’s motion. In most cases that I have seen when tracks have failed, they can usually be fixed by just increasing the search window size and then re-tracking.

That concludes this tutorial on the basics of the new Tracker in Nuke, and also an introduction into how pattern tracking works. I hope it’s been useful, thanks for watching.

Hi, my name is Dan Ring. In this tutorial, I am going to go back to basics, and show you how to spot problems with your tracks and figure out what your track is doing. This is the first step in correcting bad tracks, which we will look at in later tutorials.

In the last tutorial, we created a couple of tracks, one in particular on the girls nose. To judge how good the track was, we looked at the zoom window and scrubbed back and forth in the timeline to see a nice and steady image. This way, you can see when the track is drifting very quickly. Another way to get the same stabilize effect is to use this center viewer button here. As the name suggests, this centers the viewer around all of the selected tracks. You can see the entire Viewer is locked to this selected track. The nice thing about using the center viewer option is we get to see the rest of the picture as well as the stabilize tracks. Also if we select multiple tracks, we get to see the average, the stabilized average of both of these tracks. For a more dedicated way to stabilize the image, we can use the Tracker’s own Transform > Stabilize option. We could also create a CornerPin or Transform stabilize node from the Export menu.

Looking at stabilized image data is one of the best ways to gauge how well your track is doing. Another very good way is to look at the error values of a track. To show you what I mean, I am going to start a track and pause it halfway through. If we zoom into the bottom left hand corner of the search radical, we can see this magic error value here. The error value measures the image difference between the pattern window at the current frame 165, and the track’s pattern window at the original frame 149. If the object doesn’t change appearance much over time, we expect the error to be quite low. A high error either means the object changes appearance a lot or the track has gone wrong somewhere. The Tracker records the error values for all tracked frames. Now go into the Curve Editor and look at the Error Curves for our track. The thing to note about this Error Curve, which is typical of most Error Curves, is it starts off at around zero and gradually increases as the track goes on. We expect this behaviour as most interesting objects change appearance over time, and this error value measures the change in appearance from our grab frame and our current frame. Things really start to get interesting when we start trying to figure out what these high error value peaks correspond to in our image. Let’s keep this Curve Editor window open here in the corner.

We can scrub quickly across the Timeline by dragging this vertical yellow bar. Let’s move it over to our first peak. Looking at the error value here for frame 165, you can see it’s about half that of the maximum error value for this shot. What that means is that we expect the image patch for this frame to be fairly different from the image patch we originally grabbed at frame 149. We can verify this by quickly increasing the keyframe size. True enough we can see that the two patches are fairly different. We can see at the right at frame 149 that part of the ear is occluded, and then uncovered here at frame 165.

Let’s move on to our next peak. As expected, the two image patches are very different, even more so than the last time. For example, the hair is no longer occluding the ear, as it was in the previous example. We are also no longer tracking the center of the ear as we were - we are now tracking slightly to the left. So the difference between these two image patches is what gives us our high error value here in the Curve Editor.

You can use the Curve Editor to spot problems in the tracks by looking at the Error Curves. This only gives us part of the picture. What we want to do is get a better idea of how the track is doing over time, where it’s drifting, and by how much. And more importantly, we want that all in the Viewer. To see this, we can toggle this new switch called the show error on track button. The error is now shown on the animation path of the track in terms of traffic light colors, where red means high error, and green means low error. If we scrub along, you can see instantly how the track is doing. For example, at our high peak you can see the values are red, and on our original peak at frame 165, you can see it’s more orange. But closer to our original frame at 149, it’s more green. One thing to remember here is that there is no objective way to say if a given error value is good or bad. This means that a red color value may mean one track is proceeding nicely, but in another track it means that it has failed. For example, if we go back to frame 200, and look at this red point here, the image patches are quite different as we saw earlier. If we now go back and look at track 1, and go to the regions where the track has turned red, the image patches are still quite similar. Even though this is red, it does not necessarily mean the track has failed or there’s a problem, it just means this is the highest error value we have seen so far for this track.

In this tutorial, I have shown you the four main ways of diagnosing track quality: using the zoom window, the center viewer window, the Curve Editor, and the traffic light warning system (show error on track). Being able to spot when and why a track has failed is extremely useful and will save you a lot of time in the future. On top of that, it will help you get the hang of the new keyframe-based tracking workflow that we will present in the next tutorial.

Hope you enjoyed this, thanks for watching.

Hi, my name’s Dan Ring and in this tutorial I’m going to be introducing the new keyframe-based tracking workflow. This tutorial relies on a few things that were introduced in previous tutorials, so if you find you don’t quite get something, it’s probably worth going back to previous tutorials and having a look through them.

The idea behind keyframe tracking is to lay down keyframes and let the Tracker figure out what happens in between them. Let’s create a new track. This creates our first new keyframe. Then I will jump 25 frames and move the track in order to create a new keyframe. You can see we now have a pair of keyframes. Now, just to make sure both these image patches are the same, I will click on the keyframe patches. You can see that the border of the keyframe turns orange if we are on the same frame as that keyframe. If you look at the zoom window, you can see it’s pretty much in the right spot.

To start the keyframe tracking, we go to the top and click the key track current button (Shift+C). This starts the keyframe tracking between the current pair of keyframes. You can see the keyframe track proceeding forwards in the shot, and although it looks like it’s doing the same thing as the auto-track we showed earlier, it is actually trying to match the current pattern window against both search windows of the keyframes, but also keeping the track position of the two keyframes locked. This keyframe tracking workflow if quite powerful, and in particular it allows us to set up our entire track in advance. So, I’m going to do just that and skip on another 25 frames. Create another keyframe, using the zoom window to get a nice and precise lock on the nose. Skip back and forth to ensure we have the same position on each of the keyframes. I will set up 5 keyframes for this example. Now, if we want to track all of the keyframes, use the key track all button (Shift+V). This finds each pair of keyframes and tracks them sequentially, for example, these three pairs. I will also turn on the center viewer button. This allows us to see how the track is doing better.

So, already you can see just how powerful this new keyframe tracking workflow is for setting up your whole track in advance, but the really interesting things about this keyframe tracking workflow are yet to come. So let's now delete this track and try something more difficult.

Now, for this trickier example I am going to try and track the tattoo on the girl’s arm. Zoom in and move the Tracker to the top circle of the tattoo, and then adjust the pattern window, so it captures most of the top of the tattoo. I will also make the search window larger, so it captures the motion of the arm. I will start off with an auto-track and see how far that goes. We can look at it in the zoom window, and as you can see, it has already drifted. If you remember from the previous tutorial, we can turn on the traffic light system (show error on track) to see exactly where the track went wrong. To get a better look at our track, I am going to enlarge the zoom window. You can see around the orange and red dots, the track has really drifted. So, what we could do, is go back to our last good frame, click track forwards and hope that it works this time. But we want a better and more deterministic way of solving this problem. To do this, we can use keyframe tracks.

Click the set keyframe button, which adds a keyframe at frame 101. We can then jump onto a later frame, for this example I’m going to use frame 128. I will move the Tracker to the top circle of the tattoo. By doing this, we create a new keyframe. Now that we have set up two keyframes around the selected area, from frame 101 to frame 128, I am going to click the key track current (Shift+C) button to track over that difficult region. Instantly, from that error we can see the Tracker has done a much better job. We have a nice, smooth change between the colors of red and green. But, that red area is still not perfect. The really useful thing is now, because we have already made a key track, if we created a new keyframe or nudged the Tracker so it creates a new keyframe, the Tracker will then re-track the affected areas in using the new keyframe. I’m now going to move the tracker to the top section of the tattoo. This creates a new keyframe at frame 113, and then starts re-tracking between our new pairs of keyframes at frames 101 and 113, and 113 and 128. The area that was red has now turned green, but this has flagged up a new red area. You see, the track is not quite on the circle of the tattoo, so move the Tracker to where it should be, let go, and allow it to re-track. Instantly, the red frames have turned green. If you scrub along the timeline, you can see that this track is much better all round, so keyframe tracking has really helped us out here.

To give you a better idea of what happens when you re-track, recorded in the last tutorial we looked at the track’s error value using the Curve Editor. This is the Error Curve for our track. You can see a couple of high peaks for our track. These correspond to the yellow frames around here and the animation path. If we scrub to each of these, we can have a look to see what the track is doing. Looking at the zoom window for these two peaks here, you can see the track is a little bit off, but I’m not too concerned about that at the moment. If we go here, you can see the track is way off. It has completely drifted. What we can do now is, again, pull the track into place and allow it to re-track. Now, we have a nice smooth track. It is important to note the height of the Error Curve has now been chopped in half. I will just undo that, so you can see the old curve. You can see we chopped that mountain of error in half. This is a nice way to think about keyframe tracking: every time you move a track, you create a new keyframe. You are imparting new information into the system, which gives you a better overall track. You do this incrementally, so every time you touch the track you improve it. This is a very similar idea to how rotoscoping works: every time you move a control point or create a new keyframe, you end up with a better overall matte.

This concludes our tutorial on keyframe tracking, I hope you enjoyed it. Thanks for watching.

Hello, and welcome to this tutorial on keyframe tracking in Nuke 7.0. We have a nasty shot here - there is a lot of movement and it also looks like there’s some rolling shutter and there are two bars that wipe all the way through the whole of the shot. Before going to keyframe tracking, I would just like to approach this the normal way. I will turn off this middle icon here which creates keyframes. You would always get a keyframe on the first frame you track on, so press Shift and create your first track. The image is moving around a lot, so increase your search area and then track forwards and see what happens. And sure enough, as the bar comes through the shot, the track bombs out completely.

So let’s see how we would approach this with keyframe tracking, so bring up the Tracker as normal. We would start off in exactly the same way as we would with normal tracking. I will use the Shift key to start a track. Now I have my first keyframe on frame one. As I scroll across the timeline, you can see the bar will come and wipe through, so we know from experience that will not track. I will put a second keyframe here, and now I have two keyframes. These keyframes are giving the Tracker two different patterns to search between, so it knows what’s coming up, effectively. I am going to create another keyframe, right on the last frame of the occlusion. I will create a keyframe so the second pattern is clear, go forward a frame, and create one more keyframe. I have created four keyframes around the occlusion: my last good frame, my first next clean frame, and then two that are problem frames in the middle. Now the problem frames in the middle would throw the tracker off normally, so we would track with an offset at this point. Obviously, offset trackers only work if the area you are trying to track is within the same plane as the occluded area. Now I will adjust those two patterns to be as close as I can, and go to frame 22. Frame 22 still has an offset, and I have keyframed the offset. As soon as you don’t require the offset, use the clear offset button to get rid of it. So now we have keyframes either side of that occlusion.

We head down to a bit further down this shot, create another keyframe, again to help the Tracker. As this boat comes towards us, the pattern it is trying to track has changed, so we will just nudge it along to find the next pattern. I will do the same further down, and I go to the end and give it one last keyframe. Now, I haven't forgotten about the wipe that occurs in the middle, so the beauty of this is I don't need to do this in order, I can set keyframes as I need to. This updates and nudges the other keyframes further down. We are going to do the same thing: I am going to use my last good frame, my next frame which has a problem in it, my next good frame, and I will use my last problem frame. So, good frame, problem frame, problem frame, and good frame. And then offset on the problem frames, check my offsets are good and patterns match. Go forward a frame and I will turn off the offset, so good frame, bad frame with offset, bad frame with offset, and good frame. So, moving down now I have a keyframe quite close to the other ones, so just go to the top and remove the keyframe. Now I will set my keyframes throughout the whole shot, as there is quite a lot of movement throughout the shot.

Now, with the old Tracker I would have been able to adjust the search window and that would have worked for the whole shot, but because this is keyframe tracking, the search area is also keyframed and it only adjusts it on the frame I am on. So, consequently I would have to go through the entire lot and adjust them. Anyway, let’s see how it gets on with the search area that it has. Instead of using the normal tracking button, I will use this new button, key track all (Shift+V). It will track the entire shot. We have one next, key track current (Shift+C). This will track the keys either side of the keyframe you are on. So if I will have a keyframe on frame 1 and a keyframe on frame 30, and I was on frame 20, it would track from frame 1 to 30, for example. In this case, I want to track all of them, so I will press go. Now I want to check my track. What I like to do is use the zoom window and scroll through and see how it’s gone. It’s done a pretty good job all the way through. I can also tell on the traffic light system (show error on track), and this shows me green for low errors, red for high errors, and yellow for in between errors. An error does not necessarily mean a bad track. It can mean there was a good track, but it found the pattern to be different but still got the track correct. You need to check to see if the error is actually causing you an issue. I can see my track has gone wrong, so what can I do about it? The first thing I can do is figure out why it has gone wrong. It is clear my search area is not really wide enough at this point. Now, because I have this function on, re-track when keyframe is moved, this means if I make any adjustments to the Tracker between track keyframes, it will automatically fire off a re-track. My nearest frames are frame 100 and 128. I am just going to expand the search window at this point, and it has kicked off a re-track between those keyframes. Frame 19 has been added in the middle there.

Now I can see it has given me a good track for those bad frames. So the proof is in the pudding, so lets stabilize it. I am using Stabilize 1pt here, as if I only have 1 tracker. It will give us the best result, as it basically locks the filter to keys. It’s done a pretty good job of those occlusions. Now the shot is stabilized, we can see how bad the rolling shutter is, so it’s possible we could have sent this shot through correction before working on it. With keyframe tracking, we are effectively nudging the tracker into the right direction throughout the shot to achieve a good track.

This is Howard Jones for The Foundry, goodbye.

Hello! In this tutorial, I would like to show a combined workflow between the Tracker, CameraTracker, and Roto. Here we have a hand-held shot, essentially I would like to get a camera track out of this, but the people will interfere with the camera track, so we need to roto them out. I will use keyframe tracking to get a rough track of their positions. I don’t need to be particularly accurate, as I am only going to garbage matte out the people.

So, I have created four keyframes on my first track, first track done. Then I create a second track on this guy here, go through and make keyframes, and click key track all. The last guy is more problematic because he gets occluded. Now, I don't really need to worry too much about dealing with occlusion the way I would have done before. I am just going to switch to a different pattern halfway through and then go back because again I only need a rough garbage matte throughout. Let’s see how that does. So I am expecting this track to jump a bit. The question is if it jumping is going to cause me an issue or not? I think that will be pretty good.

So I will bring up a Roto now, and just do a quick garbage around the first guy. I will put him into his own group, and then I will select Link to > Tracker 1 > the first track. It immediately pushes the roto off, so I’m going to use its own transform to push it back again. Again, do the same for the second guy. I accidentally put him in the wrong group, so move him back to Root and create new layer, again go to Link to > Track 2, go back to its own positioning, and put the roto back. Lastly I deal with the third guy, remember this is more of a problematic track and it’s less accurate. Again I put him in his own group, then Link to > Track 3, and again put the shape back where we started. As we can see, that track is good enough to cover him throughout again, as we are only garbage matting him.

Now, one other point, what I need to do is ensure I have a track point up at the top here, within my CameraTracker. Now you can see tracks within my CameraTracker, but these don’t always remain, so I’m going to use a User Track to isolate this point. I need to be a bit more accurate with my keyframes, as I want to place CG there. I have only keyed 60 frames, not the whole lot this time, but I will key track them anyway and just track those 60 frames. Looking at the traffic lights, we can see the issues caused by motion blur, and actually even though it has gone red it’s a good track.

I’m going to bring up a CameraTracker now. You want to mask out the people, so go to Mask > Mask Alpha. We also want to create a User Track. Just expand this out, so you can read the buttons. Click the Add User Track animation button, then Link to > Track 4. However, I find the CameraTracker does not work if it is a link, so I will turn this into keyframes: Edit > Generate > Generate Keys from 1 to 60. I have a problem at the end when the interpolation carries on, so I will go to Interpolation > After/Constant, and in the CameraTracker we will use that information. Now click Track Features. Essentially, we are trying to get a lock on that point, so we know we can put an object there later on. Now that we have a point that we can create an object for, right-click on the point and select create > sphere. So lets go back into the CameraTracker and create a Camera, bring up a Scene, and through this we will be able to check the integrity of this track.

So, through using the Tracker in a combination with Roto we have been able to garbage out the people, and we have also added a User Track to the CameraTracker, so we have been able to put an object in a position of our choice.

This is Howard Jones for The Foundry, goodbye.