PointCloudGenerator

Using the PointCloudGenerator node (NukeX and Nuke Studio only) you can create a dense point cloud based on the information generated by CameraTracker and use the dense point cloud further to create 3D meshes of your 2D footage. Dense point clouds are a useful starting point for 3D modeling and positioning objects in a scene.

Note:  The analysis must be performed in NukeX, but the result can be used in Nuke.

Inputs and Controls

Connection Type

Connection Name

Function

Input

Camera

The camera track of the scene from which you intend to generate the dense point cloud.

Mask

An optional image to use as a mask. By default, the point cloud is limited to the non-black areas of the mask.

At first, the mask input appears as triangle on the right side of the node, but when you drag it, it turns into an arrow labeled mask. If you cannot see the mask input, ensure that the mask control is disabled or set to none.

Source

The source footage from which the camera information was obtained.

Control (UI)

Knob (Scripting)

Default Value

Function

PointCloudGenerator Tab

display

display

solid

Adjusts the display characteristics of points. These settings don’t affect the render output of the scene; these are for display purposes only in the 3D Viewer.

off - hides the points.

wireframe - displays only the outlines of the points.

solid - displays all points with a solid color.

solid+wireframe - displays points as solid color and includes their vertices.

textured - displays only the surface texture.

textured+wireframe - displays the wireframe plus the surface texture.

selectable

selectable

enabled

When enabled, you can make selections as normal in the Viewer. When disabled, points cannot be selected or changed.

render

render_mode

solid

Sets how the points will render. This control is independent from the display selection, but has the same settings.

cast shadow

cast_shadow

enabled

When enabled, the object can cast shadows.

receive shadow

receive_shadow

enabled

When enabled, and a material shader is added to the object, the object can receive shadows.

Principal View

principalView

left

Sets the principal view where tracks are analyzed and the Mask input is applied. The secondary camera is calculated relative to this Principal View camera.

This control is only displayed if you have set up more than one view in your Project Settings (typically, if you’re working on a stereoscopic project).

Ignore Mask

ignore

None

Set the mask type to exclude areas of the sequence during generation:

None - none of the footage is ignored.

Source Alpha - use the alpha channel of the source clip to define which areas to ignore.

Source Inverted Alpha - use the inverted alpha channel of the source clip to define which areas to ignore.

Mask Luminance - use the luminance of the Mask input to define which areas to ignore.

Mask Inverted Luminance - use the inverted luminance of the Mask input to define which areas to ignore.

Mask Alpha - use the Mask input alpha channel to define which areas to ignore.

Mask Inverted Alpha - use the inverted Mask input alpha channel to define which areas to ignore.

Set Keyframes

Analyze Sequence

analyzeSequence

N/A

Click to automatically analyse the sequence and set the required keyframes to use when generating the point cloud.

Note:  Keyframes are selected to provide a large enough camera baseline to triangulate points, but you can add and delete keyframes manually, if necessary.

Keyframes

keyframes

0

Displays the frames used to track points in the input sequence. Both automatic and manual keyframes are shown.

Add

addKeyframe

N/A

Click to set a keyframe at the current frame.

Delete

deleteKeyframe

N/A

Click to delete the keyframe at the current frame.

Frame Spacing

frameSpacing

15

Sets the interval between keyframes when you click Add All. For example, the default value creates a keyframe every 15th frame.

Add All

addAllKeyframes

N/A

Click to set keyframes throughout the sequence at intervals specified by the FrameSpacing control. For example, for a sequence with 30 frames, the default spacing value creates 3 keyframes.

Delete All

deleteKeyframes

N/A

Click to delete all keyframes in the sequence.

Calculated Accuracy

accuracy

0

Displays the depth accuracy calculated at the current frame when using the automatic SetKeyframes method.

The accuracy rating can help you to select frame ranges when generating a point cloud - higher accuracy frames produce cleaner point clouds.

Dense Tracking

Point Separation

pointSeparation

2

Sets the separation value, in pixels, for points in the dense point cloud. Reduce the separation to create denser point clouds and vice-versa.

Track Threshold

matchThreshold

0.5

Sets the correlation threshold at which point tracks are terminated. Use a higher threshold to increase the accuracy in matching points between keyframes.

Track Points

track

N/A

Click to display a frame range dialog to determine which frames are used to calculate the point cloud. Use the standard Nuke notation, for example, 1-19, 40-60, 65-100.

Points can also be added to the point cloud in stages using different frame range selections. Use CalculatedAccuracy to determine the most accurate frames for generation.

Clear Points

clearPoints

N/A

Click to clear all tracked points.

Post Filtering

Display rejected points

displayRejectedPoints

enabled

When enabled, points that fall outside the Angle and DensityThresholds are highlighted in the 3D Viewer.

When disabled, all rejected points are removed from the Viewer.

Angle Threshold

minTriAngle

1

Sets the threshold for the minimum angle to triangulate 3D points (in degrees). Points with a large triangulation angle are more accurate. Set a threshold of 0 to triangulate all points. Increase the threshold to retain the more accurate points.

As a rule of thumb, anything below 5 degrees is likely to be incorrect.

Density Threshold

densityThreshold

0

Sets the threshold for the minimum density for points. Set a threshold of 0 to output all points and increase the threshold to highlight isolated points.

Delete Rejected Points

deleteRejected

N/A

Click to permanently delete points that are rejected by the Angle and DensityThresholds.

Tip:  To clean up the point cloud, try enabling Output points per frame and Display rejected points. You can then tune the Filtering thresholds and delete points at different frames.

Output

Point Size

pointSize

2

Sets the size, in pixels, of points displayed in the 3D Viewer.

Output points per frame

outputPerFrame

disabled

When enabled, only the points generated at the current frame are displayed in the Viewer.

When disabled, points from all tracked frames are displayed.

Groups Tab

Display groups in overlay

displayGroups

enabled

When enabled, group colors are displayed as an overlay in the 3D Viewer.

Output visible groups only

outputGroups

disabled

When enabled, only points designated as part of a group on the Groups tab are displayed in the 3D Viewer.

You can label the point cloud using Groups to help visualize different parts of the scene.

Create Group

createGroup

N/A

Click to add the current vertex selections to a new group. You can also use the Viewer right-click menu to create, delete, and modify groups.

Delete Selected Group

deleteGroup

N/A

Click to delete all selected groups and deselect the group in the point cloud.

To delete points use Vertex selection mode in the 3D Viewer and press the delete key.

Groups

groups

groups

None

Lists all groups in the point cloud and allows you to set the name, color, and visibility of each group.

To create groups, enable Vertexselection mode in the 3D Viewer. You can then select points and use the right-click menu to create a new group or add the selected points to an existing group.

Bake Selected Groups

bakeGroups

N/A

Click to bake the selected groups to a separate, baked point cloud.

Bake Selected Groups to Mesh

bakeMesh

N/A

Click to convert the selected groups to a mesh using Poisson Surface Reconstruction.

Mesh creation is based on the SGP 2006 paper, Poisson Surface Reconstruction by Michael Kazhdan, Matthew Bolitho, and Hugues Hoppe. The original source code and paper can be found at http://www.cs.jhu.edu/~misha/Code/PoissonRecon/.

Samples

samplesPerNode

1

Specifies the minimum number of sample points used to calculate mesh points, using floating point values.

For noise-free point clouds, lower values in the range 1.0 - 5.0 can be used. For more noisy samples, higher values in the range 15.0 - 20.0 may provide a smoother mesh.

Python Tab (These controls are for Python callbacks and can be used to have Python functions automatically called when various events happen in Nuke.)

before render

beforeRender

none

These functions run prior to starting rendering in execute(). If they throw an exception, the render aborts.

before each frame

beforeFrameRender

none

These functions run prior to starting rendering of each individual frame. If they throw an exception, the render aborts.

after each frame

afterFrameRender

none

These functions run after each frame is finished rendering. They are not called if the render aborts. If they throw an exception, the render aborts.

after render

afterRender

none

These functions run after rendering of all frames is finished. If they throw an error, the render aborts.

render progress renderProgress none These functions run during rendering to determine progress or failure.

Step-by-Step Guides

Creating Dense Point Clouds

Video Tutorials