Planar Iop: Image Processing with 2-Dimensional Outputs

Nuke also has support for Planar Iops, in addition to the usual Row-based Iops. These are primarily intended for Iops that find producing their output in single rows to be unnatural, and would prefer instead to output data containing multiple rows at a time (either the entire image, or smaller stripes.)

Planar Iops have two levels of abstraction: there is a core PlanarI interface, which provides a mechanism that PlanarIop then uses. We encourage users to implement PlanarIop rather than PlanarI, as PlanarIop provides additional caching, without which performance can be impaired.

Render Stripe

The main rendering function for a PlanarIop is renderStripe():

void renderStripe(ImagePlane& imagePlane)

This is the equivalent to engine() for a normal Iop, but is required to fill in a two-dimensional “stripe” of the output image, rather than a single row.

The implementation of renderStripe() should output a rendered stripe to imagePlane. This image plane will already have been set up with the required bounding box and channels; all the implementation needs to do is to fill in the output data for this region and channels.

By default, renderStripe() will be called for the entire bounding box of the Iop, as set by _validate(). This is usually the most efficient way to process for a plugin that takes the entire input image as a single input and produces a single output image. However, since the majority of Nuke nodes do row-based processing, users often don’t expect to have to wait for a whole frame’s worth of data to be calculated before they begin to see any output. A node that requires the whole of its input to be calculated before it can start to produce data might not give the fast feedback they would like, particularly when there is a large and complicated node graph above it. In addition, if a node requires a large amount of memory for its processing or needs to process on a device with a relatively small amount of memory, such as a GPU, it might not be appropriate or even possible to process the whole output area at once.

For the reasons above, a PlanarIop can choose to render in stripes rather than render the whole image in one go. To do this you would implement the function

bool useStripes() const

to return true.

You can also implement the function

int stripeHeight() const

to set the height of the render stripes. The stripes will then be of the height returned by stripeHeight(), apart from the final stripe (furthest from the origin), which will be clipped to the bounding box size if necessary. The stripes will always be the full width of the image, in order to interoperate nicely with the Row-based operators in Nuke.

Note

Unlike engine(), renderStripe() is called on a single thread at a time, so if you require multi-threaded processing inside a PlanarIop then you must implement this yourself. The PlanarIop interface was designed to support custom multi-threading by plug-ins which don’t fit well with Nuke’s standard scanline-based approach, or plug-ins which do their processing on a different device such as a GPU. In the past, such plug-ins had to implement locking inside engine() in order to launch their custom processing on a single engine thread, and further implement the reading back of scanlines from the output buffer they produced, as well as caching that output buffer internally somehow. With the PlanarIop interface, now all you need to do is implement the renderStripe(ImagePlane& imagePlane) function, and Nuke will take care of caching the output plane produced and then reading back scanlines from it as required.

Request

Important: PlanarIops should not implement _request(). Instead, they should implement getRequests(), which is a similar, but non-recursive, function:

virtual void getRequests(const Box& box,
                         const ChannelSet& channels,
                         int count,
                         RequestOutput &reqData) const

This will be used for more complex render management in future. In addition, for PlanarIops only getRequests() is guaranteed to be set up correctly for planar processing; _request() might not be called with the correct information, so we strongly recommend that you don’t try to implement it for a PlanarIop.

Channels and Layers

A PlanarIop will get a separate render stripe call to fill in each set of channels that go to make up an image plane. The default behaviour is to have a plane for each layer, e.g. forward and backward motion vectors will be stored in a single image plane. An image plane will never contain more than four channels.

The following function determines how the channels will be allocated to image planes:

virtual PlaneID PlanarI::getPlaneFromChannel(Channel chan);

Since it is virtual, a plugin can change the default behaviour and decide on its own definition of the image planes by overriding this function.

PlaneID is a presently a typedef to ChannelSet (this might change in subsequent versions). The default implementation of getPlaneFromChannel() implements the following mapping

Chan_Red	Mask_RGBA
Chan_Green	Mask_RGBA
Chan_Blue	Mask_RGBA
Chan_Alpha	Mask_RGBA
Chan_U	Mask_MoVec
Chan_V	Mask_MoVec
Chan_Backward_U	Mask_MoVec
Chan_Backward_V	Mask_MoVec

As an example, if you wanted the forward and backward vectors to be calculated separately, you could override getPlaneFromChannels() to provide the following mapping

Chan_Red	Mask_RGBA
Chan_Green	Mask_RGBA
Chan_Blue	Mask_RGBA
Chan_Alpha	Mask_RGBA
Chan_U	Mask_MoVecForward
Chan_V	Mask_MoVecForward
Chan_Backward_U	Mask_MoVecBackward
Chan_Backward_V	Mask_MoVecBackward

By default, a render stripe call will not necessarily be asked to fill an image plane containing all of the channels that go to make up a plane, but only those channels that have been requested (by one or more Ops below the PlanarIop in the node graph). If your plugin needs to render all the channels in a plane (or would find it more efficient to do so), you can override the following function to return true:

virtual bool renderFullPlanes() const;

When renderFullPlanes() returns true, the plugin will never be given an image plane to fill that contains less than the full set of channels.

Packed Preference

There is a further virtual function you can implement

virtual PackedPreference packedPreference() const

The return value of this indicates whether the Iop prefers to produce packed or unpacked output planes. If it returns ePackedPreferenceNone it has no strong preference and renderStripe() can be called with either depending upon which the calling code finds more convenient; if it returns ePackedPreferencePacked or ePackedPreferenceUnpacked then renderStripe() will only be called to fill a packed or unpacked stripe.

PlanarI

We do not encourage plugins to use this directly; rather, they should inherit from PlanarIop.

Image Planes

The ImagePlane class is at the heart of the Planar code. ImagePlanes can be of various formats, with different bounding box, channels and packness. The bbox can be obtained with the accessor function bounds(); the channels with channels() and the packness with packed().

In all modes there is no padding. For example, in packed mode, with a 2x2 image, with RGBA channels, the floats would be

red 0,0	green 0,0	blue 0,0	alpha 0,0
red 0,1	green 0,1	blue 0,1	alpha 0,1
red 1,0	green 1,0	blue 1,0	alpha 1,0
red 1,1	green 1,1	blue 1,1	alpha 1,1

whereas in unpacked mode the data is represented by

red 0,0	red 0,1	red 1,0	red 1,1
green 0,0	green 0,1	green 1,0	green 1,1
blue 0,0	blue 0,1	blue 1,0	blue 1,1
alpha 0,0	alpha 0,1	alpha 1,0	alpha 1,1

ImagePlane provides several accessor functions for its data.

Low-level access

ImagePlane::readable() points at the start of the allocation. For the examples above, this will point at the red at (0, 0).

rowStride(), colStride() and chanStride() can then be used to multiply y, x, and channel values, to add to these. For example, in the first,

colStride()	4
rowStride()	8 (2 * 4)
chanStride()	1

and in the second

colStride()	1
rowStride()	2
chanStride()	4

data() points at the bottom-left corner of the image, which is not necessarily at the origin (0, 0).

There is no support for images with origins at the top-left.

Higher-level access

You can access individual pixels with

const float& ImagePlane::at(int x, int y, int z)

z is the channel number /within/ the ChannelSet for the ImagePlane, i.e. for a Mask_RGBA image plane then red = 0, green = 1, blue = 2, alpha = 3. If it was just red, green and alpha, then, then this would make red = 0, green = 1, alpha = 2.

The function ImagePlane::chanNo() can be used to look this up but this iterates over the ChannelSet and you should avoid calling it within a tight loop.

There is also a

const float& ImagePlane::at(int x, int y, Channel z)

which does the lookup of the Channel for you.

These functions give references to the actual image data. If you want to abstract some of that away, there are functions that return strided pointers.

const ImageTilePtr ImagePlane::readableAt(int y, int z);

This returns a pointer to (0, y, z). The pointer is strided: so it knows what chanStride() is, so that ++ increments it to the next x value.

Note that this returns a pointer to (0, y, z) even if the x bounds of the origin start rightwards of the origin. This means that the pointer cannot be safely deferenced without incrementing by (at a minimum) the x bounds. This is there to make multiplying in the offset slightly easier, and is consistent with the way that Row::operator[] works.

Writing to ImagePlane

ImagePlanes use shallow copying when possible.

If you are writing to an ImagePlane that might be shared with what ought to be read-only copies then you should call

void ImagePlane::makeUnique()

If you are writing to a new ImagePlane that you made, you should call

void ImagePlane::makeWritable()

One pattern that is valid, for example, is

void renderStripe(ImagePlane& outPlane) {

  input0().fetchPlane(outPlane);

  outPlane.makeUnique()

  Box box = outPlane.bounds();

  foreach(z, outPlane.channels()) {
    for (Box::iterator it = box.begin(); it != box.end(); it++) {
      outPlane.writableAt(it.x, it.y, outPlane.chanNo(z)) *= 2;
    }
  }
}

If there is another copy that shares backend data then it will copy the data and then make the modifications in-place. If there is no extra copy that is needed, because it has decided that the input does not need caching (perhaps it is really cheap to calculate), then it will not need to even do a copy, thus reducing peak memory use.

If you want to build an output from scratch, you can do something like:

outPlane.makeWritable()

foreach(z, outPlane.channels()) {
  for (Box::iterator it = box.begin(); it != box.end(); it++) {
    outPlane.writableAt(it.x, it.y, outPlane.chanNo(z)) = 0.5;
  }
}