Time & Stereo Ops¶
Architecture¶
This section gives an overview of the architecture and common functions required when dealing with temporal access and stereo.
split_input and inputContext¶
Nodes, as represented in the Node Graph (DAG), show the overall structure but do not themselves provide data. Instead, they’re being used to generate operations (Ops) that produce the actual 2D or 3D data. A node can be instantiated at a particular context, which possibly creates an Op on that node or possibly defers to another node. This context takes the form of a DD::Image::OutputContext node, which contains frame, view, and proxy scale information.
By default, Ops are time-neutral. They operate upon one input frame to produce an output frame at the same time.
If you wish to have an Op that works on a different input frame to that it produces (for example, a FrameHold), you need to implement inputContext(). Also, if you have an Op that uses two or more input frames (for example, a FrameBlend), then you need to override split_input.
These functions exist in the Op. The default implementations are as follows:
virtual int split_input(int inputNo) const
{
return 1;
}
virtual const OutputContext& inputContext(int n, int offset, OutputContext& oc) const
{
return outputContext();
}
When NUKE constructs the inputs for your Op, it calls these functions to determine which context or contexts to build those inputs for. By default, it creates one input for the Op for each input on the node, and that input Op is generated for the same time, frame, and view that your Op is itself being generated for.
Suppose you wanted to always access frame 1 on your input. You would then override inputContext like so:
virtual const OutputContext& inputContext(int n, int offset, OutputContext& oc) const {
oc = outputContext();
oc.frame(1);
return oc;
}
Note that the OutputContext reference parameter to inputContext is to a temporary working area only, and is not initialized. This implementation copies the Op’s outputContext to it, overrides the frame to return 1, and returns that.
If you were writing a FrameHold plug-in, you might have a knob that defined which frame to access. In this case, the inputContext function would look like this:
virtual const OutputContext& inputContext(int n, int offset, OutputContext& oc) const {
oc = outputContext();
oc.frame(_frameNumber);
return oc;
}
The definition of the _frameNumber knob would be:
void knobs(Knob_Callback f)
{
Float_knob(f, &_frameNumber, "frame");
SetFlags(f, Knob::EARLY_STORE);
}
It is important to set the EARLY_STORE flag on any knobs which inputContext, split_input, inputUIContext, or uses_input use, as otherwise the knob’s value will not have been stored by the time they are called.
If you wish to access two frames at once, you might have:
virtual int split_input(int inputNo) const
{
if (inputNo == 0)
return 2;
return 1;
}
virtual const OutputContext& inputContext(int n, int offset, OutputContext& oc) const
{
oc = outputContext();
if (inputNo == 0 && offset == 1)
oc.frame(oc.frame() + 1);
return oc;
}
This results in the first input of the node corresponding to two inputs on the Op: input(0) and input(1). If the node had a second input, the Op generated by this is placed at input(2).
It is possible to use a two-parameter version of Op::input() for easier access. The Ops generated for the first node input are returned by input(0, 0) and input(0, 1), and the Op generated for the second node input is input(1, 0).
Everything said here regarding times applies equally to views. It is also possible, although unusual, to change the OutputContext’s scale.
uses_input¶
Nodes might not use all of their inputs with certain knob settings. For example, a node with a “mix” setting to fade between two inputs might use both inputs usually, but only the first input if “mix” is 0, and only the second input if “mix” is 1. In this case, it’s not worth generating the unused input. To achieve this, you can override the uses_input() function. For example:
virtual float uses_input(int input) const {
if (input == 0) return _mix < 1;
if (input == 1) return _mix > 0;
return 1.0f;
}
The knob for _mix must, like knobs for split_input/inputContext, have the EARLY_STORE flag set on it.
inputUIContext¶
When a node’s control panel is open, it draws handles, and the evaluated values of knobs on it are shown. These handles and values must be shown for only one particular time, and not for all the times that the node is being pulled from.
The OutputContext used for this purpose is chosen during the Op generation process. The first context generated on that Op is usually used. This means that if an inputContext is implemented like so:
virtual int split_input(int inputNo) const
{
return 11;
}
virtual const OutputContext& inputContext(int n, int offset, OutputContext& oc) const
{
oc = outputContext();
oc.frame(oc.frame() - 5 + offset);
return oc;
}
Then the uiContext on the input node is set to frame - 5, rather than “frame”. This may not be desirable. One way to work around this is to re-order the inputs, so that input(0) always corresponds to the context that should be used. However, this can be result in complications with the code. Instead, it is possible to just implement inputUIContext as follows:
virtual const OutputContext* inputUIContext(int n, OutputContext&) const
{
return &outputContext();
}
Building The TemporalMedian Node¶
The TemporalMedian.cpp source is an excellent example of a node utilizing split_input to get access to multiple frames. As usual, grab the source and compile it up. Salient snippets are reproduced below:
class TemporalMedian : public Iop
{
public:
//...
// Tell it that the single input is now 3 inputs
int split_input(int n) const { return 3; }
//...
};
const OutputContext& TemporalMedian::inputContext(int i, int n, OutputContext& context) const
{
context = outputContext();
switch (n) {
case 0:
break;
case 1:
context.setFrame(context.frame() - 1);
break;
case 2:
context.setFrame(context.frame() + 1);
break;
}
return context;
}
//...
void TemporalMedian::engine ( int y, int x, int r,
ChannelMask channels, Row& row )
{
row.get(input0(), y, x, r, channels);
Row prevrow(x, r);
Row nextrow(x, r);
prevrow.get(input1(), y, x, r, channels);
nextrow.get(*input(2), y, x, r, channels);
foreach ( z, channels ) {
const float* PREV = prevrow[z] + x;
const float* CUR = row[z] + x;
const float* NEXT = nextrow[z] + x;
float* outptr = row.writable(z) + x;
const float* END = outptr + (r - x);
const float core = this->core[z <= Chan_Alpha ? z - 1 : 0];
while (outptr < END) {
// We use single letter variable names here to correspond with the
// text description of the algorithm above.
float A = *CUR++;
float B = *PREV++;
float C = *NEXT++;
float D = MAX(A, C);
float E = MAX(B, C);
float F = MAX(A, B);
float G = MIN(D, E);
float H = MIN(F, G);
float I = H - A;
float J = (I > core) ? MAX(2 * core - I, 0.0f) : I;
float K = (J < -core) ? MIN(-2 * core - J, 0.0f) : J;
*outptr++ = K + A;
}
}
}
As you can see, we simply set the split_input to indicate we always use three frames from the source input. Outputcontext is then set to return either the previous, current, or next frame depending on the state of the offset fed in, and the engine call uses input0, input1 and input(2) to return the data from the three frames respectively for processing.
Exercise: Build a Stereo Median¶
For this exercise, take the existing TemporalMedian and expand it so that the user can provide the desired temporal window (that is, frames) over which the median is to be performed.
For additional fun, add in the ability to set the center point of the temporal window (so that, for example, only trailing frames can be used, or only leading frames, or any mixture of the two). Now adapt it for the somewhat less useful task of providing a median between views, so that both are combined into one (whilst retaining the ability to have a temporal window of frames). This requires the context handling to set views as well as frames.