Writing a 3D Plugin

Writing a plugin for the new 3D system is quite similar to the old way, but there are some important differences. We find that plugins can be ported from the old to the new system fairly easily though.

We said earlier that an op passes a USD stage down the graph, but this was a simplification and isn’t actually true. In fact, what actually happens is that 3D ops make USD layers and these get composed to a stage on request. The current layer stack is what is passed between ops. If we have a chain of 3D ops, each op makes a new layer and these are all stacked up and composed into a stage. Each layer defines new prims or overrides existing prims. Composing a stage can be expensive, and so we only do it when necessary.

Time in Plugins

The treatment of time in USD is the biggest thing that needs to be taken into account when writing a plugin. The challenge is that Nuke works frame by frame, but a USD layer contains time samples for the whole duration of its existence and these samples need not coincide with Nuke frames. A mesh which is animated would need to produce a new time sample for each frame, but one that is not animated would only need to produce the mesh once, for the “default” time. To accommodate this, a 3D plugin in the new system is split into two parts, the Nuke Op and a GeometryEngine. The Op does all the work of creating knobs and so on, while the GeometryEngine does all the geometry work. Unlike an Op, a GeometryEngine has access to all the time samples for the Op’s knobs and can make decisions based on that information in order to make suitable time samples for the requested frames.

This is similar to the old mechanism of geometry hashes, but can be as finely-grained as the op requires. As an example, consider a primitive op such as Sphere. Changing the radius of the sphere affects the point locations but not the number of faces, while changing the number of rows changes both. The op might therefore decide that if the only animated knob is the radius knob, it can create time samples for points, but not for faceVertexCounts and faceVertexIndices.

SourceGeo Plugins

The equivalent of SourceGeo in the new system is SourceGeom. Porting a SourceGeo plugin to SourceGeom is relatively straightforward. Here’s a quick overview:

1. Change the superclass to SourceGeom

2. Define an engine class which inherits SourceEngine

3. Add a call to the constructor to build the engine

4. Remove the geometry_hash method and in the knobs method for every knob which affects the output geometry add a call to KnobDefinesGeometry.

5. Also in the knobs method, add a call to makeTransformKnob if you want one.

6. Move geometry validation code from _validate to appendGeomState.

7. Remove the create_geometry method and move all the geometry creation code into the engine class.

8. For every knob that affects the output, retrieve its KnobBinding.

9. Loop through all the requested time samples, fetch the KnobBinding values at those times and create prims in the op’s edit layer.

See the sample code GeoTriangle.cpp for an example of an op which creates geometry.

ModifyGeo Plugins Plugins

The porting procedure here is very similar. The superclass, ModifyGeom, will take care of handling the mask knob and will provide the geometry engine with a list of prim paths which you should operate on.

1. Change the superclass to ModifyGeom

2. Define an engine class which inherits ModifyEngine

3. Add a call to the constructor to build the engine

4. Move geometry validation code from _validate to appendGeomState.

5. Remove the geometry_hash method and in the knobs method for every knob which affects the output geometry add a call to KnobModifiesAttribValues.

6. Remove the modify_geometry method and move all the geometry creation code into the createPrims method in the engine class.

7. For every knob that affects the output, retrieve its KnobBinding.

8. Loop through the supplied prim paths. Fetch the existing prim for each path and any attributes you need, create an override of the prim and set its attributes.

See the sample code GeoTwist.cpp for an example of an op which modifies geometry.

Material and Shader Plugins

Material nodes have also been changed for Nuke 14 and, therefore, there are now different aspects to consider when writing Material node plugins. As previously mentioned, Nuke supports different materials for different renderers. In practice this is achieved by generating different shader networks for different render targets (for example, ScanlineRender and Arnold).

The Ndk API provides classes to describe Material Networks and Shaders:

• ShaderDescGroup

• ShaderDesc

• ShaderConnection

• ShaderProperty

The ShaderDescGroup provides the notion of a shader graph which consists of a collection of individual shaders represented by ShaderDesc objects. In turn, ShaderDesc objects can have multiple inputs with default values and can be connected to the output of other ShaderDesc objects through a ShaderConnection, overriding its default value. These inputs and outputs are generally referred to as properties and are implemented by ShaderProperty objects.

The first step to write shader plugins is to create a subclass of ShaderSchema which informs Nuke of the properties, source types (e.g. glslfx) and metadata the shader plugin provides. An instance of the ShaderSchema subclass needs to be registered with Nuke and the subclass constructor needs to pass the properties and metadata to the ShaderSchema constructor. This can be done like in the following example.

using namespace fdk;
using namespace usg;

static const ShaderPropertyArray properties = {
    { InputProp, Value::Float4, "A", Value(Vec4f(0,0,0,1)) },
    { InputProp, Value::Float4, "B", Value(Vec4f(0,0,0,1)) },
    { InputProp, Value::Int, "operation", Value(1) },
    { OutputProp, Value::Token, "surface", KeyValueMap{{ "renderType", "terminal surface" }} }}
};
static const KeyValueMap metadata = {};

class MyShaderPlugin : public SurfaceShaderSchema
{
public:
    static const SchemaDescription description;
    MyShaderPlugin() : SurfaceShaderSchema(description, properties, metadata) {}
};

static Schema* schemaBuilder() { return new MyShaderPlugin(); }
/*static*/ const SchemaDescription MyShaderPlugin::description("MyShaderPlugin", schemaBuilder);

The network of Nuke material nodes that is applied to a geometry is translated to a shader graph using the classes described above, where each individual Nuke material node usually corresponds to a single ShaderDesc object. Similarly, inputs to Nuke nodes correspond to ShaderConnection objects.

To achieve this (and as a second step) material plugins should subclass DDImage::ShaderOp class and override its createShaderGraph method to create a ShaderDesc. This class provides a static method to create an instance based on the SurfaceShaderSchema created in the first step. The returned object has all the input and output properties initialised to the default values. This greatly simplifies the process of creating the shader graph and makes the code easy to read. From the createShaderGraph method it is possible to call createShaderGraphFromOp with the input material op to create the shader graph for the upstream nodes. It is, then, easy to connect the output of the upstream ShaderDesc to the current one. To support different renderers (e.g. ScanlineRenderer and USD’s HdStorm), it is possible to generate different shader graphs. The following example puts together all of these concepts.

usg::ShaderDesc* createShaderGraph(int32_t                   output_type,
                                   const usg::RenderContext& rtx,
                                   usg::ShaderDescGroup&     shader_group) override
{
    // Build the shader name and check if it already exists.
    // This can happen if another MaterialOpI in the graph has the same input connection
    const std::string shader_name = getShaderNodeName(rtx.target);
    usg::ShaderDesc* shader_desc = shader_group.getShaderNode(shader_name);
    if (!shader_desc) {
        shader_desc = usg::ShaderDesc::createFromSchema(getOutputSchema());
        if (shader_desc) {
            usg::ShaderDesc* Bin = createShaderGraphFromOp(input(0), rtx, shader_group);
            usg::ShaderDesc* Ain = createShaderGraphFromOp(input(1), rtx, shader_group);

            // shader graph for ScanlineRenderer
            if (rtx.target == usg::GeomTokens.slrRenderContext) {
                // Connect Node/Op inputs to shader properties:
                shader_desc->connectInput("B", Bin, "pixel");
                shader_desc->connectInput("A", Ain, "pixel");
            }
            // shader graph for HdStorm
            else if (rtx.target == usg::GeomTokens.glslfxRenderContext) {
                // Connect Node/Op inputs to shader properties:
                shader_desc->connectInput("B", Bin, "surface");
                shader_desc->connectInput("A", Ain, "surface");
            }
            shader_group.addShaderDesc(shader_desc, shader_name);
        }
    }
    return shader_desc;
}

Certainly, to have a useful material plugin, some of the ShaderDesc input properties will need to have their values set from knobs. This is achieved by overriding the SurfaceMaterial updateShaderGraphOverrides method. This is easily achieved as the following example illustrates.

void updateShaderGraphOverrides(int32_t                   output_type,
                                const usg::RenderContext& rtx,
                                usg::ShaderDescGroup&     shader_group) override
{
    // Update inputs first:
    updateShaderGraphFromOp(input(0), rtx, shader_group);
    updateShaderGraphFromOp(input(1), rtx, shader_group);

    // Copy local knob values to schema instance:
    const std::string shader_name = getShaderNodeName(rtx.target);
    usg::ShaderDesc* shader_desc = shader_group.getShaderNode(shader_name);
    if (shader_desc) {
        overrideShaderDescInput("operation",  // local knob name
                                *shader_desc, // our ShaderDesc
                                "operation"); // ShaderDesc property name
    }
}

Finally, once a material plugin has created the shader graph and updated its overrides, Nuke 14 is able to generate and insert Material and Shader prims in a layer that will be composed into the final stage ready to be rendered.

Rendering with Material Plugins

So far, we have described how a material plugin is converted into a ShaderDesc and inserted as a component of a larger shader graph but we haven’t mentioned how material plugins can perform their work. Supporting different render targets means that material plugins will have to provide different implementations. In this document we describe how material plugins can provide implementations for the Scanline Renderer and for HdStorm.

Scanline Renderer

ScanlineRender has been versioned-up to ScanlineRender2, updated to be compatible with USD material networks and render the standard USD preview shader networks. It no longer relies on the Iop and Material classes in DDImage as part of the shading infrastructure and now uses the dedicated SlrShader class. SlrShader plugins are usually the concrete implementations of defined ShaderSchemas, and ScanlineRender2 uses the USD ShaderPrim info:id attribute to search for the matching named ‘slr’-prefixed SlrShader plugin. For example if a ShaderPrim info:id name is UsdPreviewSurface then ScanlineRender2 will prepend slr to the name and search for a shader plugin named slrUsdPreviewSurface. After the SlrShader plugin has been loaded and a local instance created, the ShaderPrim property connections and overrides are translated to the instance.

SlrShader is further specialised to SlrCoShader, SlrSurfaceShader, SlrGeometryShader and SlrLightShader which have the specific interfaces for those shader types. Currently SlrCoShader and SlrSurfaceShader are the most functional and the following example illustrates the basics of a simple shader that combines the results of two upstream shaders:

namespace slr {

class MyShaderPlugin : public SlrSurfaceShader
{
public:
    SlrSurfaceShader* inputB;
    SlrSurfaceShader* inputA;
    int32_t           operation;

public:
    static const PluginDescription description;
    static const InputKnobList  input_defs;
    static const OutputPortList output_defs;
    const char* shaderClass() const override { return description.shaderClass(); }
    const InputKnobList&  getInputKnobDefinitions()  const override { return input_defs;  }
    const OutputPortList& getOutputPortDefinitions() const override { return output_defs; }

public:
    MyShaderPlugin();
    void  updateProperties(const SlrRenderContext& slrtx,
                           const usg::ShaderDesc&  shader_desc) override;
    void  surfaceShader(SlrShadingContext& stx,
                        DD::Image::Pixel&  out) override;
};

static SlrShader* shaderBuilder() { return new MyShaderPlugin(); }
/*static*/ const SlrShader::PluginDescription MyShaderPlugin::description("MyShaderPlugin", shaderBuilder);

/*static*/ const SlrShader::InputKnobList MyShaderPlugin::input_defs =
{
    { "B"         , PIXEL_KNOB },
    { "A"         , PIXEL_KNOB },
    { "operation" , INT_KNOB   },
};
/*static*/ const SlrShader::OutputPortList MyShaderPlugin::output_defs =
{
    { "pixel", PIXEL_KNOB },
};

MyShaderPlugin::MyShaderPlugin() : SlrSurfaceShader(input_defs, output_defs) {}

void
MyShaderPlugin::updateProperties(const SlrRenderContext& slrtx,
                                 const usg::ShaderDesc&  shader_desc)
{
    // Bind input connections or assign fallback defaults from ShaderDesc:
    shader_desc.getInputValue("operation").get(operation, 0/*default fallback*/);
}

/*virtual*/ void
MyShaderPlugin::surfaceShader(SlrShadingContext& stx,
                              DD::Image::Pixel&  out)
{
    // Note that we're ignoring the 'operation' parameter to simplify example
    if (!inputB)
    {
        out.erase();
    }
    else
    {
        inputB->surfaceShader(stx, out); // copy B shading results to output

        DD::Image::Pixel A(out.channels); // fyi creating Pixels in shader call is currently very expensive
        inputA->surfaceShader(stx, A);
        // Add A shading results to output:
        foreach(z, out.channels) {
            out[z] += A[z];
        }
    }
}

}

Hydra and HdStorm

To be able to visualise materials in the Hydra Viewer, and more specifically using HdStorm, we need to provide glslfx code with which HdStorm can render the objects. This brings us back to the SurfaceShaderSchema subclass. The first step is to override the getSourceTypes method to declare that the plugin provides glslfx code. The sourceCode method should use the ShaderSource interface to generate a glsl function that will be part of a larger glslfx shader and called at the appropriate point and with the correct inputs. The ShaderSource interface provides methods that help in generating shader source code. All of this can be done like the following example.

class shdMergeLayers : public SurfaceShaderSchema
{
public:
  static const SchemaDescription description;

  shdMergeLayers() : SurfaceShaderSchema(description, properties, metadata) {}

  //! Declare our intention to provide glslfx source code.
  void  getSourceTypes(TokenSet& types) const override
  {
    types.insert(GeomTokens.glslfxSourceType);
    types.insert(GeomTokens.customGlslfxSourceType);
  }

  const bool sourceCode(ShaderSource& shaderSource) const override
  {
    if (shaderSource.sourceType() == GeomTokens.customGlslfxSourceType) {
      shaderSource.startFunction();

      shaderSource.emitLine("vec4 A = vec4(" + shaderSource.input("Argb") + ", " + shaderSource.input("A_opacity") + ");");
      shaderSource.emitLine("vec4 B = vec4(" + shaderSource.input("Brgb") + ", " + shaderSource.input("B_opacity") + ");");
      shaderSource.emitLine("vec4 surfaceColor = A + B;");
      shaderSource.output("rgba", "surfaceColor");

      shaderSource.endFunction();
      return true;
    }
    return SurfaceShaderSchema::sourceCode(shaderSource);
  }

  const ShaderPropertyArray& shaderProperties() const override { return properties; }
};