Introduction

Welcome to the .cat File Creation guide. This guide details how to convert your PyTorch models to TorchScript, and from TorchScript to a .cat file that can be used in the Inference node.

We start this guide by presenting a simple object detection model that can be easily converted to a .cat file using the CatFileCreator node in Nuke. We then build on this simple example in the subsequent sections to highlight some of the things you should bear in mind when writing a model that is to be converted to a .cat file. These include:

  • ensuring your model forward function has input and output tensors that are the correct shape

  • ensuring you have added any necessary normalisation/preprocessing steps to your model forward function

  • ensuring your model works with Nuke’s Use GPU if available and Optimise for speed and Memory knobs

  • ensuring your model is properly annotated so that it can be successfully converted from PyTorch to TorchScript

  • adding attributes to your model that can be controlled using a custom knob in Inference

  • how to reformat your model so that it only has one input and output tensor, which is a requirement when running your model in Nuke

For these scenarios, we also highlight how to set up the CatFileCreator and Inference nodes so that you can create and use your .cat file successfully in Nuke. Finally, we present two examples at the end of this guide that combine some of the topics discussed.

Important

To create a .cat file that can be run in all versions of Nuke from Nuke 13.x onwards, the following libraries must be used when converting a PyTorch model to TorchScript:

TorchScript models created using PyTorch 1.6 can be converted to .cat files using Nuke 13.x or Nuke 14. Note that TorchScript models intended to run in Nuke 13.x should be converted to .cat file using Nuke 13.x as those converted with Nuke 14 will not run in Nuke 13.x.

Nuke 14 now uses PyTorch 1.12 which means that PyTorch models can also be converted to TorchScript using:

TorchScript models created using PyTorch 1.12 must be converted to .cat files using Nuke 14. Note that .cat files created with Nuke 14 will not run in Nuke 13.x.

For more details on TorchScript, please see the TorchScript documentation available on the PyTorch website. It may also be useful to consider the documentation specific to your PyTorch version, for example, the 1.6 documentation.

Note

Supporting Python files and Nuke scripts for this guide are also available online or locally in your NukeX.X/Documentation/CatFileCreationDevGuide/examples folder so you can try out the models for yourself!

Throughout this guide, we’ll refer to Nuke. Nuke can be taken to mean either NukeX, NukeX Non-Commercial, NukeX Indie or Nuke Studio.

Over the course of these tutorials we will be saving TorchScript models using the .pt/.pth extension. Please keep in mind that in the wild, PyTorch checkpoints are also sometimes saved with .pt extensions.

Info

Cat File Creation guide. Copyright © 2021 The Foundry Visionmongers Ltd. All Rights Reserved. Use of this Developer Guide and the NUKE software is subject to an End User License Agreement (the “EULA”), the terms of which are accessible in the NUKE User Guide included in the NUKE product bundle. This Developer Guide and the NUKE software may be used or copied only in accordance with the terms of the EULA. This Developer Guide, the NUKE software and all intellectual property rights relating thereto are and shall remain the sole property of The Foundry Visionmongers Ltd. (“The Foundry”) and/or The Foundry’s licensors.

The EULA can be read in the NUKE User Guide Appendices.

The Foundry assumes no responsibility or liability for any errors or inaccuracies that may appear in this Developer Guide and this Developer Guide is subject to change without notice. The content of this Developer Guide is furnished for informational use only.

Except as permitted by the EULA, no part of this Developer Guide may be reproduced, stored in a retrieval system or transmitted, in any form or by any means, electronic, mechanical, recording or otherwise, without the prior written permission of The Foundry. To the extent that the EULA authorizes the making of copies of this User Guide, such copies shall be reproduced with all copyright, trademark and other proprietary rights notices included herein. The EULA expressly prohibits any action that could adversely affect the property rights of The Foundry and/or The Foundry’s licensors, including, but not limited to, the removal of the following (or any other copyright, trademark or other proprietary rights notice included herein):

NUKE compositing software © 2021 The Foundry Visionmongers Ltd. All Rights Reserved.

NUKE is a trademark of The Foundry Visionmongers Ltd.

Digital Domain® is a registered trademark of Digital Domain, Inc.

In addition to those names set forth on this page, the names of other actual companies and products mentioned in this User Guide (including, but not limited to, those set forth below) may be the trademarks or service marks, or registered trademarks or service marks, of their respective owners in the United States and/or other countries. No association with any company or product is intended or inferred by the mention of its name in this User Guide.

Linux® is a registered trademark of Linus Torvalds.

Windows® is the registered trademark of Microsoft Corporation.

Mac, Mac OS X are trademarks of Apple, Inc., registered in the U.S. and other countries.