The demise of Natron: how we got here and where we go further

The demise of Natron: how we got here and where we go further

Natron, free/libre VFX compositing software, has been in serious trouble for at least the past year. With dozen of releases, active community, and well over 300 dedicated tutorials in many languages on YouTube, some of them posted as far back as last week, Natron would seem like a healthy free/libre software project. But it's not, and here is why.

First off, there seems to be a doom (apocalyptic much?) looming over standalone free/libre compositing tools. Ramen went on and off for a while and even became a closed software project, until Esteban Tovagliari pulled a final plug on it (he's now a lot happier working on appleseed and blenderseed). It's been 7 years since we last heard of Synapse. And last year, Fabien Castan ended up admitting that ButtleOFX is done for.

So given the release frequency and persistance of Natron developers, one would think we are finally getting there. Unfortunately, nothing lasts forever.

The project was sponsored by the Inria institute and worked on mostly by just two people: Alexandre Gauthier-Foichat, principal developer of the project, and Frédéric Devernay, his mentor at Inria. For a short period of time, Ole-André Rodlie, another Inria employee, helped out.

The collaboration wasn't without its moments. At some point, the duo started developing a falling-out over things like making Natron a clone of Nuke or focusing on original work and new ideas instead. Eventually, Alexandre's contract with Inria ended, and he started to work on a closed-source application.

Frédéric continued maintaining the project and releasing updates until August 2018, but these were mostly bugfix releases, while more serious work had to be done. To give you idea, while Alexandre was working on a private tracking plug-in requested by Inria, these things stood out:

  • Natron had lots of memory issues when reading very long video files
  • A lot of race conditions were caused due to multi-threading
  • A lot of logic in the internal engine had to be re-written

Given that, Alexandre started working towards a more solid rendering engine that would become the staple of Natron 3, but says he never had the time to complete it (the code is in the master branch). His contract with Inria, in his own words, subsequently ended in December 2017.

The most up-to-date GitHub repository is now this one (and the repository of community-contributed plugins is nearby).

Now that neither original developers works for Inria or wants to continue contributing to this project, the question is: how could it have been prevented and where do we go next?

We spoke to both Alexandre and Frédéric about it.

Development of Natron was at some point paid by Inria. I think they paid for the first 12 months, then extended for at least another year. Is that so? Why did they stop supporting it?

Frédéric: Inria did not stop supporting it, since I kept maintaining Natron until I left Inria in September 2018. Inria still owns most of the Natron code (as it was my employer). In Dec 2017, Alexandre Gauthier left the project to start a closed-source commercial graphics software which I know nothing about.

What were the conditions of Ole-André's paid participation?

Frédéric: He was paid as an engineer by Inria to work on his contribution to the Natron infrastructure (source code, build and test systems). Any code he produced was property of Inria, as with the code I produced. The fact that Alexandre still owns part of the code was due to an agreement between him and Inria during his first year on Natron.

Did you try to find other funding sources?

Frédéric: Any user can offer a bounty to solve a github issue (using bountysource, there's a link at the bottom of every issue). Since when I started this in April 2018, no bounty has ever been offered. My hope was that if there were bounties, this could attract new developers, but it didn't. In April 2018, I already knew I would have to quit Natron a few months later.

Has the signing of Contributor License Agreement ever been a problem (that you know of) for potential contributors?

Alexandre: Not that I know of. The lack of contributors is mainly due to the lack of talented developer who want to and have enough skill to maintain such a code-base. Generally they already work in a company or structure of some sort.

Frédéric: I don't know. Nobody ever said "if you dump that CLA, I'll gladly contribute to Natron". People only said that the CLA was bad, without offering any help.

At some point, you had a content team of three people and an online communication guy. What happened to their involvement?

Frédéric: The communication guy (is that Ren D'vila?) was the original Natron website designer, but the web site was since totally redesigned. Ren also wrote that initial "team" page.

Francois Grassard and Jean-Christophe Levet are VFX artists, and may still be working with Alexandre on his commercial project. I had no contact with them since Alexandre left.

Alexandre: Since we were almost always 2 developers maintaining the project, we never had time to focus on communication.

Communication has to be focused on users targeted by Natron. On one end you have a small group of experienced users, who are very familiar to commercial counterparts (Fusion, Nuke) and just wish they could find the same things they are used to, but for free. They tend to push in the direction of maintaining Natron in the way it is (i.e: as close as possible to Nuke).

On the other end you have people learning with Natron, that sometimes find it very frustrating and confusing and don’t understand why we (developers) don’t make it more user-friendly (documentation, tutorials, etc…)

The reason I asked the previous question is that as far as I can tell, at some point you kept producing releases, but media stopped covering your work. It might or might not have something to do with dropping News section on the website. What's your opinion on that?

Frédéric: The Facebook page was still very active, and was actually the only reason I had a Facebook account (which I since closed). I think many people were still using Natron and following Natron development but did not talk on social media, mainly because they were also working for VFX studios (small or medium). Now that BlackMagic Fusion is being stopped, they start crying for help...

Natron's source code is very nearly half a million lines of code large. That could be a bit too much for new contributors to chew. Is the developers' guide sufficiently up-to-date (seeing how it's built from source code anyway) and does it provide information that would help newly arrived contributors to get the hang of the source code structure etc.?

Frédéric: Yes, at least a few developers managed to build Natron on Linux and Windows. I also made sure during the last few months to add sources and documentation for all our build systems (Mac, Linux, Windows) in the Natron sources (see the "tools" directory). This also includes patches for Qt4.

The source itself is hardly documented. This is often the case when using Agile software development.

A lot of the source code is also generated by shiboken (directories Engine/NatronEngine and Gui/NatronGui), so these should really not be counted as human-written source code.

Supposing, someone gets interested in continuing the development. What do you think would be a good place to start? What kind of work do you think would help establishing the feeling of ownership (and, by extension, accountability for the future of the project)? Are the any low-hanging fruits you could think of?

Frédéric: By order of difficulty (from 0 to 10):

  • 2: pyplugs, shadertoys (there are still developers for these, see https://github.com/NatronGitHub/natron-plugins)
  • 4: write an OpenFX plugin, starting from an example in openfx-misc or from the official openfx examples, for example try to make an OpenFX plugin from a widely-used PyPlug. There are a few OFX plugin developers in the community.
  • 5: build Natron locally (on any system)
  • 7: compile a redistributable Natron binary (Linux is easier, since we build and ship most dependencies using the build scripts)
  • 9: fix a simple Natron bug
  • 10: add new functionality to Natron (see issues)

There is a hard slope from building Natron to fixing a bug. But I have just started assigning difficulty levels to Natron issues, to help newcomers.

Alexandre: The number of users that are able to set up a complex node-graph, starting from scratch with a blank page is incredibly low. Most people hit a big wall with Natron, because there are very few presets and guidance for the user.

Usually, the user must at some point link parameters or nodes all over the place or create expressions in a scripting language (like Python). This makes the learning curve of the software really steep, almost exclusively reserved to experienced users.

Whoever is going to take over will have to face a decision:

  • Keep it in the same direction.
  • Strongly brainstorm the workflow and user experience to attempt to make it better and unique.

The former is easier to do, but Natron will be looked at the same way it had be to this date: a weaker, but free counterpart to commercial compositing software. If it is targeted only towards experienced users, you could say that the best to happen to Natron would be that a high-end studio takes over Natron to steer it even further in that direction.

The latter requires a lot of work with users, designers, and developers. This is extremely expensive, and few open-source applications (without a business backing them) have the luxury to do any market research. Generally development of a software is dictated by developers who anyway do this in their free time and don’t consider it a “real job”.

Is there some sort of a roadmap, or do you think a new maintainer have to come up with his/her own one? Basically, since you have a unique insight into the code of Natron, what do you see as must-do items on the TODO list, if the project would continue?

Frédéric: With the widespread of 4K, people are now finally asking for working proxy support in Natron, which allows working on low-resolution videos and rendering in full resolution. This almost works, it just has to be debugged and tested. We should have the same functionality as in Nuke.

Then, there should be an open-source movie project using Natron. This will bubble up the remaining 2D bugs and help fixing these.

I think support for metadata in the compositing graph goes next. Everyone keeps asking for "3D space", but this is not possible without metadata support. Metadata includes camera parameters, timestamps, 2D (SVG) graphics, 3D meshes.

From there on, a proper roadmap should be discussed, and would probably include that famous "3D space". 3D space should not be a big thing: I think only basic functionality should be there, such as tracking cameras, reprojecting videos using a 3D mesh, rendering simple 3D objects, and it should all be done with OpenGL. Serious 3D stuff should really be done with external software (Blender, Maya...).

Alexandre: The state I left the Natron 3 codebase in is semi-stable and requires fixing in the internal scheduling and multi-threading in the internals of the renderer. Overall, Natron 3 was mainly a re-write of the internal engine (also including the file format) to make it more stable: the whole caching system was re-written as well as better architecturing choices were made to support multi-threaded and GPU rendering.

Sadly, I don’t think there’s any low-hanging fruits in this part, as this requires a quite strong understanding and study of how it works.

One could try to implement new features (e.g: a 3D space) on top of either Natron 2 or 3, but at some point they are going to be limited by the issues that have to be fixed in the internal dependency graph scheduling.

Writing a multi-threaded dependency graph scheduler that have all the constraints that a compositing software requires is challenging and requires full-time developers to maintain. This is the core to any large-scale software found in commercial-grade solutions and this is not something that can be left aside.

In a nutshell, what do you think are the crucial skills for someone working on a compositing application?

Frédéric: I just added these to the Natron's README.md:

  • Git and GitHub
  • C++. Natron source is still C++98, but switching to C++11 or C++14 should be straightforward if needed Design patterns
  • Qt. Natron still uses Qt4 because of the lack of PySide support in Qt5, which should be integrated shortly after Qt 5.12 is released
  • Basic knowledge of OpenGL
  • Basic knowledge of Python

Alexandre: If the community wants to continue developing Natron, they should know that this is not a task for a developer that wants to help on lazy Sundays. This is a full-time job necessitating a lot of experience and trial & error.

The only realistic option is if they would find a financial structure that has the shoulders to support the needs of such a project, such as the Blender Foundation or any studio with an R&D department that masters C++ development.

It’s very challenging to find financial support for all the time and effort that needs to be put in this project, especially knowing that it is going to yield zero revenue. A full-time developer with taxes in France costs more than 70K€ a year.

Natron

Looking back, is there something you would absolutely do differently?

Alexandre: Well, at least now I know that it takes a lot more funding than what I thought it would when I started this project (I was still a student back then).

If I were to make it again open-source, I would try to either collaborate with a strong-standing open-source community (Blender) or with an industrial partner, and try to accommodate with their constraint. Being a team is always better than being isolated: the less dependency of the source-code to a single developer, the easier it is to bring new people aboard.

Frédéric: Get more people involved at the start of the project. A 2-people project rarely becomes a 10-people project, and often dies or stays a 2-people project. A 10-people project has more chances of becoming a stable 5-people project.

Inria's initial 12 months "Boost Your Code" funding was for a one-guy project (Alexandre) mentored by me, but I think this either leads to nothing or to a commercial project.

The other thing I would do differently — work more closely with Blender, maybe work on a Natron-inside-Blender as well as a standalone.

Was it useful? There's more:

0 Responses. Comments closed for this entry.