Rendering with Bosco

Example image rendered using HCC's distributed renderer

At the Nebraska Holland Computing Center, we take pride in eating our own dog food.  So I want to highlight one of our uses of Bosco to enable transparent usage of our clusters.

HCC recently has made a push for enabling the non-traditional users of clusters.  Sure it's easy to show how physics can benefit from a set of clusters, but what about the media arts?  In this case we are working with a professor whose students render short movies using Maya.

Maya can utilize another Autodesk product called Backburner, which enables rendering across multiple nodes.  The challenge was to allow Backburner to operate in a shared cluster environment.  The standard method to use Backburner expects nodes dedicated solely to running the Backburner Server daemons.  This Master-Worker fits the traditional HTC model well, therefore we felt that Bosco would be a great fit to enable submission to our clusters.

Architecture Diagram of the BOSCO enabled render (Credit: Adam Caprez)

In the architecture shown in the image above, we run a central service, hcc-render that monitors the Backburner queue in order to submit the Backburner servers to the Bosco queue.  Bosco then submits a glidein to Tusker, which in turn runs the Backburner server and renders the scene.

In this case, since we are using Backburner server for the actual processing, the data is handled internally by Backburner.  In practice this means that data is stored on the Tusker file system which is then mounted by the professors local machine that is in turn mounted by the clients.

This is a classic example of deeply integrating HTC into the user's application.
The user only needs to click the 'render' button from within Maya on their workstation, and we handle all the rest, automatically.  This is only made possible because Maya has a sane back end renderer that is designed to run on Linux.  This architecture may not work for all commercial applications.

Even though the user isn't directly using Bosco, a primary goal of Bosco, this is an excellent use of Bosco to enable HTC workflows.