PipelineFX Qube! Render Farm Basics
QUBE! COMPONENTS
Supervisor (Server Node) = Render Queue which runs on a single dedicated manager node.
Workers (Render Nodes) = Render daemons which run on all render nodes.
Client (on Artist/ User Machines) = Applications, APIs, or other interfaces that allow you to submit jobs to the render farm for processing (Nuke, Maya, 3ds Max, Mental Ray, Command Line, Python, etc)
JOBS
Job = Bundle of data, submitted by a client to the Supervisor node, that includes all of the information the Worker nodes will need to execute a job. Includes host application, host OS, file locations, user, cpu allocation, etc.
Subjob = numbered 0-N, Also called Job Processes, Defined in “Job CPUs” field. Threads initiated by the Supervisor, based on the number simultaneous frames the job is allowed to process at any given time.
Agenda Items = Frames
JOB ID NUMBERS
Example: (Although many variations are possible)
Job 12345 wants to render 4 frames (numbered 100-104) on two workers.
1) The four agenda items are: frame 100, frame 101, frame 102, and frame 103.
2) Subjob 12345.0 renders frames 100 and 102 (agenda items 1 and 3)
3) Subjob 12345.1 renders frames 101 and 103 (agenda items 2 and 4)
Important:
Number of Subjobs = the number of machines a Job is being processed on.
Number of Agenda Items = the number frames being processed.
Job Slots (or Process Slots) = how many processes a Worker can run at a given time. Job Slots do not refer to the number of processor cores on a Worker.
Job Callbacks = Any of the many action events Qube is able to trigger such as emailing the user, unblocking a job, or running a script.
SHARED STORAGE
A network fileserver accessible to the Clients and the Worker is needed as the scenes to render will be read from a network location and the resulting images written out to a network location. In order to reduce manual process of copying files back and forth, it is recommended that artists edit their scenes directly from the central fileserver instead of locally.
All users should have full read and write access to the file server. Job execution usually requires a great deal of reading and writing, so make sure that users have permission to not only read files from the server, but also create them.
If you plan to share files to both desktop users and to the render farm, double-check your fileserver licensing. Some servers have limits to the number of simultaneous client connections. A farm of machines could easily exceed the licensing limitations of a desktop machine acting as a server
Note: By default, Qube installs proxy user called qubeproxy. This user must be granted read and write access to the file server if a Worker uses the standard “proxy” mode.
INSTALLATION REQUIREMENTS
Recommended OS:
1st: Linux
2nd: OS X
3rd: WIndows
File Storage:
All Nodes must be able to access files from a shared storage location (either SAN, NAS, or Fileserver)
Administrator Access:
Required for installation
Supervisor Logs:
Reserved Space (local) for up to 4GB of log files (good for several thousand jobs)
Open TCP/IP and UDP Ports:
Turn firewall off on all Supervisor, Worker, and Client machines.
or
Open the following ports:
50001 TCP/IP and UDP
50002 TCP/IP and UDP
50011 TCP/IP and UDP
SUPERVISOR NODES
• 1-10 node renderfarm – Dedicated or multi-use server is fine, 4+GB RAM
• 11-50 node renderfarm – Dedicated Supervisor recommended, 4+ GB RAM
• 50-100 node renderfarm – Dedicated Supervisor recommended with higher memory (ie. 32GB)
• 100+ node renderfarm – Dedicated Supervisor recommended with max memory running Linux
SUPERVISOR INSTALLATION
Install the following OS components:
• MySQL (Linux only) MySQL included with Supervisor for Windows & OSX
Install the following Qube components:
• Qube Core
• Qube Supervisor
• QubeGUI Install and verify Qube License (qb.lic):
• Launch the QubeGUI and select Administration->Install License and specify the qb.lic license file. • Verify that the QubeGUI title bar shows the correct <used>/<available> licenses. (e.g. 0/100)
WORKER INSTALLATION
Install the following Qube components:
• Qube Core
• Qube Worker
• QubeGUI (optional)
• Qube Jobtypes (Maya, XSI, etc)
• [Windows only] Install Perl (if Jobtypes installed, usually requiring perl). Download version Perl 5.8 from www.activestate.com. Also supports version 5.6 on Windows.
• [Windows only] Install Python (if using Jobtypes requiring python). Download Py- thon 2.6 from www.activestate.com. Alsu supports python versions 2.3, 2.4, and 2.5 on Windows.
Configure the Worker:
• Specify Supervisor (required if not on same subnet as Supervisor) and execution mode:
-Launch the QubeGUI
-Select Administrator->Configuration to bring up a dialog
-Set the “Supervisor” parameter to the Supervisor hostname.
-Set the “proxy_execution_mode” parameter to “user” if you want to have the Worker run as the submitting user (requires ActiveDirectory or LDAP users)
-Set the “proxy_account” parameter to a network user if want to run jobs on the Worker as a single Active Directory or LDAP user.
• Disable the Firewall
Install the following Qube components:
• Qube Core
• QubeGUI
• Qube Jobtypes with in-app interfaces (Maya, MiGen, Mentalray, 3dsMax, XSI) Configure the Clients:
• Specify Supervisor (required if not on same subnet as Supervisor)
-Launch the QubeGUI
-Select Administrator->Configuration to bring up a dialog
-Set the “Supervisor” parameter to the Supervisor hostname
CLIENT INSTALLATION
Install the following Qube components:
• Qube Core
• QubeGUI
• Qube Job types with in-app interfaces (Maya, MiGen, Mentalray, 3dsMax, XSI)
Configure the Clients:
• Specify Supervisor (required if not on same subnet as Supervisor)
-Launch the QubeGUI
-Select Administrator->Configuration to bring up a dialog
-Set the “Supervisor” parameter to the Supervisor hostname
LAUNCHING QUBE!
Launching Via GUI
Windows: Start->PipelineFX->Qube
Windows 32-bit: C:/Program Files/pfx/qube/bin/qube.exe
Windows 64-bit C:/Program Files (x86)/pfx/qube/bin/qube.exe)
Linux: /usr/local/pfx/qube/bin/qube
OSX: /Applications/pfx/qube/qube.app
Launching via Python
Qube can be run directly via Python using …/qube/api/python/qb/gui/qube.py.
If using this method, the following components are required:
• Python (required)
• wxPython (required)
• numpy (optional, to display the job charts)
• PIL (optional, to provide additional internal image file formats)
• MySQLdb (optional, to provide direct MySQL database queries)
