A technical breakdown of react-three-fiber

In this article, you will learn about how React and JSX work to build our own react-three-fiber. We will cover and use the same techniques to produce the same API step-by-step.

I will not cover React’s optimizations and rendering behavior, but if you want an in-depth explanation, see Rodrigo Pombo’s article: Build your own React.

Before we dig into react-three-fiber, let’s review what React is and how it works.

What is React?

React is a library that lets you declaratively create stateful elements and components with effects, often expressed in JSX.

import React, { useState, useEffect } from 'react';

const Component = () => {
  const [state, setState] = useState();
  return null;
};

It is not a web framework; React knows nothing of the DOM and is independent of its host platform.

A renderer informs React of a host platform, its primitives, and properties. Primitives, or lower-case elements, are not imported but evaluated at run-time by the renderer.

At run-time, they are expressed in terms of their target platform. This will be HTML or <foo /> when paired with react-dom and three.js or new THREE.Foo() with react-three-fiber.

import React, { useState } from 'react';
import { render } from 'react-dom';

const Clickable = () => {
  const [clicked, setClicked] = useState(false);
  return (
    <div
      style={{ background: clicked ? 'red' : 'black' }}
      onClick={() => setClicked(clicked => !clicked)}
    />
  );
};

render(<Clickable />, document.getElementById('root'));

Examples of renderers would be:

• react-dom for the web
• react-native for native
• react-three-fiber for three.js
• react-hardware for I/O
• react-pdf for documents
• react-figma for Figma Design
• react-blessed for CLI

What is react-three-fiber?

react-three-fiber is a React renderer for three.js that renders to a canvas via a render method. It is complete with events and a performance regression system.

Inside of the package, there are hooks for accessing three.js state and a shared render-loop as well as small utilities like useLoader that enable suspense for asynchronous assets.

This is what it looks like:

import React, { useRef, useState } from 'react';
import { useFrame, render, events } from '@react-three/fiber';

function Box(props) {
  // This reference will give us direct access to the mesh
  const mesh = useRef();
  // Set up state for the hovered and active state
  const [hovered, setHover] = useState(false);
  const [active, setActive] = useState(false);

  // Subscribe this component to the render-loop, rotate the mesh every frame
  useFrame(() => (mesh.current.rotation.x += 0.01));

  // Return view, these are regular three.js elements expressed in JSX
  return (
    <mesh
      {...props}
      ref={mesh}
      scale={active ? 1.5 : 1}
      onClick={() => setActive(!active)}
      onPointerOver={() => setHover(true)}
      onPointerOut={() => setHover(false)}
    >
      <boxGeometry args={[1, 1, 1]} />
      <meshStandardMaterial color={hovered ? 'hotpink' : 'orange'} />
    </mesh>
  );
}

render(
  <>
    <ambientLight />
    <pointLight position={[10, 10, 10]} />
    <Box position={[-1.2, 0, 0]} />
    <Box position={[1.2, 0, 0]} />
  </>,
  document.querySelector('canvas'),
  { events }
);

react-three-fiber also ships with a <Canvas /> component that enables you to combine with other renderers, like react-dom or react-native.

import { Canvas } from '@react-three/fiber';
import { render } from 'react-dom';
import Box from './Box';

render(
  <Canvas>
    <ambientLight />
    <pointLight position={[10, 10, 10]} />
    <Box position={[-1.2, 0, 0]} />
    <Box position={[1.2, 0, 0]} />
  </Canvas>,
  document.getElementById('root')
);

How React Creates Elements

We know that in React, elements can have props, state, and effects. You’d express this in JSX, which later gets transformed into JS with tools like Babel.

For reference, here is the previous cube example transpiled from JSX to JS:

import React, { useRef, useState } from 'react';
import { useFrame, render, events } from '@react-three/fiber';

function Box(props) {
  // This reference will give us direct access to the mesh
  const mesh = useRef();
  // Set up state for the hovered and active state
  const [hovered, setHover] = useState(false);
  const [active, setActive] = useState(false);

  return (
    // Subscribe this component to the render-loop, rotate the mesh every frame
    useFrame(() => (mesh.current.rotation.x += 0.01)),
    // Return view, these are regular three.js elements expressed in JSX
    React.createElement(
      'mesh',
      {
        ...props,
        ref: mesh,
        scale: active ? 1.5 : 1,
        onClick: () => setActive(!active),
        onPointerOver: () => setHover(true),
        onPointerOut: () => setHover(false),
      },
      React.createElement('boxGeometry', { args: [1, 1, 1] }),
      React.createElement('meshStandardMaterial', {
        color: hovered ? 'hotpink' : 'orange',
      })
    )
  );
}

render(
  React.createElement(
    React.Fragment,
    null,
    React.createElement('ambientLight', null),
    React.createElement('pointLight', { position: [10, 10, 10] }),
    React.createElement(Box, { position: [-1.2, 0, 0] }),
    React.createElement(Box, { position: [1.2, 0, 0] })
  ),
  document.querySelector('canvas'),
  { events }
);

Elements are transformed into React#createElement calls, accepting an element type or reference, properties, and children as parameters.

We will later recreate our scene using nodes described by this method, but they need to be parented to resemble a tree or scene graph.

In the previous example, we used React’s Fragment to parent loose scene objects together. This is just a symbol to describe an empty element.

Here’s a minimal recreation of React’s createElement without optimizations. We can safely replace React’s version with our own:

const createElement = (type, props, ...childNodes) => {
  // Extract reserved keys from props
  const { key = null, ref = null, ...rest } = props || {};

  // Pass only the first child if only one is specified
  const childCollection = childNodes.length === 1 ? childNodes[0] : childNodes;
  const children = childNodes.length ? childCollection : undefined;

  const element = {
    // This tag allows React to discern this as a React Element
    $$typeof: Symbol.for('react.element'),

    // Element properties
    type,
    key,
    ref,
    props: {
      ...rest,
      children,
    },
  };

  return element;
};

const Fragment = Symbol.for('react.fragment');

const React = {
  createElement,
  Fragment,
};

Notice the structure these methods create — this tree is how react-three-fiber sees our scene.

{
  $$typeof: Symbol('react.element'),
  type: Symbol('react.fragment')
  key: null,
  ref: null,
  props: {
    children: [
      {
        $$typeof: Symbol('react.element'),
        type: 'ambientLight',
        key: null,
        ref: null,
        props: {}
      },
      {
        $$typeof: Symbol('react.element'),
        type: 'pointLight',
        key: null,
        ref: null,
        props: {
          position: [10, 10, 10],
        }
      },
      {
        $$typeof: Symbol('react.element'),
        type: function Box(props) { ... },
        key: null,
        ref: null,
        props: {
          position: [-1.2, 0, 0]
        }
      },
      {
        $$typeof: Symbol('react.element'),
        type: function Box(props) { ... },
        key: null,
        ref: null,
        props: {
          position: [1.2, 0, 0]
        }
      }
    ]
  }
}

Building a Reconciler

Now that we have a scene described, let’s create a renderer to render it to screen.

Before creating a render method, we need to create a reconciler.

We can use a reconciler to diff through state changes from React and update our scene.

import Reconciler from 'react-reconciler';

// This will centralize updates and mutations for us.
const reconciler = Reconciler({
  // three.js objects can be updated, so we inform the renderer
  supportsMutation: true,
  // We set this to false because this can work on top of react-dom
  isPrimaryRenderer: false,
  // We can modify the ref here, but we return it instead (no-op)
  getPublicInstance: instance => instance,
  // This object that's passed into the reconciler is the host context.
  // We don't need to expose it though
  getRootHostContext: () => ({}),
  getChildHostContext: () => ({}),
  // Text isn't supported in three (r133), so we skip it
  createTextInstance: () => {},
  // This is used to calculate updates in the render phase or commitUpdate.
  // Although this improves performance, it's not needed for a PoC
  prepareUpdate: () => ({}),
  // This lets us store stuff before React mutates our three.js objects.
  // We don't do anything here but return an empty object
  prepareForCommit: () => ({}),
  resetAfterCommit: () => ({}),
  // three.js elements don't have textContent, so we skip this
  shouldSetTextContent: () => false,
  // We can mutate objects once they're assembled into the scene graph here.
  // applyProps removes the need for this though
  finalizeInitialChildren: () => false,
  // This can modify the container and clear children.
  // Might be useful for disposing on demand later
  clearContainer: () => false,
  // This is where we'll create a three.js element from a React element
  createInstance(type, props) {},
  // These methods add elements to the scene
  appendChild(parentInstance, child) {},
  appendInitialChild(parentInstance, child) {},
  appendChildToContainer(parentInstance, child) {},
  // These methods remove elements from the scene
  removeChild(parentInstance, child) {},
  removeChildFromContainer(parentInstance, child) {},
  // We can specify an order for children to be specified here.
  // This is useful if you want to override stuff like materials
  insertBefore(parentInstance, child, beforeChild) {},
  // This is where we mutate three.js objects in the render phase
  commitUpdate(instance, updatePayload, type, oldProps, newProps) {},
});

There’s a lot to unpack here, but the bulk of our work with the reconciler will revolve around creating/updating, adding, and removing elements.

Let’s start with the createInstance method. Here, we’ll take our React elements and turn them into three.js elements.

import * as THREE from 'three';

/**
 * Converts camelCase primitives to PascalCase.
 */
const pascalCase = str => str.charAt(0).toUpperCase() + str.substring(1);

/**
 * TODO: apply props to three.js instance
 */
const applyProps = (instance, newProps, oldProps) => {}

//

createInstance(type, { object, args, ...props }) {
  // Convert lowercase primitive to PascalCase
  const name = pascalCase(type);

  // Get class from THREE namespace
  const target = THREE[name];

  // Validate THREE elements
  if (type !== 'primitive' && !target)
    throw `${type} is not a part of the THREE namespace.`;

  // Validate primitives
  if (type === 'primitive' && !object)
    throw `"object" must be set when using primitives.`;

  // Create instance
  const instance =
    object || (Array.isArray(args) ? new target(...args) : new target(args));

  // Auto-attach geometry and materials to meshes
  if (name.endsWith('Geometry')) {
    props = { attach: 'geometry', ...props };
  } else if (name.endsWith('Material')) {
    props = { attach: 'material', ...props };
  }

  // Set initial props
  applyProps(instance, props, {});

  return instance;
}

Now our reconciler understands how to create three.js elements, but they don’t accept props yet.

Let’s continue with our applyProps method to update elements’ props.

/**
 * Prunes keys from an object.
 */
const pruneKeys = (obj, ...keys) => {
  const keysToRemove = new Set(keys.flat());

  return Object.fromEntries(
    Object.entries(obj).filter(([key]) => !keysToRemove.has(key))
  );
};

/**
 * Safely mutates a three.js element and collects listeners.
 */
const applyProps = (instance, newProps, oldProps = {}) => {
  // Filter identical props, event handlers, and reserved keys
  const identical = Object.keys(newProps).filter(key => newProps[key] === oldProps[key]);
  const handlers = Object.keys(newProps).filter(
    key => typeof newProps[key] === 'function' && key.startsWith('on')
  );
  const props = pruneKeys(newProps, [
    ...identical,
    ...handlers,
    'children',
    'key',
    'ref',
  ]);

  // Mutate our three.js element
  if (Object.keys(props).length) {
    Object.entries(props).forEach(([key, value]) => {
      const target = instance[key];
      const isColor = target instanceof THREE.Color;

      // Prefer to use properties' copy and set methods
      // otherwise, mutate the property directly
      if (target?.set) {
        if (target.constructor.name === value.constructor.name) {
          target.copy(value);
        } else if (Array.isArray(value)) {
          target.set(...value);
        } else if (!isColor && target?.setScalar) {
          // Allow shorthand like scale={1}
          target.setScalar(value);
        } else {
          target.set(value);
        }

        // Auto-convert sRGB colors
        if (isColor) target.convertSRGBToLinear();
      } else {
        instance[key] = value;
      }
    });
  }

  // Collect event handlers.
  // We put this on an invalid prop so three.js doesn't serialize handlers
  // if you do ref.current.clone() or ref.current.toJSON()
  if (handlers.length) {
    instance.__handlers = handlers.reduce(
      (acc, key) => ({ ...acc, [key]: newProps[key] }),
      {}
    );
  }
};

Now that we can handle props, we can let the reconciler mutate three.js elements at run-time with the commitUpdate method:

// This is where we mutate three.js objects in the render phase
commitUpdate(instance, updatePayload, type, oldProps, newProps) {
  instance.busy = true;
  applyProps(instance, newProps, oldProps);
  instance.busy = false;
}

All we have to do now is handle adding/removing elements from the scene. The reconciler can do this in many ways, but we’ll use the same method throughout.

/**
 * Adds elements to our scene and attaches geometry and material to meshes.
 */
const appendChild = (parentInstance, child) => {
  if (!child) return;

  // Attach material, geometry, fog, etc.
  if (child.attach && parentInstance[child.attach] !== undefined) {
    parentInstance[child.attach] = child;
  } else {
    parentInstance.add(child);
  }
};

/**
 * Removes elements from our scene and disposes of them.
 */
const removeChild = (parentInstance, child) => {
  if (!child) return;

  // Remove material, geometry, fog, etc.
  if (child.attach && parentInstance[child.attach] !== undefined) {
    parentInstance[child.attach] = null;
  } else {
    parentInstance.remove(child);
  }

  // Safely dispose of element
  if (child.type !== 'Scene') {
    if (child.dispose) child.dispose();

    // Dispose of its properties as well
    for (const property in child) {
      if (property.dispose) property.dispose();
      delete child[property];
    }
  }
};

//

// These methods add elements to the scene
appendChild,
appendInitialChild: appendChild,
appendChildToContainer: appendChild,

// These methods remove elements from the scene
removeChild,
removeChildFromContainer: removeChild,

Lastly, let users specify an order for elements to appear in:

// We can specify an order for children to be specified here.
// This is useful if you want to override stuff like materials
insertBefore(parentInstance, child, beforeChild) {
  if (!child) return;

  child.parent = parentInstance;

  const index = parentInstance.children.indexOf(beforeChild);
  parentInstance.children = [
    ...parentInstance.children.slice(0, index),
    child,
    ...parentInstance.children.slice(index),
  ];

  // Emit an event that tells three.js the element is added
  child.dispatchEvent({ type: 'added' });
}

Creating a Render Method

Our reconciler can render React elements into a dynamic scene graph, but nothing appears on screen yet.

For that, we’ll need to create a render method. This will be the entry point for our app, where we’ll configure our rendering internals and draw to a canvas.

/**
 * Internal three.js state.
 */
const context = React.createContext(null);

// We store roots here since we can render to multiple canvases
const roots = new Map();

/**
 * This renders an element to a canvas, creating a renderer, scene, etc.
 */
const render = (element, canvas, { size, gl, camera, ...props } = {}) => {
  // If size isn't explicitly defined, we can assume it from the canvas
  if (!size) {
    size = {
      width: canvas.parentElement?.clientWidth || 0,
      height: canvas.parentElement?.clientHeight || 0,
    };
  }

  // Get store and init/update three.js state
  const store = roots.get(canvas);
  let root = store?.root;
  const state = Object.assign(store?.state || {}, { ...props, size });

  // Initiate root
  if (!root) {
    // Create renderer
    state.gl = new THREE.WebGLRenderer({
      canvas,
      powerPreference: 'high-performance',
      antialias: true,
      alpha: true,
      ...gl,
    });
    if (gl) applyProps(state.gl, gl, {});

    // Set artist-friendly color management defaults
    state.gl.outputEncoding = THREE.sRGBEncoding;
    state.gl.toneMapping = THREE.ACESFilmicToneMapping;

    // Create camera
    state.camera = new THREE.PerspectiveCamera(75, 0, 0.1, 1000);
    state.camera.position.z = 5;
    if (camera) applyProps(state.camera, camera, {});

    // Look at center by default
    if (!camera?.rotation) state.camera.lookAt(0, 0, 0);

    // Create scene
    state.scene = new THREE.Scene();

    // Create root
    root = reconciler.createContainer(state.scene, 1, false, null);

    // If an event manager is specified, connect it.
    // This lets us specify different events between platforms
    if (state.events) state.events.connect(canvas, state);

    // Keep track of elements subscribed to the render loop with useFrame
    state.subscribed = [];
    state.subscribe = ref => {
      if (state.subscribed.includes(ref)) {
        state.subscribed = state.subscribed.filter(callback => callback !== ref);
      } else {
        state.subscribed.push(ref);
      }
    };

    // Start our render loop (we use this instead of RaF for WebXR support)
    state.gl.setAnimationLoop(() => {
      state.subscribed.forEach(ref => {
        const callback = ref?.current;
        if (typeof callback === 'function') callback(state);
      });
      state.gl.render(state.scene, state.camera);
    });
  }

  // Handle resize
  state.gl.setSize(size.width, size.height);
  state.camera.aspect = size.width / size.height;
  state.camera.updateProjectionMatrix();

  // Update root
  roots.set(canvas, { root, state });

  // Update fiber and expose three.js state to children
  reconciler.updateContainer(
    <context.Provider value={state}>{element}</context.Provider>,
    root,
    null,
    () => undefined
  );

  return state;
};

At this point, we’re able to render our scene to screen, but everything is static.

Let’s create some hooks so elements can interact with our renderer and animate.

Creating Hooks

Our render method lets users subscribe to the render loop by passing ref callbacks via state#subscribe. We defined our state in a context that we can access with React#useContext anywhere in our scene.

Let’s create some hooks to make interacting with our renderer easier.

/**
 * This hooks lets users access internal three.js state.
 */
const useThree = () => {
  const state = React.useContext(context);
  // We can only access context from within the scope of the canvas
  // since a context provider is mounted at the root of our scene.
  // If used outside, we throw an error instead of returning null for DX.
  if (!state) throw 'Hooks must used inside the canvas.';
  return state;
};

/**
 * This hook lets users subscribe their elements into a shared render loop.
 */
const useFrame = callback => {
  const state = useThree();
  // Store frame callback in a ref so we can pass a mutable reference.
  // This allows the callback to dynamically update without blocking
  // the render loop.
  const ref = React.useRef(callback);
  React.useLayoutEffect(() => void (ref.current = callback), [callback]);
  // Subscribe on mount and unsubscribe on unmount (runs twice).
  // We used void in the last effect to have it only run on mount
  React.useLayoutEffect(() => state.subscribe(ref), [state]);
};

Now elements can interact with our internal state and participate in a shared render loop.

All we need is some flair — let’s create events.

Simulating Events

Events don’t exist in WebGL, but we can simulate them by subscribing to DOM events from our canvas.

We’ll keep track of these events and their corresponding props.

// Supported DOM events and their JSX keys with passive args
const EVENT_TYPES = {
  click: ['onClick', false],
  dblclick: ['onDoubleClick', false],
  pointerup: ['onPointerUp', true],
  pointerdown: ['onPointerDown', true],
  // We hijack the mousemove event to handle hover state
  mousemove: ['hoverEvent', true],
};

We can then create an events manager that raycasts through our scene whenever a DOM event is triggered.

We’ll create a raycaster and keep track of mouse coordinates to filter through elements and trigger their event handlers.

// Hovered objects will live here
const hovered = {};

const events = {
  /**
   * Creates and registers event listeners on our canvas.
   */
  connect(canvas, state) {
    // Init event state
    state.mouse = new THREE.Vector2();
    state.raycaster = new THREE.Raycaster();

    // Event handlers
    const handleEvent = (event, type) => {
      // Convert mouse coordinates
      state.mouse.x = (event.clientX / state.size.width) * 2 - 1;
      state.mouse.y = -(event.clientY / state.size.height) * 2 + 1;

      // Get elements that intersect with our pointer
      state.raycaster.setFromCamera(state.mouse, state.camera);
      const intersects = state.raycaster.intersectObjects(state.scene.children, true);

      // Used to discern between generic events and custom hover events
      const isHoverEvent = type === 'hoverEvent';

      // Trigger events for hovered elements
      intersects.forEach(intersect => {
        const { object } = intersect;
        const handlers = object.__handlers;

        if (isHoverEvent && !hovered[object.uuid]) {
          // Mark object as hovered and fire its hover events
          hovered[object.uuid] = object;

          if (handlers.onHover) handlers.onHover(event);
          if (handlers.onPointerOver) handlers.onPointerOver(event);
        } else if (!isHoverEvent && handlers[type]) {
          // Otherwise, fire its generic event
          handlers[type](event);
        }
      });

      // Clean up stale hover events
      if (isHoverEvent) {
        Object.values(hovered).forEach(object => {
          const handlers = object.__handlers;

          if (!intersects.length || !intersects.find(i => i.object === object)) {
            delete hovered[object.uuid];

            if (handlers.onPointerOut) handlers.onPointerOut(event);
          }
        });
      }

      return intersects;
    };

    // Save listeners to canvas
    canvas.__listeners = {};

    // Register events
    Object.entries(EVENT_TYPES).forEach(([name, [type, passive]]) => {
      const listener = event => handleEvent(event, type);
      canvas.addEventListener(name, listener, { passive });
      canvas.__listeners[name] = listener;
    });
  },
  /**
   * Deletes and disconnects event listeners from our canvas.
   */
  disconnect(canvas) {
    // Get listeners from canvas
    const listeners = canvas.__listeners;

    // Disconnect listeners
    Object.entries(listeners).forEach(([name, listener]) => {
      canvas.removeEventListener(name, listener);
    });

    // Remove listeners from canvas
    delete canvas.__listeners;
  },
};

Our events manager is completely modular, so we can pass it to our render method’s props or omit it entirely. This allows us to customize events and support different platforms with ease.

Cleaning Up

Now that everything’s connected with our render method, we have to let users tear it down to prevent memory leaks on unmount.

For this, we’ll create an unmountComponentAtNode method.

/**
 * This is used to remove and clean up internals on unmount.
 */
const unmountComponentAtNode = canvas => {
  const store = roots.get(canvas);
  if (!store) return;

  const { root, state } = store;

  reconciler.updateContainer(null, root, null, () => {
    // Disconnect events
    if (state.events) state.events.disconnect(canvas);

    // Clean up renderer
    state.renderer.setAnimationLoop(null);
    state.renderer.forceContextLoss();
    state.renderer.dispose();

    // Delete store
    roots.delete(canvas);
  });
};

We can combine these methods with React 18’s createRoot signature.

/**
 * The react-dom 18 API changes how you create roots, letting you specify
 * a container once and safely render/unmount later, so we mirror that.
 */
const createRoot = canvas => ({
  render: element => render(element, canvas),
  unmount: () => unmountComponentAtNode(canvas),
});

Conclusion

There we have it — we’ve created our own react-three-fiber.

We can now take the previous example from react-three-fiber and plug our renderer into it. Hooks and events will work out-of-the-box with only the change of an import.

import { useRef, useState } from 'react';
import { useFrame, render, events } from './react-three-fiber';

function Box(props) {
  // This reference will give us direct access to the mesh
  const mesh = useRef();
  // Set up state for the hovered and active state
  const [hovered, setHover] = useState(false);
  const [active, setActive] = useState(false);

  // Subscribe this component to the render-loop, rotate the mesh every frame
  useFrame(() => (mesh.current.rotation.x += 0.01));

  // Return view, these are regular three.js elements expressed in JSX
  return (
    <mesh
      {...props}
      ref={mesh}
      scale={active ? 1.5 : 1}
      onClick={() => setActive(!active)}
      onPointerOver={() => setHover(true)}
      onPointerOut={() => setHover(false)}
    >
      <boxGeometry args={[1, 1, 1]} />
      <meshStandardMaterial color={hovered ? 'hotpink' : 'orange'} />
    </mesh>
  );
}

// Render our scene to a canvas and pass our platform's event manager
render(
  <>
    <ambientLight />
    <pointLight position={[10, 10, 10]} />
    <Box position={[-1.2, 0, 0]} />
    <Box position={[1.2, 0, 0]} />
  </>,
  document.querySelector('canvas'),
  { events }
);

This article doesn’t cover the entirety of the APIs and performance optimizations that react-three-fiber uses, but hopefully, it shed some light on how React and reconcilers work to create awesome declarative interfaces.

With React 18 around the corner, more awesome things are yet to come, and I’ll be sure to document them as they come along on my Twitter.

If you have any comments, ideas, or requests for articles on this subject (or any subject), feel free to send me a message.