Socket.io, React Three Fiber Tutorial

February 5, 2024

Simple Implementation of Socket.io with React Three Fiber

A simple yet effective solution for combining React Three Fiber and Socket.ioSocket.io for multiplayer game development. The project aims to demonstrate how to integrate a .glb format 3D model into a React application using TypeScript and then synchronize this model in real-time across multiple users with the help of Socket.ioSocket.io.

The project will consist of two main steps. First, we will create the client side, which includes a .glb file that reacts to mouse clicks with animation. The second step involves creating the server side, which will be connected to the client side created earlier.

I. Basic Client Side Functionality:

Uploading and accessing the .glb file to follow the click location

1. Installing Necessary Packages

POWERSHELL
mkdir client

POWERSHELL
cd client

POWERSHELL
npm create vite@latest

Choose React + TypeScript + SWC

POWERSHELL
npm install three @types/three @react-three/fiber @react-three/drei

For future server access

POWERSHELL
npm install socket.io-client

2. Creating and Setting Up the Camera and the Basic Environment

We will create the camera, canvas, lighting, and a plane.

TYPESCRIPT
//client/src/App.tsx
import { Canvas } from "@react-three/fiber";
import { Scene } from "./Scene";

function App() {
  return (
    <>
        <Canvas
          camera={{
            fov: 60,
            near: 0.1,
            far: 300,
            position: [0, 7, 8],
          
        >
          <Scene />
        </Canvas>
    </>
  );
}

export default App;

TYPESCRIPT
//client/src/Scene.tsx
import { Environment, OrbitControls } from "@react-three/drei";
import * as THREE from "three"
export const Scene = () => {
  return (
    <>
      <Environment preset="sunset" />
      <ambientLight intensity={0.2} />
      <directionalLight
        position={[5, 5, 5]}
      />
      <OrbitControls makeDefault />
      <mesh
        rotation-x={-Math.PI / 2}
      >
        <planeGeometry args={[20, 10]} />
        <meshStandardMaterial color="#70543E" />
      </mesh>
    </>
  );
};

In the final result, we already see a plane.

3. Creating the GLB file handler

Copy the previously downloaded Dog.glb file into the public directory.

Create the GLB file handler with gltfjsx. So, run the following code:

POWERSHELL
npx gltfjsx public/Dog.glb -o src/components/Dog.tsx --transformlb --transform --types

The created file appears in the src/components directory. Afterward, we add it to the previously created Scene.tsx file.

POWERSHELL
//client/src/Scene.tsx
import { Dog } from "./components/Dog";

...

      <mesh
        rotation-x={-Math.PI / 2}
      >
        <planeGeometry args={[20, 10]} />
        <meshStandardMaterial color="#70543E" />
      </mesh>
			<Dog
          position={
            new THREE.Vector3(0,0,0)
          }
          dogColor={character.dogColor}
			/>
		</>
	);
};

4. Adding Shadows to the Elements

We will enhance the App.tsx, Scene.tsx, and Dog.tsx files by adding the necessary shadows.

The shadows attribute enables the use of shadows within the Canvas element.

TYPESCRIPT
//client/src/App.tsx
				<Canvas
          camera={{
            fov: 60,
            near: 0.1,
            far: 300,
            position: [0, 7, 8],
          }}
          shadows
        >

The castShadow attribute enables the projection of shadows by the light source. If this is not enabled, the light source will not create shadows on objects, which can reduce the visual realism and sense of space in the 3D scene. This attribute is false by default, so it must be explicitly set to true to enable shadows.

The shadow-camera-left, shadow-camera-right, shadow-camera-top, and shadow-camera-bottom attributes allow us to set the boundaries of the shadow casting camera's visibility frustum. These settings determine how wide and high the area where the shadow is calculated and displayed. If these settings are not optimized for our scene, it may result in shadows not appearing correctly or being completely absent from certain areas.

TYPESCRIPT
//client/src/Scene.tsx

...

      <Environment preset="sunset" />
      <ambientLight intensity={0.2} />
      <directionalLight
        position={[5, 5, 5]}
        castShadow
        shadow-camera-left={-10}
        shadow-camera-right={10}
        shadow-camera-top={10}
        shadow-camera-bottom={-10}
      />
      <OrbitControls makeDefault />
      <mesh
        rotation-x={-Math.PI / 2}
        receiveShadow
      >
        <planeGeometry args={[20, 10]} />
        <meshStandardMaterial color="#70543E" />

...

And add castShadow and receiveShadow to the dog as well.

TYPESCRIPT
//client/src/components/Dog.tsx
...

        <skinnedMesh
          name="Dog"
          geometry={dogGeom.geometry}
          material={materials.AtlasMaterial}
          skeleton={dogGeom.skeleton}
          rotation={[-Math.PI / 2, 0, 0]}
          scale={100}
          castShadow
          receiveShadow
        >
...

5. Implementing Cursor Tracking for the Dog

After clicking on the plane, the dog will move from its current location to the click location.

TYPESCRIPT
//client/src/Scene.tsx
import { Environment, OrbitControls, useCursor } from "@react-three/drei";
import { useState } from "react";
import { Dog } from "./components/Dog";
import * as THREE from "three";

export const Scene = () => {

  const [onFloor, setOnFloor] = useState(false);
  const [dogPosition, setDogPosition] = useState({ x: 0, y: 0, z: 0 });

  const handlePlaneClick = (e: { point: { x: number; z: number; }; }) => {
    // Frissítsük a kutya pozícióját az új értékekre, ahol a felhasználó kattintott
    setDogPosition({ x: e.point.x, y: e.point.y, z: e.point.z });
  };
  useCursor(onFloor);
	return (

...

			<mesh
        rotation-x={-Math.PI / 2}
        onClick={handlePlaneClick}
        onPointerEnter={() => setOnFloor(true)}
        onPointerLeave={() => setOnFloor(false)}
        receiveShadow
      >
        <planeGeometry args={[20, 10]} />
        <meshStandardMaterial color="#70543E" />
      </mesh>
			<Dog
        position={
          new THREE.Vector3(dogPosition.x, dogPosition.y, dogPosition.z)
        }
        dogColor={"#222222"}
      />

...

We consider the distance of the dog from its current to its future location, and if the distance between the two is greater than 0.1, then the system switches to a run animation.

TYPESCRIPT
//client/src/components/Dog.tsx
import * as THREE from "three";

import { useEffect, useRef, useMemo, useState } from "react";
import { useGLTF, useAnimations } from "@react-three/drei";
import { useFrame, useGraph } from "@react-three/fiber";
import { SkeletonUtils } from "three-stdlib";
import { DogProps } from "../lib/types";

const SPEED = 0.1;

export function Dog({ dogColor = "green", ...props }: DogProps) {
  // eslint-disable-next-line react-hooks/exhaustive-deps
  const position = useMemo(() => props.position, []);
  const group = useRef<THREE.Group>(null);
  const { scene, materials, animations } = useGLTF("/Dog-transformed.glb");
  const clone = useMemo(() => SkeletonUtils.clone(scene), [scene]);
  const { nodes } = useGraph(clone);
  const { actions } = useAnimations(animations, group);
  const [animation, setAnimation] = useState(
    "AnimalArmature|AnimalArmature|AnimalArmature|Idle"
  );
  const dogGeom = nodes.Dog as THREE.SkinnedMesh;
  useEffect(() => {
    const currentAction = actions[animation];
    if (currentAction) {
      currentAction.reset().fadeIn(0.32).play();
      return () => {
        currentAction.fadeOut(0.32);
      };
    }
  }, [actions, animation]);

  useFrame(() => {
    // Ensure the group and its target position are defined
    if (!group.current || !props.position) {
      return;
    }
    // Calculate the distance to the target position
    const distance = group.current.position.distanceTo(props.position);
    // Check if the character needs to move
    if (distance > 0.1) {
      // Calculate the direction vector towards the target position
      const direction = props.position
        .clone()
        .sub(group.current.position)
        .normalize();

      // Move the character towards the target position
      const moveStep = direction.multiplyScalar(SPEED);
      group.current.position.add(moveStep);

      // Make the character face the target position
      group.current.lookAt(props.position);

      // Switch to the "run" animation if not already running
      if (animation !== "AnimalArmature|AnimalArmature|AnimalArmature|Run") {
        setAnimation("AnimalArmature|AnimalArmature|AnimalArmature|Run");
      }
    } else {
      // Switch to the "idle" animation if not already idle and the character is close enough to the target position
      if (animation !== "AnimalArmature|AnimalArmature|AnimalArmature|Idle") {
        setAnimation("AnimalArmature|AnimalArmature|AnimalArmature|Idle");
      }
    }
  });

...

II. Creating a Server and Connecting It to the Client Side

Multiplayer game development involves setting up and managing communication between the client and server, allowing players to see and interact with each other in real-time within a shared virtual space. Below, we will demonstrate how to create a simple multiplayer infrastructure using a server and client-side application that utilizes Socket.ioSocket.io for communication.

1. Server Installation

POWERSHELL
cd ..

POWERSHELL
mkdir server

POWERSHELL
cd server

POWERSHELL
npm i socket.io nodemon

Modify the package.json to start the server

JSON
//server/package.json
"scripts": {
    "dev": "nodemon index.js"
  },

2. Creating a Socket.io Server Side System

Through Socket.ioSocket.io, we can manage our character's position, color, and handle position changes whenever one character moves. In the current example, we will create a simple Map system, which ensures that every key is unique, facilitating the management of entities, especially when dynamically adding or removing elements. This is particularly useful in multiplayer games, where each player or character has a unique identifier.

For newly created characters, we will set a random position and a random brown color.

JAVASCRIPT
//server/index.js
import { Server } from "socket.io";

const io = new Server({
    cors: {
        origin: "http://localhost:5173",
    },
});

io.listen(3001);

const characters = new Map();

const randomPosition = () => {
    return [Math.random() * 10, 0, Math.random() * 3];
};

const randomBrownHexColor = () => {
    const red = Math.floor(Math.random() * 50) + 100;
    const green = Math.floor(Math.random() * 30) + 70;
    const blue = Math.floor(Math.random() * 20) + 30;

    // Convert the components to hexadecimal and format the color
    const hexColor = `#${red.toString(16)}${green.toString(16)}${blue.toString(16)}`;

    return hexColor;
};

io.on("connection", (socket) => {
    console.log("user connected");

    // Add character to Map
    characters.set(socket.id, {
        id: socket.id,
        position: randomPosition(),
        dogColor: randomBrownHexColor()
    });

    // Emit all characters to all clients
    io.emit("characters", Array.from(characters.values()));

    socket.on("move", (position) => {
        if (characters.has(socket.id)) {
            const character = characters.get(socket.id);
            character.position = position;
            // Emit updated characters
            io.emit("characters", Array.from(characters.values()));
        }
    });

    socket.on("disconnect", () => {
        console.log("user disconnected");
        // Remove character from Map
        characters.delete(socket.id);
        // Emit updated characters
        io.emit("characters", Array.from(characters.values()));
    });
});

3. Adding the Socket.io Server to the Client

We will create an additional file to manage Socket.ioSocket.io. We'll update the characters and connection data with the server side.

TYPESCRIPT
//client/src/components/SocketProvider.tsx
import React, { useEffect, useState } from "react";
import { Character, SocketProviderProps } from "../lib/types";
import { SocketContext, socket } from "../lib/constants";

export const SocketProvider: React.FC<SocketProviderProps> = ({ children }) => {
  const [characters, setCharacters] = useState<Character[]>([]);

  useEffect(() => {
    const handleCharactersUpdate = (
      newCharacters: React.SetStateAction<Character[]>
    ) => {
      setCharacters(newCharacters);
    };

    socket.on("connect", () => console.log("Connected"));
    socket.on("characters", handleCharactersUpdate);

    // Return a cleanup function that correctly performs cleanup without returning a value
    return () => {
      socket.off("connect");
      socket.off("characters", handleCharactersUpdate);
    };
  }, []);
  return (
    <SocketContext.Provider value={{ characters }}>
      {children}
    </SocketContext.Provider>
  );
};

We will create a separate file for the constants to be exported, from where they will be easily accessible.

TYPESCRIPT
//client/src/lib/constants.ts
import { io } from "socket.io-client";
import { SocketContextType } from "./types";
import { createContext, useContext } from "react";

export const socket = io("http://localhost:3001");

export const SocketContext = createContext<SocketContextType>({
  characters: [],
});

export const useSocket = () => useContext(SocketContext);

TYPESCRIPT
//client/src/lib/types.ts
import { ReactNode } from "react";

export interface Character {
  dogColor: string;
  id: string;
  position: [number, number, number];
}

export type DogProps = {
  dogColor?: string;
  position: THREE.Vector3;
};

export interface SocketContextType {
  characters: Character[];
}

export interface SocketProviderProps {
  children: ReactNode;
}

Enhancing the App.tsx

TYPESCRIPT
//client/src/App.tsx
import { Canvas } from "@react-three/fiber";
import { Scene } from "./Scene";
import { SocketProvider } from "./components/SocketProvider";

function App() {
  return (
    <>
      <SocketProvider>
        <Canvas
          camera={{
            fov: 60,
            near: 0.1,
            far: 300,
            position: [0, 7, 8],
          }}
          shadows
        >
          <Scene />
        </Canvas>
      </SocketProvider>
    </>
  );
}

export default App;

Updating Scene.tsx to not only create one character but all of them. Thus, with the move changes, we now send the positions to the server and forward them to all other client sides.

TYPESCRIPT
//client/src/Scene.tsx
import { Environment, OrbitControls, useCursor } from "@react-three/drei";
import { useState } from "react";
import { Dog } from "./components/Dog";
import { socket, useSocket } from "./lib/constants";
import * as THREE from "three";

export const Scene = () => {
  const { characters } = useSocket();
  const [onFloor, setOnFloor] = useState(false);
  const [, setPosition] = useState({ x: 0, y: 0, z: 0 });
  const handlePlaneClick = (e: { point: { x: number; z: number } }) => {
    const newPosition = { x: e.point.x, y: 0, z: e.point.z };
    setPosition(newPosition);
    // The new updated code send to server
    socket.emit("move", [newPosition.x, newPosition.y, newPosition.z]);
  };

...

{characters.map((character) => (
        <Dog
          key={character.id}
          position={
            new THREE.Vector3(
              character.position[0],
              character.position[1],
              character.position[2]
            )
          }
          dogColor={character.dogColor}
        />
      ))}
    </>
  );
};

By opening multiple windows, it can be tested that multiple dogs appear on the website.

Where We Can Go From Here

This is a good starting point for React Three Fiber and understanding the 3D environment in Node.js. We looked into using Socket.ioSocket.io and a simple multiplayer solution during the Three.js example.

Code

The full code can be found at:

https://github.com/balazsfaragodev/socketio-react-three-fiber-tutorialhttps://github.com/balazsfaragodev/socketio-react-three-fiber-tutorial

Share this article

The Newsletter for Next-Level Tech Learning

Start your day right with the daily newsletter that entertains and informs. Subscribe now for free!