Building a RTC video chat application

Introduction to WebRTC

WebRTC __Web Real-Time Communication__ is a technology that enables peer-to-peer audio, video, and data sharing directly between browsers, without needing any plugins. This allows developers to create powerful, real-time applications, from video chat to file sharing, without relying on an intermediary server for media exchange. Let’s explore how WebRTC works and how we can build a video calling application with it.

Key Components of WebRTC

WebRTC works through several key components. First, ICE __Interactive Connectivity Establishment__ helps establish peer-to-peer connections even in complex network environments. Then, STUN __Session Traversal Utilities for NAT__ and TURN __Traversal Using Relays around NAT__ servers play crucial roles in establishing these connections across NATs and firewalls.

Finally, once connected, WebRTC enables media streaming and data transfer between users. Let's dive into each of these components.

Setting Up Peer Connections

To create a peer connection in WebRTC, I started by creating an RTCPeerConnection object in JavaScript. This allows me to handle the setup of ICE candidates, as well as add media tracks for video and audio streaming.

I also made some basic html page to be served by the express server. ((https://i.imgur.com/75eWGU1.png, My page looked like this when I started)) Basically there has to be two video elements, __the remote and local video__

javascript

const peerConnection = new RTCPeerConnection(config); 
peerConnection.onicecandidate = (event) => {
    if (event.candidate) {
        sendCandidateToOtherPeer(event.candidate);
    }
};

peerConnection.ontrack = (event) => {
    const [stream] = event.streams;
    document.getElementById('remoteVideo').srcObject = stream;
};

The ICE Process

The ICE protocol enables devices to discover each other and establish connections. ICE candidates are generated and sent to the other peer until an optimal path is found. These candidates are essential to establishing a direct connection, particularly for users behind NATs.

STUN and TURN Servers

STUN servers help each device determine its own public IP address and establish the direct connection. If direct connections fail, TURN servers step in by relaying the media data between peers. This fallback process ensures that connections are still possible when NAT traversal is challenging.

Handling Offers and Answers

The offer/answer model in WebRTC is used to negotiate connection settings between peers. One peer creates an offer __a description of its media configuration__ and sends it to the other peer, which replies with an answer to complete the negotiation.

javascript

const createOffer = async () => {
    const offer = await peerConnection.createOffer();
    await peerConnection.setLocalDescription(offer);
    sendToRemotePeer({ offer });
};

peerConnection.onicecandidate = ({ candidate }) => {
    sendToRemotePeer({ candidate });
};

// When receiving answer from remote peer
peerConnection.setRemoteDescription(new RTCSessionDescription(answer));

Handling ICE Candidates

After the initial offer and answer, ICE candidates are exchanged between peers until an optimal connection is established. Each candidate represents a potential network route, and peers will test each route until they find the most effective path.

javascript

peerConnection.onicecandidate = (event) => {
    if (event.candidate) {
        sendCandidateToOtherPeer(event.candidate);
    }
};

Adding Media Streams

WebRTC allows us to add media tracks to the peer connection. Once connected, these media tracks enable audio and video streaming between the peers. Here’s how to capture video and add it to our peer connection.

javascript

navigator.mediaDevices.getUserMedia({ video: true, audio: true })
    .then((stream) => {
        stream.getTracks().forEach((track) => peerConnection.addTrack(track, stream));
        document.getElementById('localVideo').srcObject = stream;
    })
    .catch((error) => console.error('Error accessing media devices.', error));

Error Handling and Disconnects

WebRTC offers events and error-handling mechanisms to manage disconnects, network failures, or ICE candidate issues. Using `oniceconnectionstatechange` allows us to monitor the connection state and respond accordingly if the connection drops.

javascript

peerConnection.oniceconnectionstatechange = () => {
    if (peerConnection.iceConnectionState === 'disconnected') {
        console.log('Disconnected from peer');
        // Handle reconnection or cleanup
    }
};

In conclusion, WebRTC provides a powerful set of tools for building real-time communication applications. By handling offers, answers, and ICE candidates, and integrating STUN/TURN servers, we can enable direct media streaming and data transfer between peers, opening up possibilities for a wide range of real-time web applications.