Creating an AI-based “Try Not To Laugh” Game

I always love to apply new tools to existing workflows. Connecting the ideas from different areas excites me. Having fun is just enough for me to spend time on the development. From this perspective, making a face recognition-based game came to my mind when I was learning a JS framework.

Learning a programming language or framework is generally straightforward. There are well-structured online courses that claim to get you from zero to hero. But when you complete them, you have only got a digital certificate or a new bullet on your CV. What about converting your knowledge to experience? 

Working on an idea found by you makes you better understand what you have learned and apply it by not following any instructions. At that point, I had decided to create a mini-game when I was learning the fundamentals of ReactJS. Actually, React is a general-purpose framework to build user interfaces. Therefore it is not explicitly designed for AI apps, but it is okay. Creating any unique thing via the learned tool is preferable.

Some of you may know the Youtube Series called Soğuk Savaş. In which two players ask awkward questions to each other one by one. If the player does not laugh when the funny answer is announced, he wins. I realized, when we are watching the videos, we are also trying to not laugh. If a computer can detect our smiling faces, we can play without any real player.

I have started to search on face detection APIs using Javascript and decided to use face-api.js. It is a javascript module built on top of tensorflow.js core, which implements several CNNs (Convolutional Neural Networks) to solve face-related problems for the web.

My goal was to detect face via webcam and understand whether the person is smiling or not during the game. This API presents more than my expectations. It provides the percentage of smiling, too.

In my React app, I have imported face-api and other design related libs.

import React, { useState } from 'react';
import useInterval from './customHooks/useInterval';
import * as faceapi from 'face-api.js'
import stepsDB from './data/steps.json';
import Button from 'react-bootstrap/Button';
import 'bootstrap/dist/css/bootstrap.min.css';
import './App.css';
import 'typeface-roboto';

Including tinyFaceDetector faceExpressionNet models are sufficient for my purpose. Webcam streaming is included after the models loaded.

Promise.all([
	faceapi.nets.tinyFaceDetector.loadFromUri('./models'),
	faceapi.nets.faceExpressionNet.loadFromUri('./models')
]).then(startVideo);

function startVideo() {
	navigator.getUserMedia(
		{ video: {} },
		stream => video.srcObject = stream,
		err => console.error(err)
	)
}

At the beginning of the app the states are defined.

const timeStep = 5000; //milliseconds
const tickInterval = 250;
const [gameStatus, setGameStatus] = useState('initial'); //'initial', 'playing, 'win', 'fail'
const [smilePercent, setSmilePercent] = useState(0); //0-1
const [step, setStep] = useState({ id: 0, question: '', answer: '' });
const [answerVisibility, setAnswerVisibility] = useState(false); //true, false
const [remainingTime, setRemainingTime] = useState(timeStep);
const [remainingTimeStarted, setRemainingTimeStarted] = useState(false);

Basically, when the awkward questions are displayed to the player, it checks the smiling percentage via webcam every 250 milliseconds. When the smiling rate reached 0.3 thresholds that I decided the player is smiled enough and the player fails. If he/she completes all questions without any laugh he/she won the game.

	useInterval(async () => {
		if (gameStatus === 'initial' || gameStatus === 'playing' ) {
			if (!video.srcObject) return;
			const detections = await faceapi.detectAllFaces(video, new faceapi.TinyFaceDetectorOptions()).withFaceExpressions();

			if (detections.length) {
				detections.forEach(detection => {
					setSmilePercent(detection.expressions.happy);

					if (smilePercent > 0.3 && gameStatus === 'initial') start();
					if (answerVisibility && smilePercent > 0.2 && remainingTimeStarted) {
						setAnswerVisibility(false);
						setRemainingTimeStarted(false);
						setRemainingTime(timeStep);
						setGameStatus('fail');
					}
				});
			}

			if (answerVisibility) {
				if (remainingTime <= 0) {
					setAnswerVisibility(false);
					setRemainingTimeStarted(false);
					setRemainingTime(timeStep);
					goNextStep();
				}
				else setRemainingTime(remainingTime => remainingTime - tickInterval);
			}
		}
	}, tickInterval);

	function start() {
		steps = stepsDB.slice(0);
		const randomIndex = Math.floor(Math.random() * steps.length);
		const nextStep = steps.splice(randomIndex, 1).pop();
		setStep(nextStep);
		setGameStatus('playing');
	}

	function showAnswer() {
		setAnswerVisibility(true);
		setRemainingTimeStarted(true);
	}

	function goNextStep() {
		if (steps.length === 0) {
			setGameStatus('win');
			return;
		}

		const randomIndex = Math.floor(Math.random() * steps.length);
		const nextStep = steps.splice(randomIndex, 1).pop();
		setStep(nextStep);
	}
    
    return...

It is actually that simple and about a hundred lines of code. But making and testing are entertaining. I have collected the questions and answers from the videos by manually typing them. Although it is also fun to watch those videos. Another module could automatically detect text from the videos to store a questions and answers database.

The game content is in Turkish since I have only access to the Turkish dataset. It could be added to any language later. I have shared the code from my GitHub repo, and also you can play it via the demo link below.

DEMO Link: https://try-not-to-laugh.netlify.app/

Code: https://github.com/ahmetbersoz/try-not-to-laugh

I am planning to share the projects I have made just for fun via my blog. Follow me from my Twitter address @ahmetbersoz or subscribe the blog from box below. Until the next post… Best.

ABE.