Winner project of the Hackatum 2021 wild track.

Panim is a new form of communication. It is a lot more interactive than just a phone call, but not quite as intensive as a video call. An avatar is used to represent you in the call. Your camera is used locally to detect your facial expressions and motion, but the video is not actually sent to your peers, just your face landmarks - 468 points that describe your current expression.

We use a pre-trained TensorFlow.js model.

The peer to peer communication is done with PeerJS, with connections borkered by PeerServer Cloud. The audio is sent as is, but no video is sent, just the tensor of face landmarks.

We use WebGL to render the avatar. We bind the chosen avatar image as a texture to the face geometry.