The outputs of the pipeline are the head pose, gaze angles, and image with redirected eye gaze. The last stage of the pipeline involves blending the eye patch back into the original video frame with an inverse transformation. The encoder estimates the gaze angle from the input eye patch along with a set of features, also known as embeddings.īased on these embeddings, the decoder performs redirection of the gaze in the input patch to make the face look forward.

The eye contact network has a disentangled encoder-decoder architecture. A 256 ✕ 64 px eye patch is cropped from the normalized frame and it is fed into the eye contact network. This head pose is then used to normalize the face in the video frame. This feature uses AI to apply a filter to the user’s webcam feed in real time and redirects their eye gaze toward the camera.įigure 3. To improve, augment, and enhance the user experience, we have developed NVIDIA Maxine Eye Contact. Many children and adults struggle to make and maintain eye contact. Maintaining eye contact is also sometimes challenging for various physiological reasons. This can be difficult if one is reading off a script or reviewing data on a computer screen.

It requires users continuously look directly at their cameras and not at their computer displays. To create an increased sense of presence and pick up on both verbal and non-verbal cues video conferencing technology must enable users to see and hear clearly.Įye contact plays a key role in establishing social connections and in face-to-face conversations it signifies confidence, connection, and attention. However, continuously making eye contact is not feasible in video conferencing scenarios. Video conferencing is at the heart of several streaming use cases such as vlogging, vtubing, webcasting, and even video streaming for remote work.

Download beyond eyes for free