• home
  • About the project
  • Miniature know-how
  • Language
    • Polski
    • English
Mi

NVED

Non-Verbal Expression Detector
Preview: default camera Preview: back camera
Parameters
cnf
gui
mirror
pointer
Description

NVED is one of the “tool” miniatures. It is built on a simple AI platform – a detector of emotional expressions on the faces of people detected in the field monitored by the camera. NVED detects a combination of 7 emotional states: calm (“neutral” state), joy, fear, surprise, disgust, anger, and sadness. The miniature can send the results of its actions externally and can be easily adapted to function as a component of your own work.

Naturally, this Miniature “reads” facial expressions, not internal emotional states—so it can be an interesting tool for, for example, introspective experimentation and exploring the differences between the two. Two other Miniaturas that use a similar (though slightly hacked) mechanism are “MianX” and “Empator.”

Technical documentation

Messages sent by Miniature

  • cnf 0.0 … 1.0; “confidence level” of the model detecting emotional expression
  • famo user | environment (or env); default user; [facingMode] allows forcing the use of the front (user) or rear (environment) camera on devices equipped with dual cameras (most modern mobile devices)
  • gui 0 or 1; default 1; hides/shows the graphical user interface panel
  • h vertical camera resolution
  • mirror 0 or 1; default 1; disables/enables mirroring of the image from the capture device (usually a camera)
  • pointer 0 or 1; default 1; hides/shows the mouse cursor
  • w horizontal camera resolution
  • worker 0 or 1; default 0; allows blocking the browser’s built-in mechanisms that suspend the program when the window is not visible
  • ws address; disabled by default; enables and sets the address (usually localhost:9001) for WebSocket communication (allows external control of Miniature and customization)
  • xws address; disabled by default; enables and sets the address (usually ws://localhost:9001) with protocol specification for WebSocket communication (allows external control of Miniature and customization)

Messages passed via WebSocket

Messages to which Miniature responds

Not applicable

Messages sent by Miniature

  • nved faces reports the number of recognized faces
  • nved face face parameters: index, normalized horizontal position, normalized vertical position, normalized width, normalized height, “calm” (normalized value), “happiness” (normalized value), “anger” (normalized value), “sadness” (normalized value), “disgust” (normalized value), “surprise” (normalized value), “fear” (normalized value)

MIDI

Messages to which Miniature responds

not applicable

Messages sent by Miniature

not applicable

Communication with MaxMSP

Messages to which Miniature responds

not applicable

Messages sent by Miniature

not applicable

Source code

index.html

sketch.js

gui.js

worker.js

ws.js

Cgk8ZGl2IGNsYXNzPSJqcy1tYWlubWVudSI+CgkJPGRpdiBjbGFzcz0ibWVudS1tYWlubWVudS1jb250YWluZXIiPjx1bCBpZD0ibWVudS1tYWlubWVudS0xIiBjbGFzcz0ibWVudSI+PGxpIGNsYXNzPSJtZW51LWl0ZW0gbWVudS1pdGVtLXR5cGUtcG9zdF90eXBlIG1lbnUtaXRlbS1vYmplY3QtcGFnZSBtZW51LWl0ZW0taG9tZSBtZW51LWl0ZW0tMzI3Ij48YSBocmVmPSJodHRwczovL21pLndyb2NlbnRlci5wbC9lbi8iPmhvbWU8L2E+PC9saT4KPGxpIGNsYXNzPSJtZW51LWl0ZW0gbWVudS1pdGVtLXR5cGUtcG9zdF90eXBlIG1lbnUtaXRlbS1vYmplY3QtcGFnZSBtZW51LWl0ZW0tMzI1Ij48YSBocmVmPSJodHRwczovL21pLndyb2NlbnRlci5wbC9lbi9vLXByb2pla2NpZS8iPkFib3V0IHRoZSBwcm9qZWN0PC9hPjwvbGk+CjxsaSBjbGFzcz0ibWVudS1pdGVtIG1lbnUtaXRlbS10eXBlLXBvc3RfdHlwZSBtZW51LWl0ZW0tb2JqZWN0LXBhZ2UgbWVudS1pdGVtLTk2NCI+PGEgaHJlZj0iaHR0cHM6Ly9taS53cm9jZW50ZXIucGwvZW4vbWFudWFsLyI+TWluaWF0dXJlIGtub3ctaG93PC9hPjwvbGk+CjxsaSBjbGFzcz0icXRyYW54cy1sYW5nLW1lbnUgcXRyYW54cy1sYW5nLW1lbnUtZW4gbWVudS1pdGVtIG1lbnUtaXRlbS10eXBlLWN1c3RvbSBtZW51LWl0ZW0tb2JqZWN0LWN1c3RvbSBjdXJyZW50LW1lbnUtcGFyZW50IG1lbnUtaXRlbS1oYXMtY2hpbGRyZW4gbWVudS1pdGVtLTMyOCI+PGEgaHJlZj0iIyIgdGl0bGU9IkVuZ2xpc2giPkxhbmd1YWdlPC9hPgo8dWwgY2xhc3M9InN1Yi1tZW51Ij4KCTxsaSBjbGFzcz0icXRyYW54cy1sYW5nLW1lbnUtaXRlbSBxdHJhbnhzLWxhbmctbWVudS1pdGVtLXBsIG1lbnUtaXRlbSBtZW51LWl0ZW0tdHlwZS1jdXN0b20gbWVudS1pdGVtLW9iamVjdC1jdXN0b20gbWVudS1pdGVtLTk2NSI+PGEgaHJlZj0iaHR0cHM6Ly9taS53cm9jZW50ZXIucGwvcGwvbnZlZC8iPlBvbHNraTwvYT48L2xpPgoJPGxpIGNsYXNzPSJxdHJhbnhzLWxhbmctbWVudS1pdGVtIHF0cmFueHMtbGFuZy1tZW51LWl0ZW0tZW4gbWVudS1pdGVtIG1lbnUtaXRlbS10eXBlLWN1c3RvbSBtZW51LWl0ZW0tb2JqZWN0LWN1c3RvbSBjdXJyZW50LW1lbnUtaXRlbSBtZW51LWl0ZW0tOTY2Ij48YSBocmVmPSJodHRwczovL21pLndyb2NlbnRlci5wbC9lbi9udmVkLyIgYXJpYS1jdXJyZW50PSJwYWdlIj5FbmdsaXNoPC9hPjwvbGk+CjwvdWw+CjwvbGk+CjwvdWw+PC9kaXY+CTwvZGl2PgoJPGJ1dHRvbiBjbGFzcz0ianMtYTExeSI+PGltZyBhbHQ9IkhpZ2ggY29udHJhc3QiIHNyYz0iaHR0cHM6Ly9taS53cm9jZW50ZXIucGwvd3AtY29udGVudC90aGVtZXMvbWkyMDI2L2ltZy9rb250cmFzdC5zdmciPjwvYnV0dG9uPgo=
NVED — Mi
KPO Unia Europejska #NextGenerationEU Rzeczpospolita Polska
WRO ART CENTER
  • About the project
  • Miniature know-how