Emotion Through Motion Detection

Today, more than anything else, the security apparatus of our nation is concerned over implementing impenetrable security measures at major airports to protect the flying public.  To that measure, government agencies, such as the TSA have implemented a variety of tools, mechanisms, and procedures to proactively detect threats, and mitigate any potential risks associated with mal-intent by rouge agents and terrorists.  To aid in that initiative, National Science Foundation (NSF) among other government and quasi-government organizations have dedicated intellectual resources to develop and/or leverage technology to deploy safeguards from such threats.  One such example is, NSF’s partnership with Christoph Bregler and his team of research fellows from New York University.  Chris Bregler and his team have been working on deploying the technology behind motion detection, primarily used in developing video games and animation, to detect human emotions.

Developers use special body suits with hundreds of embedded sensors in capturing body part movements to replicate them into character movements in developing video games.  Invisible to the human eye, ‘green dots’ disseminated from those motion detection sensors are read by programs give the video game characters a reference to reality.   Mr. Bregler and his team of scientists are using those ‘green dots’ to analyze the movement and intensity of facial expressions, jaw movement, the blinking and shifting of eyes and lips to interpret emotions.   The difference is that instead of relying on special body suits or technical appendages, Mr. Bregler and his team can use the program they have developed to apply it on to video footage to measure and translate facial expressions to correlate to human emotions.  To test this program out, they have taken video footage of past speeches from Fidel Castro, JFK, President Obama, and other world leaders to measure the intensity of their facial expressions and jaw line movement to relate it to their moods for those given circumstances.  Now, one might ask, why is this important in the fight against terrorism?  Actually, the answer is quite simple – this technology can assist and supplement existing mechanisms to proactively detect the most concealed weapon- intention to cause harm.  If law enforcement officials can scan individuals and their facial expressions and speech by analyzing a video footage to derive meaning, they are that much more capable of thwarting imminent threats.