The problem of emotion detection based on the measured physiological changes in the human body had received a significant attention lately (Nie et. al. 2011)[1]. However, the instant detection of each human’s emotions had not been thoroughly studied. The problem is that each person manifests emotions in a manner different from others. In social networks, for example, a person sending a message across the net may experience certain emotional status which need to be transmitted to the other party in the same manner voice or image are transmitted.
The human emotional status is rather intangible[2], and therefore cannot be directly measured. However, these emotions can be correlated to external and/or internal factors, which are rather tangible things, and hence they can be measured and analyzed. The internal factors come from different parts of the body in several forms such as electroencephalography (EEG), heart rate (HR), heart rate variability (HRV), pre-ejection period (PEP), stroke volume (SV), systolic blood pressure (SBP), diastolic blood pressure (DBP), skin conductance response (SCR), tidal volume (Vt), oscillatory resistance (Ros), respiration rate (RR), nonspecific skin conductance response rate (nSRR), skin conductance level (SCL), finger temperature (FT), and others (Kreibig 2010)[3].
These factors’ measurements are provided in wide ranges and often their impacts vary from a person to a person and for different postures for the same person. For example, a given measurement of some factors may relate to a person being happy, while the same measurements may reveal a rather “sad” status for another person. This kind of behavior lends itself naturally to fuzzy sets and fuzzy logic (zero and one, true and false or black and white cannot present this kind of data)[4].
In this paper, we will use fuzzy operations[5] to represent the knowledge about each factor. This will enable us to detect the emotion of a person using fuzzy inputs of the various factors. For example, we can use a fuzzy rule such as “IF (Temperature is High) AND (Heart Rate is High) THEN (Person is Excited).” Although fuzzy sets and operations are useful for representing the knowledge base, they fail to model the individual behavior of each and every person. Obviously, a model that is able to adapt to various categories of human responses would be preferred. Consequently, an adaptive learning mechanism is required to adjust the model if we were to cater for the differences in emotions between various humans. This requirement calls for the use of an adaptive learning system such as artificial neural networks (ANN) (Abraham 2005)[6]. However, the ANN model does not allow the use of fuzzy sets or rules, which is the more natural way of representing the relation between human emotions and human physical and physiological parameters. ANN uses exact and crisp values for representing the model’s input.
In order to utilize the benefits of both fuzzy logic and artificial neural networks, we will use the hybrid approach, which combines fuzzy logic and artificial neural networks in a single model.
The analysis and detection of human emotions using an expert system has a direct impact on several fields of the human life such as health, security, social networks, gaming, entertainment, commercials and others[3, 7]. The system will enable social networks (SN) participants to exchange emotions in addition to text, images, and videos.
In health related applications, for example, the interaction between a patient and doctor (in some critical cases) may become difficult or impossible[3], where the patient cannot explain his/her feelings to the doctor (e.g. coma infants, autism). The proposed system would enable the doctor to analyze and detect the patient’s emotions, even when the patient is unable to correctly define his emotional status.
In social networks people exchange all types of information such as text, video, images, and audios. The proposed model would enable communicating parties to detect the emotional status of their partners in a seamless automatic manner. In essence, a person chatting with a friend on the social network would be able to tell whether the other partner is sad, angry, embarrassed, afraid or happy without the need for the partner to explicitly state the emotional status.
Security is another area where the proposed model can be of significant impact. The model can be used to predict a crime before it occurs by detecting a criminal behavior based on the emotional status of the person attempting to commit a crime or breach the security at given facilities. This is based on the psychological status of the criminal before committing a crime. At an airport facility, for example, the system can identify individuals with certain emotional postures based on perceived measures of the individual’s heart rate, EEG frequencies, body temperatures and other measurable factors.
Gaming and entertainment is yet another area where the prediction of a person’s current emotion status is very useful. The system can detect the modes of customers based on the various factors studied and analyzed in this paper.
The rest of this paper is organized as follows. Related work is presented in Section 2. Section 3 provides an overview of the various factors which impact the human emotions. Section 4 presents the ANFIS model, used to build the neuro/fuzzy model. Section 5 presents and analyzes the model results. Conclusions are presented in Section 6.