Skip to main content

Table 1 Summary of recent studies on human activity recognition through data fusion

From: Multi-sensor fusion based on multiple classifier systems for human activity identification

Authors

Activities

Sensors/position

Algorithms/evaluation

Fusion type

Strength

Weakness

Tolstikov [34], Amoretti et al. [35], Sebak et al. [38], Tunca et al. [10], Qui et al. [4]

Static: sitting, standing, lying, sleeping, idle

Mobility based: walking, climbing stairs, leaving home

Household: prepare breakfast, prepare dinner, drink

Daily hygiene: shower, use toilet

Sensors: Camera, accelerometer, gyroscope, magnetometer, binary sensor

Position: ankle, toilet flush, cupboard

Sampling rate: ~ 100 Hz

Algorithm: dynamic Bayesian network, Kalman filtering, Dempster–Shafer

Evaluation metrics: error rates, accuracy, computation time, precision, recall, F-measure

Data fusion

Provide a simple and real-time, computationally efficient and independent implementation of human activity recognition

Inability to handle a long sequence of activities. Moreover, the approach is sensitive to sensor position, noise and sometimes impractical to implement

Nishida et al. [47], Spinsante et al. [5], Shoaib et al. [9], Xu et al. [50], Chen and Wang [8], Berenguar et al. (2017), Zdravevski et al. [74], Fong et al. [45], Dobbins et al. [6], Köping et al. [46], San-Segundo et al. [48], Li et al. [49], Pires et al. [29]

Static: sitting, lying down, standing, reading, making calls, kneeling

Mobility based: walking, jogging, ascending descending stairs, biking, stretching, object-lifting, bending, falling forward, falling left, falling backward, talking, cycling, nordic walking, jumping, car, running

Household: eating, watching TV, open door, close door, open fridge, close fridge, open dishwasher, close dishwasher, open drawer, close drawer, close table, drink form cup, vacuuming, ironing, shopping, cooking

Daily hygiene: brush teeth, bathing, washing clothes, drying clothes, house cleaning

Transition activities: stand up from sitting, stand up from laying, going up, going down, laying down from standing

Office: Typing, Writing, walking at computer

Harmful habit: smoking

Sensor: accelerometer, gyroscope, magnetometer, linear acceleration, gravity, heart rate, location sensor, air pressure, video

Position: front pocket, chest, ankle, thigh, forearm, wrist, waist, back, feet, Right shoulder

Sampling rate: 20–200 Hz

Algorithms: k-NN, ANN, DT, SVM, NB, LR, RF, Hoeffding tree, HMM, RNN, CNN, LSTM, LDC, QDC, POLYC, PARZENC, Gaussian Mixture Model, MLP, FNN, DNN

Evaluation metric: AUC, accuracy, precision, recall, F-measure, specificity, computation time, error rate, Kappa, FP, FN, Confusion Matrix

Feature fusion

Use to fuse sensor of diverse modalities and less sensitive to noise

Feature incompatibility, instability to sensor failure and signal variation reduce performance

Catal et al. [52], Gjoreski et al. [27], Peng et al. [54], Chowdhury et al. [19], Garcia-Ceja et al. [18], Saha et al. [16], Peng et al. [59]

Static: lying, sitting, standing, kneeling, sleeping, recreational activities

Mobility based activities: running, walking, cycling, ascending stairs, descending stairs, jogging, travel, sports

Household: washing dishes, mop floor, sweep the floor, eat chips, watch TV, shopping, brush teeth

Hygiene based: cleaning, wash hand

Office activities: working on the computer, meeting

Sensor: accelerometer, calorimeter, biosensor, body-media, accelerometer, sound, accelerometer, location data, vital signs

Positions: chest, wrist, mouth, pocket, table (sound), shirt pocket, belt, bag

sampling rate: 20–200 Hz

Base classifiers: MLR, SVR, GPR, M5P, MLP, SVM, RF, BDT, DNN, Adaboost, LR, J48, KNN

Fusion method: posterior probability, majority voting, single classifier based multi-view stacking

Evaluation metrics: F-measure, recall, precision, accuracy, Confusion Matrix

Multiple classifier systems

Can handle complex activity details, high dimensional sensor data and uncertainty by systematic classifier fusion. Combine heterogeneous and homogeneous classifier to reduce variance and ambiguity that are likely to occur in the single classifier

External knowledge dependencies and may be computationally complex based on the base classifier