Table 9 Classification performance of the Bayesian based classifier in identifying legitimate and adversarial users for 3 different attack scenarios (against naive adversaries, against strong adversaries, and against both strong and naive adversaries) for given question types for \(n=1\), \(n=3\), and \(n=6\) questions respectively
From: Evaluating smartphone-based dynamic security questions for fallback authentication: a field study
Against naive | Against strong+naive | Against strong | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
TPR (%) | FPR (%) | F1 score (%) | Accuracy (%) | TPR (%) | FPR (%) | F1 score (%) | Accuracy (%) | TPR (%) | FPR (%) | F1 score (%) | Accuracy (%) | ||
Call | n = 1 | 87.0 | 3.1 | 89.7 | 91.8 | 79.3 | 6.6 | 82.1 | 88.4 | 75.2 | 10.2 | 80.4 | 82.1 |
n = 3 | 95.9 | 4.7 | 95.9 | 96.0 | 90.7 | 2.1 | 92.8 | 95.7 | 89.1 | 2.7 | 92.7 | 93.1 | |
n = 6 | 100.0 | 4.2 | 98.2 | 97.8 | 98.1 | 0.6 | 98.3 | 98.9 | 95.9 | 0.8 | 97.4 | 97.5 | |
SMS | n = 1 | 76.8 | 5.0 | 82.3 | 86.2 | 69.7 | 10.7 | 73.7 | 81.6 | 65.1 | 14.3 | 72.1 | 74.5 |
n = 3 | 93.4 | 4.4 | 94.7 | 94.1 | 88.1 | 8.2 | 87.4 | 90.2 | 86.2 | 12.6 | 87.4 | 86.8 | |
n = 6 | 97.8 | 4.0 | 97.3 | 96.8 | 94.2 | 5.5 | 92.3 | 94.3 | 92.8 | 7.8 | 93.1 | 92.6 | |
Location | n = 1 | 84.3 | 7.1 | 86.0 | 89.1 | 75.1 | 14.4 | 73.4 | 82.3 | 70.8 | 17.0 | 74.0 | 77.0 |
n = 3 | 96.3 | 3.2 | 96.8 | 96.3 | 93.2 | 10.5 | 88.3 | 90.7 | 88.7 | 16.7 | 87.3 | 85.8 | |
n = 6 | 99.2 | 0.0 | 99.6 | 99.5 | 97.1 | 7.7 | 92.1 | 93.8 | 95.2 | 14.5 | 91.8 | 90.4 | |
App | n = 1 | 89.6 | 3.7 | 93.0 | 92.2 | 88.3 | 12.9 | 85.9 | 87.3 | 86.5 | 21.3 | 85.7 | 82.7 |
n = 3 | 95.0 | 0.0 | 97.3 | 97.1 | 93.8 | 2.8 | 94.7 | 95.7 | 92.7 | 5.0 | 94.3 | 93.4 | |
n = 6 | 94.7 | 0.0 | 97.2 | 97.1 | 94.0 | 0.2 | 96.5 | 97.6 | 92.8 | 0.6 | 95.8 | 95.9 | |
Music | n = 1 | 83.1 | 6.5 | 87.0 | 87.7 | 79.9 | 16.7 | 76.1 | 81.6 | 78.2 | 28.7 | 76.2 | 74.2 |
n = 3 | 94.2 | 0.2 | 96.7 | 96.8 | 91.9 | 9.3 | 88.7 | 91.1 | 88.6 | 15.6 | 87.7 | 85.9 | |
n = 6 | 96.5 | 0.0 | 98.2 | 98.2 | 95.9 | 5.7 | 93.3 | 94.8 | 95.2 | 9.3 | 93.4 | 92.6 | |
Activity | n = 1 | 93.3 | 6.2 | 93.4 | 93.2 | 87.2 | 17.2 | 81.5 | 83.9 | 83.1 | 27.1 | 81.5 | 78.1 |
n = 3 | 95.9 | 0.0 | 97.8 | 98.0 | 93.5 | 7.4 | 90.8 | 92.6 | 91.4 | 17.1 | 90.1 | 86.8 | |
n = 6 | 96.5 | 0.0 | 98.2 | 98.0 | 95.5 | 4.0 | 94.4 | 95.6 | 93.7 | 7.6 | 94.2 | 93.2 | |
Battery | n = 1 | 89.3 | 7.2 | 91.8 | 90.3 | 86.4 | 8.0 | 87.9 | 89.5 | 84.9 | 8.8 | 88.5 | 87.7 |
n = 3 | 94.0 | 1.8 | 96.1 | 95.9 | 90.7 | 2.0 | 93.5 | 95.3 | 90.5 | 3.0 | 94.0 | 93.6 | |
n = 6 | 95.3 | 0.0 | 97.5 | 97.3 | 92.9 | 0.0 | 96.0 | 97.4 | 93.1 | 0.0 | 96.2 | 96.4 | |