Skip to main content

Table 1 Summary of classifier accuracy and AUC results after 100 samples, for machine-driven, user-driven, and collaborative selection

From: Visual analytics for collaborative human-machine confidence in human-centric active learning tasks

Method User relabelled Highest accuracy Area under curve (AUC)
SIL IL IDA CBA SIL IL IDA CBA
Machine-driven
 RS 41 77 94 86 87 54.95 77.15 64.05 63.90
 DS 55 71 89 83 86 38.35 70.75 50.55 53.10
 LCRS 75 82 94 93 92 52.70 77.85 64.85 66.10
 LCDS 68 72 84 71 85 45.75 67.05 52.65 57.55
 MCRS 61 79 94 92 93 56.00 77.40 71.20 70.65
 MCDS 72 64 90 80 88 44.20 74.65 56.65 58.00
 ECRS 68 81 92 94 94 54.60 77.55 73.20 73.40
 ECDS 69 67 86 81 79 45.10 71.25 59.75 59.10
User-driven 37 80 96 89 88 60.05 84.45 73.20 73.30
Collaborative 48 82 95 90 94 54.50 83.05 72.60 74.80
  1. The number of user re-labelled samples, and accuracy scores for the four labelling schemes: single-instance labelling (SIL), inferred labelling (IL), image data augmentation (IDA), and confidence-based augmentation (CBA). Results in italic highlight where CBA improves on the IDA approach