Visual analytics for collaborative human-machine confidence in human-centric active learning tasks

Table 1 Summary of classifier accuracy and AUC results after 100 samples, for machine-driven, user-driven, and collaborative selection

Method	User relabelled	Highest accuracy				Area under curve (AUC)
Method	User relabelled	SIL	IL	IDA	CBA	SIL	IL	IDA	CBA
Machine-driven
RS	41	77	94	86	87	54.95	77.15	64.05	63.90
DS	55	71	89	83	86	38.35	70.75	50.55	53.10
LCRS	75	82	94	93	92	52.70	77.85	64.85	66.10
LCDS	68	72	84	71	85	45.75	67.05	52.65	57.55
MCRS	61	79	94	92	93	56.00	77.40	71.20	70.65
MCDS	72	64	90	80	88	44.20	74.65	56.65	58.00
ECRS	68	81	92	94	94	54.60	77.55	73.20	73.40
ECDS	69	67	86	81	79	45.10	71.25	59.75	59.10
User-driven	37	80	96	89	88	60.05	84.45	73.20	73.30
Collaborative	48	82	95	90	94	54.50	83.05	72.60	74.80

The number of user re-labelled samples, and accuracy scores for the four labelling schemes: single-instance labelling (SIL), inferred labelling (IL), image data augmentation (IDA), and confidence-based augmentation (CBA). Results in italic highlight where CBA improves on the IDA approach