Human-in-the-loop

Human-in-the-loop means a person confirms or corrects what a system proposes, before anything is acted on.

Definition

Human-in-the-loop is a way of working in which an automated system makes proposals and a person decides. The system puts something forward, the person confirms, corrects or rejects it, and that decision is the result. It stands against fully autonomous execution, where a system acts with nobody in between.12

What it means for the calculation

In estimating, this is the difference between a number you can defend and a number you have to believe. A proposal the estimator can check and adjust stays their own quotation; a number that goes to the customer unchecked is a risk that lands on them the moment it turns out wrong. On top of that, a correction carries information: where a person steps in is exactly the point where an automated proposal and that shop's reality diverge.

Where this comes up

In the description of AI systems that take part in a work process, and in regulation about human oversight of automated decisions. Subduxion uses the term for its own position: the estimator remains responsible for the decision.

Common mistakes

Using the term as reassurance without making it real: if in practice a person clicks everything through with no way to see what a proposal rests on, there is no oversight, only a button. Also assuming human oversight moves the responsibility; the decision stays with whoever signs it.

What it means for your calculation · most relevant in How Blake works

In estimating, this is the difference between a number you can defend and a number you have to believe. A proposal the estimator can check and adjust stays their own quotation; a number that goes to the customer unchecked is a risk that lands on them the moment it turns out wrong. On top of that, a correction carries information: where a person steps in is exactly the point where an automated proposal and that shop's reality diverge.

Read the research

Frequently asked questions

Is human-in-the-loop the same as working by hand?

No, the system does the groundwork; the person reviews and decides.

Does it slow a process down?

Reviewing costs time, but less than repairing a mistake that passed unseen.

What is the estimator's role in it?

They confirm or correct the proposal, and remain responsible for the quotation that comes out.

Notes & references

  1. Fails, J.A. & Olsen, D.R. (2003). Interactive Machine Learning. Proceedings of the 8th International Conference on Intelligent User Interfaces (IUI '03), 39–45. https://doi.org/10.1145/604045.604056 The paper most widely credited with establishing the interactive/human-in-the-loop machine learning paradigm; still cited as the field's foundational reference two decades later.
  2. Mosqueira-Rey, E., Hernández-Pereira, E., Alonso-Ríos, D., Bobes-Bascarán, J. & Fernández-Leal, Á. (2022). Human-in-the-loop machine learning: a state of the art. Artificial Intelligence Review, 56(4), 3005–3054. https://doi.org/10.1007/s10462-022-10246-w Recent, highly cited peer-reviewed survey mapping the field (active learning, interactive ML, machine teaching) — used here to ground that human-in-the-loop is an established research area, not a phrase Subduxion coined.