Foundation-Models: Without the “Human in the Loop”?

For most use cases, foundation models need a “human in the loop”. This means that human experts or users need to be involved in the process of developing, adapting and validating the models. As already described, the great leap in quality from GPT-3 to ChatGPT was achieved with reinforcement learning based on human feedback.

Since a “human in the loop” is currently necessary for most use cases, the question arises as to whether this process is even possible without humans.

Human in the loop” plays a particularly important role in model adaptation and validation. The best-known foundation models such as GPT or DeBERTa can be used in many areas on the basis of the training data used and must be adapted for specific use cases. Here, human experts usually have to help adapt the model to the user’s needs and validate it to ensure that it works correctly and reliably.

If foundation models are to be used for inspiration and brainstorming to get im-pulses and ideas, the output of the model is not directly used as finished content. Rather, the model supports people in creating new content through inspiration and ideas.

If the model is used for imitations, such as “write the text in the style of Shake-speer”, little or no quality assurance by the human is usually required. Similarly, content such as blog posts that have a certain tolerance for error can be used without much human adaptation. In this case, it is a matter of teaching the model company-specific topics and expressions (cf. Figure 2). If, on the other hand, robust end-to-end processes or company applications are to be developed, specific fine-tuning by humans is necessary. In addition, the error tolerance is particularly low here; an incorrect specification in a product description or in a software code can have serious consequences.