The New Kid on the Block – Short Introduction into Foundation Models

Foundation models are at the heart of the third wave of artificial intelligence (AI).

These are a new generation of neuronal AI models that have been trained on gigantic amounts of data. The special feature of these models is that the knowledge of the AI acquired in the so-called pre-training is suitable as a basis for various applications and the AI is therefore versatile. This makes it possible to successfully use and deploy an AI across problems.

They allow the generation of any multimedia content on the basis of natural language text input from users (so-called prompts). The quality of the generated results represents a quantum leap in the evolution of AI, so that these models are also called foundation models. The main difference to existing AI methods lies in three points:

The focus of the Foundation models is on generating new content in the form of text, images or videos. They thus enable a multitude of new use cases that go beyond the primarily analytical application (such as the recognition of patterns, the classification of data points or the prediction of values). Foundation models now allow to automate creative tasks such as the generation of arbitrary text content, topic-specific illustrations or even programming code in high quality (illustration with use cases & examples).
The big breakthrough of the Foundation models came with the GPT-3 language model, launched in 2020 by the company OpenAI.
The interface for using these models has radically simplified, especially for non-technical users: control is not based on code, but on a prompt – a natural language paraphrase of the desired result – which is used by the models as input to generate the content. Many applications can be used as pay-per-use models with simple API or web-based interfaces that do not require a dedicated IT infrastructure.
The effort required to introduce and use these models has been drastically reduced. In contrast to classical AI, which usually requires extensive pre-processing of data and training of the models, these language models are “pre-trained” and many are freely available as open source. This means they can be used quickly – even for companies that do not have sufficient amounts of data – and the often very computationally and therefore cost-intensive training costs are eliminated.