If you read through the relevant trade media and business networks such as LinkedIn, you can’t help but get caught up in one of the countless reports and discussions on ChatGPT. The ease of use of ChatGPT turns language models into a kind of “people AI”.
To generate content, ChatGPT uses a technique called “Generative Language Modelling”. In doing so, the model analyses the previous text and determines which words and phrases are most likely to follow. Based on this information, ChatGPT generates new sentences that follow the context of the original text.
ChatGPT is a special version of the Generative Pre-trained Transformer (GPT)-3.5 model, optimised for the creation of chatbots and text dialogue systems. The main differences between GPT-3.5 and ChatGPT lie in the training data used and the fine-tuning of the model to specific tasks. The similarities between GPT-3 and ChatGPT lie in the underlying architecture and technology. Both models are based on the Transformer architecture and have been trained on large amounts of text. They are capable of performing a wide range of language tasks and can be applied to a variety of applications.
Language models such as ChatGPT learn firstly from the underlying base of training data, which typically consists of large portions of the World Wide Web as well as digi-tal editions of books and professional publications. Providers of language models are systematically trying to increase the learning base. Thus, more and more online sources are being tapped and integrated into the knowledge base.
The limitations of AI-based learning lie in the lack of evaluation of results. An AI model also reproduces right-wing extremist, sexist or other undesirable results on the basis of the training data. In order to correct this, a feedback loop by human acutators is required (so-called reinforceent learning from human feedback). Thus, the great leap in quality from GPT-3 to ChatGPT is due to the fact that a large number of click-workers evaluated and corrected the results of GPT-3 at a low hourly rate of 1-2 dollars. Thus, AI in the sense of “augmented intelligence” means a combination of human and machine.
