The man who invented ChatGPT

A large part of the French poststructuralists would certainly have been delighted with the modern chatbots of the "Artificial Generative Intelligence" (AGI) generation. As an expression of the evidence of their language theories, of course.

In ChatGPT, the name says it all and is undoubtedly based on the findings of the Francophone classics. The G P T in GPT-3 means "Generative Pre-Trainend Transformer". The whole thing is based on deep learning - the attempt to process or craft natural language.

So what connects the ideas of literary scholars and language artists such as Roland Barthes, Julia Kristeva or Jacques Derrida with the Large Language Models (LLM) of today, which are used in chatbots such as ChatGPT from the American company Open AI? So. Similar to Barthes' "generative conception", LLMs attempt to generate an infinite set of sentences from a finite supply of elements and rules.

In Die Lust am Text (1973) Barthes states: “Text means tissue; but while this weave has hitherto always been understood as a product, a finished veil behind which the meaning (truth) resides, more or less concealed, we now emphasize the generative notion of the weave, that the text is formed by a constant Lichen arises and processes itself”.

In any case, the writer and literary critic's view of the constant interaction between reader, text and author corresponds quite precisely to the functional description of the chatbot technology.

The basic idea of ​​the computer-generated language games actually turns out to be quite simple. The AGI is first fed with words in the truest sense of the word. In the case of GPT-3, approximately one trillion (1012) characters found in words, phrases, and statements on the Internet, in books, and other documents.

The bot uses this pool of written language when the AGI formulates the best possible sentence as an answer to a question with the expected probability. Derrida understands his concept of "ecriture" in a similar way, which as a medium and mediator always semantically defines something new. "The text exists solely in a continuum of texts, with the preceding text being transformed into the succeeding one (intertextuality)". The words of the literary scholar Axel Schmitt read as if Derrida had advised the developers when calculating their models.

In Pre-Trained, however, lies the everlasting difference between author and reader, according to Barthes in "Death of the Author" (1967), the meaning of the text is not defined by the author but by the reader. For the Irish poet Paul Muldoon, this idea involves the "notion that reading also involves, to some extent, writing or even rewriting a text".

Viewed against this background, the technicians with their computational modulations appear as if they are following the directions, the puns staged by the novelists. It is no coincidence that in the pre-trained process, in countless iterative loops, the chatbot is asked questions and, in the follow-up, people evaluate the AGI's answers as right or wrong, useful or useless. In addition, each correction of an unacceptable answer improves the basis of the results.

Everything serves the solution strategy of transformation. Spoken in the mind of Derrida and Kristeva, the intertextuality. So it's no wonder that computer scientist Marcel Waldvogel sounds like something out of the writings of language theorists when he explains how the programs work.

The AGI is trained to recognize words from previous texts; analyze sentence structures; to anticipate which object relates to which verb; knowing what a pronoun refers to; to keep the subject of the text in view, more precisely, not to lose the point of view; To adhere to structures - such as the form of an essay, poem or recipe.

In the language-philosophical reflections of the Bulgarian-French literary theorist Kristeva, each text is transformed without limits, crystallizes as a transit point, a semantic intersection of a multitude of texts. Every single word building block, be it literal, stylistic or structural, becomes a "mosaic of quotations". She even talks about the fact that the author practically dissolves during this process.

And this is where the two worlds meet. Because this is exactly the experience that ChatGPT users have when they look at the screen in fascination after a question has been asked, how the letters are written by a magical hand, bit by bit creating a meaningful connection for a reader.

For the computer technicians, the key to success lies in "attention" - a technical method in which statistical connections are automatically established between the bigrams used and their direct and indirect ancestor symbols. So it's also about the frequency and relevance of even letters, similar to how it is handled in cryptography. Numerous neural language models based on the Transformer architecture such as the Megatron-Turing Natural Language Generation Model (MT-NLG), Google LaMDA or BERT make use of it.

This philological meticulousness in word construction is reminiscent of the literary impressionist Gustav Flaubert, who for his contemporary subjects scoured joke sheets, court files, engravings, fashion journals, laundry slips, address books, street maps, read to himself aloud in order to examine the sensual timbre of the words used, such as his contemporary and literary critic Émile Faguet knows how to report.

Ongoing interaction between author and reader through the text. Everything merged together into a dynamic, holistic system. This is how the poststructuralists imagined not only the literary but also the philosophical world. According to Barthes, words “weave” “a text of life”, are ultimately “work and a play”, “a volume of shifting traces”.

Anyone who uses the chatbots to engage in mutual question-and-answer collaboration should therefore have a clear goal in mind. If the question asked is vague, the answer will also reflect this character. Being brief is a good practice, as is avoiding unnecessary words or information.

Good results are achieved by writing in a way that is easy to understand and uses everyday language. And while Derrida, with his understanding of deconstruction, sees the irrepressible power of ambiguity as the driving force behind his shift, the bot makes it clear quite quickly that ambiguous words and phrases only confuse an AGI.

If you do not provide enough context, you will ensure that there is nothing adequate to be found in the provided arsenal of words and sentences. One of the possible consequences that bots hallucinate because the models draw inaccurate conclusions about a topic or situation when not included in the training dataset.

That is the moment when the finished veil no longer lifts, “behind which, more or less hidden, the meaning (the truth) resides.”