ChatGPT: why is everyone talking about it?

The release of ChatGPT on November 30 marked a new milestone in the development of language processing technologies. In any case, it is the first time that an AI system directly resulting from research has aroused such enthusiasm: numerous articles have been published in the specialized as well as the general press. ChatGPT (or the company that developed OpenAI, ChatGPT, or both) continues to trend on Twitter today.

Why such an echo? Are ChatGPT’s capabilities different from previous technologies capable of text generation?

ChatGPT can chat

Clearly, ChatGPT has made a leap in quality: it can answer almost any query in natural language, often appropriately. GPT2, then GPT3 – previous language models developed by OpenAI – were very good at generating text from a few words or sentences given as primers (called “data”): we often talked about “parrots” (stochastic parrots), as suggested by Emily Bender and many other researchers who criticize these methods. Indeed, it can be said that these systems have produced text from fragments of sentences compiled from the huge body at their disposal… even if this parrot has a descriptive quality: the systems do not reproduce the extracts word for word, but produce them in general. original text by taking, expressing, combining elements seen in different bodies.

ChatGPT accepts these features, but the “chat” part (dialog) adds a completely different dimension and often looks amazing.

The system no longer simply extracts a paragraph from a sentence given as input. He can now answer both simple and complex questions correctly, present elements of reasoning, speak different languages, analyze or generate computer code, and more.

For example, when asking if two characters can meet, the system can determine the dates that correspond to their existence. compare these dates and deduce the answer. This is trivial to a human, but until now no system could answer these types of questions so accurately without being highly specialized. Apart from language, ChatGPT can also provide elements of mathematical thinking (But is often mistaken) and analyze computer code especially.

In this regard, ChatGPT is one step ahead of its competitors.

ChatGPT, a closed system

As for the operation of the system, it is difficult to say more, because OpenAI, despite its name, develops closed systems. That said, the computer code (source code) used is not available, and the research on ChatGPT remains largely a trade secret for now – although of course other teams are working on similar systems based on deep learning. So we have an idea of ​​what OpenAI uses.

Other companies have maintained a more open model, such as Meta, with work conducted at FAIR generally open and published at major conferences. But, more and more, we are seeing more and more studies being closed. For example, while Apple has always had a private and highly secretive development model, Deepmind has had an open model, and probably a little less since the company is under Google’s control.

The ChatGPT system itself could be commercialized: OpenAI is funded by Microsoft, which may decide to close access to the system one day soon if it suits its interests.

By testing ChatGPT, you contribute to its improvement (for free).

In the meantime, OpenAI is benefiting from the great publicity its tool has brought it, as well as all the user interaction with it. If the user reports that the answer is not good, or asks the tool to reformulate their answer to take into account an additional element, this is as much information as the system stores and can use again to refine their answer the next time they are asked. or by similar request. By testing ChatGPT, we work for OpenAI for free !

To get back to the system itself, the dialog part is what makes ChatGPT so powerful and special (compared to, say, GPT3). It is no longer a “simple” language model capable of generating “kilometers” of text, but a real dialogue system. It has probably benefited from millions or billions of human-rated examples, and the current stage—where hundreds of thousands of users are “playing” with the system almost for free—still allows it to be continually improved, as all interactions are publicly recorded and is used for

At least for languages ​​widely used on the Internet, it is now quite simple to access a corpus of billions of words to develop a “GPT”-type language model.

But the data that enables the development of ChatGPT (dialogues, human interactions) is not public, and this is what gives OpenAI a significant competitive advantage.

For example, Google has different data, but it can certainly be used for this type of system – especially since Google has been developing a knowledge graph for several years that allows it to generate more reliable answers than ChatGPT. In particular, analyzing query sequences from Google’s search engine can provide valuable information to guide user interaction… But, in the meantime, it’s OpenAI that owns ChatGPT, not Google: OpenAi has the upper hand.

Besides ChatGPT, what are the future applications?

Although it is actually possible Bypass ChatGPT restrictions, the system refuses to explain how to create bombs, create erotic tales or give it feelings (ChatGPT often replies that it is a machine, it has no feelings or personality). OpenAI clearly took care of its communication. The company has also carefully “armored” the system to allow it to avoid most of the pitfalls that can destroy this type of application in a matter of hours, as with systems open to the general public.

For example, Meta had to take back its Galactica system two days after putting it online last November. Galactica is designed to offer services to scientists trained and researched. It was introduced for the first time can quickly write scientific articles automatically…before the company clarified that it was just editorial assistance. But this failed communication strategy led to an argument that forced the Meta to quickly separate Galactica.

On the contrary, ChatGPT is still online and still sparks passion among a large audience. However, questions remain: What impact will ChatGPT have? What applications will be available from it? What economic model is OpenAI (and its main investor Microsoft) targeting?

Thierry Poibeau, DR CNRS, Ecole Normale Supérieure (ENS) – PSL

This article is republished from The Conversation under a Creative Commons license. Read the original article.