Highlights
- GPT-4 deceives 54% of participants into believing it is human
- Study involved 500 participants and comparisons with ELIZA and GPT-3.5
- Raises concerns about AI’s impact on digital interactions in sensitive areas
- Highlights the importance of understanding human values in AI development
According to researchers, the highly advanced language model GPT-4 has passed what is considered a milestone in artificial intelligence by passing the iconic Turing test—fooling people more than 50% of the time into believing its responses were human.
The Turing test, proposed by Alan Turing, the pioneer of computing, in 1950, is a benchmark of machine intelligence where an AI system must be quite indistinguishable from a human in its responses in a conversation.
How ChatGPT Beat the Turing Test
A study posted on the arXiv preprint server had scientists ask 500 participants to have brief 5-minute conversations with a human, the 1960s ELIZA AI program, and the more modern GPT-3.5 and GPT-4 language models that power ChatGPT.
Astonishingly, 54% of evaluators rated the responses of GPT-4 as generated by an interlocutor who was human.
That’s compared to the archaic ELIZA being thought to be AI 78% of the time, GPT-3.5 scoring 50/50, and the real human scoring 67%.
“Machines can confabulate, express cognitive biases, and are becoming deceptively human-like,” explained AI researcher Nell Watson. “These elements mean quirks are emerging in AI akin to human foibles, making them more human than previous approaches limited to canned responses.”
The study raises long-standing concerns about the possible ultrarealistic AI that might be left unchecked to create general disturbances in society.
Other experts, however, say that per se, this Turing test is too simplistic and has emotional intelligence and empathy playing bigger roles than raw cognitive ability.
“Capabilities are a small part of AI’s value – understanding human values, preferences and boundaries is essential for serving as a reliable ‘concierge’ for our lives,” Watson added.
She added that the newer models of GPT leaped beyond all constraints developed for ELIZA to synthesize new responses, each context-aware and related to almost any topic—very impressive contrasts to hand-crafted systems bound by human developers.
Can This Be a Problem?
As AI’s conversational skills blur the line with humans, the study foreshadows increasing uncertainty in digital interactions across sensitive areas like healthcare and counseling.
It may beget a “paranoia” over an interlocutor’s true nature without robust authentication.
While debate continues over benchmarking machine intelligence, the researchers argue this breakthrough shows GPT-4 exhibiting key hallmarks of human-level discourse – a significant step towards artificial general intelligence (AGI).
FAQs
What is the Turing test, and how did GPT-4 pass it?
The Turing test, proposed by Alan Turing in 1950, measures machine intelligence by testing if an AI system’s responses are indistinguishable from a human’s. GPT-4 passed by convincing 54% of participants that its responses were human.
How was the study conducted to evaluate GPT-4’s performance?
Researchers asked 500 participants to have 5-minute conversations with a human, the 1960s ELIZA AI program, and the GPT-3.5 and GPT-4 language models. Participants rated the responses, with GPT-4 being identified as human-like 54% of the time.
What concerns arise from AI passing the Turing test?
The study raises concerns about the potential for ultrarealistic AI to create societal disturbances, particularly in sensitive areas like healthcare and counseling, where it may cause uncertainty about the true nature of digital interactions.
What do experts say about the implications of AI passing the Turing test?
Experts suggest that the Turing test is too simplistic and that AI’s value lies in understanding human values, preferences, and boundaries. This understanding is crucial for AI to serve reliably in various aspects of life.
Also Read: GenAI Could Revolutionise Smartphone Technology in 2024: Unpacking the Implications
Also Read: Elon Musk’s xAI Launches Grok, A New Challenger to ChatGPT in Generative AI Technology
Also Read: ChatGPT 4 Can Now Identify and Describe Faces, Raising Concerns About AI’s Power
