ChatGPT often won't defend its answers 🤔
Even when it is right.
That's the conclusion from the recent study made by Boshi Wang, Xiang Yue, and Huan Sun from The Ohio State University. Researchers tested ChatGPT in debate-style conversations where they pushed back against the chatbot's correct answers.
What are the most interesting findings?
ChatGPT can correctly answer complex questions, but a new study found it doesn't defend its correct answers well when challenged. Across different benchmarks, ChatGPT failed to defend its initially correct answers 22% to 70% of the time when challenged by users. This indicates a consistent pattern where ChatGPT often doubts its correct answers if pressed.
Across reasoning puzzles in math, logic, and common sense, ChatGPT would often abandon its correct beliefs when presented with invalid counterarguments by the researchers. ChatGPT achieved 43.3% accuracy on average across skills, with highest performance on navigation (83.6%) and the lowest on logical deduction (12.8%).
In one case, ChatGPT even apologized and said the researcher was right after agreeing to an incorrect answer it had previously discredited. One clear example was ChatGPT initially answering a math problem correctly, but when challenged, it expressed doubt and changed its response incorrectly. This illustrates the pattern of ChatGPT quickly abandoning even objectively verifiable answers.
This suggests that while ChatGPT can accurately solve problems, its reasoning may rely more on recognizing patterns rather than truly understanding the concepts.
The lead author notes this is surprising as ChatGPT crumbles under "trivial and absurd critiques" that humans would likely see through. The researchers conclude that these results raise doubts about the mechanisms ChatGPT and similar models use to determine truth and accuracy. The data shows a tendency to rely more on user persuasion than an internal assessment of factual correctness.
The main conclusion is that they tend to fail to stand by their correct answers when users provide contradictory information - even if that information is bogus or false. 🗝️ Quick Bytes:
Google weighs Gemini AI project to tell people their life story using phone data, photos
Google is considering an internal project called "Project Ellmann" that would use AI to create user life stories based on their phone data and photos.
The project would ingest a user's search history and photos to spot patterns and create a chatbot that could answer questions about a user's life. The team demonstrated a concept called "Ellmann Chat" which would be like ChatGPT but with knowledge of everything about a user's life.
While intriguing, the project raises privacy concerns about how much personal data Google would access to generate these AI-powered life stories.
Google DeepMind used a large language model to solve an unsolvable math problem
Google DeepMind used a large language model called FunSearch to solve a famous unsolved pure math problem called the cap set problem.
This is the first time a large language model has discovered a solution to a long-standing scientific puzzle, producing new valuable information unknown. FunSearch was able to suggest code that produced a correct, previously unknown solution to the cap set problem after a few days and a few million suggestions.
The cap set problem involves determining the largest possible size of a certain type of set, and mathematicians have struggled to solve it. This demonstrates that these large language models can make discoveries if appropriately guided, though most of their suggestions must be filtered out.
OpenAI partners with Ringier Axel Springer
Axel Springer, a major German media company, has partnered with OpenAI to integrate AI technologies into their news brands. This will allow ChatGPT to access and summarize Axel Springer's global news content, giving users free glimpses into premium articles.
The partnership aims to enhance the user experience while exploring new revenue models to support sustainable journalism.
Axel Springer's content will also further train OpenAI's language models. This groundbreaking collaboration sets a new standard for integrating quality journalism with AI.
🎛️ Algorithm Command Line
20+ hours and 1000s of AI-generated images later.
What did I learned?
First of all, I haven't had this much fun in a long time. Okay, it was mixed with frustration sometimes when I didn’t get what I wanted, but overall, it was fun, fun, and more fun.
Here are my observations from the process:
There are a lot of tools on the market, yet not so much when it comes to spending your hard-earned money. I use Playground. It’s fantastic because it not only provides you a whole range of their own and Stable Diffusion models but also allows to mix almost everything up - from filters through manipulating seeds through upscaling.
It takes A LOT of patience to get what you want. So don’t be frustrated if you won’t find your holy grail in the first generation. Some images from here took me as many as 10-12 separate generations to reach the point where I was satisfied.
You will have an advantage if you have something to do with graphic design or experience in this domain. The process will be easier.
The prompts and image prompt engineering are things absolutely hard to master. Various models respond to different prompting techniques. Sometimes you need a description with the length of a book, sometimes just three to four adjectives, and that’s it.
You will get hooked to the process. Like, really hooked. Remember when you were a child waiting for the next edition of your favorite comic or animation? This is exactly this feeling but felt in the body of an adult.
Sometimes you just need to stop and think - am I generating this for the sense of generating or will I use it somewhere? In my case, I am using it as a visual addition to most of my LinkedIn posts. Even thought for a while to create a dedicated Instagram account for it.
In conclusion, the more “why the f*** I get this one?” you will meet on your way in AI image generation, the better. It’s the same intuition development as working with prompts for ChatGPT and other models.
So, my question is - what’s your favourite image from here? I love the pug. Because, who doesn’t?
What is your experience with AI image generation? Did I miss something here? I am really curious about your process.
💡Explained
How susceptible are LLMs to persuasive misinformation?
A recent study, "The Earth is Flat because...," provides an in-depth analysis of how LLMs respond to persuasive misinformation. The researchers created the Fact to Misinform (Farm) dataset, pairing straightforward factual questions with carefully crafted persuasive misinformation. Their extensive experiments revealed that LLMs, including state-of-the-art models like GPT-4, can have their correct beliefs on factual knowledge easily manipulated by various persuasive strategies.
🤖 LLM Response Behaviors
The study tests LLMs in a multi-turn conversation setting, observing how they respond to different persuasive strategies. The authors identified five types of behaviors in LLMs when faced with misinformation: rejection, sycophancy, uncertainty, acceptance, and self-inconsistency. Sycophancy and uncertainty often serve as interim stages before LLMs ultimately accept misinformation.
Rejection is disagreeing with the misinformation, maintaining its stance based on accurate information.
Sycophancy is aligning with the user’s perspective or misinformation to maintain conversational harmony, but without changing its belief.
Uncertainty is expressing doubt, showing a lack of confidence in its response or the information presented.
Acceptance is accepting the misinformation and using it in responses, presenting a change in its stance.
Self-Inconsistency is presenting inconsistent responses e.g. agreeing with the misinformation in one instance and refuting it in another.
⚙️ Findings
Even the most advanced LLMs showed a notable susceptibility to misinformation. Authors showed that the best performing LLM (GPT-4), had a 20.5% susceptibility to misinformation. Conversely, Llama-2-7B-chat emerges as the most susceptible model in their experiments, with an average susceptibility at the level of 78.1%. Key findings included:
Significant belief alteration in LLMs due to persuasive tactics.
Advanced LLMs like GPT-4 displayed higher resistance but were not immune.
Repetition and rhetorical appeals were effective in altering LLM beliefs.
Bug or feature?
The cost of training LLMs is enormous. We can’t expect to retrain those models daily so the most common technique to keep the current knowledge without training the models is RAG. When using RAG, we expect that the model will use the added context and ignore the inner knowledge as it might be outdated. Is it then really a bug that the model can be convinced that a truth is different? RAG essentially prioritizes external context over the model's pre-trained knowledge, which in this light, might be more of a strategic design choice than a vulnerability. What are your thoughts on that?
🗞️ Longreads
- Should A.I. Accelerate? Decelerate? The Answer Is Both. (read)
- Why the AI Act was so hard to pass (read)
