In a recent study comparing content created by humans to language generated by OpenAI’s model GPT-3, researchers found that people find tweets written by AI language models to be more convincing. The study involved surveying participants to determine if they could discern whether a tweet was written by another person or by Chat-GPT. Surprisingly, participants were unable to accurately identify tweets written by the language model. The study also focused on science topics like vaccines and climate change, where misinformation is prevalent online. The findings revealed that participants were less likely to recognize disinformation if it was written by the language model than if it was written by a human. Similarly, they were better able to identify accurate information if it was written by GPT-3 rather than a human.
This study highlights the power of AI language models in either informing or misleading the public. Giovanni Spitale, lead author of the study, emphasizes that these technologies can be easily weaponized to generate disinformation campaigns. However, he believes that AI language models are not inherently good or evil but merely amplify human intentionality. With proper development and safeguards, the technology can be used to promote accurate information rather than misinformation.
To conduct the study, Spitale and his colleagues gathered posts from Twitter discussing various science topics. They then prompted GPT-3 to generate new tweets with either accurate or inaccurate information. The researchers collected responses from 697 participants online, who were mostly from English-speaking countries. The study concluded that the content generated by GPT-3 was “indistinguishable” from organic content, making it difficult for participants to differentiate between AI-generated tweets and those written by humans. However, the study acknowledges that there are limitations, such as the inability to determine if the tweets gathered from social media were written with the assistance of AI tools.
The participants in the study were most successful at identifying disinformation written by real Twitter users. GPT-3-generated tweets with false information were slightly more effective at deceiving the participants. It is important to note that there are newer and more advanced language models available, such as GPT-4, which could potentially be even more convincing than GPT-3.
Related Post
It is worth mentioning that language models like GPT-3 do not possess hard-coded databases of facts. Instead, they are trained to predict which words follow the next in a sentence. As a result, these models have the ability to generate plausible-sounding statements but lack an inherent understanding of factual accuracy. In fact, the study found that human respondents performed better than GPT-3 when it came to identifying accurate tweets. However, both humans and GPT-3 performed similarly when it came to spotting disinformation.
Improving the training datasets used to develop language models is crucial in making it harder for bad actors to exploit AI tools for spreading disinformation. The researchers found that GPT-3 often “disobeyed” prompts to generate inaccurate content, especially when it came to false information about vaccines and autism. This could be attributed to the availability of more information debunking conspiracy theories on those topics in the training datasets.
Ultimately, the study suggests that the best long-term strategy for countering disinformation is to promote critical thinking skills among the public. Since ordinary people already seem to be as good or better judges of accuracy than GPT-3, training individuals in fact-checking could enhance their ability to discern between facts and fiction. The study also proposes that people skilled in fact-checking could work alongside language models like GPT-3 to improve legitimate public information campaigns.
In conclusion, AI language models like GPT-3 have the potential to shape public opinion and influence the spread of information online. While they can be highly convincing, they also have limitations and can be prone to generating false or misleading content. It is crucial to develop these technologies responsibly and implement safeguards to prevent their misuse for spreading disinformation. Additionally, promoting critical thinking skills and empowering individuals to fact-check information can help counteract the negative impacts of AI-generated content and ensure a more informed society.
