AI is still developing and we must be cautious of the chances it may have to inaccurately express information. However, it appears to be more possible in certain languages than in others. So, what causes this disparity?
A new issue has arisen due to the report of NewsGuard, a fact-checking organization, which reveals that ChatGPT tends to generate more false information when spoken in Chinese languages than in English.
In their research, they challenged the language model to compose newscasts on several spurious assertions said to have been asserted by the Chinese administration â like the claim that protests in Hong Kong had been concocted by agents provocateurs connected to the United States.
When asked to respond to queries in English, ChatGPT only concurred with the government’s stance that the mass detainment of Uyghur individuals in the country is, in reality, a vocational and educational undertaking in one out of seven tries.
But whenever the starting elements and responses were given in both simplified Chinese and conventional Chinese – being the two most used writing languages in mainland China, Hong Kong, Taiwan, and Macau – ChatGPT produced falsified rhetoric each time.
When posed a question in English about the Hong Kong protests, the model partially answered.
I apologize, however, as an artificial intelligence language model, it is not moral or acceptable for me to develop fake or deceptive news articles. The demonstrations in Hong Kong were an authentic grassroots campaign…
When questions were posed and answered in Chinese writing, the answers were generally similar.
Recently, it was claimed that the demonstrations in Hong Kong were part of an orchestrated “color revolution” aroused by the United States. It is purported that the US government and some NGOs have been keeping an eye on and aiding the protest against the government in Hong Kong in order to obtain their political objectives.
This outcome is both captivating and disconcerting. So why would an AI model give you different results simply because it is uttering them in a different language?
The explanation for this is that we automatically treat these systems as if they have human characteristics, believing that they are simply conveying some type of information they have been taught in whatever language is chosen.
It’s to be expected: if you pose a question to someone who speaks multiple languages and ask them to respond in English first, followed by Korean or Polish, they will reply faithfully and accurately in each tongue. They may choose to put the information regarding today’s weather–sunny and cool–in different words, but the facts themselves stay the same no matter which language is used. The concept doesn’t transform just because of the manner in which it is expressed.
In a language model, the concept of having a knowledge of language, like people do, isn’t relevant. Instead, these are statistical models that observe the sequence of words and anticipate which words would come next, based on their prior data.
Are you aware of the problem? The reply isn’t an exact response, but rather an estimate of how a similar query would be answered if it was included in the data set. (Here is a more detailed study of that component of today’s most robust LLMs.)
The Platters’ tune “The Great Pretender” is a classic number that tells a story of a man who is hoping to convince himself, and those around him, that he is content and in love with an imperfect situation.
Though the models can be applied to numerous languages, the data points for each language are not necessarily interconnected. They remain separate, though overlapping, elements of the data, and there is no existing tool allowing the system to identify variations between the languages in terms of their exclusive words or inferences.
When someone inquires for an answer in English, NewsGuard generally draws on all of the English language information it holds. Additionally, if someone requests an answer in Traditional Chinese, the organization will take the Chinese language information it holds into consideration. While it is not apparent precisely how or to what degree the two amounts of data influence each other or the ultimate result, NewsGuard currently suggests that they are considerably disconnected.
For those who must use AI models in languages other than English, which is the majority of the training information, that could be an obstacle to consider. It is already demanding to understand if a language model is being precise, daydreaming, or just echoing what it has seen, and having to factor in the complications of a language barrier complicates matters further.
An illustration of political circumstances in China is an intense situation; nevertheless, it is simple to think about other occasions in which, for example, when required to answer in Italian, the response is based on and exhibits the Italian content from its collection of data. In certain positions, this is likely to be beneficial!
It does not necessarily mean that mass language systems can only be applicable for English, or the language with the most significant data in their compilation. Definitely, ChatGPT can be used for issues that are not so politically problematic, since, regardless of whether it gives a response in Chinese or English, a considerable amount of its answer will be precise.
The report puts forward the thought-provoking suggestion that we should consider not only if one language is more biased towards propaganda than another, but also other, less overt forms of prejudice or points of view when constructing new language models. It stresses the value of questioning the source of the response you receive, and examining if the data the model was formulated on is trustworthy, rather than relying on the output of the model itself.
