Baidu claims Ernie 3.5 outperformed ChatGPT and GPT-4 in important metrics.

Competition in the artificial intelligence (AI) market is intensifying, as China’s Baidu announced that its AI model, Ernie 3.5, outperformed the popular OpenAI’s ChatGPT and GPT-4 in key tests. Baidu unveiled the Ernie bot at an event in March and CEO Robin Li said that the new product was not perfect and would continue to improve as people use it and give feedback. Within an hour of revealing the Ernie bot, around 30,000 corporate clients joined the waitlist to access the chatbot.

Baidu has been publicly testing the Ernie bot since its unveiling in March. The chatbot, built on the Chinese search engine’s foundational AI model called Ernie, is trained on extensive data. On the other hand, ChatGPT, which Baidu said Ernie 3.5 outperformed, is based on OpenAI’s GPT 3.5 model. Baidu also claimed that its AI model beats OpenAI’s latest and more advanced model, GPT-4. It noted that Ernie 3.5 performed better than OpenAI’s product in Chinese language tests.

Baidu Claims Ernie 3.5 Is Better than ChatGPT in Multiple Key Areas

The Chinese company made the claim citing a report by China Science Daily. According to the report, a “Few-Shot evaluation” reveals that Ernie 3.5 outperformed ChatGPT in multiple test sets. The three evaluation benchmarks are AGIEval, C-Eval, and MMLU. Microsoft Research developed the AGIEval evaluation benchmark to examine the model’s performance level in the “human-oriented” standardized test. The focus is on 20 official, public, and distinct qualifying exams, such as the SAT exam in the US and college entrance examinations in China. More include Bar exams, American GMAT, GME, and so on. Additionally, Berkeley University, Columbia University, the University of Illinois at Urbana-Champaign, and the University of Chicago jointly released MMLU. The large-scale multi-task language understanding test measures the models’ English interdisciplinary professional ability. This test covers different educational areas like social sciences, humanities, science, technology, engineering, and mathematics (STEM), and more.

The c-Eval evaluation is a Chinese basic model evaluation containing 13,948 multiple-choice questions covering 53 subjects. The evaluation benchmark was created and released by the joint effort of Tsinghua University, the University of Edinburgh, and Shanghai Jiaotong University.

The results of the AGIEval and C-Eval tests show that Ernie 3.5 achieved higher scores than other large models, including ChatGPT, and surpassed GPT-4. Also, the Baidu AI model also outdid ChatGPT’s 40.27 points and GPT-4’s 56.96 points. Ernie 3.5 scored a whopping 64.37 points, taking first place. For the Chinese c-Eval evaluation, Ernie 3.5 outperformed ChatGPT. While the Chinese AI model scored the highest at 71.93 points, ChatGPT measured 51.70 points, and GPT-4 got 68.57 points. In addition, Baidu mentioned more results that showed that Ernie 3.5 has “outstanding Chinese ability” and outperformed ChatGPT and GPT-4.

We will continue to update Phone&Auto; if you have any questions or suggestions, please contact us!


Was this article helpful?

93 out of 132 found this helpful

Discover more


Whale buys $38M worth of SHIB, rebound expected?

The Shiba Inu (SHIB) token continues to experience massive transactions, generating excitement within its vibrant com...


Dogwifhat: A Memecoin Tale of Fortunes

A savvy trader saw a 7x return on their investment in just five days after purchasing a Solana-based memecoin featuri...


Shibarium Hype: Shiba Inu Whales Grab 1 Trillion SHIB Tokens

Shiba Inu, a highly popular alternative cryptocurrency in the market, has experienced a remarkable surge in its price...


SHIB surges 14% in a day with Shiba Inu Worldpaper nearing.

The value of Shiba Inu (SHIB) is currently increasing rapidly and consistently, with significant progress made in the...


Shiba Inu Price Prediction A 10% 24-Hour Rise to Unleash Optimism, Are We Barking Up the Tree of New Highs?

The Shiba Inu cryptocurrency has surged by 8.5% today, reaching a value of $0.00000803. This increase is part of a 1%...


Shiba Inu's burn rate increased by 1500% in 24 hours, but the price still struggled in the red

While the Shiba Inu (SHIB) token seems to be facing competition from another meme coin that is currently gaining hype...