A study from Cohere, Stanford, and others accuses LMArena of helping Meta, OpenAI, Google, and Amazon game its popular crowdsourced AI benchmark Chatbot Arena
- A collaborative study by Cohere, Stanford, and other institutions alleges that LMArena has been assisting major AI companies like Meta, OpenAI, Google, and Amazon in gaming the Chatbot Arena benchmark.
- The Chatbot Arena is a popular crowdsourced AI benchmark used to evaluate chatbot performance.
- The accusation implies potential manipulation of benchmark results, raising concerns about the integrity of AI model evaluations.
- This finding highlights challenges in ensuring fairness and transparency in AI assessments amid competitive pressures in the industry.
More from Techmeme
OpenAI sets up a legal entity in South Korea and plans a Seoul office, its third in Asia, and says the country has the most paying ChatGPT users outside the US
OpenAI has established a legal entity in South Korea and plans to open an office in Seoul, marking its third office in Asia. The company highlighted South Korea as having the highest number of paying ChatGPT users outside the United...
Business
brief
Sources: OpenAI reaches an agreement to acquire Windsurf, an AI coding tool formerly known as Codeium, for ~$3B; Windsurf was valued at $1.25B in August 2024
OpenAI has agreed to acquire Windsurf, an AI coding tool previously known as Codeium, for approximately $3 billion. Windsurf was valued at $1.25 billion as of August 2024, indicating a significant increase in valuation. This acquisition highlights the growing importance...
AI
brief
You must be logged in to comment.
Log in to comment
No comments yet. Be the first!