Seen is an example of the Open Ko-LLM leaderboard, which Upstage will create in collaboration with the National Information Society Agency (NIA) to evaluate Korean large language models. Upstage said ,Monday, that it will launch the leaderboard in partnership with the NIA, and that the platform will help continue to expand the reach of Korea’s AI ecosystem. Courtesy of Upstage
Upstage, a local AI startup, will create a large language model (LLM) evaluation platform with the state-run National Information Society Agency (NIA) in order to improve the competitiveness of Korean-based AI platform companies, the company said Monday.
Upstage said it will launch Open Ko-LLM leaderboard with the NIA on Sept. 27. The company, which is recognized for its competitive LLM technology, said it has been working with the NIA to prepare for the launch of the leaderboard, helping AI developers raise the level and competitiveness of Korean language model research.
LLM leaderboard is a tool for researchers to evaluate the performance and diversity of LLMs by assessing the competitiveness of language models using the Korean language.
Upstage said it has built its own high-quality data to evaluate the Korean language models more effectively than Hugging Face leaderboard, a popular measurement platform overseas.
“Open Ko-LLM leaderboard is an open platform where anyone can register their Korean LLM model and compete against other models,” the company said.
One of the advantages of the platform is the addition of a criterion that evaluates the ability to generate commonsense, a helpful tool to check the performance and diversity of Korean LLM models, the company added. This criterion measures whether the AI’s output for a given set of orders can match the commonsense that a Korean speaker would have.
“The ability to generate commonsense is a dataset built by Upstage in collaboration with a team led by Prof. Heui-seok Lim of Korea University that evaluates a wide range of historical distortion types, hallucination errors, morphemic errors and more,” the company said.
Kim Sung-hoon, CEO of Upstage, said “we will continue to strive to expand the Korean AI ecosystem and promote its development by sharing high-quality Korean data.”
![[INTERVIEW] 'Now is time for Koreans to unlock potential in Africa'](https://img.koreatimes.co.kr/upload/thumbnailV2/9236a17d88ea472b862b237c9788069d.jpg/dims/resize/178/optimize)
![[INTERVIEW] Meet the man behind giant rubber ducks that once took over Seoul](https://img.koreatimes.co.kr/upload/thumbnailV2/df32c60c8204448e98ccdb9841366bec.jpg/dims/resize/178/optimize)
![[REVIEW] Musical 'Monte Cristo' returns with riveting tale of vengeance, love](https://img.koreatimes.co.kr/upload/thumbnailV2/3934be1fa6844478b619a015d610bc5b.jpg/dims/resize/178/optimize)
