Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena(opens in a new tab)
Using a strong model to grade other models' answers, checked against human votes on MT-Bench and Chatbot Arena. GPT-4 as judge agreed with humans over 80 percent of the time, about as often as humans agree with each other.