> This strongly suggests that the model is able to memorize solutions from its training set
I'm not sure why this is a problem - surely in systems like chatGPT we want the specifics that was in the training set not a generalization. It's not learning/reasoning from the training data its 'cleverly regurgitating' things its seen.
For some applications, yes, but it comes at the cost of not knowing how powerful ChatGPT really is. So the claim from OpenAI that ChatGPT 4 can pass the bar exam are deceptive since it will likely fail any future bar exam.
Agreed, but apart from the novelty factor I'm not sure what the practical use is of ChatGPT passing the bar exam. In fact I think it's a good thing that it can't do well in any future tests as its likely to be used by the unscrupulous
I'm not sure why this is a problem - surely in systems like chatGPT we want the specifics that was in the training set not a generalization. It's not learning/reasoning from the training data its 'cleverly regurgitating' things its seen.