![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
The so-called case of data overfitting
reminds me of an old joke:
Once, two men who had a major dispute decided to seek a resolution from a village rabbi. The older man went to the rabbi first and carefully outlined his side of the argument. The rabbi listened intently and finally said, “You are right.” The man went away satisfied. Later in the day, the other party to the dispute arrived and told the rabbi his side of the story. The rabbi again listened carefully, was impressed with the arguments, and replied after some thought, “You are right.” Later, the rabbi’s wife, who had overheard the rabbi’s conversations with both men, said to him, “Rabbi, you told both men that they were right. How can this be?” To which the rabbi replied, “And you are right too!”
Data overfitting can be a rational strategy when being wrong is a low-cost option, while missing an opportunity is a high-cost one.
Once, two men who had a major dispute decided to seek a resolution from a village rabbi. The older man went to the rabbi first and carefully outlined his side of the argument. The rabbi listened intently and finally said, “You are right.” The man went away satisfied. Later in the day, the other party to the dispute arrived and told the rabbi his side of the story. The rabbi again listened carefully, was impressed with the arguments, and replied after some thought, “You are right.” Later, the rabbi’s wife, who had overheard the rabbi’s conversations with both men, said to him, “Rabbi, you told both men that they were right. How can this be?” To which the rabbi replied, “And you are right too!”
Data overfitting can be a rational strategy when being wrong is a low-cost option, while missing an opportunity is a high-cost one.