Two things - firstly you need to understand the concept of overfitting : http://en.wikipedia.org/wiki/Overfitting .
Two things - firstly you need to understand the concept of overfitting : http://en.wikipedia.org/wiki/Overfitting . If teams were allowed to train on the full dataset, it would be possible to get a 100% score, yet still not have a model that was useful on any images that were not in the training set. Furthermore, if you allowed infinite submissions, teams could just train a million models with slightly different hyperparameters, and submit the one that did best (which may be what Baidu was trying to do here). This is a problem because now there is the possibility that you are overfitting the test data - there would be no way to tell if the accuracy generalised to other images without coming up with more labelled data, i.e. making a new test set. Second of all, cross validation : http://en.wikipedia.org/wiki/Cross-validation_%28statistics%... . You don't HAVE to submit to the test set to get a feel of how well your model is performing. On datasets this large, cross validation should be effective, if more time consuming method (unless your model is extremely unstable).
Posted by sweezyjeezy on Jun 4, 2015, 1:51 PM
Tags
Related issues
- The biggest problem with having an LLM do this, from what I've observed of at least frontier models, is that the way they are trained...
- The biggest problem with MOND is that it was hand-crafted with additional parameters simply to fit the data.
- (My comment is about detached houses).
- The biggest problem with this is that I can't run my company's own internal benchmarks against this API, because they will be trained int...
- > steer Claude away from over-engineering parts of the system in response to hypothetical concerns.
- Why is there no comparison with FLUX 3 Image?
Raise real-world problems, find people facing the same issue, and discuss solutions.6,000 issues0 comments
hackernews (Other)
The biggest problem with having an LLM do this, from what I've observed of at least frontier models, is that the way they are trained...
The biggest problem with MOND is that it was hand-crafted with additional parameters simply to fit the data.
(My comment is about detached houses).
The biggest problem with this is that I can't run my company's own internal benchmarks against this API, because they will be trained int...
> steer Claude away from over-engineering parts of the system in response to hypothetical concerns.
Why is there no comparison with FLUX 3 Image?
> our Git monorepo was developing an annoying problem Sentences like this are so common from people using monorepos that I am convinced ...
Why is there no contact form?
Two things - firstly you need to understand the concept of overfitting : http://en.wikipedia.org/wiki/Overfitting .
Two things - firstly you need to understand the concept of overfitting : http://en.wikipedia.org/wiki/Overfitting . If teams were allowed to train on the full dataset, it would be possible to get a 100% score, yet still not have a model that was useful on any images that were not in the training set. Furthermore, if you allowed infinite submissions, teams could just train a million models with slightly different hyperparameters, and submit the one that did best (which may be what Baidu was trying to do here). This is a problem because now there is the possibility that you are overfitting the test data - there would be no way to tell if the accuracy generalised to other images without coming up with more labelled data, i.e. making a new test set. Second of all, cross validation : http://en.wikipedia.org/wiki/Cross-validation_%28statistics%... . You don't HAVE to submit to the test set to get a feel of how well your model is performing. On datasets this large, cross validation should be effective, if more time consuming method (unless your model is extremely unstable).
Comments (0)
No comments yet. Start the discussion.