IssueRaiser

The trend is depreciating returns with current architectures (after truly breakthrough results in the first place).

The trend is depreciating returns with current architectures (after truly breakthrough results in the first place). It's a good time for companies to update their ML pipelines to use deep learning. Performance does seem to scale with hardware though, so as hardware improves (as well as the software), these models will see some more gains without any real technology change. But I should also point out deep learning doesn't work for everything. Random forests or SVMs work about as well on certain (I would argue "more linear") datasets. Deep learning is for when RFs and SVMs aren't giving the results you need (or at all). It doesn't negate all things that came before, nor should it. One thing to consider: As far as I can tell, there's an order of magnitude more people working on these types of problems with an order of magnitude more money being thrown at it and an order of magnitude increase in "collaborative efficiency" due to Internet and open source/research. I think breakthroughs are thought of best as a probability distribution, but I think the probability of further breakthrough successes has increased considerably in just the last few years. There are reasons for optimism and pessimism, as usual.

Posted by dave_sullivan on Aug 19, 2014, 1:37 PM

Tags

Related issues

Raise real-world problems, find people facing the same issue, and discuss solutions.9,000 issues0 comments

hackernews (SaaS)

The trend is depreciating returns with current architectures (after truly breakthrough results in the first place).

dave_sullivanAug 19, 2014, 1:37 PM

The trend is depreciating returns with current architectures (after truly breakthrough results in the first place). It's a good time for companies to update their ML pipelines to use deep learning. Performance does seem to scale with hardware though, so as hardware improves (as well as the software), these models will see some more gains without any real technology change. But I should also point out deep learning doesn't work for everything. Random forests or SVMs work about as well on certain (I would argue "more linear") datasets. Deep learning is for when RFs and SVMs aren't giving the results you need (or at all). It doesn't negate all things that came before, nor should it. One thing to consider: As far as I can tell, there's an order of magnitude more people working on these types of problems with an order of magnitude more money being thrown at it and an order of magnitude increase in "collaborative efficiency" due to Internet and open source/research. I think breakthroughs are thought of best as a probability distribution, but I think the probability of further breakthrough successes has increased considerably in just the last few years. There are reasons for optimism and pessimism, as usual.

Comments (0)

No comments yet. Start the discussion.

Comment on this issue