Thank you for putting in the time to try to explain this. I’ve also been working with ML for a long time and have the intuition that there is a fundamental limit to what can be achieved with Neural Networks and probably any other Machine Learning technique.
Your cake analogy is probably better, but let me try another one. It’s like having a machine that is trying to solve a gigantic puzzle with random pieces from different puzzles. Yeah, if you have enough puzzle pieces and the machine is fast enough it can give you suboptimal solutions but the puzzle will not ever be solved as it is supposed to be solved.
Thank you for putting in the time to try to explain this. I’ve also been working with ML for a long time and have the intuition that there is a fundamental limit to what can be achieved with Neural Networks and probably any other Machine Learning technique.
Your cake analogy is probably better, but let me try another one. It’s like having a machine that is trying to solve a gigantic puzzle with random pieces from different puzzles. Yeah, if you have enough puzzle pieces and the machine is fast enough it can give you suboptimal solutions but the puzzle will not ever be solved as it is supposed to be solved.