• 0 Posts
  • 798 Comments
Joined 3 years ago
cake
Cake day: June 13th, 2023

help-circle




  • It needed the rules, and it needed carefully worded questions that matched the parameters set by the rules. I bet if the questions’ wording didn’t match your rules so exactly, it would generate worse answers. Heck, I bet if you gave it the rules, then asked several completely unrelated questions, then asked it your carefully worded rules-based questions, it would perform worse, because it’s context window would be muddied. Because that’s what it’s generating responses based on - the contents of it’s context window, coupled with stats-based word generation.

    I still maintain that it shouldn’t need the rules if it’s truly reasoning though. LLMs train on a massive set of data, surely the information required to reason out the answers to your container questions is in there. Surely if it can reason, it should be able to generate answers to simple logical puzzles without someone putting most of the pieces together for them first.


  • I can be convinced by contrary evidence if provided. There is no evidence of reasoning in the example you linked. All that proved was that if you prime an LLM with sufficient context, it’s better at generating output, which is honestly just more support for calling them statistical auto-complete tools. Try asking it those same questions without feeding it your rules first, and I bet it doesn’t generate the right answers. Try asking it those questions 100 times after feeding it the rules, I bet it’ll generate the wrong answers a few times.

    If LLMs are truly capable of reasoning, it shouldn’t need your 16 very specific rules on “arithmetic with extra steps” to get your very carefully worded questions correct. Your questions shouldn’t need to be carefully worded. They shouldn’t get tripped up by trivial “trick questions” like the original one in the post, or any of the dozens of other questions like it that LLMs have proven incapable of answering on their own. The fact that all of those things do happen supports my claim that they do not reason, or think, or understand - they simply generate output based on their input and internal statistical calculations.

    LLMs are like the Wizard of Oz. From afar, they look like these powerful, all-knowing things. The speak confidently and convincingly, and are sometimes even correct! But once you get up close and peek behind the curtain, you realize that it’s just some complicated math, clever programming, and a bunch of pirated books back there.













  • Alright I’ll spell it out for you. For some context, the article in the post (which you probably didn’t read) describes how schools are sending tablets and laptops home with elementary and middle school children. I specifically stated that I didn’t use a laptop for school until I was in college, and implied that my technology literacy did not suffer despite such “late exposure”.

    I did not say that I didn’t use a computer until college. You made that up. I’m not advocating to remove all technology from school. That’s a strawman you’ve built to argue against. I used computers all throughout my time in school, starting in like 2nd grade. We had these things called computer labs, where a teacher that specialized in technology would teach us the ins and outs of using a computer, how to be safe on the internet, and provide adult supervision and guidance. In middle school, we had designated computer lab time to work on book reports, lab reports, research projects, etc. I carried a usb stick around with me to save things onto, which I would then take home, where I could continue working on my assignments on our family computer. My parents established rules and boundaries for using the home computer, and were another resource I could go to for help and guidance.

    But we also wrote stuff down. Like with pencils, on paper. And had teachers up at the front of the room giving lectures, helping us through example problems, teaching. That was the primary way we learned. We weren’t sent home with an iPad and some edutainment games and told “good luck!” like the kids described in the posted article.

    I’ll say it again, but I’ll reword it in more plain language so there’s less chance of misunderstanding: sending school children home with corpoware-riddled tablets and laptops with little to no guidance and expecting them to use that for the bulk of their schoolwork (the thing described in the article) is not a good way to foster technology literacy.