• 5 Posts
  • 401 Comments
Joined 3 years ago
cake
Cake day: September 11th, 2023

help-circle
  • By modern standards, you didn’t really get ghosted. That text was him calling it off. It sucks, but I’m afraid that’s about as good as you’re probably gonna get.

    I’ve been in your shoes, wracked my brain trying to think about everything I might have done wrong and wishing they’d tell me exactly why they weren’t interested so I could try to fix it.

    I’ve also been the guy in this situation, having figured out that I’m not interested in the person and struggling to put it into words. Maybe the vibe wasn’t right, maybe we had different expectations, maybe there’s nothing wrong with them physically but yet for some reason I’m just not attracted to them. I’m ashamed to admit I have just ghosted people because it’s easier than telling them the truth. Maybe the times I got ghosted in return was some sorta karma.

    You shouldn’t blame yourself or assume anything is wrong. It sounds like this guy just had different expectations about some things, and that’s perfectly fine. You shouldn’t try to look for anything you need to change about yourself just to find a partner. Keep looking for someone who’s willing to accept you as you are.










  • Seems like a positive feedback loop:

    1. Netflix creates show that’s pretty good
    2. People watch it
    3. Netflix takes too long to release season 2
    4. People forget show exists, stop watching
    5. Netflix drops a season years later
    6. No one watches it because they forgot what happened in season 1 and Netflix shows are so serial that you have to know what happened
    7. Netflix cancels show due to low viewership
    8. Fans of the show feel jilted and slam Netflix on social media
    9. Rinse and repeat








  • Not sure what this internal state you are referring to is. Are you talking about all the values that come out of each step of the computations?

    It would need to be able to form memories like real brains do, by creating new connections between neurons and adjusting their weights in real time in response to stimuli, and having those connections persist. I think that’s a prerequisite to models that are capable of higher-level reasoning and understanding. But then you would need to store those changes to the model for each user, which would be tens or hundreds of gigabytes.

    These current once-through LLMs don’t have time to properly digest what they’re looking at, because they essentially forget everything once they output a token. I don’t think you can make up for that by spitting some tokens out to a file and reading them back in, because it still has to be human-readable and coherent. That transformation is inherently lossy.

    This is basically what I’m talking about: https://www.comicagile.net/comic/context-switching/

    But for every single token the LLM outputs. The fact that it’s allowed to take notes is a mitigation for this context loss, not a silver bullet.




  • The size of the context window is fixed in the structure of the model. LLMs are still at their core artificial neural networks, so an analogy to biology might be helpful.

    Think of the input layer of the model like the retinas in your eyes. Each token in the context window, after embedding (i.e. conversion to a series of numbers, because ofc it’s just all math under the hood), is fed to a certain set of input neurons, just like the rods and cones in your retina capture light and convert it to electrical signals, which are passed to neurons in your optic nerve, which connect to neurons in your visual cortex, each layer along the way processing and analyzing the signal.

    The number of tokens in the context window is directly proportional to the number of neurons in the input layer of the model. To make the context window bigger, you have to add more neurons to the input layer, but that quickly results in diminishing returns without adding more neurons to the inner layers to be able to process the extra information. Ultimately, you have to make the whole model larger, which means more parameters, which means more data to store and more processing power per prompt.