It’s more likely a combination of both, it’s permanent lights-out, but the universe doesn’t meaningfully exist without a conscious observer, so you’ll be dead for infinity and you won’t sense any of that time passing because you’ll be dead, but even the most unlikely events become 100% certain given enough time and there are calculations for how long it takes for another universe to pop into existence. (It’s a huge number, but again, effectively zero if you’re dead.)

This may mean it’s impossible to really die forever. You won’t have memories or anything, but if space and physics is constant then at some point things will repeat and a sense of “self” will exist again. And again. And again.