• 1 Post
  • 55 Comments
Joined 1 year ago
cake
Cake day: October 4th, 2023

help-circle












  • So far, there has been zero or one[1] lab leak that led to a world-wide pandemic. Before COVID, I doubt anyone was even thinking about the probabilities of a lab leak leading to a worldwide pandemic.

    So, actually, many people were thinking about lab leaks, and the potential of a worldwide pandemic, despite Scott’s suggestion that stupid people weren’t. For years now, bioengineering has been concerned with accidental lab leaks because the understanding that risk existed was widespread.

    But the reality is that guessing at probabilities of this sort of thing still doesn’t change anything. It’s up to labs to pursue safety protocols, which happens at the economic edge of of the opportunity vs the material and mental cost of being diligent. Reality is that lab leaks may not change probabilities, but yes the events of them occurring does cause trauma which acts, not as some bayesian correction, but an emotional correction so that people’s motivations for atleast paying more attention increases for a short while.

    Other than that, the greatest rationalist on earth can’t do anything with their statistics about label leaks.

    This is the best paradox. Not only is Scott wrong to suggest people shouldn’t be concerned about major events (the traumatic update to individual’s memory IS valuable), but he’s wrong to suggest that anything he or anyone does after updating their probabilities could possibly help them prepare meaningfully.

    He’s the most hilarious kind of wrong.


  • Ah, if only the world wasn’t so full of “stupid people” updating their bayesians based off things they see on the news, because you should already be worried of and calculating your distributions for… inhales deeply terrorist nuclear attacks, mass shootings, lab leaks, famine, natural disasters, murder, sexual harassment, conmen, decay of society, copyright, taxes, spitting into the wind, your genealogy results, comets hitting the earth, UFOs, politics of any and every kind, and tripping on your shoe laces.

    What… insight did any of this provide? Seriously. Analytical statistics is a mathematically consistent means of being technically not wrong, while using a lot of words, in order to disagree on feelings, and yet saying nothing.

    Risk management is not a statistical question in fact. It’s an economics question of your opportunities. It’s why prepping is better seen as a hobby, a coping mechanism and not as viable means of surviving apocalypse. It’s why even when a EA uses their super powers of bayesian rationality the answer in the magic eight ball is always just “try to make money, stupid”.


  • In practice, alignment means “control”.

    And the the existential panic is realizing that control doesn’t scale. So rather than admit that goal “alignment” doesn’t mean what they think it is, rather than admit that darwinian evolution is useful but incomplete and cannot sufficiently explain all phenomena both at the macro and micro levels, rather than possibly consider that intelligence is abundant in systems all around us and we’re constantly in tenuous relationships at the edge of uncertainty with all of it,

    it’s the end of all meaning aka the robot overlord.




  • seems a bit like a younger person who is now going through some trauma about how much influence they really have vs how much they imagined they had when they were 12

    This really resonates. Unfortunately I think that’s right. Having this epiphany, this existential correction, about ones self, has either the possibility to create true life long wisdom or, irrecoverable life long self loathing, and from my experience it comes down to the quality of this person’s relationships to lean on when confronting the internal fear of mortality.

    So it’s sad to see but this is another example of the latter and not the former.


  • For what it’s worth then, I don’t think we’re in disagreement, so I just want to clarify a couple of things.

    When I say open system economics, I mean from an ecological point of view, not just the pay dollars for product point of view. Strictly speaking, there is some theoritical price and a process, however gruesome, that could force a human into the embodiment of a bird. But from an ecosystems point of view, it begs the obvious question; why? Maybe there is an answer to why that would happen, but it’s not a question of knowledge of a thing, or even the process of doing it, it’s the economic question in the whole.

    The same thing applies to human intelligence, however we plan to define it. Nature is already full of systems that have memory, that can abstract, reason, that can use tools, that are social, that are robust in the face of novel environments. We are unique but not due to any particular capability, we’re unique because of the economics and our relationship with all the other things we depend upon. I think that’s awesome!

    I only made my comment to caution though, because yes, I do think that overall people still put humanity and our intelligence on a pedestal, and I think that plays to rationalist hands. I love being human and the human experience. I also love being alive, and part of nature, and the experience of the ecosystem as a whole. From that perspective, it would be hard for me to believe that any particulart part of human intelligence can’t be reproduced with technology, because to me it’s already abundant in nature. The question for me, and our ecosystem at large, is when it does occur,

    what’s the cost? What role, will it have? What regulations, does it warrant? What, other behaviors will it exhibit? And also, I’m ok not being in control of those answers. I can just live, in a certain degree of uncertainty.


  • Yes, and ultimately this question, of what gets built, as opposed to what is knowable, is an economics question. The energy gradients available to a bird are qualitatively different than those available to industry, or individual humans. Of course they are!

    There’s no theoritical limit to how close an universal function approximator can get to a closed system definition of something. Bird’s flight isn’t magic, or unknowable, or non reproduceable. If it was, we’d have no sense of awe at learning about it, studying it. Imagine if human like behavior of intelligence was completely unknowable. How would we go about teaching things? Communicating at all? Sharing our experiences?

    But in the end, it’s not just the knowledge of a thing that matters. It’s the whole economics of that thing embedded in its environment.

    I guess I violently agree with the observation, but I also take care not to put humanity, or intelligence in a broad sense, in some special magical untouchable place, either. I feel it can be just as reductionist in the end to demand there is no solution than to say that any solution has its trade offs and costs.