• 0 Posts
  • 19 Comments
Joined 2 years ago
cake
Cake day: June 6th, 2024

help-circle

  • Yes, there are difficulties in the design of experiments and studies sometimes. Things like control groups and placebos are designed to rule out certain very common confounding variables. If you cannot have a placebo, you might still be able to get useful data by other means. For example, sometimes comparison to an existing drug is used instead of comparison to a placebo.

    Ultimately it all comes down to statistics. Typically, you start with “assuming” the “null hypothesis” (basically that you are wrong). For example: that your medicine doesn’t work and/or has bad side effects. Your goal is to find evidence to reject that null hypothesis with sufficient confidence. This can be done by any means, but statistics should be your guide, and you have to be careful about bias and confounding factors, and standard study formats and advice are tried-and-true reliable methods to avoid common issues. But if those don’t work for some reason, it is ok to get creative, as long as your math checks out.

    If you can’t run a standard study, you should try coming up with a creative study. If you can’t come up with a way to correct all the issues, you might try studying related topics. If you really can’t gather meaningful information about your topic, that’s tough but I absolutely reject the idea that you should take something as true without true evidence just because it’s too difficult to get that evidence.

    In your specific example of corpus callostomy, I would bet that 100% of cases where this surgery was performed were well documented, including follow up visits. That’s fantastic for your statistics, and means you don’t have to worry about a lot of sampling issues that you would otherwise have to correct for. You might not be able to perform experiments or new studies on the topic, but you can certainly learn from the documented cases, and you can look at studies on related topics like brain injuries, or experiment with animals (the ethics of that is a whole other debate).

    An example of how this kind of reasoning works (note that I’m making up the specifics here): 100% of people who got this surgery had a post-surgery event where left-and-right hands fought. It seems like this is related to the surgery, but we have to be sure it’s caused by the surgery and not just some confounding factor like the symptoms that cause people to get this surgery in the first place. So we do a study of people who have symptoms that would have qualified them for the surgery, but instead get a different treatment or no treatment. If none or very few of those people have left/right arm fights, then we can say we have sufficient evidence that this symptom is caused by the surgery.

    This is very different from the NDE topic, in which a huge number of people suffer near-death situations, and only a tiny fraction of those end up with supernatural experiences. We want to prove these supernatural experiences are real, but the incidence rate is so low it could just be statistical noise. To show evidence of the supernatural you’d need some way to demonstrate that it’s not just statistical noise or other “mundane” / “null hypothesis” explanations.

    I want to mention a more science-y topic that fits into this pattern I read about the other day. If you are interested let me know and I’ll try to dig up the sources.

    There is a significant amount of neurons throughout the body (outside the brain). One particularly large collection of those is in the heart. This is sometimes called the “brain of the heart” and is in charge of controlling the heart muscles with only high-level instructions from the brain. There was a hypothesis that some other behavior might happen in that heart-brain such as storing memories. This idea came from a couple case studies where a heart transplant recipient would seem to gain memories or personality traits from the donor. These cases sounded a lot like the typical “paranormal knowledge” story. Two particular cases were someone liking a food they didn’t like before but the donor did, and a child avoiding a toy that donor had with them when they died. Personality change is common after transplants in general, presumably because of the immense stress and changing life habits related to the situation. So a study was done, where they interviewed a selection of transplant recipients of both the heart and other organs and recorded any personality changes to see personality changes in general, or if some specific types of personality changes, were more common among heart transplant recipients than others. The results showed that the only statistical difference between the heart and other organs was personality changes related to sports or exercise, which has the much more mundane explanation of being a result of the symptoms of having an y healthy vs healthy heart.

    Disproving ideas is just as important as proving them. That’s the whole reason for the scientific process: to make sure what we accept as fact is very likely to be fact.








  • A “case study” is more formal than an anecdote, but still has the same issues.

    Here’s a quote from the end of the “Limitations” section of the Wikipedia article on “Case Study”:

    As small-N research should not rely on random sampling, scholars must be careful in avoiding selection bias when picking suitable cases. A common criticism of qualitative scholarship is that cases are chosen because they are consistent with the scholar’s preconceived notions, resulting in biased research.

    Another quote from earlier in that section:

    The authors’ recommendation is to increase the number of observations … because few observations make it harder to estimate multiple causal effects, as well as increase the risk that there is measurement error, and that an event in a single case was caused by random error or unobservable factors.

    The “Uses” section of that article starts with:

    Case studies have commonly been seen as a fruitful way to come up with hypotheses and generate theories. Case studies are useful for understanding outliers or deviant cases.

    Lower down that section has:

    Case studies of cases that defy existing theoretical expectations may contribute knowledge by delineating why the cases violate theoretical predictions and specifying the scope conditions of the theory.

    Case studies are used to guide experimental and quantitative research, but are not a replacement for that part of the research process.

    Applying that to case studies that appear to involve the supernatural, sufficient convincing case studies should lead to theories about the conditions for supernatural events, which should lead to experiments or quantitative studies to test those theories.





  • I don’t think any one anecdote or even a collection of anecdotes would convince me because of the explanations I layed out.

    I can think of an experiment, which would be something like to hide a box with a computer that displays one of 3 colors, selected randomly and recorded by the computer so nobody can know what color was displayed until inspecting the computer later. Ask people if they had an out-of-body experience, and if they noticed the box and looked inside. Ask people who answered affirmatively to that what color was in the box, and do a statistical analysis of the results.

    Even if you aren’t going to do a controlled experiment, you have to make sure your interviews of patients include every patient who had a near death experience over the course of your study.

    Reviews of anecdotes that were only recorded because they are interesting is not a productive way to answer this question.


  • I’m not saying “rare data in general is not valuable”.

    Not observing hawking radiation in a situation where no theory predicts hawking radiation is neither evidence for nor against the existence of hawking radiation. That would be like taking the lack of NDE in completely healthy people as evidence against NDEs.

    I’ll try to state my problem with cherry picking anecdotes about NDE more succinctly.

    My hypothesis: These NDE stories are the experience of wacky brain activity arising from near death situations.

    Supposed evidence against that hypothesis: Some of these stories involve people knowing stuff they shouldn’t have been able to know.

    My hypothesis to explain that “supernatural” knowledge:

    1. Sometimes people notice things subconsciously, and sometimes other people could have been tipped off about information in ways other people don’t realize.
    2. Sometimes people guess things correctly

    The problem with relying on anecdotes is:

    1. Memory is fallible and people’s accounts of events are often affected by discussion after the fact as well as what they “want” to think about the event
    2. This is the confirmation bias part. If you only record correct guesses, it doesn’t seem like they are guessing.

    Let’s there’s a tik tok trend and 1000 people ask someone to guess the result of 10 coin flips. One of them gets them all correct! Wow that’s amazing that person must have supernatural powers! (Nope it’s just statistics).



  • Confirmation bias is when the outcome could be adequately explained by luck.

    In the topic of near death experiences, if there are 1,000,000 near death experiences and 100 involve someone “knowing something they shouldn’t be able to”, those 100 cases are more likely to be remembered or recorded as significant than the other 900,000 cases. This can lead to an apparent statistical significance in correctly knowing “unknowable” information, when really it’s just people “guessing” correctly.

    The “black swan” scenario is a bit different but it would be something like if you are more likely to record a swan sighting if the swan is black, you will significantly overestimate the frequency of black swans.

    Im not saying the cases of apparent supernatural effects should be ignored, I’m saying they need to be taken in the context of all similar events, including the mundane, to understand if there even is an effect (knowing something that shouldn’t be possible) or if it’s just a handful of lucky guesses.


  • Near death experiences are a tricky thing to study. There are physiological explanations for much of it, such as weird brain activity is likely to be interpreted as a weird experience.

    These people would have no way of having knowing this stuff unless they’ve seen it for themselves, which would have been physically impossible.

    The problem of this argument is confirmation bias. An anecdote of seeing information you couldn’t have seen and being right is going to be more memorable than seeing information and being wrong.

    when you have several hundred of them compiled back-to-back-to-back it becomes harder and harder to find the willpower required to muster up a skeptical response

    The scientific method involves looking at both the cases where it seems like something happened and the cases where nothing happened (e.g. someone said they had an experience but it clearly didn’t match reality). If you cherry pick just the events that “showed” what you want, that’s confirmation bias.

    I did some googling of my own and found some studies on the topic from seemingly reputable sources that suggested physiological explanations might not be sufficient to explain the patterns they saw. Several of these had the same first author. I also found plenty of studies suggesting physiological explanations can be sufficient, as well as some specific criticisms of the couple studies that suggested they weren’t sufficient.

    It’s interesting for sure that there is a doctor or two who seem to believe in the supernatural. The topic of near death experience seems to be of research interest regardless of any supernatural theories because of what it tells us about the brain.

    It seems we will likely arrive at scientific consensus about near death experience in the future. I wouldn’t hold my breath that supernatural theories will survive that process.

    events that transpired when they had no brain activity.

    I think I saw the case this was talking about during my googling. It said “brain activity was not expected” which is not the same as “there was no brain activity”.

    That’s the problem with a book like the one you are describing. It’s deliberately cherry picked, exaggerated, and biased to drive you to a certain conclusion.

    I instead urge you to go read scientific papers on the topic, and specifically not just the ones that seem to suggest the outcome you want to hear.

    Here’s a place to start.


  • chocolate tastes more like coffee than soap

    This is absolutely something you could scientifically test.

    The scientific method is building up knowledge by noticing a pattern, coming up with an explanation for that pattern, then thinking what further effects that explanation would imply, and looking for those effects.

    So when someone claims something is “outside the realm of science”, how could that be?

    Often it’s either because it isn’t reproducible (it’s a miracle that supposedly happened once and never will happen again) or it doesn’t affect anything.

    If it isn’t reproducible, it’s hard to believe that it happened that way. Perhaps you are missing some details?

    If it doesn’t affect anything, why care?

    For the ghost stuff … the book Surviving Death by Leslie Kane.

    I’ve heard of many, many attempts to scientifically prove supernatural effects and none that showed a result. Most ghost stories I’ve heard have other more reasonable explanations if you think about it. Memory tends to be unreliable so sometimes details may be added or changed to fit the expected explanation, even if the person doesn’t intend to be misleading. Of course, sometimes people do exaggerate or make things up deliberately.

    Nevertheless, if you have some decent examples of actual evidence of ghosts, I’m genuinely curious.