WordPress.com



Hi-Phi Nation Season OneWritten, Produced, and Edited by Barry LamEpisode 7: Hackademics II: The HackersBarry: previously on Hi-Phi Nation.Kruth: parapsychologists are scientists.Barry: we looked at the scientific study of ESP.Anita: And I was like damn. You’re good.Barry: what I found was that in the best studies in parapsychology people do perform better than random guessing at mind to mind communication or reading the minds of their future self. But is the right conclusion that there is ESP out there? if you accept the statistics of their research and this rule that statistical significance means you found something real, it's very hard to resist the conclusion that there are small effects of ESP all around us. today finally I'm gonna look at how this rule works in the mainstream human sciences. first: psychology, the study of the human mind.Brian: I’m Brian Nosek, I am the executive director at the center for open science and a professor of psychology at the University of Virginia. we did a study where we were thinking about what might people on the extremes -- either the extreme left or extreme right -- how might they be different in interesting ways from people that are moderate and one of the studies was led by Matt Motel.Barry: Matt Motyl was a graduate student Brian's lab and he devised this technique to try and test people's color vision. you give them a colored word on a page and you have them look on a color swatch and have them find the color of the word and you measure how accurate they are. you have extreme right wing people do it, extreme left-wing people do it, and then you have moderates do it. you're trying to find out if being on a political extreme affects the accuracy of your color vision. Brian: and what he found astonishingly was that people who were further out on the left and right were less able to identify accurately the shades of grey and people who are in the political center. so literally people who are politically extreme are not able to see shades of grey, they see the world and more black and white terms.Barry: in academic psychology this kind of finding is a big deal Brian: we were like oh my god this is going to make his career as a senior grad student so he’s looking, you know getting close to being on the market. he is really enthused, we are both really enthused, the lab sees it and is very enthused we say well okay that is a crazy finding but totally cool. Barry: I mean think about it. they already had the best title for their paper: Fifty Shades of Grey. that book was an enormous bestseller at the time and it's an age of extremist politics in America so you're basically just handing print media and science bloggers clickbait on a platter.Brian: the easy thing to do which would be playing into the incentives of what is at stake for Matt and his career, what's at stake for my career is to have taken that initial finding and publish just that. not bother with doing a replication because why would we do a replication? the only thing that we can do by doing a replication is lose this golden nugget.Barry: there's no requirement of researchers in psychology that before they publish a finding they have to run the study again to make sure it works again. the original study has almost 2000 participants too so it's not like they had a small sample. and last week I talked about this rule of statistical significance, p less than .05, as the standard for drawing a conclusion from your study. Nosek and his student Matt Motel got p less than .01. by any of the standards at the time they did fine.Brian: and so we ran another study did it again and we got nothing. didn't replicate it. that's the end of the story.Barry: well it was almost the end of the story. Brian Nosek got worried. how many other flashy findings out there in psychology were like this?Brian: a key aspect of a scientific claim becoming credible is that you don't have to rely on me, the originator of that finding, to say it's true. a scientific claim becomes credible because you can reproduce it. an independent person can follow the same methodology and obtain that result. and if that doesn't happen claims that are supposed to be scientific claims become less credible. it's just a core value of what makes science science. in the daily practice of science it isn't often how researcher is done.Barry: it's not how research is done. the way that research is done that you publish the 50 shades of grey finding and then enter the pantheon of cool psychological findings. a lot of what you hear reported in the popular science press is like this. not everything, but a lot. a flashy finding supported by a single study based on statistical significance.Andrew: I’m Andrew Gelman, professor of statistics and political science at Columbia University. there’s a term someone came up with called the myth of the single study and if you look at statistics textbooks, econometrics textbooks, quantitative science textbooks they have these little studies which are supposed to be definitive. there's this paradigm in social science we call it the stylized fact. stylized sounds negative but it's not, it's supposed to be sort of a true fact that's presented in a clear way so an example with stylized fact would be that presidential incumbent candidates do better when the economy is better. so in social science we do these studies and we get these stylized facts and then we build our theories around them.Barry: flashy results that make for a good story around a stylized fact. it's just candy to us, the everyday consumer of news. it makes us feel like there was this ever so slightly hidden fact about the world that we have now uncovered but having uncovered it we understand it because of the story. here are some recent examples of these stylized facts. hurricanes with female names kill more people than hurricanes with male names. because of sexism people don't take female named hurricanes as seriously as male named hurricanes so they don't prepare or evacuate. her-icanes are deadlier than him-icanes. or this one: calming background music on the playground reduces bullying among children because background music affects mood. moods that lead to bullying. or how about this one from the same researcher: energetic happy background music that you like increases your chances of complying when asked to do mean things to people. maybe because you're so happy from the music that you're more likely to think that the mean things that you're doing aren't going to be that bad. both results hold up to the rule of statistical significance. for Brian Nosek, it wasn't just about one failed replication. how many other stylized facts out there don't hold up to scrutiny under replication? so Brian Nosek tried to find out.Brian: this was a community effort of 270 co-authors and another 85 people that contributed to try to get an initial estimate of the reproducibility of psychological science. people volunteered their time to do a replication of a sample of studies from three journals that were published in 2008 in the psychological literature.Barry: Brian Nosek and his team took a sample of 100 studies from 2008. and each study increase their sample size or their measurement accuracy or both. measurement accuracy just means that. you're trying to find better ways to measure what you're looking for. for example, and I'm just making this up, if you are trying to measure an emotional response to music you might decide that counting foot tapping and head bobbing is more accurate a measurement than say a questionnaire or it could be the other way around. either way these kinds of improvements in a study are called increasing the power of the study, which means that you're trying to design your study to better detect the thing that you're looking for. and finally most importantly the open science center pre-registered the replications and all this means is that you're making open to the entire community ahead of time what you're doing, how you're doing it, and putting all your methods and analyses on the table before you run the experiment and analyze it. and with this setup the open science center had some predictions about how many studies ought to be replicated. Brian: just with the basic standards we would anticipate that about 5% of published results are false positives.Barry: in other words you would expect 5% not to replicate?Brian: that’s sort of our willingness to tolerate false positives.Barry: if the rule of statistical significance -- p less than .05 -- were as accurate as everybody thought you should have ninety-five percent of the study you publish replicate successfully. in fact, it should be more than that because if you remember some of the results are p less than .01, like the Fifty Shades of Grey study. some are even lower than that so really even more should replicate. out of a hundred studies they should have gotten almost all of them replicated by the established standards in the field. Brian: I had thought I was making a pessimistic prediction when I said we’d probably get fifty percent. Barry: Brian Nosek was way less optimistic because he knew things that we don't. more on that later there's also a disputable what counts as a successful replication. do you have to have the same standards as last time, p less than .05, or can something count as a replication if it's close? Nosek and his team just decided to be as inclusive as possible Brian: with different measures of deciding whether the replication result was successful or not in reproducing the original result, we got between thirty and forty percent of the original results successfully reproduced in the replicationsBarry: in the large-scale crowd-sourced replication with high standards for what counts as replication 30 of the studies were replicated and with lower standards 40. so the accepted standards of publication in the psychological sciences is about thirty to forty percent accurate...sort of.Brian: the replication effect sizes were about half the size one average as the original studies. so that's a large magnitude change, is that we really see reduction almost across the board, there were very few of the experiments that were able to obtain the same size of a result as the original studies.Barry: remember that the effect is like whether a drug works in making you better, the effect size is how much sooner the drug gets you better. if the drug gets you better in six days rather than seven without the drug it has an effect but its effect size is small. if it gets you better in two days that's a large effect size so even for the studies that did replicate, the effect sizes were cut in half. that's the kind of inaccuracy, even if it's less than accurate than not having a replication at all. that's not good Barry: you're listening to Hi-Phi Nation, a show about philosophy that turns stories into ideas. I'm Barry Lam. what does the failure to replicate even mean? it doesn't mean that your earlier study is false. it could mean that the replication was the bad study and the first study was a good one. that's possible. it could mean that your original finding just held in a very, very narrow circumstance in the world and that in broader circumstances it's not going to hold. that's possible too. these are all legitimate questions but I think they missed the key takeaway. non replication doesn't show that you're wrong. it shows you didn't know that you were right in the first place. the point is that the rule that everyone settled on to make inferences about the world from your experiment is much less accurate than you thought it was. much much less accurate. you’re way overconfident in your original findings even if you still think they're true. what's remarkable is the next part of the story. even before one attempted replication took place Brian Nosek had another idea. Brian: so we ran prediction markets on replications Barry: he had teams of psychologists and researchers not involved in the replications to bet on which studies would replicate and which wouldn't.Brian: gave them one hundred dollars each to bet in the markets. they had a paragraph description of what the question was and what the finding was and then what the criterion of success would be, which is p-value less than .05 for this test. essentially they're buying shares in different replications and if the replication was successful then the share would pay out a dollar. if it was unsuccessful it would pay out zero.Barry: these prediction markets tell us a lot. most importantly I think they tell us what scientists really think about the results of these studies. you're allowing people to use any opinion or bias whatsoever they want to make a decision about what they really think. there's this really long tradition in the philosophy of belief that says if you really want to know what people believe see how they'd bet. I think Nosek’s betting market gives this alternative universe in which science is conducted in a different way, where there isn't just one rule and standard for everyone to follow as to what to conclude from a study. instead the scientific community as a whole uses whatever background knowledge, suspicions, biases, all of these good and bad things that go into human judgments to render their verdicts about the soundness of an experiment and its findings. it's way more subjective yes. but is it worse?Brian: there are a couple of interesting things. one was that there's variation across the findings. some were high confidence right? the market prices were 83 cents and others were low confidence, market price twenty cents, and all of them still had uncertainty right? there were no market prices that were nearly a dollar or nearly nothing.Barry: so all this means is that there was no one study where there was a consensus view about whether it would or wouldn't replicate. so once people had finished betting you could see how accurate they were at predicting which studies would replicate.Brian: if the price was anywhere over fifty cents then it's predicting replication success and the actual outcomes were right on target. 71 percent of time the market successfully predicted the replication outcome and so that was an amazing result to us which was that you can actually predict these outcomes. Barry: what is going on? Psychologists are betting against their own standards for accepting studies. They're shorting themselves and winning. From the outside one can vary uncharitably state the finding of your paper as the finding that psychologists as a whole can detect the bullshit that's going on in their in their discipline. You don't have to respond to that.Brian: it is a challenging result for sure. but you can still have a charitable conclusion which is they can identify which ones are going to be hard to replicate even though they're true. Barry: Brian Nosek is being very nice here. he could be right but the judgments people are making are that they won't find the effect in a replication but it's still there. but what reason do people have for concluding that? are the original studies so good that your conclusion should still stand even when a replication attempt fails? Andrew Gelman has an argument that this can't be right. Andrew: the trick is that you do a mirror reflection.Barry: This is an excerpt from a talk he gave in Britain. he’s asking us to think of what we would say if the replication came first and the original study came second.Brian: in that case what we have is a controlled pre-registered study consistent with no effect followed by a small sample study that was completely uncontrolled where the researchers who are allowed to look for anything and they happen to find something that was statistically significant. Barry: the replications have larger samples that are measurements and controls and were specifically designed to find the effects that the original studies claim to have found. if the people betting in the markets really thought that the effects were there but the better design studies would fail to detect them they'd be putting a lot of weight on the original studies just because they came first. you're listening to hi-phi nation, a show about philosophy that turns stories into ideas. I'm Barry Lam. I’ve been talking as though these problems of replication are unique to psychology. but they're not they're everywhere in the human sciences.Brian: in every place that reproducibility has been examined systematically, there are challenges of reproducing original results. It's not psychology, it's not social sciences, it's not life sciences; it's everywhere that it's been investigated. Now, that doesn't mean that there's a ton of data. In many places, it hasn't been on the map. The reason that the or project and psychology got so much attention was it was the first large-scale attempt to do this with actual independent replications but there are a lot of teams that are starting to investigate this systematically across different disciplines Barry: since Brian Nosek and the open science center published their large-scale replication study other fields in the human sciences are coming on board. people have now done it for economic research which is about sixty percent replicable based on p less than .05. but similarly the effect sizes and the replications were all much lower just like in psychology. and just like Nosek’s betting markets when the field of economics bet on replications they were 75% percent accurate. something is up. entire fields of science are betting against the reliability of their own standards. they know as a whole that something is off with the consensus standards for publishing research but individually research like this is continuing. as a whole these fields are pretty good at telling us which research can be independently verified and which can't. then why don't they just tell us that? why is there this whole song and dance around statistical significance p less than .05? publications, press releases to the media and the public turns out to believe all this stuff that they know can't be independently verified. I try to find out coming up on hi-phi nation.Barry: You're listening to Hi-Phi Nation, a show about philosophy that turns stories into ideas. I’m Barry Lam. andrew Gelman of columbia university has a blog about all of the issues i've presented so far on this show. it's at Andrew Gelman dot com. there's a link to it on our website. Gelman believes that science as a whole should discard this rule of drawing conclusions on the basis of statistical significance. one of his reasons is that statistical significance, he thinks, is responsible for an illusion. an illusion that affects almost all of the research in the human sciences. Andrew: statistical based studies tend to overestimate effect sizes because of what we call the statistical significance filter and that's just a mathematical bias.Barry: it's a subtle but rather profound point who explains a lot of what Brian Nosek found. imagine that there really are one percent of people in the country that would change their vote on Election Day if their favorite football team won the weekend before. how would we find this out? in reality people aren't always honest if you ask them who they're going to vote for and they're not always honest or even knowledgeable about why they're voting for the person they're voting for but that's the best question you have to find out so it'll have to be good enough and you don't have enough time to talk to everyone in the country so you can only pick a few hundred. this is an example of a noisy measurement and a small sample. a noisy measurement means you're going to get all of these differences in how people answer. that roughly picks out what you're looking for but isn't very exact. When you combine all of this now you have to compare two groups. groups where their football team didn't win and groups where their football team did win. when you compare those two groups and you found in your study a one percent difference between them even though that's in reality the truth, that wouldn't be statistically significant because you wouldn't be able to separate that from random chance. mathematically the only way you would get significance is if you got a huge difference, like you had a group where ten percent of the people would change their vote because their football team won. if you saw that, you get statistical significance and you report it but you'd be reporting an illusion. it would be a result of the sample you happen to pick out or due to the noise in your measurement or both.Andrew: if you get statistical significance, you lost...because you became very confident in something that is just noise.Barry: but it's even worse than that.Andrew: if you have a noisy study, there can be a high chance that the true effect is in the opposite direction of where you think of it, even if it's statistically significant.Barry: what this means is that if in reality people on average got better just a little bit sooner from taking a drug your samples and measurements could make you conclude the opposite. that the drug just prolongs the illness. mathematically there can be a good chance with small samples and noisy measurements that if you found significant differences between two groups it's because you happen upon the group who took the drug but stayed sicker longer for all kinds of other reasons. Gelman thinks that this search for statistical significance is presenting researchers with an illusion, a distorted picture of reality which turns out to be exactly the distortion that Brian Nosek found when he tried higher quality replication.Barry: unlike Andrew Gelman Brian Nosek doesn't conclude from the problems with statistical significance that we should get rid of it as a method of drawing scientific conclusions. it'd be paradoxical if he thought that. he used those very methods in his replication studies. they work when used correctly. Deborah: my name is Debra Mayo. I am a philosopher of science and i'm a professor at Virginia Tech and also a visiting professor at the London School of Economics Barry: philosopher Deborah mail is a leading researcher in the philosophy of statistics and is the most staunch defender of the original ideas behind the rules of statistical significance and p-values in making scientific inferences. her main diagnosis is that, well, there's abuse and cheating going on. Deborah: it's typical that people say well it’s too easy to get low p values but notice that when they do have pre-registration, pre-registration where they know in advance and they even agree how they're going to do the study, then they find it difficult. I call this the paradox of replication, that somebody says it’s easy to get p-values well why how come they only got about thirty three percent replication. is it easier is it hard? well it's it's hard if you don't cheat.Brian: the currency of science is publication and so to the extent that getting a publication is advancing my career then there is a challenge, a conflict of interest when what gets published is not necessarily what's most accurate. when I'm doing my research I'm finding things I'm finding different ways to analyze data and looking at different findings I'm getting results here not getting them there and my incentives are to make that the most beautiful clean wonderful story I can in order to make it publishable and science’s interests are just tell it how it is. but because we're working on hard problems because we're pushing out at the boundaries of knowledge it isn't always a nice clean tidy amazing interesting story. a lot of times it's a bit of a mess but I have to get it to be as publishable as possible. so that conflict of interest is in when I'm faced with different kinds of data what's best for me and what's best for science may not be the same thing.Barry: it's a fundamental assumption of Brian Nosek’s and the Open Science Center, but the problem of replicating results in the sciences have to do with careerism: getting funding, getting a job, a promotion. These pull away from the impartial pursuit of the truth. The problem for Nosek is not the rule of statistical significance or p-values; it's that careerism incentivizes people to bend the rules.Andrew: here you are doing researching; you want to make discoveries and your p-value is like 0.01 instead of 0.05; well that’s too bad. So you can do things called p-hacking Mayo: p-hacking Brian: p-hacking Andrew: to get the p-value less than 0.05.Mayo: we all know that if you report data selectively, you will be able to find evidence for any hypothesis you want.Brian: so for example, you can throw away cases.Mayo: cherry-picking.Andrew: this happens a lot in the context of medical trials. It's a well-known problem that they like to drop out people who are getting better under the control.Barry: if it's your incentive to show that a drug works, you have to have the treatment group do significantly better than the control group, but if there's someone doing really well in the control group, that messes up your comparison. So you find some way to exclude them from the experiment; maybe they weren't sick, after all. Like in psychology, if you're trying to show that listening to Beyonce makes people happier than people who listen to Kenny G and you're almost there, but there's this person who gets really happy while they're listening to Kenny G ,you find a reason to exclude them, like maybe they're happy because they like to listen to music ironically and that's not what you're really testing. But then if someone is listening to Beyonce ironically and really happy,Andrew: maybe find a way to keep her in. So that's p-hacking.Barry: that’s a kind of cheating and then there's another kind.Deborah: what they'll do is, let's say we try the test once and we found that you couldn't reject the hypothesis, but then finally, the third, the fourth time we find some and we ignore the cases that didn't show the result and we only report the ones that did. That's typically what goes on. In fact, the probability of finding at least one statistically significant result by chance alone can be very high, depending on how hard you try.Barry: this is called a file drawer effect. A single researcher run to study multiple times all of a failed attempts to stick back into their file drawer and the successful ones they end up publishing from the outside it looks like you have one study and got a significant result when in reality you very selectively showed your wins and not your losses the file drawer effect works at a large scale without any nefarious intentions from researchers journals just aren't interested in publishing studies that failed so there's a good reason for that it isn't true that eating bacon makes you earn more money do we need a published study to tell us that if any researchers tried and failed to find that and stuck the failed study in their file drawer then the people who happen by chance to find the population of bacon munchers that made a lot of money they're going to look like they found something original and interesting Deborah: the p-values are no longer legitimate the the ones that you report our spurious p-values Barry: so that's cheating. and then there are things that are not quite cheating but also not quite kosher either.Deborah: Optional stopping Barry: Optional stopping is like playing rock paper scissors with a friend, and then when you lose, you say “okay best two out of three” and then when you lose that you say hey best three out of five, and then when you finally win you get congratulated for a fair victory. [Music] In experimental science sometimes this is called data peeking. you run an analysis after studying a hundred people, but your results are just iffy, right on the borderline, not quite statistically significant. so then you add another fifty subjects, and then boom you get statistical significance. so you report that you ran a study of 100 people and got a result. Is it cheating or is it not? Well here's two ways to look at it. the first way is, yeah it's cheating. you're selecting your sample size for the sole purpose of succeeding in your study just like in the case of rock-paper-scissors. There's another way to look at it: I could have just as easily decided to study 150 people rather than a hundred. as long as those people are randomly chosen subjects also, what's the big deal. Multiple comparisons. Suppose you're trying to find out if Beyonce makes you happier than listening to Kenny G and you don't find anything.Andrew: another thing you can do is break up your data, so analyzed all people, and we found the effect for men but not for women, or just for women or not for men. there was a study that didn't find statistical significance. this was the study saying that women were more likely to wear red clothing during a certain time of the month. they looked and they found that they had data on two different days and one day was a warm day one day was a cold day, and it turned out there was a difference between the two days.Barry: and once you find a difference in the data when you break it up, you can then report women are likelier to wear red on cold days when they're ovulating, even though originally you are trying to figure out whether they're likelier to wear red in general when they're ovulating.Andrew: you can change what your threshold is. there was the example of the study where the sociologists claim that more beautiful parents were more likely to have girl babies. the attractiveness of the parents was rated on a one-to-five scale and he compared the 5s to the 1-4s, and he got statistical significance. but if he compared the 1-3s to the 4s and 5s he wouldn't have got it, or the one and twos to the three fours and fives he wouldn't have found if, he found the one comparison Barry: again iffy if it's cheating or not. I mean on the one hand you can say it's cheating. you were looking for something, you didn't find it, so you're looking to your data and you're finding something to publish. another way of looking at it is to say, well you gathered all this data, you didn't end up finding something that you wanted to find, but there's all these interesting patterns that i did find. i'm going to go and report that, because those patterns are definitely real.You're listening to Hi-Phi Nation, a show about philosophy that turns stories into idea i'm Barry Lam. These multiple comparisons, optional stopping, or excluding people from your study, they're not always explicit attempts after an analysis to hack your way to a result that achieves statistical significance. Sometimes you make decisions about this stuff beforehand or you make your decisions as you run an experiment before you even analyze the data. people have very good reasons for making the decisions they do. maybe when people rate couples on a scale of one through five, you really should only count the fives as attractive because they're also much more attractive than the fours. Maybe you should exclude people from your study who mark I'm extremely happy on every question in the survey because they weren't paying attention Andrew: it's like you're you're walking through a garden right and you choose different paths like each time it seems to make sense.Barry: Gelman's worry is that even when the decisions make sense other decisions would have made sense to if they would have helped you had your experiment turned out differently Andrew: you look at your data and you do just one analysis, but if your data had been different you would have done the different analysis, and if your data had been even different you would have done a different out Barry: Are attractive parents the ones rated a five or the ones rated four as well. you might have good reason to think in your study the fives. But would you have had good reason to think it was the fours and the fives, if it turned out that that was the most convenient way to classify attractiveness when your data came out? There's no dishonesty here, it's just a kind of motivated reasoning. To fight that you have to consider the decisions you didn't make, and how they would have affected your final results. So gelman has a solution to this problem which people call the problem of research or degrees of freedom [Music] after you run your numbers for statistical significance, pretend you made different decisions, and run the numbers for those decisions. then you can see just how fragile your results turn out to be. Deborah: We're not interested in isolated significance results. you have to show that you can generate significant results at will. You can generate results that rarely fail to be statistically significant to show that you have demonstrated an actual effect.Barry: this is the deeper problem. when you're trying to detect things, real things, real effects, among human beings, and your particular test says “yeah it's there.” To be sure, or at least sure enough to tell everyone in the world, it can't be too easy for you to have failed to detect it. in all of these cases of single studies where people find significant results, the problem is never that the effects aren't there. even if you're right and they are there, your test isn't good enough for you to know that. it's not a problem of truth, it's a problem of knowledge. knowledge can't be that's fragile [Music] You're listening to Hi-Phi Nation a show about philosophy at turns story into ideas, I'm Barry Lam. so back to our question from our previous episode. where is parapsychology in all of this? here's Matthew Makel, psychology researcher at Duke University, and former student of Daryl Bem, the parapsychologist who claimed to have found ESP in 2011 Matt: the general consensus of the skeptics of the field is that what Bem really demonstrated was researcher degrees of freedom, because in his multiple experiments in the 2011 paper, they have different numbers of participants. why? was it that he 50 and then looked, and said I got something significant and I’ll stopped and then if he collected fifty and it wasn't significant, he said “let’s collect more” then peak again. I think he was doing what has largely been the accepted practice of the field for decades.Barry: these practices are changing with the advances at the Open Science Center and the move towards pre-registering studies, and a more knowledgeable application of statistics. I heard a rumor, which I can't substantiate, that there will be pre-registered high-powered studies in parapsychology very soon as well. I'll keep you updated. [Music] I want to close today with some thoughts. There's a very old debate in philosophy about how to trade off being right and not being wrong. one thing we can do is drown ourselves in a sea of ideas mostly false ones, in the hopes that we find that one insightful true nugget of wisdom we wouldn't otherwise find if we were too cautious. if we did that we end up believing more true things but a lot of false things as well. we'd be credulous. Or we can build a very strong border wall and do extreme vetting of ideas to keep out all the false ones, so that only the true ones get in, and we're never duped or harmed. if we did that, we'd be skeptical. we're not going to be wrong very much, but we might not have much to believe either. Raising the standards of publishing results in science, finding out the right rules to draw conclusions in science, it's the same issue. how much is too much falsehood to allow in pursuit of the truth? Scientists are arguing right now about the right trade off, but philosophers haven't settled this issue amongst themselves either. the only rule of thumb i can give is when you read the next cool scientific finding think about whether you want to be more credulous or more skeptical. ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download