I ran into an interesting incident of this recently. I got a Google Scholar alert about a paper with some experiments related to a paper I had published a while ago, by one "N. Tvlg". I read the paper with interest but I started noticing that although the arguments sounded good, they didn't really make sense, and also the descriptions of the results didn't really match the figures. Eventually I came across a cluster of citations to completely unrelated papers---my field is computational linguistics and these were citations to, like, studies of battery technologies for electric cars. I looked up "N Tvlg" on Google Scholar and they had "published" several articles very recently in totally divergent fields, and upon inspection, all of them had citations back to this materials science research buried deeply somewhere. Clearly these were LLM generated papers trying to build up citation count and h-rank for someone's career.
It kinda skips over how large mainstream journals, with their restrictive and often arbitrary standards, have contributed to this. Most will refuse to publish replications, negative studies, or anything they deem unimportant, even if the study was conducted correctly.
So much of this started with the rise of the peer-review journal cartel, beginning with Pergamon Press in 1951 (coincidentally founded by Ghislaine Maxwell's father). "Peer review" didn't exist before then, science papers and discussion was published openly, and scientists focused on quality not quantity.
I'm not sure that the system was ever that near to perfection: for example, John Maddox of Nature didn't like the advent of pre-publication peer review, but that presumably had something to do with it limiting his discretion to approve and desk-reject whatever he wanted. But in any case it (like other aspects of the cozy interwar and then wartime scientific world) could surely never have survived the huge scaling-up that had already begun in the post-war era and created the pressure to switch to pre-publication peer reivew in the first place.
I know a PhD professor doing post doc or something, and he accepted a scientific study just because it was published in Nature.
He didn't look at methodology or data.
From that point forward, I have never really respected Academia. They seem like bottom floor scientists who never truly understood the scientific method.
It helped that a year later Ivys had their cheating scandals, fake data, and academia wide replication crisis.
When I read something in a textbook I blindly believe it, depending on the broader context and the textbook in question. Is that a bad thing?
People are constantly filtering everything based on heuristics. The important thing is to know how deep to look in any given situation. Hopefully the person you're referring to is proficient at that.
Keep in mind that research scientists need to keep abreast of far more developments than any human could possibly study in detail. Also that 50% of people are below average at their job.
There is a vast difference between a student reading from a textbook and a researcher / scientist reading studies and/or papers.
As a student you are to be directed* in your reading by an expert in the field of study that you are learning from. In many higher level courses a professor will assign multiple textbooks and assign reading from only particular chapters of those textbooks specifically because they have vetted those chapters for accuracy and alignment with their curriculum.
As a researcher and scientist a very large portion of your job is verifying and then integrating the research of others into your domain knowledge. The whole purpose of replicating studies is to look critically at the methodology of another scientist and try as hard as you can to prove them wrong. If you fail to prove them wrong and can produce the same results as them, they have done Good Science.
A textbook is the product of scientists and researchers Doing Science and publishing their results, other scientists and researchers verifying via replication, and then one of those scientists or researchers who is an expert in the field doing their best to compile their knowledge on the domain into a factually accurate and (relatively) easy to understand summary of the collective research performed in a specific domain.
The fact is that people make mistakes, and the job of a professor (who is an expert in a given field) is to identify what errors have made it through the various checks mentioned above and into circulation, often times making subjective judgement calls about what is 'factual enough' for the level of the class they are teaching, and leverage that to build a curriculum that is sound and helps elevate other individuals to the level of knowledge required to contribute to the ongoing scientific journey.
In short, it's not a bad thing if you're learning a subject by yourself for your own purposes and are not contributing to scientific advancement or working as an educator in higher-education.
* You can self-study, but to become an expert while doing so requires extremely keen discernment to be able to root out the common misconceptions that proliferate in any given field. In a blue-collar field this would be akin to picking up 'bad technique' by watching YouTube videos published by another self-taught tradesman; it's not always obvious when it happens.
> There is a vast difference between a student reading from a textbook and a researcher / scientist reading studies and/or papers.
Not really. Both are learning new things. Neither has the time or access to resources to replicate even a small fraction of things learned. Neither will ever make direct use of the vast majority of things learned.
Thus both depend on a cooperative model where trust is given to third parties to whom knowledge aggregation is outsourced. In that sense a textbook and prestigious peer reviewed journals serve the same purpose.
> If you fail to prove them wrong and can produce the same results as them, they have done Good Science.
Not really in my humble opinion. Sure, the Popperian vibe is kind of fundamental, but the whole truncation into binary-valued true/false categories seldom makes sense with many (or even most?) problems for which probabilities, effect sizes, and related things matter more.
And if you fail to replicate a study, they may have still done Good Science. With replications, it should not be about Bad Science and Good Science but about the cumulation of evidence (or a lack thereof). That's what meta-analyses are about.
When we talk about Bad Science, it is about the industrial-scale fraud the article is talking about. No one should waste time replicating, citing, or reading that.
This is a good point. It is not humanly possible to verify every claim you read from every source.
Ideally, you should independently verify claims that appear to be particularly consequential or particularly questionable on the surface. But at some point you have to rely on heuristics like chain of trust (it was peer reviewed, it was published in a reputable textbook), or you will never make forward progress on anything.
> When I read something in a textbook I blindly believe it, depending on the broader context and the textbook in question. Is that a bad thing?
It is if what you read is factually incorrect, yes.
For example, I have read in a textbook that the tongue has very specific regions for taste. This is patently false.
> Keep in mind that research scientists need to keep abreast of far more developments than any human could possibly study in detail. Also that 50% of people are below average at their job.
So, we should probably just discount half of what we read from research scientists as "bad at their job" and not pay much attention to it? Which half? Why are you defending corruption?
The problem is that you can't just verify everything yourself. You likely have your own deadlines, and/or you want to do something more interesting than replicating statistical tests from a random paper.
Academia has problems, like everywhere else. But that seems like a big extrapolation from just one professor.
Fake data—you can only get that type of scandal when people are checking the data. I’d be more skeptical of communities that never have that kind of scandal.
Fun fact, he almost got the worldwide console rights to Tetris back in the 80s, and tried going to Soviet officials to get those rights. To the point he's the antagonist of a recent "Tetris" movie that came out.
I believe by saying it is coincidental they are saying there is probably no relevance, just an interesting piece of trivia, why put out this interesting piece of trivia? Because maybe someone will be able to make an argument of relevance.
I imagine it's the interesting peculiarity that the same people seem to crop up over and over and over again. Six degrees of Kevin Bacon or something, except it's like one or two degrees. As George Carlin said, "it's a big club, and you ain't in it"
For example Donald Barr (father of twice-former US Attorney General Bill Barr) hiring college-dropout Jeffrey Epstein whilst headmaster at the elite Dalton School
Additional fun facts about Donald Barr: he served in US intelligence during WWII, and wrote a sci-fi book featuring child sex slaves
Also the Epstein-Barr virus causes Mono, the clone of .NET, which was created by Bill Gates, known associate of Epstein, whose father was president of the Washington State Bar Association. And you know who else works in Washington? Join the dots, people.
We call people who make connections like these "conspiracy theorists," until they're right, at which point we call them "right". And somewhere in between, if they manage to get a job, we call them "Simpsons writers."
Ghislaine's father (Robert Maxwell) was also a terrible person but for different reasons.
Robert Maxwell was a crook, he used pension funds (supposed to be ring-fenced for the benefit of the pensioners) to prop up his companies, so, after his slightly mysterious death it was discovered that basically there's no money to pay people who've been assured of a pension when they retire.
He was also very litigious. If you said he was a crook when he was alive you'd better hope you can prove it and that you have funding to stay in the fight until you do. So this means the sort of people who call out crooks were especially unhappy about Robert Maxwell because he was a crook and he might sue you if you pointed it out.
If you want to know more about the history of Pergamon Press there's a great Behind the Bastards episode on Robert Maxwell (Ghislaine Maxwell's father) - who himself was a scumbag in a variety of ways that were entirely distinct from Ghislaine Maxwell's brand of scumbaggery - that covers this. Might even be a multipart episode - it's a while since I've listened to it, but I have a feeling it's at least a two parter.
tl;dr He is the bridge that uncomfortably links Biden's former Secretary of State, Antony Blinken, to Jeffrey Epstein and Mossad. Hence, *gestures at the last couple of weeks and years*. Dude was just, like, Fraud Central, apparently.
Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies. It'll be great if I can look them up when needed but im not looking forward to email ToC alerts filled with them.
Also who's funding you for replication work? Do you know the pressure you have in tenure track to have a consistent thesis on what you work on?
Literally every single know that designs academia is tuned to not incentivize what you complain about. Its not just journals being picky.
Also the people committing fraud aren't ones who will say "gosh I will replicate things now!" Replicating work is far more difficult than a lot of original work.
> Do you want issues of Nature and cell to be replication studies?
Of course I do! Not all of course, and taking (subjectively measured) impact into account. "We tried to replicate the study published in the same journal 3 years ago using a larger sample size and failed to achieve similar results..." OR "after successfully replicating the study we can confirm the therapeutic mechanism proposed by X actually works" - these are extremely important results that are takin into account in meta studies and e.g. form the base of policies worldwide.
Honestly even if they didn't publish the whole paper, if there was just a page that was a table of all the replication studies that were done recently, that would be pretty cool.
I know you got a ton of responses already but not caring about replicability just invalidates science as a method. If we care only about first to publish we end up in the current situation where we don't even know that we know is actually even remotely correct.
All because journals prefer novelty over confirmation. It's like a castle of cards, looks cool but not stable or long-term at all.
Maybe nature and cell and a few other journals should be exceptions: they should be the place that the most advanced scientists publish interesting ideas early for the consumption by their competitors. At that level of science, all the competitors can reproduce each other's experiments if necessary; the real value is expanding the knowledge of what seems possible quickly.
(I am not seriously proposing this, but it's interesting to think about distinguishing between the very small amount of truly innovative discovery versus the very long tail of more routine methods development and filling out gaps in knowledge)
Advanced groups usually replicate their competitor's results in their own hands shortly after publication (or they just trust their competitor's competence). But they don't spend any time publishing it unless they fail to replicate and can explain why they can't replicate. From their perspective, it's a waste of time. I think this has been shown to be a naive approach (given the high rate of image fraud in molecular biology) but people who are in the top of the field have strong incentives to focus on moving the state of the art forward without expending energy on improving the field as a whole.
"strong incentives to focus on moving the state of the art forward without expending energy on improving the field as a whole"
That sort of Orwellian doublethink is exactly the problem. They need to move it forward without improving it, contribute without adding anything, challenge accepted dogma without rocking the boat, and...blech!
Top journals are not inherently prestigious. They are prestigious because they try to publish only the most interesting and most significant results. If they started publishing successful replication studies, they would lose prestige, and more interesting journals would eventually rise to the top. (Replication studies that fail to replicate a major result in a spectacular way are another matter.)
> Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies.
Actually, yes, I do. The marginal cost for publishing a study online at this point is essentially nil.
I think archives with pretty low standards for notability are a good idea. At some point though you have to pick what actually counts as interesting enough to go in the curated list that is actually suggested reading, where the prestige is attached. If there's no curation by Nature then it falls to bloggers or another journal to sift through the fire-hose and make best-of lists. Most of the value is in the curation, not the publishing. Without exclusivity there's very little signal.
Even if that negative study could save you one, two, three+ years of work for the same outcome (which you then also can't really do anything with)? Shouldn't there BE funding for replication studies? Shouldn't that count towards tenure? Part of the problem is that publications play such a heavy role in getting tenure in the first place.
I'm sure you can more narrowly tune your email alerts FFS.
"Original research" isn't worth much unless replicated, which is the entire problem being discussed in this thread. Replicating studies are great though because they tell you if the original research actually stands and is valid.
> Replicating work is far more difficult than a lot of original work.
Only if the original work was BS. And what, just because it's harder, we shouldn't do it?
I must be missing something, surely the argument isn't "other systems also disincentivize solving the problem, therefore we shouldn't work to fix this one"
If you're thoroughly debunking a previous Nature paper they just might publish that. But the expectation is that you'll succeed. Publishing that sort of mundane article would reduce the prestige of getting something into the journal. Publishing in a high impact journal is only seen as an achievement in the first place because of what it implies about the content of your paper.
Realistically, everyone will say “yes” to the “do you want” question because if you’re not a reader or a subscriber you benefit from the readers reading replication studies.
I believe people will enthusiastically say yes but that they do not routinely read that journal.
> Replicating work is far more difficult than a lot of original work.
I don’t regularly read scientific studies but I’ve read a few of them.
How is it possible that a serious study is harder to replicate than it is to do originally. Are papers no longer including their process? Are we at the point where they are just saying “trust me bro” for how they achieved their results?
> Do you want issues of Nature and cell to be replication studies?
Not issues of Nature but I’ve long thought that universities or the government should fund a department of “I don’t believe you” entirely focused on reproducing scientific results and seeing if they are real
> How is it possible that a serious study is harder to replicate than it is to do originally.
They aren't. GP was on point until that last sentence. Just pretend that wasn't there. It's pretty much always much easier to do something when all the key details have been figured out for you in advance.
There is some difficulty if something doesn't work to distinguish user error from ambiguity of original publication from outright fraud. That can be daunting. But the vast majority of the time it isn't fraud and simply emailing the original author will get you on track. Most authors are overjoyed to learn about someone using their work. If you want to be cynical about it, how else would you get your citation count up?
This isn’t about honest researchers resorting to fraud to publish their null results
because they were blocked by big bad Nature. It’s about journals and authors churning out pure junk papers whose only goal is to game metrics like citation count.
Is there a viable career path for researchers who choose to focus on replication instead of novel discoveries? I assume replications are perceived as less prestigious, but it's also important work.
Right, it seems that many of the weaknesses in the system exist because they serve the interests of journal publishers or of normal, legitimate-ish researchers, but in the process open the door to full-time system-hackers and pure fraudsters.
Mainstream journals are complicit, but are not the biggest problem.
The biggest problem by far is modern society: Tenure, getting paid a livable wage as a researcher, not getting stack-ranked and eliminated from your organization all overindex on positive research results that are marketable. This "loss function" encourages scientific fraud of sorts.
This is Goodhart's law at scale. Number of released papers/number of citations is a target. Correctness of those papers/citations is much more difficult so is not being used as a measure.
With that said, due to the apparent sizes of the fraud networks I'm not sure this will be easy to address. Having some kind of kill flag for individuals found to have committed fraud will be needed, but with nation state backing and the size of the groups this may quickly turn into a tit for tat where fraud accusations may not end up being an accurate signal.
Also, Brandolini's law. And Adam Smith's law of supply and demand. When the ability to produce overwhelms the ability to review or refute, it cheapens the product.
> Number of released papers/number of citations is a target
There was this guy, well connected in the science world, that managed to publish a poor study quite high (PNAS level). It was not fraud, just bad science. There were dozens of papers and letters refuting his claims, highlighting mistakes, and so... Guess what? Attending to metrics (citations, don't matter if they are citing you to say you were wrong and should retract the paper!), the original paper was even more stellar on the eyes of grants and the journal itself.
There’s an accurate way to confirm fraud: look for inconsistencies and replicate experiments.
If the fraudsters “fail to replicate” legitimate experiments, ask them for
details/proof, and replicate the experiment yourself while providing more details/proof. Either they’re running a different experiment, their details have inconsistencies, or they have unreasonable omissions.
Of course this is slightly messy too. Fraudsters are probably always incorrect, of course they could have stolen the data. But being incorrect doesn't mean your intentionally committing fraud.
That would be great if journals bothered publishing replication studies. But since they don't, researchers can't get adequate funding to perform them, and since they can't perform them, they don't exist.
We can't look for failed replication experiments if none exist.
>>95% of the time, the fraudsters get off scot-free. Look at Dan Ariely: Caught red-handed faking data in Excel using the stupidest approach imaginable, and outed as a sex pest in the Epstein files. Duke is still giving him their full backing.
It’s easy to find fraud, but what’s the point if our institutions have rotten all the way through and don’t care, even when there’s a smoking gun?
What do you think it is about machine learning that makes it hard to replicate? I'm an outsider to academic research, but it seems like computer based science would be uniquely easy - publish the code, publish the data, and let other people run it. Unless it's a matter of scale, or access to specific hardware.
A lot of things are easy if you ignore the incentive structure. E.g. a lot of papers will no longer be published if the data must be published. You’d lose all published research from ML labs. Many people like you would say “that’s perfectly okay; we don’t need them” but others prefer to be able to see papers like Language Models Are Few-Shot Learners https://arxiv.org/abs/2005.14165
So the answer is that we still want to see a lot of the papers we currently see because knowing the technique helps a lot. So it’s fine to lose replicability here for us. I’d rather have that paper than replicability through dataset openness.
But the lab must publish at least the general category of data, and if that doesn't replicate, then the model only works on a more specific category than they claim (e.g. only their dataset).
Even with the exact same dataset and architecture, ML results aren't perfectly replicable due to random weight initialisation, training data order, and non-deterministic GPU operations. I've trained identical networks on identical data and gotten different final weights and performance metrics.
This doesn't mean the model only works on that specific dataset - it means ML training is inherently stochastic. The question isn't 'can you get identical results' but 'can you get comparable performance on similar data distributions.
Then researchers should re-train their models a couple times, and if they can't get consistent results, figure out why. This doesn't even mean they must throw out the work: a paper "here's why our replications failed" followed by "here's how to eliminate the failure" or "here's why our study is wrong" is useful for future experiments and deserves publication.
As per my previous comment - we are discussing stochastic systems.
By definition, they involve variance that cannot be explained or eliminated through simple repetition. Demanding a 'deterministic' explanation for stochastic noise is a category error; it's like asking a meteorologist to explain why a specific raindrop fell an inch to the left during a storm replication.
and select "Mathematics and Computer Science", you'll find the top-ranked university is the University of Electronic Science and Technology of China.
My Chinese colleagues have heard of it, but never considered it a top-ranked school, and a quick inspection of their CS faculty pages shows a distinct lack of PhDs from top-ranked Chinese or US schools. It's possible their math faculty is amazing, but I think it's more likely that something underhanded is going on...
It's strange to me that in places full of smart people, it seems to be well understood that this happens and there are lots of anecdotes relating to it; yet the same people will be confused that their political adversaries don't trust "the science" on one issue or another.
That’s the beautiful thing about science: You do not have to (and should not) trust any individual. And even if you don’t trust “the consensus” of “the scientific community”, you can empirically verify yourself.
Once you move from abstract to practical - like say having legislators or regulators make rules based on The Science, or relying personally on more facts than you have time to independently verify - yes you do need to have trustworthy people.
No, but the literature is open for you to read. Thus you can judge the stated reasoning for yourself. You can also assess how many independent groups are making the same (or closely related) claim.
If only one person claims X then it might be fraud. If large numbers of seemingly unrelated people all claim X then you're forced to decide between X and a global conspiracy to misrepresent X.
To your example. Importantly, even if you deemed one of the global mean temperature datasets to be untrustworthy there are other related (but different) datasets. There are also other pieces of evidence related to the downstream claims that don't look directly at temperature.
More broadly, an incredible amount of our society's systems are built around actors being uncoordinated. Redesigning institutions to resist networks of coordinated action between seemingly unlinked individuals will, in my opinion, be one of the great social challenges of this era.
It is useful to distinguish between "effective" scientific fraud, where some set of fraudulent papers are published that drive a discipline in an unproductive direction, and "administrative" scientific fraud, where individuals use pseudo-scientific measures (H-index, rankings, etc) to make allocation decisions (grants, tenure, etc). This article suggests that administrative scientific fraud has become more accessible, but it is very unclear whether this is having a major impact on science as it is practiced.
Non-scientists often seem to think that if a paper is published, it is likely to be true. Most practicing scientists are much more skeptical. When I read a that paper sounds interesting in a high impact journal, I am constantly trying to figure out whether I should believe it. If it goes against a vast amount of science (e.g. bacteria that use arsenic rather than phosphorus in their DNA), I don't believe it (and can think of lots of ways to show that it is wrong). In lower impact journals, papers make claims that are not very surprising, so if they are fraudulent in some way, I don't care.
Science has to be reproducible, but more importantly, it must be possible to build on a set of results to extend them. Some results are hard to reproduce because the methods are technically challenging. But if results cannot be extended, they have little effect. Science really is self-correcting, and correction happens faster for results that matter. Not all fraud has the same impact. Most fraud is unfortunate, and should be reduced, but has a short lived impact.
The distinction between effective and administrative fraud is useful and I think underappreciated. A lot of the conversation in these threads conflates the two, which makes it hard to reason about what actually needs fixing.
I want to push back a little on "science is self-correcting" though. It's true in the limit, but correction has a latency, and that latency has real costs. In fields like nutrition, psychology, or pharmacology, a fraudulent or deeply flawed result can shape clinical guidelines, public policy, and drug development pipelines for a decade or more before the correction lands. The people harmed during that window don't get made whole by the eventual retraction.
The comparison I keep coming back to is fault tolerance in distributed systems. You can build a system that's "eventually consistent" and still have it be practically broken if convergence takes too long or if bad state propagates faster than corrections do. The fraud networks described in TFA are basically an adversarial workload against a system (peer review) that was designed for a much lower rate of bad input. Saying the system self-corrects is accurate, but it's not the same as saying the system is healthy or that the current correction rate is adequate.
I think the practical question isn't whether science corrects itself in theory but whether the feedback loops are fast enough relative to the rate of fraud production, and right now the answer seems pretty clearly no.
My wife completed her PhD two years ago and she put a LOT of work into it. Many sleepless nights, and it almost destroyed our marriage. It took her about 6 years of non-stop madness and she didn’t even work during that time. She said that many of her colleagues engaged in fraudulent data generation and sometimes just complete forgery of anything and everything. It was
obvious some people were barely capable of putting together coherent sentences in posts, but somehow they generated a perfect dissertation in the end. It was common knowledge that candidates often hired writers and even experts like statisticians to do most of the heavy lifting. I don’t know if this is the norm now, but I simultaneously have more respect and less respect for those doctoral degrees, knowing that some poured their heart and soul into it, while others essentially cheated their way through. OTOH, I also understand that there may be a lot of grey area.
I found the article and your third-hand anecdotes troubling. The good news is that it does not match any of the years of experience in my field. Fraud is just not that rampant. At PhD-granting institutions, the level of fraud you describe here is very seriously punished. It's career-ending. The violations that you are serious enough that any institution would expel said students (or harshly punish faculty--probably firing them). She did no one any favors by not reporting them.
Unfortunately I don't think a dialogue around vague anecdotes is going to be particularly enlightening. What matters is culture, but also process--mechanisms and checks--plus consequences. Consequences don't happen if everyone is hush-hush about it and no one wants to be a "rat".
That is where being good at politics come into play. And if you are good at it, instead of being career-ending, fraud will put you in the highest of the positions!
No one wants a "plant" who cannot navigate scrutiny!
> The good news is that it does not match any of the years of experience in my field.
I worked for exactly one academic, and he indulged in impossible-to-detect research fraud. So in my own limited experience research fraud was 100%.
It was a biology lab, and this was an extremely hard working man. 18 hours per day in the lab was the norm. But the data wasn't coming out the way he wanted, and his career was at stake, so he put his thumb on the scale in various ways to get the data he needed. E.g. he didn't like one neural recording, so he repeated it until he got what he wanted and ignored the others. You would have to be right in the middle of the experiment to notice anything, and he just waved me off when I did.
This same professor was the loudest voice in the department when it came to critiquing experimental designs and championing rigor. I knew what he did was wrong, because he taught me that. And he really appeared to mean it, but when push came to shove, he fiddled, and was probably even lying to himself.
So I came away feeling that academic fraud is probably rampant, because the incentives all align that way. Anyone with the extraordinary integrity to resist was generally self-curated out of the job.
I had a somewhat similar experience- was a postdoc for a pre-tenure professor at berkeley. after writing up a paper based on her methods, with poor results, I handed the draft to her. She rewrote it- basically adding carefully worded/presented results that made it look as good as possible. And then submitted it (to a niche conference where the editor was a buddy of hers). When I read her submission I asked her to remove my name from it and she immediately withdrew the submission. I left her lab shortly after because I am not going to tarnish my publication record with iffy papers like that.
Over time I learned that most papers in my field (computational biology) are embellished to some extent or another (or cherry-picked/curated/structured for success) and often irreproducible- some key step is left out, or no code is provided that replicates the results, etc. I can see this from two perspectives:
1) science should be trivially reproducible; it should not require the smartest/most capable people in the field to read the paper and reproduce the results. This places a burden on the people who are at the state of the art of the field to make it easy for other folks, which slows them down (but presumably makes overall progress go faster).
2) science should be done by geniuses; the leaders in the field don't need to replicate their competitors paper. it's sufficient to read the paper, apply priors, and move on (possibly learning whatever novel method/technique the paper shows so they can apply it in their own hands). It allows the field innovators to move quickly and discover new things, but is prone to all sorts of reliability/reproducibility problems, and ideally science should be egalitarian, not credentials-based.
I have repeated it many times on this site but here’s the reality of human experience: if the rate of fraudulent labs is even as high as 10% you should expect that any viewpoint that it’s widespread would be drowned out by views that it’s not real.
Also, the phenomenon you observed where people are champions till the rubber meets the road is more common than one thinks.
yeah - skeptical here. Among certain departments, at large schools, under certain leaders.. The combination of "my marriage almost crumbled" for motivated reasoning, and "I have never seen any of this before" total inexperience with actual process.. the post shows itself to be biased and unreliable.
However, among certain departments, at large schools, under certain leaders.. yes, and growing
firstly, there are basically no legal repercussions for scientific misconduct (e.g. falsifying data, fake images, etc.). most individuals who are caught doing this get either 1) a slap on the wrist if they are too big to fail or in the employ of those who are too big to fail or 2) disbarred, banned, and lose their jobs. i don't see why you can go to jail for lying to investors about the number of users in your app but don't go to jail for lying to the public, government, and members of the scientific community about your results.
secondly, due to the over production of PhD's and limited number of professorship slots competition has become so incredibly intense that in order to even be considered for these jobs you must have Nature, Cell, and Science papers (or the field equivalent). for those desperate for the job their academic career is over either way if they caught falsifying data or if they don't get the professorship. so if your project is not going the way you want it to then...
sad state of things all around. i've personally witnessed enough misconduct that i have made the decision to leave the field entirely and go do something else.
One approach is more integration of researchers with businesses. Fraud (or simple incompetence) by researchers negatively affects businesses, as they expend effort on things that aren't real. I understand this is a constant problem in the pharmaceutical industry.
It's quite possible to be very successful marketing and selling things that aren't real. The market consists of humans, not perfectly rational machines.
Almost as if capitalism makes everything into a market, and the profits make it self sustaining.
How many will see the connections between this and our capitalist mode of production? Probably few since modern lit/news is allergic to systemic analysis.
The blatant flaws of capitalism can't be ignored for much longer.
What I get from this is that the professional academic community -- as a whole -- has hit critical mass, which has produced a cottage industry of paper mills and fraudulent services to support said surplus.
Socialism wouldn't be the answer to this because socialism is famous for struggling with surpluses and shortages. All socialism would do is clamp down (hard) on academic's, which case you wind up with the famous shortage where not enough PHD's are available to produce research for an industry.
And that's not a problem specific to just socialism, that's the fallacy of central-planning. The US government clamped down on welfare fraud and the result were freak government social workers sniffing people's bed sheets and rooting through drawers and forcing everyone to document partners.
This is the situation where there needs to be a market correction because the alternative could be far worse.
The future of science, the Internet, and all things: The Library of Babel by Jorge Luis Borges.
Some things should not have been democratized. Silicon Valley assumes that removing restrictions on information brings freedom, but reality shows that was naïve.
The Library of Babel comparison is too fatalistic imo, even granting that it's maybe just an extreme example. The real world doesn't quite resemble a closed system with no metadata. We can still establish chains of trust.
Whether or not people will build resilient chains is another story, contingent on whether the strength of that chain actually matters to people. It probably doesn't for a lot of people. Boo. But inasmuch as I care, I feel I ought to be free to try and derive a strong signal through the noise.
In what way was it was democratised? We're not talking about Substacks and YouTube channels here, we're not even talking about arXiv preprints and the like, we're talking about peer-reviewed journal publications, and that system remains gated in much the same way that it was in the 1980s when it comes to trying to publish in it. If anything this system is the poster child for top-down gatekeeping by the recognised authorities, and it's precisely the value of that official recognition that makes people so desperate to break into it. The major changes seem to have been the easy availability of author publication lists and the advent of publication metrics, not things which have been or were ever meant to be particularly democratising for would-be authors; and an increase in the number of people playing the game, driven to a large extent by increasing participation from developing countries, and hopefully not many people would have the gall to argue for a ban on developing-country participation.
There is a saying in my native language that goes something like "If you mix poison and milk, the milk will turn poisonous, instead of poison becoming milk (aka beneficial)".
I guess, to convert it into this context, we can say that if you mix the high minded and infantile (which I think is what Internet and social media did), the high minded becomes infantile, instead of the other way around.
there is no 'sin of maximal inclusivity here', the gate is broken, but primarily because it was largely an honor system before, and no one has the motivation or resources to really dig into a lot of these papers.
in no sense was it corrupted by the desire to include a larger population in journal publications.
I ran into an interesting incident of this recently. I got a Google Scholar alert about a paper with some experiments related to a paper I had published a while ago, by one "N. Tvlg". I read the paper with interest but I started noticing that although the arguments sounded good, they didn't really make sense, and also the descriptions of the results didn't really match the figures. Eventually I came across a cluster of citations to completely unrelated papers---my field is computational linguistics and these were citations to, like, studies of battery technologies for electric cars. I looked up "N Tvlg" on Google Scholar and they had "published" several articles very recently in totally divergent fields, and upon inspection, all of them had citations back to this materials science research buried deeply somewhere. Clearly these were LLM generated papers trying to build up citation count and h-rank for someone's career.
It kinda skips over how large mainstream journals, with their restrictive and often arbitrary standards, have contributed to this. Most will refuse to publish replications, negative studies, or anything they deem unimportant, even if the study was conducted correctly.
So much of this started with the rise of the peer-review journal cartel, beginning with Pergamon Press in 1951 (coincidentally founded by Ghislaine Maxwell's father). "Peer review" didn't exist before then, science papers and discussion was published openly, and scientists focused on quality not quantity.
I'm not sure that the system was ever that near to perfection: for example, John Maddox of Nature didn't like the advent of pre-publication peer review, but that presumably had something to do with it limiting his discretion to approve and desk-reject whatever he wanted. But in any case it (like other aspects of the cozy interwar and then wartime scientific world) could surely never have survived the huge scaling-up that had already begun in the post-war era and created the pressure to switch to pre-publication peer reivew in the first place.
>scientists focused on quality not quantity.
I know a PhD professor doing post doc or something, and he accepted a scientific study just because it was published in Nature.
He didn't look at methodology or data.
From that point forward, I have never really respected Academia. They seem like bottom floor scientists who never truly understood the scientific method.
It helped that a year later Ivys had their cheating scandals, fake data, and academia wide replication crisis.
When I read something in a textbook I blindly believe it, depending on the broader context and the textbook in question. Is that a bad thing?
People are constantly filtering everything based on heuristics. The important thing is to know how deep to look in any given situation. Hopefully the person you're referring to is proficient at that.
Keep in mind that research scientists need to keep abreast of far more developments than any human could possibly study in detail. Also that 50% of people are below average at their job.
There is a vast difference between a student reading from a textbook and a researcher / scientist reading studies and/or papers.
As a student you are to be directed* in your reading by an expert in the field of study that you are learning from. In many higher level courses a professor will assign multiple textbooks and assign reading from only particular chapters of those textbooks specifically because they have vetted those chapters for accuracy and alignment with their curriculum.
As a researcher and scientist a very large portion of your job is verifying and then integrating the research of others into your domain knowledge. The whole purpose of replicating studies is to look critically at the methodology of another scientist and try as hard as you can to prove them wrong. If you fail to prove them wrong and can produce the same results as them, they have done Good Science.
A textbook is the product of scientists and researchers Doing Science and publishing their results, other scientists and researchers verifying via replication, and then one of those scientists or researchers who is an expert in the field doing their best to compile their knowledge on the domain into a factually accurate and (relatively) easy to understand summary of the collective research performed in a specific domain.
The fact is that people make mistakes, and the job of a professor (who is an expert in a given field) is to identify what errors have made it through the various checks mentioned above and into circulation, often times making subjective judgement calls about what is 'factual enough' for the level of the class they are teaching, and leverage that to build a curriculum that is sound and helps elevate other individuals to the level of knowledge required to contribute to the ongoing scientific journey.
In short, it's not a bad thing if you're learning a subject by yourself for your own purposes and are not contributing to scientific advancement or working as an educator in higher-education.
* You can self-study, but to become an expert while doing so requires extremely keen discernment to be able to root out the common misconceptions that proliferate in any given field. In a blue-collar field this would be akin to picking up 'bad technique' by watching YouTube videos published by another self-taught tradesman; it's not always obvious when it happens.
> There is a vast difference between a student reading from a textbook and a researcher / scientist reading studies and/or papers.
Not really. Both are learning new things. Neither has the time or access to resources to replicate even a small fraction of things learned. Neither will ever make direct use of the vast majority of things learned.
Thus both depend on a cooperative model where trust is given to third parties to whom knowledge aggregation is outsourced. In that sense a textbook and prestigious peer reviewed journals serve the same purpose.
> If you fail to prove them wrong and can produce the same results as them, they have done Good Science.
Not really in my humble opinion. Sure, the Popperian vibe is kind of fundamental, but the whole truncation into binary-valued true/false categories seldom makes sense with many (or even most?) problems for which probabilities, effect sizes, and related things matter more.
And if you fail to replicate a study, they may have still done Good Science. With replications, it should not be about Bad Science and Good Science but about the cumulation of evidence (or a lack thereof). That's what meta-analyses are about.
When we talk about Bad Science, it is about the industrial-scale fraud the article is talking about. No one should waste time replicating, citing, or reading that.
This is a good point. It is not humanly possible to verify every claim you read from every source.
Ideally, you should independently verify claims that appear to be particularly consequential or particularly questionable on the surface. But at some point you have to rely on heuristics like chain of trust (it was peer reviewed, it was published in a reputable textbook), or you will never make forward progress on anything.
> When I read something in a textbook I blindly believe it, depending on the broader context and the textbook in question. Is that a bad thing?
It is if what you read is factually incorrect, yes.
For example, I have read in a textbook that the tongue has very specific regions for taste. This is patently false.
> Keep in mind that research scientists need to keep abreast of far more developments than any human could possibly study in detail. Also that 50% of people are below average at their job.
So, we should probably just discount half of what we read from research scientists as "bad at their job" and not pay much attention to it? Which half? Why are you defending corruption?
You don't seem to be engaging in good faith.
The problem is that you can't just verify everything yourself. You likely have your own deadlines, and/or you want to do something more interesting than replicating statistical tests from a random paper.
> The problem is that you can't just verify everything yourself.
So the problem is reduced to "I believe what I want! This person said it and so I think it's true!"
Sounds like politics in a nutshell.
Academia has problems, like everywhere else. But that seems like a big extrapolation from just one professor.
Fake data—you can only get that type of scandal when people are checking the data. I’d be more skeptical of communities that never have that kind of scandal.
> coincidentally founded by Ghislaine Maxwell's father
A crazy world we live in where Robert Maxwell's daughter is more notorious than he is.
Fun fact, he almost got the worldwide console rights to Tetris back in the 80s, and tried going to Soviet officials to get those rights. To the point he's the antagonist of a recent "Tetris" movie that came out.
This is a fun fact, thank you.
Never knew of the guy but what a terrible sounding person from his Wikipedia at least.
Shit apple doesn’t fall far from the shit tree I guess.
Peer review existed before 1951 in the US at least. See for example Einstein’s reaction to negative reviews when he tried to publish in Physical Review in 1935 https://paeditorial.co.uk/post/albert-einstein-what-did-he-t...
>Pergamon Press in 1951 (coincidentally founded by Ghislaine Maxwell's father)
perhaps a bit off-topic, but what is coincidental about this and/or what is the relevance of Ghislaine Maxwell here?
It's useless, but I'm ashamed to admit I found this tiny piece of trivia interesting.
Like the paywall blocking many scientific arti6, perhaps it would be best if we released also the Epstein Files?
I believe by saying it is coincidental they are saying there is probably no relevance, just an interesting piece of trivia, why put out this interesting piece of trivia? Because maybe someone will be able to make an argument of relevance.
It's more than coincidental, but tangential to the point. It shows crime runs in families.
I imagine it's the interesting peculiarity that the same people seem to crop up over and over and over again. Six degrees of Kevin Bacon or something, except it's like one or two degrees. As George Carlin said, "it's a big club, and you ain't in it"
For example Donald Barr (father of twice-former US Attorney General Bill Barr) hiring college-dropout Jeffrey Epstein whilst headmaster at the elite Dalton School
Additional fun facts about Donald Barr: he served in US intelligence during WWII, and wrote a sci-fi book featuring child sex slaves
Also the Epstein-Barr virus causes Mono, the clone of .NET, which was created by Bill Gates, known associate of Epstein, whose father was president of the Washington State Bar Association. And you know who else works in Washington? Join the dots, people.
This might be my fav HN comment ever. Well done!
We call people who make connections like these "conspiracy theorists," until they're right, at which point we call them "right". And somewhere in between, if they manage to get a job, we call them "Simpsons writers."
Ghislaine's father (Robert Maxwell) was also a terrible person but for different reasons.
Robert Maxwell was a crook, he used pension funds (supposed to be ring-fenced for the benefit of the pensioners) to prop up his companies, so, after his slightly mysterious death it was discovered that basically there's no money to pay people who've been assured of a pension when they retire.
He was also very litigious. If you said he was a crook when he was alive you'd better hope you can prove it and that you have funding to stay in the fight until you do. So this means the sort of people who call out crooks were especially unhappy about Robert Maxwell because he was a crook and he might sue you if you pointed it out.
If you want to know more about the history of Pergamon Press there's a great Behind the Bastards episode on Robert Maxwell (Ghislaine Maxwell's father) - who himself was a scumbag in a variety of ways that were entirely distinct from Ghislaine Maxwell's brand of scumbaggery - that covers this. Might even be a multipart episode - it's a while since I've listened to it, but I have a feeling it's at least a two parter.
Some "fun" reading on the subject of Mr. Maxwell:
https://sarahkendzior.substack.com/p/red-lines
tl;dr He is the bridge that uncomfortably links Biden's former Secretary of State, Antony Blinken, to Jeffrey Epstein and Mossad. Hence, *gestures at the last couple of weeks and years*. Dude was just, like, Fraud Central, apparently.
Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies. It'll be great if I can look them up when needed but im not looking forward to email ToC alerts filled with them.
Also who's funding you for replication work? Do you know the pressure you have in tenure track to have a consistent thesis on what you work on?
Literally every single know that designs academia is tuned to not incentivize what you complain about. Its not just journals being picky.
Also the people committing fraud aren't ones who will say "gosh I will replicate things now!" Replicating work is far more difficult than a lot of original work.
> Do you want issues of Nature and cell to be replication studies?
Of course I do! Not all of course, and taking (subjectively measured) impact into account. "We tried to replicate the study published in the same journal 3 years ago using a larger sample size and failed to achieve similar results..." OR "after successfully replicating the study we can confirm the therapeutic mechanism proposed by X actually works" - these are extremely important results that are takin into account in meta studies and e.g. form the base of policies worldwide.
Honestly even if they didn't publish the whole paper, if there was just a page that was a table of all the replication studies that were done recently, that would be pretty cool.
I know you got a ton of responses already but not caring about replicability just invalidates science as a method. If we care only about first to publish we end up in the current situation where we don't even know that we know is actually even remotely correct.
All because journals prefer novelty over confirmation. It's like a castle of cards, looks cool but not stable or long-term at all.
> Do you want issues of Nature and cell to be replication studies?
More than anything. That might legitimately be enough to save science on its own.
Maybe nature and cell and a few other journals should be exceptions: they should be the place that the most advanced scientists publish interesting ideas early for the consumption by their competitors. At that level of science, all the competitors can reproduce each other's experiments if necessary; the real value is expanding the knowledge of what seems possible quickly.
(I am not seriously proposing this, but it's interesting to think about distinguishing between the very small amount of truly innovative discovery versus the very long tail of more routine methods development and filling out gaps in knowledge)
> that level of science, all the competitors can reproduce each other's experiments if necessary
But they don't, and that's the problem!
Advanced groups usually replicate their competitor's results in their own hands shortly after publication (or they just trust their competitor's competence). But they don't spend any time publishing it unless they fail to replicate and can explain why they can't replicate. From their perspective, it's a waste of time. I think this has been shown to be a naive approach (given the high rate of image fraud in molecular biology) but people who are in the top of the field have strong incentives to focus on moving the state of the art forward without expending energy on improving the field as a whole.
"strong incentives to focus on moving the state of the art forward without expending energy on improving the field as a whole"
That sort of Orwellian doublethink is exactly the problem. They need to move it forward without improving it, contribute without adding anything, challenge accepted dogma without rocking the boat, and...blech!
All that makes it more important for top journals to reward replication, not less!
Top journals are not inherently prestigious. They are prestigious because they try to publish only the most interesting and most significant results. If they started publishing successful replication studies, they would lose prestige, and more interesting journals would eventually rise to the top. (Replication studies that fail to replicate a major result in a spectacular way are another matter.)
> Do you want issues of Nature and cell to be replication studies? As a reader even from within the field, im not interested in browsing through negative studies.
Actually, yes, I do. The marginal cost for publishing a study online at this point is essentially nil.
I think archives with pretty low standards for notability are a good idea. At some point though you have to pick what actually counts as interesting enough to go in the curated list that is actually suggested reading, where the prestige is attached. If there's no curation by Nature then it falls to bloggers or another journal to sift through the fire-hose and make best-of lists. Most of the value is in the curation, not the publishing. Without exclusivity there's very little signal.
If you're a reader within the field, then you are the one person in the world who should be most interested in negative replication studies.
Even if that negative study could save you one, two, three+ years of work for the same outcome (which you then also can't really do anything with)? Shouldn't there BE funding for replication studies? Shouldn't that count towards tenure? Part of the problem is that publications play such a heavy role in getting tenure in the first place.
I'm sure you can more narrowly tune your email alerts FFS.
"Original research" isn't worth much unless replicated, which is the entire problem being discussed in this thread. Replicating studies are great though because they tell you if the original research actually stands and is valid.
> Replicating work is far more difficult than a lot of original work.
Only if the original work was BS. And what, just because it's harder, we shouldn't do it?
Why blame just the journals when every other system also disintivizes the same.
I must be missing something, surely the argument isn't "other systems also disincentivize solving the problem, therefore we shouldn't work to fix this one"
> Do you want issues of Nature and cell to be replication studies?
Hell yeah. We’re all trying to get that Nature paper. Imagine if you could accomplish that by setting the record straight.
If you're thoroughly debunking a previous Nature paper they just might publish that. But the expectation is that you'll succeed. Publishing that sort of mundane article would reduce the prestige of getting something into the journal. Publishing in a high impact journal is only seen as an achievement in the first place because of what it implies about the content of your paper.
Realistically, everyone will say “yes” to the “do you want” question because if you’re not a reader or a subscriber you benefit from the readers reading replication studies.
I believe people will enthusiastically say yes but that they do not routinely read that journal.
Tagging seems like an option here.
Suggesting that people would stop reading Nature if they also included replication studies send like an incredible leap.
It would directly undermine the reason that people read Nature in the first place.
Not really.
"It ain’t what you don’t know that gets you into trouble. It’s what you know for sure that just ain’t so."
Knowing that something I thought was true was actually false would have saved me years in several situations.
That is a novel interpretation of my comment certainly.
> Replicating work is far more difficult than a lot of original work.
I don’t regularly read scientific studies but I’ve read a few of them.
How is it possible that a serious study is harder to replicate than it is to do originally. Are papers no longer including their process? Are we at the point where they are just saying “trust me bro” for how they achieved their results?
> Do you want issues of Nature and cell to be replication studies?
Not issues of Nature but I’ve long thought that universities or the government should fund a department of “I don’t believe you” entirely focused on reproducing scientific results and seeing if they are real
> How is it possible that a serious study is harder to replicate than it is to do originally.
They aren't. GP was on point until that last sentence. Just pretend that wasn't there. It's pretty much always much easier to do something when all the key details have been figured out for you in advance.
There is some difficulty if something doesn't work to distinguish user error from ambiguity of original publication from outright fraud. That can be daunting. But the vast majority of the time it isn't fraud and simply emailing the original author will get you on track. Most authors are overjoyed to learn about someone using their work. If you want to be cynical about it, how else would you get your citation count up?
>Also who's funding you for replication work? Do you know the pressure you have in tenure track to have a consistent thesis on what you work on?
This is partly why much of today's science is bs, pure and simple.
This isn’t about honest researchers resorting to fraud to publish their null results because they were blocked by big bad Nature. It’s about journals and authors churning out pure junk papers whose only goal is to game metrics like citation count.
Maybe we need a journal completely dedicated to replication studies? It would attract a lot of attention I think.
Economics has the Journal of Comments and Replications in Economics: https://jcr-econ.org/
We already have archival journals. What's missing is funding and any prospect of career advancement.
And funding dedicated to replication studies.
paid by the original authors if their study fails to replicate
Is there a viable career path for researchers who choose to focus on replication instead of novel discoveries? I assume replications are perceived as less prestigious, but it's also important work.
sadly no, this is not a thing and it's critically needed.
top on my list of things to do if i were a billionaire: launch an institute for the sole purpose of reproducing other's findings.
Right, it seems that many of the weaknesses in the system exist because they serve the interests of journal publishers or of normal, legitimate-ish researchers, but in the process open the door to full-time system-hackers and pure fraudsters.
Mainstream journals are complicit, but are not the biggest problem.
The biggest problem by far is modern society: Tenure, getting paid a livable wage as a researcher, not getting stack-ranked and eliminated from your organization all overindex on positive research results that are marketable. This "loss function" encourages scientific fraud of sorts.
This is Goodhart's law at scale. Number of released papers/number of citations is a target. Correctness of those papers/citations is much more difficult so is not being used as a measure.
With that said, due to the apparent sizes of the fraud networks I'm not sure this will be easy to address. Having some kind of kill flag for individuals found to have committed fraud will be needed, but with nation state backing and the size of the groups this may quickly turn into a tit for tat where fraud accusations may not end up being an accurate signal.
May you live in interesting times.
> This is Goodhart's law at scale.
Also, Brandolini's law. And Adam Smith's law of supply and demand. When the ability to produce overwhelms the ability to review or refute, it cheapens the product.
> Number of released papers/number of citations is a target
There was this guy, well connected in the science world, that managed to publish a poor study quite high (PNAS level). It was not fraud, just bad science. There were dozens of papers and letters refuting his claims, highlighting mistakes, and so... Guess what? Attending to metrics (citations, don't matter if they are citing you to say you were wrong and should retract the paper!), the original paper was even more stellar on the eyes of grants and the journal itself.
It was rage bait before Facebook even existed.
There’s an accurate way to confirm fraud: look for inconsistencies and replicate experiments.
If the fraudsters “fail to replicate” legitimate experiments, ask them for details/proof, and replicate the experiment yourself while providing more details/proof. Either they’re running a different experiment, their details have inconsistencies, or they have unreasonable omissions.
Of course this is slightly messy too. Fraudsters are probably always incorrect, of course they could have stolen the data. But being incorrect doesn't mean your intentionally committing fraud.
That would be great if journals bothered publishing replication studies. But since they don't, researchers can't get adequate funding to perform them, and since they can't perform them, they don't exist.
We can't look for failed replication experiments if none exist.
that approach is accurate, but not scalable.
the effort to publish a fraudulent study is less (sometimes much less) than the effort to replicate a study.
Yeah, but this happens all the time.
>>95% of the time, the fraudsters get off scot-free. Look at Dan Ariely: Caught red-handed faking data in Excel using the stupidest approach imaginable, and outed as a sex pest in the Epstein files. Duke is still giving him their full backing.
It’s easy to find fraud, but what’s the point if our institutions have rotten all the way through and don’t care, even when there’s a smoking gun?
Is it that easy?
Machine Learning papers, for example, used to have a terrible reputation for being inconsistent and impossible to replicate.
That didn't make them (all) fraudulent, because that requires intent to deceive.
What do you think it is about machine learning that makes it hard to replicate? I'm an outsider to academic research, but it seems like computer based science would be uniquely easy - publish the code, publish the data, and let other people run it. Unless it's a matter of scale, or access to specific hardware.
A lot of things are easy if you ignore the incentive structure. E.g. a lot of papers will no longer be published if the data must be published. You’d lose all published research from ML labs. Many people like you would say “that’s perfectly okay; we don’t need them” but others prefer to be able to see papers like Language Models Are Few-Shot Learners https://arxiv.org/abs/2005.14165
So the answer is that we still want to see a lot of the papers we currently see because knowing the technique helps a lot. So it’s fine to lose replicability here for us. I’d rather have that paper than replicability through dataset openness.
But the lab must publish at least the general category of data, and if that doesn't replicate, then the model only works on a more specific category than they claim (e.g. only their dataset).
Even with the exact same dataset and architecture, ML results aren't perfectly replicable due to random weight initialisation, training data order, and non-deterministic GPU operations. I've trained identical networks on identical data and gotten different final weights and performance metrics.
This doesn't mean the model only works on that specific dataset - it means ML training is inherently stochastic. The question isn't 'can you get identical results' but 'can you get comparable performance on similar data distributions.
Then researchers should re-train their models a couple times, and if they can't get consistent results, figure out why. This doesn't even mean they must throw out the work: a paper "here's why our replications failed" followed by "here's how to eliminate the failure" or "here's why our study is wrong" is useful for future experiments and deserves publication.
As per my previous comment - we are discussing stochastic systems.
By definition, they involve variance that cannot be explained or eliminated through simple repetition. Demanding a 'deterministic' explanation for stochastic noise is a category error; it's like asking a meteorologist to explain why a specific raindrop fell an inch to the left during a storm replication.
Perhaps relevant to this - if you go to this global ranking of publications:
and select "Mathematics and Computer Science", you'll find the top-ranked university is the University of Electronic Science and Technology of China.My Chinese colleagues have heard of it, but never considered it a top-ranked school, and a quick inspection of their CS faculty pages shows a distinct lack of PhDs from top-ranked Chinese or US schools. It's possible their math faculty is amazing, but I think it's more likely that something underhanded is going on...
It's strange to me that in places full of smart people, it seems to be well understood that this happens and there are lots of anecdotes relating to it; yet the same people will be confused that their political adversaries don't trust "the science" on one issue or another.
Maybe it's the scientists they don't trust?
That’s the beautiful thing about science: You do not have to (and should not) trust any individual. And even if you don’t trust “the consensus” of “the scientific community”, you can empirically verify yourself.
Once you move from abstract to practical - like say having legislators or regulators make rules based on The Science, or relying personally on more facts than you have time to independently verify - yes you do need to have trustworthy people.
Are you going to build a competitor to CERN?
There are many things that cannot be feasibly verified empirically without access to rare resources.
Can ordinary civilians feasibly measure, for example, global trends in mean temperature without relying on the data of others?
No, but the literature is open for you to read. Thus you can judge the stated reasoning for yourself. You can also assess how many independent groups are making the same (or closely related) claim.
If only one person claims X then it might be fraud. If large numbers of seemingly unrelated people all claim X then you're forced to decide between X and a global conspiracy to misrepresent X.
To your example. Importantly, even if you deemed one of the global mean temperature datasets to be untrustworthy there are other related (but different) datasets. There are also other pieces of evidence related to the downstream claims that don't look directly at temperature.
More broadly, an incredible amount of our society's systems are built around actors being uncoordinated. Redesigning institutions to resist networks of coordinated action between seemingly unlinked individuals will, in my opinion, be one of the great social challenges of this era.
This is what happens when people argue past each other on "Trust the science".
Science is good, but it's mediated via corruptible humans.
It always comes back to Goodhart's Law and our apparent inability to create sustainable incentive structures.
It is useful to distinguish between "effective" scientific fraud, where some set of fraudulent papers are published that drive a discipline in an unproductive direction, and "administrative" scientific fraud, where individuals use pseudo-scientific measures (H-index, rankings, etc) to make allocation decisions (grants, tenure, etc). This article suggests that administrative scientific fraud has become more accessible, but it is very unclear whether this is having a major impact on science as it is practiced.
Non-scientists often seem to think that if a paper is published, it is likely to be true. Most practicing scientists are much more skeptical. When I read a that paper sounds interesting in a high impact journal, I am constantly trying to figure out whether I should believe it. If it goes against a vast amount of science (e.g. bacteria that use arsenic rather than phosphorus in their DNA), I don't believe it (and can think of lots of ways to show that it is wrong). In lower impact journals, papers make claims that are not very surprising, so if they are fraudulent in some way, I don't care.
Science has to be reproducible, but more importantly, it must be possible to build on a set of results to extend them. Some results are hard to reproduce because the methods are technically challenging. But if results cannot be extended, they have little effect. Science really is self-correcting, and correction happens faster for results that matter. Not all fraud has the same impact. Most fraud is unfortunate, and should be reduced, but has a short lived impact.
The distinction between effective and administrative fraud is useful and I think underappreciated. A lot of the conversation in these threads conflates the two, which makes it hard to reason about what actually needs fixing.
I want to push back a little on "science is self-correcting" though. It's true in the limit, but correction has a latency, and that latency has real costs. In fields like nutrition, psychology, or pharmacology, a fraudulent or deeply flawed result can shape clinical guidelines, public policy, and drug development pipelines for a decade or more before the correction lands. The people harmed during that window don't get made whole by the eventual retraction.
The comparison I keep coming back to is fault tolerance in distributed systems. You can build a system that's "eventually consistent" and still have it be practically broken if convergence takes too long or if bad state propagates faster than corrections do. The fraud networks described in TFA are basically an adversarial workload against a system (peer review) that was designed for a much lower rate of bad input. Saying the system self-corrects is accurate, but it's not the same as saying the system is healthy or that the current correction rate is adequate.
I think the practical question isn't whether science corrects itself in theory but whether the feedback loops are fast enough relative to the rate of fraud production, and right now the answer seems pretty clearly no.
>methods are technically challenging.
And finanacially too..
>Science really is self-correcting..
When economy allows it....
My wife completed her PhD two years ago and she put a LOT of work into it. Many sleepless nights, and it almost destroyed our marriage. It took her about 6 years of non-stop madness and she didn’t even work during that time. She said that many of her colleagues engaged in fraudulent data generation and sometimes just complete forgery of anything and everything. It was obvious some people were barely capable of putting together coherent sentences in posts, but somehow they generated a perfect dissertation in the end. It was common knowledge that candidates often hired writers and even experts like statisticians to do most of the heavy lifting. I don’t know if this is the norm now, but I simultaneously have more respect and less respect for those doctoral degrees, knowing that some poured their heart and soul into it, while others essentially cheated their way through. OTOH, I also understand that there may be a lot of grey area.
My eyes have been opened!
I found the article and your third-hand anecdotes troubling. The good news is that it does not match any of the years of experience in my field. Fraud is just not that rampant. At PhD-granting institutions, the level of fraud you describe here is very seriously punished. It's career-ending. The violations that you are serious enough that any institution would expel said students (or harshly punish faculty--probably firing them). She did no one any favors by not reporting them.
Unfortunately I don't think a dialogue around vague anecdotes is going to be particularly enlightening. What matters is culture, but also process--mechanisms and checks--plus consequences. Consequences don't happen if everyone is hush-hush about it and no one wants to be a "rat".
>It's career-ending..
That is where being good at politics come into play. And if you are good at it, instead of being career-ending, fraud will put you in the highest of the positions!
No one wants a "plant" who cannot navigate scrutiny!
> The good news is that it does not match any of the years of experience in my field.
I worked for exactly one academic, and he indulged in impossible-to-detect research fraud. So in my own limited experience research fraud was 100%.
It was a biology lab, and this was an extremely hard working man. 18 hours per day in the lab was the norm. But the data wasn't coming out the way he wanted, and his career was at stake, so he put his thumb on the scale in various ways to get the data he needed. E.g. he didn't like one neural recording, so he repeated it until he got what he wanted and ignored the others. You would have to be right in the middle of the experiment to notice anything, and he just waved me off when I did.
This same professor was the loudest voice in the department when it came to critiquing experimental designs and championing rigor. I knew what he did was wrong, because he taught me that. And he really appeared to mean it, but when push came to shove, he fiddled, and was probably even lying to himself.
So I came away feeling that academic fraud is probably rampant, because the incentives all align that way. Anyone with the extraordinary integrity to resist was generally self-curated out of the job.
I had a somewhat similar experience- was a postdoc for a pre-tenure professor at berkeley. after writing up a paper based on her methods, with poor results, I handed the draft to her. She rewrote it- basically adding carefully worded/presented results that made it look as good as possible. And then submitted it (to a niche conference where the editor was a buddy of hers). When I read her submission I asked her to remove my name from it and she immediately withdrew the submission. I left her lab shortly after because I am not going to tarnish my publication record with iffy papers like that.
Over time I learned that most papers in my field (computational biology) are embellished to some extent or another (or cherry-picked/curated/structured for success) and often irreproducible- some key step is left out, or no code is provided that replicates the results, etc. I can see this from two perspectives:
1) science should be trivially reproducible; it should not require the smartest/most capable people in the field to read the paper and reproduce the results. This places a burden on the people who are at the state of the art of the field to make it easy for other folks, which slows them down (but presumably makes overall progress go faster).
2) science should be done by geniuses; the leaders in the field don't need to replicate their competitors paper. it's sufficient to read the paper, apply priors, and move on (possibly learning whatever novel method/technique the paper shows so they can apply it in their own hands). It allows the field innovators to move quickly and discover new things, but is prone to all sorts of reliability/reproducibility problems, and ideally science should be egalitarian, not credentials-based.
My cousin (with whom I am very close) had a similar experience that I posted about years ago. https://news.ycombinator.com/item?id=32969092
I have repeated it many times on this site but here’s the reality of human experience: if the rate of fraudulent labs is even as high as 10% you should expect that any viewpoint that it’s widespread would be drowned out by views that it’s not real.
Also, the phenomenon you observed where people are champions till the rubber meets the road is more common than one thinks.
What field? I am aware this kind of stuff happens, but I don't really see it among any of my colleagues.
yeah - skeptical here. Among certain departments, at large schools, under certain leaders.. The combination of "my marriage almost crumbled" for motivated reasoning, and "I have never seen any of this before" total inexperience with actual process.. the post shows itself to be biased and unreliable.
However, among certain departments, at large schools, under certain leaders.. yes, and growing
$0.02
Fucking hilarious to me when people claim academics are motivated by the "money", eg, when claimed by climate deniers.
Undoubtably climate science is the exception and immune from fraudulent data generation and sometimes complete forgery
If you get paid by the government to do research you should make all your raw data, code, results etc, accessible to the public.
If it then turns out any of it is fabricated, you should be personally liable for paying it back
the problem is two-fold in my opinion.
firstly, there are basically no legal repercussions for scientific misconduct (e.g. falsifying data, fake images, etc.). most individuals who are caught doing this get either 1) a slap on the wrist if they are too big to fail or in the employ of those who are too big to fail or 2) disbarred, banned, and lose their jobs. i don't see why you can go to jail for lying to investors about the number of users in your app but don't go to jail for lying to the public, government, and members of the scientific community about your results.
secondly, due to the over production of PhD's and limited number of professorship slots competition has become so incredibly intense that in order to even be considered for these jobs you must have Nature, Cell, and Science papers (or the field equivalent). for those desperate for the job their academic career is over either way if they caught falsifying data or if they don't get the professorship. so if your project is not going the way you want it to then...
sad state of things all around. i've personally witnessed enough misconduct that i have made the decision to leave the field entirely and go do something else.
I unironically agree, p-hacking should be a criminal offense.
why would anyone actually interested in scientific research come to this, since it literally undermines the whole practice of science?
Publish or perish. Academia requiring PhDs to publish or be fired. It's made entire fields echo chambers and prone to political influence.
One approach is more integration of researchers with businesses. Fraud (or simple incompetence) by researchers negatively affects businesses, as they expend effort on things that aren't real. I understand this is a constant problem in the pharmaceutical industry.
It's quite possible to be very successful marketing and selling things that aren't real. The market consists of humans, not perfectly rational machines.
Industry >> Academia
Profits are the deciding factor, not honor.
Almost as if capitalism makes everything into a market, and the profits make it self sustaining.
How many will see the connections between this and our capitalist mode of production? Probably few since modern lit/news is allergic to systemic analysis.
The blatant flaws of capitalism can't be ignored for much longer.
What I get from this is that the professional academic community -- as a whole -- has hit critical mass, which has produced a cottage industry of paper mills and fraudulent services to support said surplus.
Socialism wouldn't be the answer to this because socialism is famous for struggling with surpluses and shortages. All socialism would do is clamp down (hard) on academic's, which case you wind up with the famous shortage where not enough PHD's are available to produce research for an industry.
And that's not a problem specific to just socialism, that's the fallacy of central-planning. The US government clamped down on welfare fraud and the result were freak government social workers sniffing people's bed sheets and rooting through drawers and forcing everyone to document partners.
This is the situation where there needs to be a market correction because the alternative could be far worse.
The future of science, the Internet, and all things: The Library of Babel by Jorge Luis Borges.
Some things should not have been democratized. Silicon Valley assumes that removing restrictions on information brings freedom, but reality shows that was naïve.
You shouldn't just assume that the inverse would be free from fraud. The incentives for fraud still apply even when the system is not democratized.
Except with AI, a fraudulent gatekept world would still be a smaller percentage of fraud than what is coming. Infinite scale fraud.
The soviets may have rigged a few studies; but the democratized world now faces almost all studies being rigged.
I think it'd be a different form of fraud that would be much harder to discredit. Think sugar industry blaming fat for health issues. More of that.
The Library of Babel comparison is too fatalistic imo, even granting that it's maybe just an extreme example. The real world doesn't quite resemble a closed system with no metadata. We can still establish chains of trust.
Whether or not people will build resilient chains is another story, contingent on whether the strength of that chain actually matters to people. It probably doesn't for a lot of people. Boo. But inasmuch as I care, I feel I ought to be free to try and derive a strong signal through the noise.
In what way was it was democratised? We're not talking about Substacks and YouTube channels here, we're not even talking about arXiv preprints and the like, we're talking about peer-reviewed journal publications, and that system remains gated in much the same way that it was in the 1980s when it comes to trying to publish in it. If anything this system is the poster child for top-down gatekeeping by the recognised authorities, and it's precisely the value of that official recognition that makes people so desperate to break into it. The major changes seem to have been the easy availability of author publication lists and the advent of publication metrics, not things which have been or were ever meant to be particularly democratising for would-be authors; and an increase in the number of people playing the game, driven to a large extent by increasing participation from developing countries, and hopefully not many people would have the gall to argue for a ban on developing-country participation.
Tearing down gatekeeping (i.e. "high standards") in pursuit of maximal inclusivity is just another way of saying "regression to the mean."
The gate has been removed from the signal chain, and now the noise floor is at infinity.
There is a saying in my native language that goes something like "If you mix poison and milk, the milk will turn poisonous, instead of poison becoming milk (aka beneficial)".
I guess, to convert it into this context, we can say that if you mix the high minded and infantile (which I think is what Internet and social media did), the high minded becomes infantile, instead of the other way around.
there is no 'sin of maximal inclusivity here', the gate is broken, but primarily because it was largely an honor system before, and no one has the motivation or resources to really dig into a lot of these papers.
in no sense was it corrupted by the desire to include a larger population in journal publications.