@NAChristakisさんのツイート20180828：再現性評価論文20180827の出版を受けて

sakanoue no fusigi @seeker98747

スレッド。 twitter.com/DanTGilbert/st…

Daniel Gilbert @DanTGilbert

@NAChristakis @talyarkoni @kinggary @hardsci Publication is not canonization. Journals are not gospels. They are the vehicles we use to tell each other what we saw (hence “Letters” & “proceedings”). The bar for communicating to each other should not be high. We can decide for ourselves what to make of each other's speech.

2018-09-20 07:38:01

Nicholas A. Christakis @NAChristakis

A 2017 article about the replicability of findings in cancer biology, noting similar levels of replication in that field (2 of 5 studies replicated easily, and others had certain issues): sciencemag.org/news/2017/01/r… via @zakkohane 10/

Nicholas A. Christakis @NAChristakis

And here is one bettor, @JoeHilgard, explaining how betting markets work and sharing his bets on which scientific papers would replicate. twitter.com/joehilgard/sta… Spoiler: he made money. 9/

Joe Hilgard, data guy @JoeHilgard

My results from betting on the latest replication project. I thought the high-priced stuff was priced about right, so I left them alone. But I saw profit in shorting the unlikely stuff, so I sandbagged the ones least likely to replicate. pic.twitter.com/kGzhTZeQyo

2018-08-28 01:22:21

Nicholas A. Christakis @NAChristakis

And here is coverage by @KellyServick in @sciencemagazine of this effort to evaluate replicability of claims in high-profile social science papers (also well noting that only one finding per paper was chosen -- a key concern of mine, too). sciencemag.org/news/2018/08/g… 8/

Nicholas A. Christakis @NAChristakis

It’s clear both extremes wouldn't serve us well: only doing replications would mean we don't allocate funds to make new discoveries; never doing replications means we may lose confidence in what we know. In a way, it’s a type 1/type 2 error issue. But I lean towards discovery 7/

Nicholas A. Christakis @NAChristakis

i also think that the scientific community will have to come to some rough consensus about the balance between resources allocated to new discovery-based science and to replication efforts. Both are clearly important. But what that balance should be, we do not yet know... 6/

Nicholas A. Christakis @NAChristakis

Detailed coverage of this replication effort by @edyong209 in @TheAtlantic with a particular focus on the cool prediction market aspect of this study, demonstrating that scientists can be strong assayers of reality: theatlantic.com/science/archiv… 5/

Nicholas A. Christakis @NAChristakis

Article in @nature @NatureNews by @philipcball provides more context for this replication effort in the social sciences (lately facing similar replicability challenges as some branches of the biological and physical sciences). nature.com/articles/d4158… 4/

Nicholas A. Christakis @NAChristakis

I did have some concerns with the way this replication project was accomplished (though I realize that methodological decisions were inevitable). My note to @NatureHumBehav is attached. 3/ pic.twitter.com/uttPpsN1Ri

拡大

Nicholas A. Christakis @NAChristakis

Our 2015 paper from @nature -- on inequality and visibility of wealth in experimental social networks -- was one of these 21 (and it was replicated). nature.com/articles/natur… Other prior replications of this work include: sciencedirect.com/science/articl… 2/

Nicholas A. Christakis @NAChristakis

New paper in @NatureHumBehav evaluates replicability of (portions of) 21 high-profile experiments in @nature & @sciencemagazine. It finds 62% replicate nature.com/articles/s4156… via @CFCamerer @BrianNosek et al. Interestingly, scientists could predict replicabiity in a market. 1/ pic.twitter.com/fxiosxvkwH

拡大