In this guest post, PLOS Neuro Community Contributing Editor, Neuroskeptic, discusses a novel proposal for simplifying and improving the dissemination of scientific research, using insights from social psychology.
In order to motivate scientists to adopt best practices such as data sharing and improve the reproducibility of research, will it be necessary to radically overhaul the systems by which the business of science is conducted?
I spoke to Brett Buttliere, of the Knowledge Media Research Center at the University of Tübingen in Germany (and formerly of the Department of Methodology and Statistics, Tilburg University in the Netherlands). He’s the author of “Using science and psychology to improve the dissemination and evaluation of scientific work,” in Frontiers in Computational Neuroscience.
In that paper, Buttliere proposes that science would run better if we had a single online network, a “Facebook for science”, instead of the many different publishers and academic tools that we currently use.
NS: In your Frontiers paper, you argue for the creation of a “centralized, easy to use platform” for scientists that would implement a new kind of peer review and discussion. Could you briefly explain how this would differ from the current system?
Absolutely. I believe science would benefit from having one online platform for people to do basically all aspects of science in, including review. Such a system would probably involve: a user friendly profile, a feed of (science) stories based upon previous viewing behavior, the ability for users to like, comment, and interact with content (e.g., papers, datasets, materials) within the system, and some sort of impact metrics that quantify the individual’s contribution into the system; basically, something like a Facebook or Twitter for science.
The system would take the most laborious and time consuming aspects of the research process and facilitate them within the system, making it better for everyone involved. Especially the data associated with this system would improve the situation by encouraging researchers to make contributions that are appreciated by the community (e.g., statistical reanalysis, replications, insightful comments, curating good content).
Researchers would be encouraged to reanalyze data or make insightful comments because they would gain a reputation by doing so. Researchers would want to upload their data because it would enable these trusted individuals to reanalyze it and leave their stamp of approval (thus drawing others in). Readers would benefit from seeing these efforts and the system could even examine network maps of papers, authors, and keywords to find the best paper(s) to inform their own research.
More generally, I understand that many are already working toward these goals, so in the paper I focused more on trying to take what we know about psychology into the design and implementation plans for such a system. People are not rewarded under the current system for doing the things outlined above, so they generally don’t; the current system actually rewards doing bad science, and we are all worse for it. The system needs to be focused on rewarding good behaviors rather than punishing bad ones, which experience shows tends to make the problem worse by making better cheaters (think of what happened with alcohol during the Prohibition Era.)
NS: You discuss ‘questionable research practices’ (QRPs) such as publication bias, and how these practices distort science. You say that we need to change the incentive system to reward scientists for adopting better approaches. What kind of behavior would you want to see rewarded?
Yes, my point here is that researchers will do whatever it is that most efficiently gets them to their goals (including use QRPs); the goal then becomes to make what is best for the group also best for the individual. For me, good practice is something like doing honest enough research that we are happy when others follow up on our work and look closely at our methods and raw data, rather than be upset like it is now.
The next question is how to make that happen, because sharing materials (e.g., data, materials, and code) and doing high quality research obviously involves extra effort and risk. So there needs to be some incentive to offset this extra work and risk. One way to solve this problem would be to follow the example of the most successful social media platforms to reward both uploading materials and having people follow up on them.
Again, the public comment section and impact factors are really the keys to the idea, as they are a means to reward both the researcher who is examining the data and the researcher whose data are being examined.
NS: The online system you describe would be centralized. Is there a danger that this would put too much power in the hands of the developers and administrators of the system?
No, I don’t believe so. It seems like the system in the paper, or one pretty similar to it, is coming whether we like it or not; the question is who will lead it, science or business? I think we, the scientists, should take control before someone else comes in and does it for us with who knows what intentions. No matter who builds such a platform, it should be open; the idea of an ‘open science’ platform that isn’t open about its own operations is silly to me. Researchers could thus use the data within the system (about who is commenting and linking where, for instance) to continually improve the feed, the impact metric system, and simultaneously help spot and minimize anyone who is trying to rig or ‘game’ the system for their own benefit.
The reason centralization is important is because it brings the different utilities that current open science initiatives have into a coherent and functional whole. Current open science initiatives have been successful, but not as successful as Twitter or Facebook. The reason for this, I think, is because they do not offer enough value for the cost necessary to achieve that value. Having so many different teams fragments the resources for both the team and the market, keeping any single team from reaching the critical mass it needs – it’s a classic common good dilemma. People do a cost benefit analysis before engaging with any new tool and oftentimes they find the value is not worth the cost. I mean, each new system costs the time to set up a profile, remember a new password, learn a new interface, and build up a new network.
The goal is to put the pieces together so that, at the same time, the value is increased and the effort needed to achieve that value is decreased.
NS: One thing you don’t discuss in your paper is money. Scientific publishing today involves a lot of money in the form of subscription and author fees. In your proposed system, who would pay for what? Would anyone make a profit?
Well, if this becomes like I envision (a Facebook or Twitter for science), it shouldn’t be too hard to pay the bills. I mean, Facebook is accessed 14 times a day by the average user, that is something like 20 billion minutes a day!
The main funding source that I can see so far (though I am no businessman) is subtle advertising in the newsfeed that alerts academics about new opportunities, e.g., conferences, professional organizations, information outlets, open positions, funding opportunities, software or hardware used for science.
This way, hopefully, they are providing some utility; certainly the advertisements should not be lowering the valuation of the platform. It is pretty much imperative that it be free for authors to post to and free for the public to access. So that is how it could make money; as for spending money, obviously there would need to be employees. Though hopefully, the majority of the work in terms of maintaining the feed and impact metrics would be done by the users, i.e. academics via the community attention / reward available.
Honestly though, there is probably a lot of money to be made here; it would provide a lot of value to the world. But even supposing that the system was allowed to make 200 million a year in pure profit, it would still cost massively less than the current publication system (Elsevier alone reportedly made $1.2 billion profit in 2011). Plus, there’s the time that would be saved in the research process. This profit and the huge efficiency savings could (probably should) be fed back into the system in the form of scholarships, travel grants, etc. or even just given back to society to show how effective psychology for the public good can be.
NS: What do you see are the main challenges to overcome to making this system a reality?
The main challenges to overcome are, in my opinion, almost all human. Technically, all the pieces are already there. We just need to convince the companies that it is worth it to put them together. Of course, then we need to get people to use it, but, if it is of high enough value, it is almost sure to succeed, and if it is not, it will fail.
Potentially the largest hurdle to both of those outcomes is simply getting the word out (and for your help here, I would like to sincerely thank you!). If the companies hear about it, maybe they organize of their own accord. If they don’t, maybe we can get enough consumer support to nudge the companies to act in the betterment of all, rather than themselves.
So, in conclusion, let’s create an online system that is fun, easy to use, and conducive to science. Such a system should probably incorporate all that we know about science and psychology into its design and implementation.
I’d also like to encourage readers to reach out to us (me) to discuss how it might look or play out. People can reach me on Twitter @BrettButtliere, on Facebook, or send me an email at BrettButtliere@gmail.com. Thank you again!
Neuroskeptic is a U.K. neuroscientist who takes a skeptical view of his profession. As a Contributing Editor to The PLOS BLOGS Network and PLOS Neuro Community, Neuroskeptic does in-depth interviews with fellow scientists on neuroscience and issues in scientific communication . If you missed any of these, check out NS interviews with Srivas Chennu, Michael Corballis and Cordelia Fine. The regular Neuroskeptic blog at Discover Magazine is here.