More publishing and transparency could go a long way toward cleaning up the climate “science” mess.
8 thoughts on “Academic Journals”
Comments are closed.
More publishing and transparency could go a long way toward cleaning up the climate “science” mess.
Comments are closed.
There’s simply no reason to not move to a blog/wiki model. It’s much cheaper, more transparent, and allows for more feedback from fellow scientists. Some type of “real name and reputation” system would discourage trolls.
This is probably the most important reform in science in a generation….
I agree.
The current journals reflective a 19th Century legacy solution for the distribution of information. The emergence of technology like the Internet really renders them outdated as a means of distributing scientific information, but they survive because much of reward system and as well as the accreditation of academic institutions revolve around publication in “respected” academic journals. The move by the British government is a good one, and one hopefully that will be followed by the NSF, NIH, NOAA, NASA and other government agencies that fund research. But even more important will be the DOE pushing for universities and accrediting bodies to move away from journal publication as part of the process of determining the qualifications of faculty into a more broader measure that takes the new communication technology into account.
The bits I want are:
1) Data & actual code online. If it relies upon commercial software, exact versions etc.
2) A check of third-party ‘data verification’. That is: I plug their number into their software and get their results.
3) An indication of replication. That is: a third party has set their own equipment up and demonstrated the results of the paper to their own satisfaction.
It’s all well and good for someone to read my work and say “Looks fine.” It’s something else to say “Your math is correct, and your description allowed me to replicate your work.”
Neither of those steps appear to happen much in the wild. And when they do (I’ve done some) there is no official avenue to report that.
I’d rather see a paper with two notations “Data and results congruent” and “Results independently verified” than a paper with 2,000 citations.
They just aren’t the same thing.
Al,
That would be good. The problem is that current journals will usually not publish studies that are “mere” replications of already published studies. And if research is not publishable faculty generally won’t do it because they will not be rewarded or even funded to do so.
Yes, I’ve been there.
Even confirmation (or refutations!) of “controversial” results. If you say “I did exactly what he did, and here are my results.” it isn’t pretty.
Something like arxive, with just a couple extra key lines. ‘List of people replicating this work’ etc.
If a system like that is in place, -then- it can be used for judging people for tenure etc.
But as it is, it’s a flamewar over citations. “Gosh, I loved this paper… except he seems to have completely overlooked Al2003, Al2005, and Al2007. Recommended for publication with those fixes.”
(Ok, not that blatant generally. But that’s the thrust of a ridiculous number of reviewer’s notes.)
Al,
Yes, and then is spins off into a closed little world of its own with half a dozen researchers arguing among themselves and splitting hairs via published papers. Great for building resumes for tenure but not really advancing the field very much. And of course no one else dares intrudes into the subject as the group will likely reject them unless they have a member of the group as the sponsor or co-author. And its at this process that “blind” reviews break down is it becomes easy for members of the “group” to recognize each others work by its style and focus. And of course their cites of others in the group.
It depends on where you are publishing. Some places require your results to be reproducible while others do not. There is a trend toward more open publishing models like arXiv or PLoS but in computer science for some weird reason we still do not use that model to any measurable degree. Which we should since traditional publishers do nearly none of the work and get all the profit. The reviewers and even the editor usually work for free and at best the publisher will work on typesetting and printing. Given modern typesetting tools the actual authors of today end up doing the typesetting and people today usually prefer to have electronic copies because they are easier to store and search. The publishers have simply turned into gatekeepers and rent seekers.
Currently a lot of people prefer not publish their code or data because they do not want to cede their current research edge to 3rd parties. If you are a researcher today you are expected to have a certain amount of knowledge production measured in publications or patents per year so it can get pretty competitive at times. If someone else beats you at your own game you are toast so by not publishing the code and data you are basically giving yourself a head start of several months research time. Other people only publish one research paper when they are long in the process of writing the succeeding papers.