User:Christopher Thomas/Reputable research

This is based on a comment I'd posted on the WikiProject Physics talk page. I've adjusted it to remove references to the specific debate that sparked it. --Christopher Thomas (talk) 06:46, 17 July 2012 (UTC)

Distinguishing reputable research from less-reputable research

WP:UNDUE and WP:FRINGE work just fine for scientific topics. A contrived example to illustrate how to judge these:

• E. Spengler publishes an article in a respectable journal. It's cited in a review/survey article, and has quite a few citations by unrelated research groups. E. Spengler's work is clearly notable, and is probably not fringe (though it may still be a minority viewpoint). (Weight isn't established, and the threshold for "quite a few" is very field-specific, but hopefully you get the idea.)

• R. Stantz publishes an article in a respectable journal. It has a moderate number of citations, including one review/survey article mention, but most of the citations are from Prof. Stantz's research group or close affiliates. There are very few citations by people not affiliated with Prof. Stantz. Prof. Stantz's work is probably not notable. (Whether it's "fringe" or not depends on whether it's technically accurate - which it may be, if it passed peer review - and whether Prof. Stantz is claiming that it's more notable than it actually is.)

• P. Venkman publishes multiple articles in less-respected journals, and at conferences. While all of these have some form of peer review, the standards are far lower than for top-tier journals. Prof. Venkman's work is cited extensively by his own research group, and occasionally by other publications in less-respected journals, but the sole citation by an article in a respectable journal poked holes in it (and several similar publications). Prof. Venkman's publication is clearly fringe, and probably non-notable (given that it hasn't made much impact citation-wise even within the fringe community). (Be advised that even reputable conferences tend to rubber-stamp conference papers/presentations. I _wish_ they'd caught an embarrassing error in one of mine a few years back. There's a reason why journal publications carry most of the weight.)

Most of the headaches from fringe science patrol are from Venkman-type publications. The difficulty on Wikipedia is that it's very hard to find mainstream/reputable articles that criticize them (as they're beneath serious notice), so critical statements tend to get challenged under WP:V. Editors going strictly by publication count without being familiar with venues may even mistake a Venkman-type publication for a Spengler-type.

A less-frequent source of headaches, but still a significant one, is editors pushing Stantz-type publications. These are works written by a scientist who's published in respectable venues but who hasn't made significant impact with their views. Weight for purposes of WP:UNDUE is minimal. Evaluating this rigorously requires knowledge of the journals used by the researcher, knowledge of what the field's handful of most prestigious journals are, and knowledge of what citation counts are typical and what citation counts indicate a particularly noteworthy work.

You can find experts with the level of knowledge to judge this at appropriate wikiprojects (WT:PHYS, WT:AST, and WT:ASTRO for the subjects I usually work with). Even without that level of knowledge, a bit of careful research can get you a rough idea of how notable a work is.

(Before anyone jumps on me, please also note that a single researcher can fall into multiple categories depending on what project they're working on. I doubt the "for fun" paper I'm planning will get taken very seriously, but the optics one should.)

Long story short, yes, editors can evaluate what is and is not fringe and what a concept's weight is, using publicly-accessible citation databases, as long as you have a bit of knowledge of the field. We have plenty of people on Wikipedia with the required knowledge, and many will also have full access to the journals in question (in addition to the citation databases). Applying WP:UNDUE and WP:FRINGE is feasible for scientific topics. --Christopher Thomas (talk) 06:30, 17 July 2012 (UTC)