Lawyers developed a solution (albeit imperfect) to this problem in the 19th century.

In common law systems like the United States, attorneys rely on previous legal opinions that support their arguments. Consequently, much depends on the continuing validity of this opinion. When a lawyer finds an important case, he/she will always determine whether the same argument, test, rule, etc. is still cited approvingly in the apropriate jurisdiction. How do we do this? Well, in the US there are several services (Shepard's is the oldest and best) whereby an attorney will enter a case citation and the service will return with every single subsequent case, law review article, etc. that has cited to that case. Furthermore, the service will frequently tell you if these subsequent citations support, criticize, distinguish, or overrule the original case. Before the computer age, these services were published in large, unhappy books. But today, it's as simple as clicking a mouse button and failure to do so may be malpractice.

The system is not perfect. Sometimes, two separate threads of legal reasoning develop independently of one another despite the fact that they deal with the same issue. As a result, a lawyer might miss a completely valid and contradictory set of case law. However, this situation (while embarassing) is a rare occurrence.

I don't see why a conglomerate of doctors (the AMA for example) could not devise a similar system that would simply cross references all research papers and articles published by the myriad of medical journals that exist today. Consequently, a doctor could easily determine whether or not an important study has been superceded by more recent findings.

It's just a thought.

Funny, I was just about to make the same suggestion. There are some key differences that might make it difficult to import Shepards into the medical profession whole cloth; issues of money and motivation would have to be worked out. Westlaw and Lexisnexis are extremely expensive*, but the costs are passed on to clients and the personal consequences for not shepardizing are immediate and unpleasant. Every law student in America is shown a video, by representatives of LexisNexis, of Judge Ito embarrassing Marcia Clark for not shepardizing properly in the O.J. Simpson trial. Doctors and researchers may not have the same cost-spreading or motivational mechanisms in place. I'd be interested to hear from someone with a medical background on the subject - I find it hard to believe that no one has attempted something like this before.

* but see Slate's Tim Wu's new website, created with Paul Ohm: altlaw.org, an 'alternative' to the twin behemoths of lexisnexis and westlaw.