Evaluating Treatment Products

Many of us are interested in taking steps to prevent diseases, stay healthy, and to preserve and improve our quality of life. But deciding what diets and nutritional products are best, and what medications to take is not easy. There are a bewildering number of nutritional products and medications that purportedly can prevent diseases and keep us healthy. While some of these products have proven safety and effectiveness, many have not been subjected to any scientific evaluation, and some are downright harmful. Increasingly, doctors are being asked to advise patients on how to prevent diseases. However, the medications and measures doctors recommend can be quite different from those recommended by nutritionists, chiropractors, complementary or alternative health providers, etc. What is the reason for these differences? Why do doctors recommend some products but not others? How do doctors determine whether a product is good or bad? How good are doctors' recommendations? What are the doctors' weaknesses and limitations in making these assessments?

The process (steps) doctors commonly adopt in evaluating any disease prevention medication or product is imperfect. And the resultant doctors' recommendations are not always right (most of the recommendations in fact change over time as more scientific evidence becomes available). But the process itself is "honest"; it is there to minimize harm while trying to do some good.

These are the questions doctors ask when evaluating a new health product, medication, treatment, or prevention measure:

1. Has its effectiveness and safety have been demonstrated by prospective, double blind, randomized, placebo-controlled clinical trials? In prospective, double-blind, randomized, placebo-controlled trials, patients who are similar in age, sex, genetic background, and other characteristics such as health status, and diet are randomly assigned to receive either the test medication or a placebo. The study has to be conducted in a double blind fashion, meaning neither the patients nor the researchers (those who are administering the drug or treatment) know who is receiving the test medication or the placebo. At the end of the trial, treatment results from the medication-treated group are compared to the placebo-treated group to determine if the test medication is more effective than the placebo. Randomly assigning study subjects and double blinding of subjects and researchers are important to eliminate human bias from these trials. Comparing test medications to placebo eliminates the placebo effect (see below).
2. Has its effectiveness and safety have been critically reviewed by independent experts? Independent experts (not affiliated with the company manufacturing the product or drug and who are not in a position to receive monetary gain from the product) need to critically review the trial design, methodology, results, and researchers' conclusions before the trial can be accepted for publication in a reputable medical journal.
3. Has its effectiveness and safety have been validated by other researchers? Having the trial published in reputable journals is still not enough. Most doctors want to see validation (confirmation) of the trial results by several independent research teams. There is comfort in numbers.
4. How many subjects have been treated and for how long? Adverse side effects of medications or products may not show up until a large number of people have used the product over long periods of time.

Ideally all products used in treating and preventing diseases should undergo prospective, randomized, double blind placebo-controlled trials involving a large number of study subjects. But these studies are costly, and not all proposed studies can be funded. Furthermore, disease prevention studies generally take many years to complete. Therefore, doctors often have to make recommendations based on incomplete knowledge and scientific data. When conclusive proof of effectiveness and safety are not available, doctors sometimes rely on less reliable information from observational studies and even anecdotes.

Even expertly designed and carefully conducted clinical trials may not detect all the adverse effects of a drug or product. Some adverse side effects of a new product may not show up until many (often millions) of users have used it for a long time (months to years).

In medicine, an anecdote is a treatment response observed on a single person or a number of people. The observations are made outside of a formally conducted, double-blind, randomized, placebo-controlled trial.

Anecdotal benefits cannot be distinguished from placebo effects. More often than not, anecdotal benefits cannot be confirmed by double-blind randomized placebo-controlled trials. Therefore, doctors generally are hesitant to recommend treatments solely based on anecdotal evidence. Moreover, potentially serious adverse side effects may only be uncovered by a carefully conducted placebo controlled clinical trial.

However, anecdotal reports of benefits, especially if they seem plausible in a scientific sense, may induce medical researchers to conduct clinical trials to determine the effectiveness and safety of a given product or treatment. In addition, some cases are quite unique and cannot be adequately studied in a trial. In these instances, anecdotal data may provide benefit to those patients.

A placebo is a biologically inert substance that does not have any effect on the disease under investigation Placebos are given to persons participating in double-blind controlled clinical trials. For example, a placebo can be sugar powder or salt placed in capsules that are made to look exactly like the medication being tested, so that neither the person giving the drug, or the person taking the drug know whether a placebo or the test drug is being administered.

A placebo effect is a beneficial response of a disease and its symptoms to a placebo. The placebo effect can be found in almost every clinical trial. For example, in all the randomized double-blind, placebo-controlled trials involving ulcer medications, a consistent number of placebo-treated patients will experience ulcer healing and symptom improvement. Thus the only way to prove that a test medication is effective is by demonstrating its superiority over a placebo or to another medication used for the same purpose.

An observational study is a retrospective analysis comparing the health status of one group of subjects (for example, women who took folic acid supplements) to another group (for example, women who did not take folic acid supplements). Observational studies can only provide circumstantial evidence. Conclusive proof of treatment benefit has to come from prospective, randomized, and placebo-controlled trials.

• Example: Vitamin A is an antioxidant that was believed to be beneficial in preventing cancer and heart diseases. But randomized prospective placebo-controlled trials not only could not demonstrate any benefit of vitamin A, some studies actually found it harmful in some subjects.

QUESTION

What are bed bugs? See Answer

The decision-making process in disease prevention is necessarily imperfect; it is a combination of judgment, experience, and science. It is a balancing act between being cautious (by doing no harm) versus being proactive.

Safety first

Preventing disease is different from treating diseases. In treating diseases, doctors and patients are often willing to accept a finite degree of risk of side effects in order to achieve a cure or improvement in symptoms. In preventing diseases, doctors are extremely risk adverse. Remember, the first priority in doctoring is to "do no harm". Thus when prescribing an agent for prolonged periods of time to prevent a disease that may or may not occur, the doctor would not want that agent to cause adverse side effects in a healthy person.

• Example: NSAIDs (nonsteroidal anti-inflammatory drugs, a class of medications used for arthritis and other inflammatory processes in the body) have been known to inhibit the growth of colon polyps. Colon polyps are precursors to colon cancer. Why aren't doctors recommending NSAIDs to prevent colon cancer? Because prolonged NSAIDs use can have unwanted side effects such as ulcers, intestinal bleeding, and aggravation of liver and kidney diseases. Without prospective randomized placebo controlled trials involving a large number of patients, doctors will not recommend NSAIDs for colon polyp and cancer prevention except in very special and limited situations.

Do not always insist on conclusive proof

Sometimes doctors are willing to recommend a long-term preventive treatment in the absence of any conclusive proof of benefit, provided that we know the treatment is safe. This is especially true if the treatment also has a sound scientific basis and has been found beneficial by observational studies.

• Example: Observational studies have shown that people who take folic acid supplements have lower blood levels of homocysteine. Observational studies have also shown that higher blood levels of homocysteine increase the risk of coronary arteriosclerosis and heart attacks. Scientific studies have also shown that homocysteine can cause injury to the inner lining of arteries thus promoting atherosclerosis. Even though there is not yet conclusive proof from prospective placebo-controlled trials that taking folic acid actually prevents heart attacks, doctors are recommending that all adults take a daily multivitamin that contains folic acid because it is known to be safe over the long term. In this situation, doctors do not want to miss an opportunity to recommend something safe that possibly can prevent heart attacks while waiting for absolute proof of its effectiveness, which can be many years away.

Learn from history

Consider blood cholesterol as another example. Thirty years ago, observational studies suggested that high blood cholesterol (like homocysteine) could cause coronary artery disease and heart attacks. Even though there were no double-blind, placebo-controlled trials available, doctors in those days suspected (correctly) that lowering blood cholesterol could reduce heart attacks. They were recommending low fat diet and exercise to lower blood cholesterol, and medications such as statins only when diet and exercise failed. They also did one very important thing-they started numerous, large scale randomized, placebo-controlled trials to determine if lowering cholesterol actually prevents heart attacks.

Today the prospective trials they started have been completed. These trials have conclusively shown that lowering cholesterol (especially the bad LDL cholesterol) reduces heart attack risks and prolongs life. These trials further showed that the benefits of lowering cholesterol outweigh the risks of side effects of the statin medications. Therefore doctors today are much more aggressive than doctors of yesteryears. Doctors are much more willing to use medications such as statins to lower cholesterol, and the "normal cholesterol level" has been rapidly reduced.