December 18, 2012

Nutritional Biochemistry

Is a low fat diet best for weight loss?

Read Time 6 minutes

I know many of you are awaiting Part II of the mini-series on ketosis, but I’d like to digress briefly to comment on a study published last week, which a number of you have asked about.

In this study, by Hooper et al., titled Effect of reducing total fat intake on bodyweight: systematic review and meta-analysis of randomised controlled trials and cohort studies, the authors take aim at addressing one of the most important questions underpinning our current epidemic of obesity and, by extension, its related diseases: Is there a preferred dietary intervention that can lead to a long term reduction in body fat?

To address this important question the authors conducted an exhaustive meta-analysis of 33 randomized controlled trials (RCTs) and 10 cohort studies in which patients were treated with a low-fat diet for outcomes beyond weight reduction. For example, they examined studies which treated women at risk for breast cancer (e.g., due to abnormal mammography) with a low-fat diet to test if the diet reduced their likelihood of progressing to breast cancer.  Or studies where subjects at risk for heart disease (e.g., due to biomarkers or a strong family history) were randomized to a low-fat diet versus a standard diet to examine the impact on biomarkers for heart disease.

Before getting into the details of this analysis I’d like to reiterate a point I made in a previous post. James Yang, one of my mentors when I was in medical school and again in fellowship, always reiterated, “a hundred sow’s ears makes not a pearl necklace” when talking about meta-analyses.  Stated another way, Dr. Samuel Shapiro, a distinguished professor in Cape Town, South Africa, made this comment on meta-analyses:

“As a matter of logic, it is fallacious to argue that a series of inadequate studies taken together cancel out their inadequacies.”

In other words, a meta-analysis, no matter how large, no matter how elaborate in statistical tools, no matter how erudite in authorship, can be no better than the sum of its parts. To quote a good friend of mine, James Lambright, the former Chief Investment Officer of the TARP program, a meta-analysis “is somewhat analogous to ratings agencies looking at a huge pile of mortgages and concluding that collectively they deserve an ‘A’ rating, without noticing that the underlying mortgages might all be lousy.”  Nice.  A meta-analysis is basically a CDO. Some good…many not.

A close analysis of the 33 randomized controlled studies included in this systematic review reveals a common trend in most, though not all, of them. (I’ve not looked at the 10 cohort studies for the obvious reason – we will never glean cause and effect from such studies.)

The studies, almost without exception, followed a pretty typical pattern.  The subjects were divided into two (sometimes more) groups and randomized into a treatment arm (or arms) and a control arm.  Here is a typical example of how the investigators interact with the subjects in the treatment and control arms:

Treatment arm: Patients received individualized and/or group counseling to reduce fat intake and increase consumption of fruits and vegetables on a weekly and then monthly basis, often with cooking classes, behavioral interventions, and newsletters. In some trials, the counseling intervention for the low-fat diet arm included weekly or monthly calls from a study dietitian to troubleshoot dietary challenges.

Control arm: Patients received no, or very little, dietary counseling or interventions. Controls were simply instructed to consume their standard diet for the duration of the study.

To my counting, about 99.2% of the nearly 74,000 subjects (all but about 600 subjects) across the 33 RCTs examined were subjected to this treatment bias, referred to more specifically as performance bias. (Yes, this required actually reading — very quickly — each of the studies used in this analysis.)

According to the Cochrane methodology, the gold standard for research methodology, “performance bias refers to systematic differences between groups in the care that is provided, or in exposure to factors other than the interventions of interest. Randomisation of subjects and even blinding the investigators does not eliminate this (performance) bias.”

(I include this last comment about blinding because the BMJ study authors list the absence of blinding as a potential weakness of their study, though they don’t mention performance bias.)

In other words, it is not clear if the pooled effects observed in this meta-analysis reflect the low-fat dietary intervention, the counseling effect, or some combination of both.

Let me illustrate with an example.  Assume you’re one of the subjects in the treatment group.  Upon enrollment in the study, you undergo a lengthy assessment with a study dietician, where you provide a 5-day log of everything you’ve been eating for evaluation.  You are given hours of counseling on how to avoid dietary fat.  You are provided with menus and recipes to cook low-fat (and presumably healthy) dishes. Every few weeks you meet alone (or in group with other subjects in the low-fat arm) to receive additional support and counseling.  Every month a study dietitian calls you to answer any questions you might have and to provide encouragement.

Does anyone think this intervention, regardless of what you’re being prescribed to eat, does not make a difference? If the guidance of the Cochrane group isn’t enough, I can absolutely attest to this from my experience working with people.  We all benefit from encouragement, and the encouragement we get has an effect beyond the dietary composition.

This suggests a slightly different conclusion than that proposed by Hooper et al. Rather than concluding that lower total fat intake leads to small but statistically significant and clinically meaningful, sustained reductions in body weight in adults, it seems a more accurate conclusion would be that lower total fat intake, coupled to an intensive counseling and support regimen, leads to small but statistically significant and clinically meaningful, sustained reductions in body weight in adults compared to a standard diet without counseling and support.

Because the counseling effect could have other unintended effects, (for example, consumption of less sugar, fewer highly refined or processed foods, or more exercise), we can’t be sure what caused the measured effect.

The best studies in this space normalize for intervention effect across all treatment arms.  This way the investigators have some way of assessing the impact of the actual intervention in question – the diet.

A couple of other oddities about this study

If you really wanted to understand the impact of a low-fat diet on weight loss (or some better marker of actual fat loss), it seems that actually looking at trials where this was being tested might be a better place to look. There is no shortage of such trials out there, including the work of Gardner, Foster, Ludwig, Dashti, Shai, and others.  The advantage of looking at diet studies include:

  1. They usually remove the performance bias I described above.  Typically (though not always) in these studies all subjects are given the same support and dietary counseling.
  2. Such studies are statistically powered to detect meaningful differences in the outcome or endpoint of interest – fat loss (or some proxy of it).

I’ve written before about the difference between statistical significance and statistical power, so I won’t repeat the explanation.  But, I’d like to point out the problem of looking at endpoints that were not powered.  The largest study included in this BMJ meta-analysis was the Women’s Heath Initiative (WHI), a study of nearly 50,000 post-menopausal women.  The WHI, which I talked about at length in the presentation in this post, tested a low-fat dietary intervention over an average follow-up of nearly 6 years. The study was powered to detect hard outcomes like cancer incidence, heart attacks, and death.  That’s why the study was so large.  But when a study is this large, it’s actually quite easy to find statistical significance in any number of parameters, even if they are not clinically significant or relevant.  Look at Table 3 from the JAMA report of the WHI:

JAMA table 3

You’ll notice that in waist circumference, for example, some of the subsets showed a statistically significant difference.  Overall (bottom right section of table) the control (normal fat) group increased their waist circumference over the study from 89 cm to 90.4 cm, while the low-fat intervention group increased from 89 to 90.1 cm, a difference of 3 mm, which achieved a p-value of 0.04.   While this is statistically significant (and therefore included in the BMJ paper as a study for consideration), it’s not really clinically significant.  In fact, it’s not clear if any of the statistical differences in the WHI are clinically significant.  Waist-to-hip measurements didn’t change, which of all the anthropometric changes is probably the most relevant to consider (absent actual body composition data).

Interesting side-note: both the treatment and intervention group lost a bit of weight, yet both groups saw an increase in BMI, suggesting they got shorter over the duration of the study. Furthermore, despite scrutinizing this table and the paper for longer than I care to admit, I cannot for the life of me explain the arithmetic. The authors logically define “change” as the difference between follow-up and baseline.  However, examination of this table reveals the arithmetic to be incorrect more often than it is correct.  Some instances may be attributable to rounding errors and significant figures, but many are not. It is possible drop-out accounts for this, I suppose.

My micro point

Notwithstanding the fact that the WHI suffered from performance bias:

“Women assigned to the control group received a copy of the Dietary Guidelines for Americans as well as other diet- and health-related educational materials, but otherwise had no contact with study dietitians. In contrast, women randomized to dietary intervention were assigned to groups of 8 to 15 participants for a series of sessions structured to promote dietary and behavioral changes that would result in reducing total dietary fat to 20% and increasing intake of vegetables and fruit to 5 or more servings and grains (whole grains encouraged) to 6 or more servings daily.”

The differences observed, despite this bias, were statistically significant because of the sample size, but were not clinically significant.

My macro point

A meta-analysis based on studies like this is not the best way to answer the question at hand.  Sadly, I suspect, most providers (e.g., physicians, dietitians, nutritionists) may not appreciate this given how the media reports on this kind of publication.

 

Photo by Beatriz Pérez Moya on Unsplash

Disclaimer: This blog is for general informational purposes only and does not constitute the practice of medicine, nursing or other professional health care services, including the giving of medical advice, and no doctor/patient relationship is formed. The use of information on this blog or materials linked from this blog is at the user's own risk. The content of this blog is not intended to be a substitute for professional medical advice, diagnosis, or treatment. Users should not disregard, or delay in obtaining, medical advice for any medical condition they may have, and should seek the assistance of their health care professionals for any such conditions.

114 Comments

  1. Excellent article Peter. I love your work and appreciate the data you share.

    To everyone’s health!

    Cathie.

  2. Good artical tons of data.
    I have done the Keto diet a few times, high fat, low to no carbs. From personal experiance it works wonders. I can lose @10lb in 2-3weeks. but , you really start missing carbs with in the first few days

  3. New to your blog and really enjoying the new information and perspective.
    As Michael Pollan points out in In Defense of Food, the whole idea behind these studies is pretty flawed. You put people on a low-fat diet, but you can’t change just one variable. When you lower fat, you’re increasing protein, or carbs, and then if the diet is working for some people – there is no way to determine if it’s because of the increase of protein, or maybe due to the extra fibre you’re eating (or a combination of many things.) et cetera.
    We want to simplify things down to fat = bad and ‘eat this, not that’ when food and nutrition is incredibly complex .

  4. If you really wanted to understand the impact of a low-fat diet on weight loss or some better marker of actual fat loss, it seems that actually looking at trials where this was being tested might be a better place to look.

  5. I am just blown away- found you on my daily ted talk- I have had PCOS weight dramas and all the other insulin issues that go with it dismissed by Drs my entire life. I’m taking Metformin 1700mg a day, so the effect that has is great for some things, like regular ovulation, but I still do not seem able to move weight- and in recent high-stress times I put on those 10 kgs I have killed myself over the last year to loose. I need a user-manual and easy steps- talk to me like I’m 5- what book do I need to start implementing meaningful diet changes? I already cut out sugars- and fruit. The nutritionist at my diabetes clinic meetings hates me. I cook in coconut oil and she freaks out and says this is why I’m fat. Hopeful and glad to have found this blog- best wishes! E

  6. Peter, thank you for putting this together.
    I want to know if it’s good to add drinking tea in the diet plan. There are saying tea is a great method for weight loss, can you please check https://bestteaforweightloss.com/ and give some thoughts?
    I also point my name to the site to avoid being removed in this comment.

    Thank y ou

  7. Sadly each study seems to disprove the last. Or perhaps we are just constantly learning?
    I recall thinking in Science class at College that science is by no means perfect and not at all complete, so we need to take everything we hear and learn with a grain of salt!
    Mike

  8. I believe that the basic rules of keeping shape are the same for everyone. Though, everyone should adapt them for himself. For me fitness is the best way. When I stop going to the gym I immediately gain weight (unless I starve myself). For me all the diets are very depressive, I can not live without sweets and cookies, that’s why I choose fitness. Regular training is very rewarding: I’ve noticed my first results within 1 month, and it was an awesome stimulation for future trainings. When I feel tired I take Super Army Formula by Military Grade and it quickly restores my strength and enthusiasm. It provides the necessary nutritional supply, which is vital when you are training intensively. Thus, nothing prevents me from eating occasional cookie or a bar of chocolate when I want it so much.

  9. Dear Dr. Attia:

    I’m new to your blog but appreciate your emphasis on using one’s own body as a “laboratory” to test various approaches in order to discover those that work. So many “experts” emphasize the correctness of their approach and its applicability to broad segments of the population. Not sure if you caught Richard Smiths catchy titled Feature in this week’s BMJ, “Are some diets “mass murder”?”, highlighting just how nearly impossible it is to conduct meaningful dietary research in free-ranging populations. https://www.bmj.com/content/bmj/349/bmj.g7654.full.pdf

    As a psychologist, I’m primarily interested in the behavioral elements of diet and weight management. One relatively new area of research is Palatable Food Relapse, the mechanisms by which almost any effort to restrict food intake either in volume or type, almost immediately triggers obsessive and compulsive drives to return to pre-existing patters of feeding. A fascinating, yet somewhat discouraging explanation for the astonishingly high recidivism rate among dieters (ranging from 80-98% depending on the study).

    Another new area, findings for which were just published in Cell Biology, is Time Restricted Feeding (TRF) and its impact not only on weight regulation but many metabolic cofactors such as insulin, b.p., cholesterol, etc. While the work at this point is in animals, the findings are very compelling that restricting feeding to a 9-12 hour window has a huge impact on hormonal activity and their sequelae. It ties in to what I remember you once saying about the chicken and egg question of obesity as cause vs. effect. TRF piece at: https://www.cell.com/cell-metabolism/abstract/S1550-4131(14)00498-7

    I enjoy your approach and writing. Thanks.

  10. This may be an obvious question, however is insulin the only/primary mechanism that the body uses to store and accumulate fat? If there are other mechanisms, what are they and how much effect (compared to insulin) do they have in the regulation of the body accumulating fat? I hope someone can guide me to this answer. Thanks.

  11. Hi Dr. Attia~ I’ve read most everything on your site with great interest. I began a Keto diet 2 weeks ago and started searching up the effects of keto and athletic performance right away (mainly cycling and running). I am struggling right now with the symptoms (aka “keto flu) but what’s most disturbing is that I’ve had terrible runs and rides for the last 2 weeks. My legs are lead … and I did make the mistake of getting dehydrated last night … caused me to gain 6 lbs and have some unpleasant side effects. I guess I just wanted to hear from you or someone that athletic performance would return and that I would be able to “climb” again on the bike without dead legs or feel a good return of energy for sustained efforts? I am under the impression that I should take it easy with workouts and training right now? Any thoughts or suggestions will be greatly appreciated!
    Bests!! Christi

  12. Reading Original Sources For Myself:

    I am a social scientist but of course much less familiar with medical literature and language. On reading some studies (OK, I confess, starting with abstracts) I can access about HCLF vs VLCHF diets in PubMed, I came across a 2008 study entitled, “Metabolic effects of weight loss on a very-low-carbohydrate diet compared with an isocaloric high-carbohydrate diet in abdominally obese subjects.” When I read the Results and Conclusions (abstract version), I am stumped. I thought higher HDL & lower triglycerides/triacylglycerols (as observed in VLCHF participants) would be the better set of outcomes? Yes, VLCHF had higher (but unchanged!) LDLs, but it doesn’t look like particle density was measured. Am I completely off-base to think that the authors concluded something more subjectively (and rather erroneously) than objectively here?

    Find article abstract here: https://www.ncbi.nlm.nih.gov/pubmed/18174038

  13. Up to this point I’ve personally seen Cochrane data with high regards. And yet, your point on meta-analyses is spot-on and it clicked on me (‘of course!’). I feel silly I haven’t noticed their limitations sooner.

    You say (not sure if ironically or not) that Cochrane is the gold standard of methodology. Supposing it’s not ironic, what is the good part, then? Because when I think ‘Cochrane’ I think ‘meta-analyses’. My view of Cochrane has normally been ‘aw, so the Cochrane people finally came together to do an exhaustive (meta)analysis of all things published about the theme, ever. it was about time’.

  14. Hey there! before I ask my question, I want to say that your blog was the primary reason why I finally decided to switch to a low carb diet. I was on the fence a while but now I’ve been practicing a ketogenic diet for a little over two months!

    Anyway, I went from 162 pounds to 144 pounds in these two months which is a great change for my 5’5 body, but I’m still a long way from my goal of 125. For the past two weeks I’ve been bouncing in between 143-145 and haven’t seen any real weight loss. I’m not sure how to handle this. Do you have any tips? Did you experience this type of stall on your journey?

    I eat between 70-85% fat a day, at least 65g-100g protein, below 25g carbs (but usually below 20), between 1320-1600 calories (at the moment my life is most sedentary aside from working out at the gym for about an hour a day), I do use some sweeteners but not often, I use MCT oil every morning due to a suggestion I saw on your blog. I’m not sure what else I could say but it feels like I’m not doing anything wrong.

    Any advice would be great!

  15. Hi! I have been hypoglycemic and high cholesterol since early 90s. In 2010 I was diagnosed with hashimotos, hypothyroid, goiter, and follicular carcinoma. I have had numbness and tingling in my hands and feet the Dr’s can’t explain since 2005. Now the Dr is saying I am insulin resistant and prediabetic and wants me to eat beans and go on a raw/vegan/low glycemic diet. Right now im on a lean meat and vegetable diet with limited carbs. I have seen 4 nutritionists and even spent a year on weight watchers. I still am hypoglycemic so my sugar drops low. Especially if I eat in the morning. I do better if I wait til about 10 or 11 to eat, then eat protein to start my day. I don’t understand how I could be insulin resistant if I’m still being flooded by insulin. And I can’t get this weight off me no matter what I do. I’m prior military. Exercise doesn’t get it off. It just builds more muscle under the fat. Right now I’m at 32% body fat. Trying to get down to 24%. I was just watching your ted talk from 2013 and thought maybe you could help.
    Thanks for any help you can offer.

  16. This smacks of handwaving to dismiss evidence contradicting your preconceived notions.

Facebook icon Twitter icon Instagram icon Pinterest icon Google+ icon YouTube icon LinkedIn icon Contact icon