Critical evaluation of physical activity and BMI study design

¶ … Physical Activity and Body Mass Index in Adolescents.

Is the Abstract Clearly written?

The test as it was conducted and some of the results were clear. The conclusion did not tie into the methods, results and purpose statement, as it was more of a political prescription than a true conclusion from the study.

Also, the n was low, and the conclusions should have said something like "a larger study needs to be done."

Although the total number of study participants was indicated, they did not indicate the relatively low compliance (less than 50%), so the actual numbers should have been reduced to the compliant n.

Why was the study done?

In order to demonstrate the inverse relationship between BMI and exercise, with an aim to demonstrate amongst different ethnicities that exercise is essential to lower BMI.

Is the research design appropriate to the stated aims?

No, because there is such a small n in each of the three groups that it is not possible to derive statistical significance. With a total of about 80 compliant subjects split into 3, the 2-SD variation is greater than 50%

Who exactly has been studied?

These were all high school students with a roughly equal split between Mexican-, Black- and White Americans. They were self-selected; that is, although the school may have been diverse, the fact that respondents to a questionnaire and flyer enrolled may mean that that population suffers from selection bias. A control group in the same school may have helped to define if there was selection bias.

Was a control group used. Should one have been used?

Yes, because although there were different groups within the study, all came from the same school and therefore may have had similar home, nutritional and experiential backgrounds.

How adequate was the follow up?

Adequate follow-up should have included another evaluation of the same girls over a longer period of time. As it is, there were only 3 days per girl, 50% who were compliant. We don't know what the actual exercise experienced over a longer period of time is (see comments below).

Was the sample size justified?

This is one of the biggest weaknesses of the study. In order to draw meaningful conclusions about each subgroup, a larger group needed to be chosen. If, for example, they had 300 compliant subjects, 1/3 in each group, they could have predicted to a +/- 6% degree of certainty. By then breaking the group down further, to pre- and during puberty, the groups were even smaller.

Are the measurements likely to be valid & reliable?

I am concerned about the low compliance. Perhaps if people got water on it, the measurements were compromised. Also, we need to see if there is actually a correlation between movements and calories burned, or the amount of cardiovascular challenge exercise actually done.

We also don't have an indication of the importance of water sports -- if that was a significant activity, we need to factor that in.

10 Was the exposure intervention accurately measured?

No, we have no indication that it was accurately measured. We know that the results were hindered by water -- so those who sweat may have disabled the device or changed the result.

11 Were the relevant outcome measures ignored?

There was no analysis of the results of sexual maturity on BMI or exercise. Since all three groups could be aggregated for this portion of the study, the results might have been interesting and suggestive.

12 Are the statistical methods described?

Yes and no. We need to have an indication of the n. For each compliant subgroup, not just the total group.

13 Did untoward events occur during the study?

Yes, and the non-compliance should have been explained. Also, moving from the target of 7 days to 3 days smacks of a failed study -- either the measurement technique was too fussy, or the girls didn't want to comply. That implies even more selection bias than with the initial selection by brochure.

14 Was the basic data adequately described?

No. If there were additional data points, such as sexual maturity, they should have been presented.

15 Did the analysis allow for the passage of time?

No. 7 days establishes a point in time. It would be more interesting to understand the longer-term changes in exercise levels and BMI. The fact that there was only a 6% difference between those who exercised and those who didn't wasn't helpful.

16 Do The numbers add up?

The comparison between the "actigraph" and heart rate, oxygen uptake and heart rate are all too low. Although they mention the r, or correlation, they should use r2, which would look even worse. That is, the oxygen uptake, which has aan r=.73, is actually an r2 of about 0.50, which is way below the level needed for statistical significance.

17 Was the statistsical significance assessed?

"p" factor was given, but should have been indicated as to whether or not it met the definition of statistical significance. See above note on correlation with the actigraph and the amount of exercise actually done...very poor correlation.

18 What do the main findings mean?

Not much. We don't have a correlation between the movement and the amount of calories burned, such that we could create a graph showing the relationship between the two. A 6% difference between the "more active" and "less active" groups in BMI doesn't seem to meet statistical significance, even cumulated, with only ~80 total compliant subjects.

19 How are the null findings interpreted?

They are not interpreted

20 Are important effects overlooked?

Yes, the effects of puberty on BMI and on exercise levels.

21 What else might influence the observed outcomes?

We have a 7-day snapshot which was then reduced to 3 days within the 7-day window (in order not to lose all the data). We don't know if this corresponds to the amount of activity that girls are experiencing more broadly. If, for example, a girl was having her period during the 7-day period, it could be that she is less active.

How do results compare to other reports?

It is clear from the study/literature review that there is a correlation between BMI and exercise. It is also clear that Hispanics and African-Americans exercise less and have higher BMI. Therefore, there were no surprises and no new questions raised by the study.

23 What are the implications of the study?

The most interesting new work would have been on the impact of puberty on exercise levels. Although the data was taken, it wasn't revealed or explored.

Relationship between Physical Activity and Body Mass Index in Adolescents.