Review of the 1990 CDC Study on
Head Injury Deaths and Helmet Laws

Reviewed by
Jonathan P. Goldstein, Ph.D
Bowdoin College

What the Study Does

In "Head Injury - Associated Deaths From Motorcycle Crashes: Relationship to Helmet-Use Laws" by D. M. Sosin, et. al, (JAMA, Nov. 14,1990 - Vol. 264, No. 18), the authors estimate the rate of motorcycle-related deaths associated with head injury for states with comprehensive (full) helmet-use laws, states with partial (youth) helmet-use laws, and states with no laws. Two different rates are calculated for each state for the years between 1979- 1986: a population-based rate (motorcycle deaths associated with head injuries divided by state population) and a registration- based rate (deaths associated with head injuries divided by the number of state motorcycle registrations).

The end result is an average population-based rate and an average registration-based rate of motorcycle deaths associated with head injuries for each of the three helmet law groupings of states during the entire 1979-1986 period.

The authors find that the population-based rate in states with partial laws (10.2 fatalities per million residents) and no helmet-use laws (11.4) were almost twice those in states with comprehensive helmet-use laws (5.5). Their registration-based rates show dramatically less contrast between the three groupings: partial law (3.7 per 10,000 registered motorcycles), no law (3.5), and comprehensive law (3.0).

On the basis of comparisons of these rates, primarily the population-based rates, across groups of states with different helmet law coverage, the authors conclude that comprehensive helmet-use laws are an extremely effective mechanism for reducing the severity of nonfatal head injuries and the rate of fatal injuries. Their policy prescription is the adoption of comprehensive motorcycle helmet-use legislation.

The contribution of this study is the creation and use of an alternative data set on motorcycle fatalities. In particular, the authors are able to separate out motorcycle-related head injury deaths from all motorcycle-related deaths. Thus, isolating a fatality variable which can more directly be used to test propositions on motorcycle helmet effectiveness. Unfortunately, the statistical methodology employed -- comparing rates of head-injury related deaths for states with different helmet-use law coverage in order to infer the effectiveness of such legislation -- is fundamentally flawed. In summary, the authors should be praised for their data collection techniques/innovations, but criticized for their inappropriate statistical methods and thus invalid conclusions.

The Major Weakness of the Study

The main conclusion of the study--helmet-use laws and thus helmets are extremely effective for reducing the fatality rate associated with head injuries--is derived from a statistical technique known as correlation analysis. Using this approach, the authors simply observe that states with comprehensive laws have lower fatality rates than states without comprehensive laws and thus conclude that the existence of comprehensive helmet laws are the sole cause of these differences in fatality rates. But as all beginning statistics students know, correlation does not imply causality.

The method employed in the CDC study fails to control for differences (across comprehensive and noncomprehensive helmet law states) in other factors -- speed, alcohol consumption, and registrations/motorcycle usage -- that can dramatically impact the population-based fatality rates used to arrive at the CDC study's conclusion. Thus the higher fatality rates in the CDC study observed in noncomprehensive law states could be explained by higher average driving speeds, more alcohol consumption, and more motorcycle usage per person, rather than by the lack of a comprehensive helmet-use law.

The data in Table I clearly establishes that states without comprehensive helmet laws have these critical characteristics over the same period, 1979-86, used in the CDC study. Thus the CDC study fails to determine which of higher speeds, more alcohol consumption, more use of motorcycles per person, and the lack of a comprehensive helmet law are responsible for the higher fatality rates. By not controlling for the impact of these other factors, the CDC study erroneous assigns their impact to the one factor that is considered in the analysis -- nonexistence of a comprehensive helmet-use law. Given that these other factors cause higher fatality rates, the CDC study clearly and dramatically overstates (distorts) the effectiveness of helmet use laws by assuming that higher fatality rates in noncomprehensive law states are determined by the absence of a law rather than by higher speed, alcohol and usage. The level of distortion could be large enough that if it is corrected, the end result would show that helmet-use laws do not have a statistically significant effect on head-injury related fatalities. Other studies that employ the correct statistical methods (Goldstein (1985,1986)) by controlling for the impact of speed, alcohol, etc. find that helmets and helmet-use laws have no statistically significant effect on the probability of fatality or fatality rates.

Referring to Table I, the heart of the problem in the CDC study can be addressed. The number of motorcycle registrations per 1000 population is dramatically higher in states without a comprehensive law--in these states, people are more likely to own/register/ride a motorcycle. Thus, motorcycle usage per capita is higher as will be fatality rates. In particular, in states with comprehensive laws, there are 18.7 registrations per 1000 population during 1979-86, in states with no laws there are 35.9, in states with partial laws 33.9 and in states without a comprehensive law (states with no laws or partial laws) there are 34.6.

This implies that part, if not all, of the differences in the population-based rates of the CDC study could be explained by the existence of more motorcycles per capita rather than by the nonexistence of a helmet law.

One way to correct for the influence of higher registration rates is to calculate a registration-based fatality rate associated with head injuries. The CDC study does this and finds that the differences between comprehensive law sates and other states all but disappear. In particular, the comprehensive states have 3.0 fatalities per 10,000 registrations while partial law states and no law states have 3.7 and 3.5 respectively. Yet, the CDC study ignores that the differences between states with comprehensive laws and without laws have been dramatically reduced and continues to rely on the distorted population-based rates to draw its conclusions. The study fails to conduct statistical tests (discussed below) to see if the 3.0 and 3.5 rates are different by more than one would expect to occur by chance sampling error. Thus, there may not exist any statistically significant difference in fatality rates between comprehensive law states and other states (unfortunately not enough information is provided in the article to calculate the appropriate statistical tests).

Another disturbing aspect associated with the registration based rates is that the CDC study ignores the fact that partial law states have a higher fatality rate than states with no laws. Using the same flawed logic contained in the CDC study, one could conclude from this result that helmet laws do not work because they result in higher fatality rates. Yet the study just ignores this finding. Of course, the contradictory result supports more the notion that the CDC study has produced distorted estimates of helmet law effectiveness than it supports the notion that helmets cause fatalities.

We have just seen that when the CDC study controls for registrations, the differences between comprehensive and noncomprehensive law states virtually disappear (in a statistical sense they may totally be eliminated). If the CDC study controlled not only for registration/usage differences across states but also for the speed and alcohol differences exhibited in Table I, the differences in fatality rates would be reduced further. The higher average driving speed and alcohol consumption found in the noncomprehensive law states certainly explains some, if not all, of the remaining differences in fatality rates between states with and without comprehensive laws., Other studies (Goldstein (1985, 1986)) have shown that speed and alcohol are the major determinants of deaths in motorcycle accidents. In addition, it is also a well-known fact that excessive speed and alcohol consumption are a primary cause of accidents. Thus, once all relevant factors are controlled for, it is highly likely that no statistically significant impact of helmet laws on fatality rates will exist. In order to control for all relevant factors and thus sort out the unbiased (undistorted) effect of helmet laws on fatality rates requires the application of multiple regression analysis. The CDC study does not employ this statistical methodology. Studies that have used this technique (Goldstein (1985,1986)) generate an undistorted estimate of helmet-use law effectiveness which shows that these laws have no statistically significant effect.

In conclusion, the CDC study dramatically overstates the effectiveness of helmet-use laws on fatality rates related to head injuries because it fails to control for all of the relevant factors that affect fatality rates (motorcycle usage, speed, alcohol). When all factors are controlled for, the differences between comprehensive helmet-use law states' fatality rates and all other states' fatality rates disappear implying that helmet-use laws are ineffective in reducing fatality rates.

Other (More Technical) Problems in the Study

The CDC study claims that it is not necessary to address sampling error and thus statistical tests of significance because the entire population of death certificates of motorcycle-related deaths is analyzed. First, the entire population of motorcycle related deaths associated with head injury, the relevant variable in the study, is not analyzed. In the data, 23% of all motor vehicle deaths were not classified by the type of vehicle. This implies that 23% of motorcycle deaths were not classified. Given that in the classified deaths, motorcycle deaths associated with head injuries accounted for 53% of motorcycle deaths, this implies that 12% of motorcycle deaths associated with head injuries are not considered. This further implies that a sample is being used-- head injury-associated deaths is a random variable. State fatality rates are subject to a sampling error. In addition, there exists a random measurement error -- deaths are recorded by residence instead of location -- implying that state fatality rates will be measured with an additional sampling error. Finally, the appropriate sample size when considering state fatality rates is the number of states, not the number or percent of the population of head injury related deaths.

The existence of sampling error implies that statistical tests of significance are appropriate. Given that the appropriate sample size is the number of states in each group (a small sample), it is highly unlikely that the differences in registration-based fatality rates reported in the CDC study are statistically significant implying that helmet-use laws have not be shown to be effective.

The analysis in the CDC study that compares the before and after fatality rates in the 3 states that changed their laws between 1979-1986 is invalid. These comparisons are based on a sample size of 2 and 1 and cannot be used to reach any statistically valid conclusion.

The data on fatalities in the CDC study includes deaths from motorized scooters, tricycles and mopeds. These types of fatalities are not representative of motorcycle deaths and should be excluded. They could distort the results. Of course, no data set is perfect and this limitation of the data is not a serious problem.

Finally, many little facts in the CDC study point to the major problem in the study: (1) the higher registration-based fatality rate for partial law states; (2) the significant drop in the differences between fatality rates when registration-based rates are employed and (3) the decline in all fatality rates between 1979-1986. (1), (2), and (3) all suggest that important factors that influence fatality rates have not been controlled for. (2) points to the importance of motorcycle usage and (3) points to the importance of stricter speed enforcement and stricter OUI laws, factors not considered in the CDC's calculation of the causes of fatality rate differences.

Conclusion

In conclusion, the CDC Study attributes differences in population-based fatality rates (associated with head injury) across states with different helmet-use law coverage solely to differences in a state's helmet-use law. The study concludes that lower fatality rates are the direct result of comprehensive helmet- use laws. The study fails to control for important differences in registration/usage, average driving speed, and alcohol consumption in determining fatality rates. In particular, states with comprehensive laws have lower registration/usage rates, lower driving speeds, and lower alcohol consumption all of which lower the fatality rate. Failure to control for these factors leads to distorted estimates of helmet law effectiveness which systematically and dramatically overstate their effectiveness. Once these factors are appropriately controlled for, no statistically significant differences in fatality rates across states with different helmet law coverage can be found.

Footnote

¹While the speed and alcohol figures in Table I are respectively for all vehicles and for the entire adult population and the differences between groups of states seem small, these differences are relevant Higher average speeds (on 55 mph highways) are associated with less strict enforcement of speed limits on all highways Given the higher relatives rates of acceleration of motorcycles when compared to other vehicles, a small speed difference for all vehicles can translate into higher speed differentials for motorcycles In addition, the alcohol differences and could be magnified in the motorcycling community

References

Goldstein, J. 1986. "The Effect of Motorcycle Helmet Use on the Probability of Fatality and the Severity of Head and Neck Injuries," Evaluation Review, 10 3, June.

Goldstein, J. 1985. "The Effect of Motorcycle Helmet Use on Motorcycle Fatalities An Econometric Approach," Discussion Paper #85-115, Public Affairs Research Center, Bowdoin College.

| Home | Goldstein Study | Other Reviews by Goldstein |

© Copyright Jonathan P. Goldstein Ph.D. 1986. All Rights Reserved.