This week I participated in an annual task of making decisions about the question items that will be included in the annual Behavioral Risk Factor Surveillance System (BRFSS) data collection. Before I write about this process, some background would be helpful.
Shortly after the publication of the watershed document "Healthy People: The Surgeon General's Report on Health Promotion and Disease Prevention" in 1979, it became more apparent that, as in all public health work, better data for planning purposes was critical. The important role played by behavior had certainly been recognized for quite some time, but health related behavior and lifestyles as causes of our dismal health status came to a prominence not seen before. Public Health leaders in the federal Health & Human Services and CDC set about building a data infrastructure for behavior change efforts, and so in the early 1980s, the BRFSS project was born. It started with only 15 states; since the mid 1990s, all states and the District of Columbia are participating. The system provides a standardized set of questions, with data reported on a national basis, data for each state, as well as to some local communities. Comparisons among levels as well as time trends are possible.
Survey items include a set of "core" questions that are uniform throughout all jurisdictions, and from year to year. There are then questions called "optional modules" that will be used less frequently, such as every other year. There are also "state-added" questions that will be inserted only in the questionnaire used for the data collection in a specific state. State-added questions may be standardized CDC questions not being used in the national survey, but for local reasons are thought important enough to be included in the state-specific survey instrument. States can also add questions not from CDC, but that address a local concern.
In addition to the importance of the actual questionnaire content, there also has to be thought given to survey administration. The data collection method is telephone survey, with most interviews being done on land-line phones. A growing proportion of phone interviews is done with cell phone numbers, since an ever larger segment of the population is shifting to cell phone use only. Kentucky's cellphone sample in 2013 will be 25% of the total sample. In addition to the cell phone interviews, states can mail questionnaires to people who either have no phones or for other reasons cannot be interviewed by phone. In Kentucky, about 300 mailed questionnaires have been completed in recent years. Survey administrators and interviewers must be concerned about the length of interviews and questionnaires, as well as the nature of sensitive questions. A typical BRFSS phone interview will take about 20 minutes, including about 140 questions - a lot to ask from busy householders who receive no specific benefit. In addition, some questions might be considered too personal or perhaps offensive to some respondents. For example, in the HIV module respondents are asked to indicate whether "You have given or received money or drugs in exchange for sex in the past year." Not everyone is pleased to get that questions from a complete stranger on the other end of the telephone.
At the state level, Departments of Public Health will typically have an administrator who manages the state BRFSS data collection. In Kentucky, an RFP process is used, so that individuals and groups are invited to submit requests for individual or groups of questions to be added to the data collection for the following year. As mentioned at the beginning, I was part of a group that reviewed the question proposals that came to our BRFSS administrator. The proposals must demonstrate three things: 1) need for questions, based on the importance of the issue (e.g. oral health, adolescent sexuality); 2) a demonstrated data gap, with information not available through other existing state-wide data collection; 3) validity and reliability of the proposed questions, and capability to use data findings. Agencies successful with their question requests are charged, in Kentucky, $2500 per question added.
The reviewers considered seven question proposals - three were rejected. In one case, the question requested, about people buying fruits and vegetables in nontraditional venues such as farmer's markets and you-pick farms, seemed to the reviewers to be not important enough information to justify inclusion. In the second case, a set of questions about adverse childhood experiences, such as being physically or sexually abused, left doubts about the practical value of the information. In the third case, a set of questions about parental support for various aspect of school-based sex education, was considered too hot politically by a majority of reviewers, and of dubious value for community interventions. Decision makers strive to be evidence-based and guided by critical thinking, but there are undoubtedly some subjective considerations.
While I left the BRFSS meeting feeling satisfied about making a small contribution to health promotion and the public's health, I also felt as though I had seen some health promotion sausage being made.
Shortly after the publication of the watershed document "Healthy People: The Surgeon General's Report on Health Promotion and Disease Prevention" in 1979, it became more apparent that, as in all public health work, better data for planning purposes was critical. The important role played by behavior had certainly been recognized for quite some time, but health related behavior and lifestyles as causes of our dismal health status came to a prominence not seen before. Public Health leaders in the federal Health & Human Services and CDC set about building a data infrastructure for behavior change efforts, and so in the early 1980s, the BRFSS project was born. It started with only 15 states; since the mid 1990s, all states and the District of Columbia are participating. The system provides a standardized set of questions, with data reported on a national basis, data for each state, as well as to some local communities. Comparisons among levels as well as time trends are possible.
Survey items include a set of "core" questions that are uniform throughout all jurisdictions, and from year to year. There are then questions called "optional modules" that will be used less frequently, such as every other year. There are also "state-added" questions that will be inserted only in the questionnaire used for the data collection in a specific state. State-added questions may be standardized CDC questions not being used in the national survey, but for local reasons are thought important enough to be included in the state-specific survey instrument. States can also add questions not from CDC, but that address a local concern.
In addition to the importance of the actual questionnaire content, there also has to be thought given to survey administration. The data collection method is telephone survey, with most interviews being done on land-line phones. A growing proportion of phone interviews is done with cell phone numbers, since an ever larger segment of the population is shifting to cell phone use only. Kentucky's cellphone sample in 2013 will be 25% of the total sample. In addition to the cell phone interviews, states can mail questionnaires to people who either have no phones or for other reasons cannot be interviewed by phone. In Kentucky, about 300 mailed questionnaires have been completed in recent years. Survey administrators and interviewers must be concerned about the length of interviews and questionnaires, as well as the nature of sensitive questions. A typical BRFSS phone interview will take about 20 minutes, including about 140 questions - a lot to ask from busy householders who receive no specific benefit. In addition, some questions might be considered too personal or perhaps offensive to some respondents. For example, in the HIV module respondents are asked to indicate whether "You have given or received money or drugs in exchange for sex in the past year." Not everyone is pleased to get that questions from a complete stranger on the other end of the telephone.
At the state level, Departments of Public Health will typically have an administrator who manages the state BRFSS data collection. In Kentucky, an RFP process is used, so that individuals and groups are invited to submit requests for individual or groups of questions to be added to the data collection for the following year. As mentioned at the beginning, I was part of a group that reviewed the question proposals that came to our BRFSS administrator. The proposals must demonstrate three things: 1) need for questions, based on the importance of the issue (e.g. oral health, adolescent sexuality); 2) a demonstrated data gap, with information not available through other existing state-wide data collection; 3) validity and reliability of the proposed questions, and capability to use data findings. Agencies successful with their question requests are charged, in Kentucky, $2500 per question added.
The reviewers considered seven question proposals - three were rejected. In one case, the question requested, about people buying fruits and vegetables in nontraditional venues such as farmer's markets and you-pick farms, seemed to the reviewers to be not important enough information to justify inclusion. In the second case, a set of questions about adverse childhood experiences, such as being physically or sexually abused, left doubts about the practical value of the information. In the third case, a set of questions about parental support for various aspect of school-based sex education, was considered too hot politically by a majority of reviewers, and of dubious value for community interventions. Decision makers strive to be evidence-based and guided by critical thinking, but there are undoubtedly some subjective considerations.
While I left the BRFSS meeting feeling satisfied about making a small contribution to health promotion and the public's health, I also felt as though I had seen some health promotion sausage being made.
No comments:
Post a Comment