SESSION 2C: Workforce-Data Collection & Analysis
Room: Thoroughbred 3
Wednesday, April 9, 2014, 9-10:15 am
Back to Conference Agenda
AUDIO
KC14 Session 2C on Workforce: Data Collection & Analysis by NCC for PHSSR
MODERATOR
Michele Issel, Ph.D, M.S.N., B.N.
PRESENTERS
Kyle Bogaert, M.P.H.
A Longitudinal Analysis of State Health Agency Workforce Characteristics
Co-Investigators: Rivka Liss-Levinson, Ph.D., Katie Sellers, Dr.P.H., CPH, Paul Jarris, M.D., M.B.A.
Background: State health agencies (SHAs) serve an essential function in promoting and protecting the health of Americans. To most effectively perform this function, SHAs must have a sufficient workforce comprised of individuals with diverse backgrounds and expertise. Additionally, SHAs must recruit and retain qualified public health workers.
Research Objective: The current study’s goal is to examine longitudinal trends in the SHA workforce using the results of the 2012 and 2010 ASTHO Profile Surveys. Data Sets and Sources: The Association of State and Territorial Health Officials (ASTHO) conducts an online survey of SHAs every two to three years to document their structure, functions, and resources. In the 2010 and 2012 surveys, respondents were asked about the number of employees in the SHA, salary and fringe benefits by occupational classification, demographic information about their workforce, and recruitment and retirement eligibility.
Study Design: The 2012 ASTHO Profile survey was administered to the 50 states, D.C., and U.S. territories and freely-associated states from October 2012-January 2013; the study is cross-sectional.
Analysis: Forty-eight states and D.C. responded to the 2012 survey. Data cleaning was completed and unusual values were clarified. Descriptive statistics were calculated using SPSS.
Principal Findings: While some characteristics of the SHA workforce have remained relatively consistent from 2010 to 2012, others have changed. Trends show increasing numbers of employees leaving SHAs and employees eligible for retirement, as well as increased recruitment. Enumeration of the public health workforce continues to be challenging.
Conclusions: Examining longitudinal trends in the SHA workforce provides insight into how SHAs continue to try and improve the quality of their workforce in the face of reduced resources. Results will be discussed in the context of challenges in enumerating and collecting certain data about the workforce.
Implications for Public Health Practice and Policy: A longitudinal analysis of the SHA workforce contributes to the PHSSR research agenda by utilizing current data to articulate changes in the supply and diversity of the workforce, and issues of recruitment and retention. The analysis identifies potential implications of the loss of institutional knowledge at SHAs with increasing proportions of the workforce eligible for retirement.
Rivka Liss-Levinson, Ph.D.
Results from Cognitive Interviews Exploring the Reliability of the ASTHO Profile Survey
Co-Investigators: Richard Ingram, Dr.P.H., M.Ed., Katie Sellers, Dr.P.H., CPH, Kyle Bogaert, M.P.H., Paul Jarris, M.D., M.B.A.
Background: State health agencies play a critical role in improving the health and well-being of Americans. Valid and reliable surveys that collect information on state health agency structure, finance, activities and workforce are integral to these efforts.
Research Objective: The goal of the current study was to explore the reliability of the questions on ASTHO’s Profile Survey by conducting cognitive interviews with a variety of personnel from twenty state health agencies.
Data Sets and Sources: The data source for this study is the written transcripts of the interviews with the state health agency personnel from the twenty state health agencies.
Study Design: All state health agencies were invited to participate in cognitive interviews assessing the reliability of the ASTHO Profile Survey. Phone interviews with twenty state health agency representatives were conducted by researchers from ASTHO and the University of Kentucky using interview guides.
Analysis: All interviews were digitally recorded and transcribed, with the transcripts uploaded into qualitative software and a detailed summary created for each interview. Each interview summary was reviewed by two team members - a primary and a secondary reviewer. Each transcript was coded and the software program was used to generate themes and key findings from the set of interview transcripts.
Principal Findings: Several themes emerged from the interviews confirming that there are specific sections or questions on the ASTHO Profile Survey that can be further refined to increase the reliability of individual states’ responses to the questions and the overall results for the survey.
Conclusion: By identifying questions in the survey that may be unclear to respondents or difficult for them to answer, ASTHO can continue to improve its ability to be the “go-to” resource on state public health.
Implications for Public Health Practice and Policy: This study contributes to the field of PHSSR by examining the reliability of a commonly-used instrument in the field of PHSSR. ASTHO’s Profile Survey plays an integral role in describing the state public health agencies. Improvements to the reliability of the ASTHO Profile Survey will increase the credibility and value of the data to researchers and practitioners alike.
Adam Atherly, Ph.D.
Reliability and validity of self-report questionnaires completed by public health agency staff
Co-Investigators: Sarah Lampe, M.P.H., Lisa VanRaemdonck, M.P.H., Julie Marshall, Ph.D., Melanie Mason, M.S.
Background: Much of what we know about governmental public health services is derived from self-report questionnaires completed by public health agency staff, such as the NACCHO Profile of Local Health Departments. Yet, little is known about the statistical reliability and validity of such data.
Research Objectives: The Colorado Public Health Practice-Based Research Network examined the reliability of data from a survey of Colorado local public health agencies conducted as part of a broader research project.
Data Sets and Sources: Two to four weeks after respondents completed the initial survey, they were asked to retake the survey. A total of 19 agencies participated in the reliability survey and were compensated for their time.
Study Design and Analysis: Test-retest reliability was examined using a Kappa statistic, with weighted Kappas for questions with more than two possible response categories. Kappas were calculated for a total of 167 different measures. Validity was tested using convergent and divergent validity.
Principal Findings: Results show variation in the reliability of individual service measures. Reliability for Communicable Disease activities was high except for questions about immunizations performed by others. For Population Health Promotion, reliability was high on questions about funding, tobacco programs and oral health, in the middle for pregnancy prevention and but lower for nutrition and physical activity. Some categories of measures showed higher reliability. For example, within Emergency Preparedness and Response, Kappas were high, with a median score of 0.61 and a range of 0.4 to .092. For Administration and Governance, all Kappas were above 0.71. For Assessment and Planning, only one Kappa out of eleven exceeded 0.45, with a median score of 0.40. Validity analysis will be completed in time for presentation.
Conclusions: Overall, reliability was highest for Administration and Governance and lowest for Assessment and Planning. Within other measures, capacity questions tended to have lower reliability while questions about the direct provision of services were higher.
Implications: As PHSSR continues to grow, and many researchers use survey methodology to collect data, it is important for us to embrace validity and reliability testing and to share its importance with practice partners.