SESSION 3B: Information & Technology-Measurement & Evaluation
Room: Thoroughbred 2
Wednesday, April 9, 2014, 10:45 am-12pm
Back to Conference Agenda
AUDIO
KC14 Session 3B on Information & Technology: Measurement & Evaluation by NCC for PHSSR
MODERATOR
Anna G. Hoover, Ph.D.
PRESENTERS
Britney Johnson, M.P.H.
What Gets Measured is What Gets Managed: Evaluating HIV/STD Partner Services Performance
Co-Investigator: April Richardson-Moore, M.P.H
Background: Previous research by the New York State (NYS) Public Health Practice-Based Research Network into NYS's integrated HIV/STD Partner Services (PS) program indicated a need for quality evaluation assessment of PS intervention activities. In response, NYS designed and implemented a program management application (PMA) to better measure the processes and outcomes of HIV/STD PS work from a quality improvement perspective.
Research Objective: To utilize data collected through PMA to accurately assess staff workload, monitor program performance in relation to SMART objectives, and identify areas for improvement.
Data Sets and Sources: Eligible HIV, syphilis, gonorrhea, and chlamydia index cases worked by PS disease investigators (N=36) in five regional offices across NYS were selected for analysis. Data were collected and entered in the PMA by regional staff between January and December 2013.
Study Design: Preliminary analyses assessed the association between the rate of successful index patient interviews and the timeframe from case assignment to index patient interview. Program metrics were assessed against performance objectives established in the NYS Tasks and Standards for HIV/STD Partner Services.
Analysis: Eligible cases (N=5,676) were stratified by disease (HIV, early syphilis, gonorrhea, or chlamydia), interview status, and interview time frame. Chi-square analyses were conducted in SAS 9.2 to identify statistically significant differences at the bivariate level.
Principal Findings:HIV index cases were significantly less likely (P
Conclusions: HIV index cases assigned for PS had significantly worse outcomes than those for other STDs. Interventions to improve the quality of PS program delivery should focus on improving HIV case interview rates.
Implications: Despite operating under an integrated program, HIV/STD PS investigations still have disparate case outcomes. Identification of barriers to effective HIV PS are critical to improving program performance and reducing disease transmission.
Jenine Harris, Ph.D.
Local health department engagement with other local health departments on Twitter
Co-Investigators: None Listed
Background: Local health departments (LHDs) using Twitter have few followers relative to jurisdiction size, reaching the equivalent of only approximately three in every 1,000 constituents living in the jurisdiction as of 2012. However, LHDs tend to be followed by other LHDs on Twitter at a relatively high rate, forming a large national network with the potential to facilitate information exchange among LHDs. Because follower relationships on Twitter are passive, it is unclear whether LHDs are engaging with information sent by other LHDs on Twitter.
Research Objective: To understand whether the network of LHDs on Twitter is facilitating active information sharing among LHDs.
Data Sets and Sources: All tweets sent by 284 LHDs across the United States were collected using the NVivo NCapture tool. Tweet data were integrated with NACCHO Profile Study data for LHD jurisdiction size, expenditures, and staffing.
Study Design: Mixed-methods were used to examine two things: (1) the structure and composition of the retweeting network among LHDs on Twitter, and (2) the content of LHD tweets retweeted by other LHDs.
Analysis: Descriptive, visual, and statistical network methods were used to examine the retweeting network. Thematic analysis was used to examine tweet content.
Principal Findings: Of 162,670 tweets and retweets sent by 284 LHDs as of August 2014, 1,124 were originally sent from one LHD and retweeted by another LHD. The network of retweets included 140 LHDs, 107 of which had retweeted from another LHD. This indicates that more than one-third of LHDs on Twitter are actively engaged with information other LHDs are sending.
Conclusions: Many LHDs are engaged with information sent by other LHDs on Twitter.
Implications for Public Health Practice and Policy: Given high rates of interconnectedness among LHDs on Twitter, and moderately high rates of engagement of LHDs with information tweeted by other LHDs, the current use of Twitter by LHDs to disseminate health behavior information to individuals may be inefficient. Instead, existing connections between LHDs could facilitate information-sharing about effective programming across a national network, providing LHDs with practice-based evidence from peer organizations to improve local public health.
Nancy Winterbauer, Ph.D., M.S.
Challenges to local health department – media engagement for health promotion: The North Carolina County Health Rankings as a case example
Co-Investigators: Katherine Jones, Ph.D., Ann Rafferty, Ph.D., Mary Tucker-McLaughlin, Ph.D., Colleen Bridger, Ph.D.
Background: Media attention is a central component of the County Health Rankings (Rankings) logic model. However, the role traditional media play in health promotion is not well known. Constraints to media – local health department (LHD) engagement, using the North Carolina 2012/2013 Rankings as an example are explored.
Objectives: We sought to describe: 1. Statewide variation in media presence; 2. LHD capacity to engage media; 3. LHD interaction with media; 4. Media representations of the Rankings.
Data Sources: Media coverage was obtained from the NC Press Association and FCC. LHD data were obtained from an online survey of leadership (n=55). News stories were derived from the Standard Rate and Data Service media database.
Design: We used an integrated mixed-method approach and a cross-sectional study design.
Analysis: Media coverage was analyzed at the county level based on the presence of newspapers and television signal coverage areas, measured via static FCC maps and a GIS analysis of overlapping signal coverage areas. Descriptive statistics depicted LHD staff capacity and interaction with media. Multiple coders examined news stories through quantitative and qualitative content analysis.
Findings: 42% of counties have at least 1 daily and 76% have at least 1 weekly newspaper. 24% have no newspaper at all. Counties range from 2 to more than 10 television stations. 38% of LHDs attempted to engage media, 40% issued a press release and 36% gave interviews. 11% reported that media staff devoted half or more of their time to media relations. 25% of counties received no media coverage; 61% of news stories did not contain an interview with LHD staff. In the majority of stories (n=76%), personal responsibility for poor health was emphasized.
Conclusions: Variation in media density and under-resourced LHD staff capacity likely impacts the potential for LHD engagement with traditional media for successful health promotion efforts.
Implications: Traditional media can be a powerful ally in health promotion. LHDs should consider communication specialists as central to health promotion strategies. However, health communication is under-resourced at the local level and ought be considered in workforce development. LHDs in media-poor environments may need to emphasize non-traditional communication strategies.
Ramakanth Kavuluru, Ph.D.
Diffusion of Health Related Information in the Twitterverse
Co-Investigator: Gokhan Bakal
Background: The asymmetric network structure and the 140 character limit per tweet have made Twitter popular for spreading health related information online. Users can choose to 'retweet' and spread particular tweets from other users regardless of whether they follow them on Twitter. As such, a tweet can spread through the Twitter network using the follower-friend edges.
Research Objective: To quantitatively measure how health related tweets that contain URLs spread in the directed follower-friend Twitter graph using the retweet mechanism.
Datasets: Using a selected list of nearly 50 health related keywords and hashtags, we curated a large set of over 50 million tweets. However, not all tweets are relevant owing to polysemy, homonymy, and sarcasm. So we manually chose nearly 200 tweets that contain a URL (since URLs indicate the source) each with at least 25 retweets.
Study Design: For each tweet, we built the corresponding retweet graph with the original user and all users who retweeted it. The edges correspond to the follower-friend connections between these users where the retweet time of a follower is chronologically later than that of a friend in the graph.
Analysis: For each of these graphs, we computed node distributions and a diffusion metric at each level of the tree formed by using breadth first search. We also computed the proportion of nodes in the main connected component that involves the original tweet author.
Principal Findings: On average 30% of the retweets are NOT in the main connected component that involves the original tweet author which indicates that people sometimes seek health information on Twitter by searching for topics and then retweet essential information even if they are not following the person who tweeted it originally. Also, 98% of the retweets in the main connected component do not spread beyond the 2nd level and only 0.85% of followers retweet at the first level.
Conclusions and Implications for Public Health Practice and Policy: Users actively seek health related information on Twitter. A more automated way of identifying health related tweets can help generate strategies for health agencies in maximizing diffusion of health related information online.