The evaluation findings indicate that the heat illness prevention campaign was effective in reaching non-English speaking workers, community organizations and employers. Since this summer’s campaign took place over just a few months and evaluation was conducted immediately afterwards, this outcome evaluation focused on intermediate targets that need to be achieved on the path towards long-term change. This outcome evaluation looked at measures commonly used to evaluate health communication programs, including awareness of the communication, recall of the main messages, comprehension of the messages, and positive attitude toward the messages and the behaviors being promoted.

As a direct result of the heat prevention campaign, worker awareness of heat illness prevention has significantly increased. The large majority of workers (87%) who were interviewed about their exposure to the media campaign reported that they had seen or heard advertisements about heat at work, protecting oneself from heat or heat illness. When asked to recall main messages, nearly two-thirds mentioned drinking water on the job. About half also mentioned messages about taking breaks or stopping to rest, and nearly a third mentioned resting or taking a break in the shade.

The vast majority of respondents responded positively toward the heat illness prevention messages developed for the media campaign. Workers also thought the media campaign was relevant to them. Almost all (99%) thought the media campaign advertisements were useful. They also thought the ads were believable (93%) and important to their work (80%). Many workers who were interviewed expressed thanks and gratitude for the advertisements.
Workers also reported changes in behavior. More workers are drinking water, resting in the shade and talking with employers and supervisors about heat prevention. While it’s probably not realistically to presume that 90% of workers are in fact following these behaviors, as reported in the interviews, there was a statistically significant change between behaviors reported this summer and previous behaviors. Even with the recognized limitations of the data based on self-report and response bias (wanting to provide the favorable response), the increase in the percentage of these behaviors indicates the summer’s activities have had a positive effect on actions taken by workers.

Employers and community organizations also reported awareness of the media campaign. About half of the employers interviewed and 83% of community representatives reported exposure to campaign. The findings may be limited by the fact that the evaluation methods used did not directly target supervisors and crew leaders, and that 30% of the community organizations responding were from areas in Southern California in which media was not placed. Response to effectiveness of media varied between these groups. Over 70% of community representatives thought the ads were effective. Almost all (98%) thought the radio and billboard ads were useful and caught people’s attention. The majority also thought they were believable (95%) and relevant to workers (93%).
Employers reported mixed opinions with almost half stating the campaign was effective. There are a variety of reasons for which employers may not have felt it was effective. Those who provided feedback stated that they should have been more radio and billboard ads, as well as ads on television. Employers could also have been influenced by the fact that ads were worker-focused and included a direct number to Cal/OSHA.

The campaign was very effective in involving community organizations and others who serve as good access points to workers, as well as employers, in providing training and promotion of heat illness prevention. The majority of employers and community organizations found all of the materials developed for the campaign either extremely or mostly useful. After exposure to the campaign, between 75 and 91% of employers groups who responded to the online survey reported considerable understanding of the standard, their responsibilities, signs/symptoms of heat illness, what to do if someone suffers from heat illness and how to prevent heat illness. Some employer groups were proud of their participation and role in preventing heat illness. In addition, 91% of employers reported using campaign materials to train and provide information to their employees. There was also evidence that some employers and those responsible for work conditions had changed their work practices as a result of the campaign.

The outreach conducted to engage organizations in the campaign resulted in the involvement of many different types of organizations in extending the effort to reach workers. Almost all the organizations who participated in the Train-the-Trainer programs were involved in distributing materials, conducting workshops, showing the DVD and leading discussions, among others. Some were also able to assist workers to resolve heat problems by contacting Cal/OSHA, informing them of the law and referring them to other organizations for help. These organizations reached an estimated 6,000 – 8,000 workers.
The evaluation also identified challenges that constrain workers’ ability to adopt heat illness measures. Some workers reported that conditions had not changed this summer because their supervisors did not take the advertisements seriously or refused to provide the required water, shade and/or rest. Others said they were afraid to speak up or that they did not know how to report a problem. One worker reported that he/she was fired for helping a coworker who was experiencing heat illness symptoms. Community representatives commented that workers are still at risk of losing their jobs if they complain that their employers are not following the heat standard and that trainings and campaign materials did not address this fear. These comments indicate the need to continue efforts directed at employers, as they have direct control of conditions and promotion of a safety culture at their work site. In open-ended comments and in key informant interviews, both employer groups and community-based organizations recognized the important role of a consistent Cal/OSHA presence in the enforcement of the heat standard.

Finally, the heat illness prevention campaign appears to have raised the profile of Cal/OSHA so that employees and community representatives are more aware of who to contact if laws are violated. While the evaluation was not able to review actual number of calls made to the 877-99CALOR line, 9% of workers involved in intercept interviews stated they had called this number, and 6% of community representatives called the heat hotline themselves or knew a worker that called the number. About 20% of these same community representatives knew someone other than a worker that had called the heat hotline. Data is not available to assess worker satisfaction with the hotline, but 36% of the community representatives who called said the heat hotline was extremely or very helpful, while 46% said the hotline was only somewhat useful.