Journal of the NACAA
Volume 11, Issue 2 - December, 2018
Incorporating Evaluation into Program Development Processes for Women in Ag Conferences
- Schultz, M. M., Women In Ag Program Manager, Iowa State University Extension And Outreach
Hyde, C.J. , Graduate Student , Iowa State University
de la Mora, A. , Research Scientist , Iowa State University
Women in agriculture are an important audience for research-based educational experiences in Iowa. Women own nearly half of all Iowa farmland. Incorporating evaluation into the extension and outreach program development processes of needs assessment, educational design, scheduling and delivery of programs helps educators maintain a focus on audience needs. This article reviews the processes used to host a leadership conference for women. Evaluation was designed to gather program impacts and inform future program improvements. Survey respondents indicated the conference positively impacted them by inspiring their leadership journeys, increasing confidence, providing useful resources, helping them learn practical skills, and offering opportunities for peer support. Based on the conference evaluation, an advisory committee recommended lengthening breakout sessions to 75 minutes to allow for more in-depth discussion of topics important to the audience.
Women in agriculture are an important audience for Iowa State University (ISU) Extension and Outreach. Women work in, manage and own complex farms and agribusiness. Women own nearly half (47%) of all Iowa farmland. From 2004 to 2018, the ISU Extension farm management team provided 114 Annie’s Project courses and 61 other local multi-session farm management courses reaching 2,838 women. This experience with the audience of women in agriculture was the genesis for a conference focused on leadership. Whether leading from the farm family kitchen table or the agribusiness boardroom, women have important and changing roles in agriculture.
With thoughtful program development processes that include needs assessment, educational design, program scheduling, delivery of programs, and evaluation; educators can empower women to confidently take on new roles. With education and support, women involved in the business of agriculture can become influential leaders as they contribute to local and global food security.
To help inform extension educators interested in incorporating evaluation into planning activities or working with the audience of women, this article reviews some of the key program development processes ISU used to host the 2017 Women in Ag Leadership Conference. The focus of this article is on program evaluation and the lessons learned for future educational conferences designed for women in agriculture.
Rationale for Supporting Women as Leaders
As the roles of women in the agriculture industry continue to evolve, women are becoming more visible on family farms as well as throughout agribusiness and land grant universities. These women are involved in decision making on and off the farm and often have unrecognized expert knowledge (Alston, 1998). Women make significant contributions in the agriculture industry (Chayal and Dhaka, 2010).
In the state of Iowa, one in four (25%) farmers is a woman (USDA, 2014) and just under half (47%) of all Iowa farmland is owned by women (Zhang et al., 2018). This trend is likely to continue, for example, in 2017, more than half (52%) of undergraduates enrolled in the College of Agriculture and Life Sciences at Iowa State University were women (ISU Institutional Research, 2017).
Women are stepping into more leadership and decision making roles. A recent survey of more than 2,000 men and women found 90% or greater of both genders felt women were an integral part of agribusiness. A significantly higher percentage of women (70%) than men (60%) said they hope to advance to a higher-level position one day. (AgCareers.com, 2015). The 2017 Iowa farm and rural life poll shows women participated in financial management and estate planning at higher rates, while men tended to be more involved in more crop management and conservation decisions (Arbuckle, 2017). Men and women participated in livestock management, marketing decisions, and day-to-day decisions at similar rates.
With the increased demand from women for educational programming and a greater understanding of the value of women-centered education, more extension educators are designing research-based educational experiences for this audience. Women appreciate being part of program planning and they want to attend events and conferences designed with hands-on, participatory workshops and sessions. Viewing women and men as equal contributors to the agriculture industry and allowing women to have space to network and learn from each other are important factors when developing programs for women. Program evaluation can inform best practices for supporting women in the agriculture industry (Barbercheck et al., 2009).
Evaluation outcome data from extension-led conferences and courses demonstrate the importance of supporting women’s learning as well as connecting them to a broader network. A five-year study of participants in the University of Nebraska’s women in ag conferences concluded the mentoring occurring through lasting friendships among conference participants improved agricultural operations, enhanced personal well-being and strengthened agricultural communities (Schlake and Vogt, 2017).
Role of Evaluation in Program Development
The ISU program development process follows the basic concepts of product development and instructional design. Below are the key steps used to develop the women in ag leadership conference.
1. Needs assessment
- Bring together stakeholders
- Describe intended audience and their needs
- Identify resources
2. Educational design
- Create overall goals
- Develop measurable learning objectives
- Plan research-based educational experiences
3. Program scheduling
- Design agenda, timeline and activities
- Interview/vet possible speakers
- Communicate about and promote program
4. Program delivery and implementation
- Present information and engage learners
- Incorporate discussion and activities
- Create opportunities for networking and informal learning
5. Evaluation and reporting
- Measure learning, actions and changes in life conditions
- Analyze and summarize data
- Publish stakeholder report
Evaluation is a beneficial and exciting component of each program planning step, beginning with needs assessment and continuing through to analyzing data and sharing the results. It plays an important role in the development and implementation of programs through continuous improvement, advancement and building of knowledge, and determining impact. Various evaluation approaches can be utilized in guiding the development and implementation of programs. Decision-oriented approaches emphasize the use of evaluation outcomes to help make informed decisions (Fitzpatrick et al., 2011). This can provide information about the value and impact of the program, and whether the program is on target to meet goals. Evaluation outcomes can be used to consider the effectiveness of the education provided and share program impacts with stakeholders. Survey data from previous educational programs can be used as building blocks for future women in agriculture programs.
Involving Stakeholders in Planning and Evaluation
Stakeholders were brought together through a leadership conference advisory committee. The committee was made up of 20 women of different ages and types of experiences including niche and commodity farmers, women working in agribusinesses, cooperatives and USDA agencies, and women working in extension.
The advisory committee was given the following tasks.
- Establish goals for the conference;
- Guide types of conference activities;
- Recommend topics and speakers;
- Promote conference within sphere of influence;
- Take visible leadership role during conference; and
- Review results and make recommendations for next year.
The advisory committee established the following conference goals.
- Inspire women’s leadership journeys;
- Teach practical skills;
- Increase leadership confidence;
- Provide resources and tools; and
- Grow networks and friendships;
To further develop the educational content of the conference, the advisory committee shaped the speaker topics by identifying two to three key learning objectives. The conference goals and leaning objectives were central in planning and evaluating the conference.
The Iowa State University Extension and Outreach Women in Ag Leadership Conference was held November 27-28, 2017, in Ames, Iowa. Monday (Nov. 27) included an afternoon tour of the ISU Veterinary Medical Center and central campus. In the evening, 98 registered participants and speakers attended one of the three 3-hour intensive workshops offered, which included dinner. An evening social concluded Monday’s activities. On the following day, 200 registered participants and speakers attended the main day-long conference. The program included individual speakers, panel speakers, awards, luncheon and networking opportunities. There were 60-minute breakout sessions in the morning and 30-minute breakout sessions in the afternoon.
The leadership conference evaluation was designed to assess participants’ perceptions regarding their overall conference experiences and to determine how well the conference met the goals and learning objectives established by the advisory committee. Following the conference, participants were contacted by email and invited to complete an online survey consisting of 13 general questions and 42 knowledge questions. General questions were asked around each program development process to determine the effectiveness of conference planning as well as to inform planning for future conferences. The survey used skip logic to assess whether respondents attended a session and if they did, how they rated their knowledge gained.
Questions on Needs Assessment
To better understand the audience, the evaluation asked survey respondents to identify their role in agriculture (e.g., farmer, working in agribusiness, student,) and the number of years involved in agriculture. To assess the targeted audience outreach, respondents were asked to indicate the extent with which they agreed the women-centered environment of the conference was important to them, to others, and to them in the future, on a Likert scale of 1 (strongly disagree) to 4 (strongly agree.). To gain further insight regarding respondent’s experiences, they were asked an open-ended question about why they attended the conference.
Questions on Educational Design
Survey respondents were asked about their learning outcomes from attending the various sessions during the conference. Questions for each session were designed to assess knowledge changes for two or three key learning objectives. A total of 16 sessions (e.g., Becoming a Cooperative Board Influencer, Discover Your Ag Leadership Strengths) were offered during the conference. Respondents were asked to rate their change in knowledge as a direct result of the session on a Likert scale from 1 (no change) to 4 (greatly expanded). Another way to assess knowledge gains and value of the research-based educational experiences was to ask respondents an open-ended question about their top three take-away messages.
Questions on Scheduling
Two questions asked for preferences on the month and day of the week for next year’s conference. Participants were also asked in an open-ended question to share how they heard about the leadership conference to help assess marketing strategies. Other open-ended questions asked for suggestions on speakers or topics for next year and suggestions for improving the conference.
Questions on Program Delivery
An important aspect of the conference was the goal of advancing and building leadership capacity and knowledge. Respondents were asked to indicate the extent with which the conference strengthened their leadership capacity on a Likert scale of 1 (No) to (4 (Significantly) in each of the following areas.
- Inspiring your leadership journey;
- Giving you practical skills you can use;
- Increasing your confidence;
- Giving you leadership tools or resources; and
- Expanding your network and friendships.
Assessing the quality of professional development provides information that allows organizers to make informed decisions towards continuous improvement. In particular, the conference advisory committee was interested in determining whether the conference met the needs and expectations of participants. Participants were asked to rate the quality of professional development at the conference on a Likert scale ranging from 1 (poor) to 4 (excellent) in each of the following areas.
- Quality of resources and materials provided
- Interaction among participants
- Interaction with speakers
- Effective use of time
- Selection of topics
- Applicability of information
- Organization of the conference
- Overall quality of the professional development
The final open-ended question asked if there was anything else respondents wanted to share.
Following the leadership conference frew 200 registered speakers, committee members and attendees. Following the conference, there were 120 women who responded to the online conference survey. The following section presents the results of the responses. The respondents offer insights on why women want to attend these types of events. They tell a story of what the most important aspects of the conference were to them and how to improve future conferences.
Results Relevant to Needs Assessment
When asked to identify their roles, nearly half (49%) indicated they worked in agribusiness, and almost one-third (32%) indicated their role was as a farmer and/or landowner, and a smaller portion were students (10%) and/or non-profit employees and educators (9%; Figure 1). Of these participants, more than half (58%) had been involved in farming, agribusiness or agriculture 10 years or less (Figure 2).
Figure 1. Participants’ Role in Agriculture.
Figure 2. Number of Years Involved in Agriculture.
Overall, the majority (90.1% - 99.0%) of participants indicated the conferences’ women-centered environment was important to them, to other participants, and to them in the future (Figure 3).
Figure 3. Participant’s Perceived Importance of a Women-centered Environment.
Participants were asked to share why they attended the women in ag leadership conference. Nearly 40% of participants commented networking was their primary interest in attending the conference; with many stating they looked forward to visiting with other women in the agricultural industry. About 22% of respondents indicated learning more about leadership and having an interest in specific topics offered was the top reason they attended. Several participants indicated they wanted a better understanding of the changing roles of women in agriculture, especially leadership roles. Representative responses about why women attended:
- “Working professionally outside of the ag industry, I don't have too many chances to connect with women in agriculture, this was a conference that allowed me to do that and still receive professional development to help me in my off-the-farm job.”
- “I enjoy attending women in agriculture conferences, but this one had more learning experiences than others I have been to. The sessions all really interested me and I knew that it would provide learning experiences and networking opportunities.”
- “I am interested in interacting further within the agriculture community in Iowa and felt this would be a welcoming environment to do so.”
Results Relevant to Educational Design
Participants were asked to rate their level of knowledge improvement for any of the 16 conference sessions they attended. They responded to specific learning objectives for each session on a Likert scale of 1 (no change) to 4 (greatly expanded). Overall, participants’ knowledge improved across all sessions (mean ranged from 2.36 to 3.35). To further describe the questions about knowledge changes for the stated learning objectives, the survey results are shown below for the keynote and capstone sessions (Table 1).
Table 1. Level of Participant Knowledge Improvement For Keynote and Capstone Sessions.
Participants were asked to share their top three takeaway or key messages from the conference. The results were analyzed by grouping common themes or topics found among responses. Many participants shared specific quotes from speakers and several commented they were surprised by things they had not realized were important to other women in agriculture. The most common takeaway message, indicated by more than two-thirds (36%) of respondents, was having greater confidence to get involved in leadership roles. The second most noted takeaway from the conference was an appreciation of women’s leadership roles in agriculture, with nearly one-third (30%) of participants referencing this. Participants indicated supporting other women in agriculture is very important. Communication, critical conversations, listening, asking questions, and saying no when needed were also common themes identified by respondents. About one-fourth (25%) of respondents indicated improved communication skills were one of their top takeaways from the conference. Below are representative participant statements.
- “Confidence, if you are invited to a meeting you are meant to be there and your voice should be heard.”
- “Women's contributions are visible in an array of agricultural fields.”
- “Three rounds of ‘Listen and Ask’ before you share, to build trust and relationships.”
Results Relevant to Scheduling
Participants heard about the conference primarily through social media outreach efforts (e.g., Facebook). Many participants also indicated they received emails, saw advertisements, or received an invitation in the mail. Others indicated they learned about the conference through word of mouth (e.g., they were told about the event or invited to attend by others). When asked about the month and day of the week for future conferences half (50%) of the respondents preferred November, and nearly half (43%) of the respondents preferred Tuesday.
When asked about how the conference can be improved for next year, respondents most often indicated timing and agenda preferences. While some women commented they enjoyed the 30-minute breakout sessions, a majority indicated these sessions were too short to fully engage in learning and benefit from the leadership skills being discussed. Respondents indicated a preference for beginning Monday’s conference activities earlier in the day and ending earlier in the evening (the intensive workshops were from 5:30 to 8:30 pm). Women also made suggestions related to deepening the learning. They liked the strong calls to action some speakers made and suggested more speakers motive next steps and leadership growth in that way. Respondents also indicated the take home handouts and worksheets were helpful and they wanted more of these.
Respondents suggested having an overarching theme for each conference would be helpful. They suggested a variety of timely and wide-reaching topics for future conferences. Some wanted to hear from more women managing farm businesses and how to work with family members. Others were interested in retirement and financial planning, estate planning, and conflict resolution skills. It was also suggested to include a session on human resources, maternity leave, and regulations or rights specific to women. Many echoed the sentiment of having strong successful women as role models to show it can be done.
Results Relevant to Program Delivery
Overall, the majority of participants indicated the conference provided positive experiences (Figure 4). The majority (82.4% - 93.9%) of participants indicated the quality of resources provided at the conference, opportunities to interact with other participants and speakers, effective use of conference time, selection of topics offered, the applicability of information provided, and conference organization were good (41.2%-51.8%) or excellent (30.7%-47.4%). The majority (90.4%) of participants indicated the overall quality of the conference was good (43.9%) or excellent (46.5%).
Figure 4. Participants' Ratings of the Quality of their Conference Experience.
Overall, participants indicated they strengthened their leadership capacity as a result of their conference experiences (Figure 5). The majority (72.0% - 75.5%) of participants indicated the conference inspired their journey (or next steps), gave them practical skills they can use, increased their confidence, gave them leadership tools/resources, and helped to expand their network and friendships moderately (31.6% - 40.4%) or significantly (31.6% - 42.1%).
Figure 5. Changes in Strengthening Aspects of Leadership Capacity after Conference.
Women offered intriguing comments when asked what else they would like to share in a final open-ended question. There were very different perceptions of how the overall topic of women in agriculture was addressed as shown in the following representative quotes.
- “I think it's important to explicitly talk about the barriers that women face and how to overcome them…and invite respect for women's contributions. This is a key part of women's leadership, and I feel uncomfortable having a leadership conference of, by, and for women without at least acknowledging that we face specific barriers.”
- “I was very disappointed in the tone of this conference. It felt like we were all being stereotyped as women who are afraid to speak up for ourselves, lack confidence in our abilities and that the men in agriculture are trying to keep us down. As a woman in the ag industry I can say that this has not at all been my experience.”
Overall, women completing the 2017 ISU Extension and Outreach Women in Ag Leadership Conference surveys valued the educational and social opportunities afforded through the event. Women strengthened their leadership capacity, valued the women-centered learning, gained knowledge, networked with others in the industry and rated the quality of the conference highly. The survey results indicate incorporating evaluation into the program development processes of needs assessment, educational design, scheduling and program delivery was effective in helping the conference planners meet the professional development and peer support needs of the audience in 2017.
It was important to involve stakeholders in the conference planning activities. This assured the conference topics were relevant to the intended audience of women farmers, women in agribusiness, and others interested in women’s agricultural leadership. Bringing together an audience with varied backgrounds, roles, and interest in agriculture created a rich learning environment and heightened awareness of women’s changing roles. The different perceptions conference participants had about the overall topic of women in agriculture indicates that for some women, women's changing roles in the industry can still be uncomfortable.
Survey respondents indicated the conference's research-based educational experiences positively impacted them by inspiring their leadership journeys, increasing confidence, providing useful resources, helping learn practical skills, and offering opportunities for friendships and peer support. Discussion, hands-on activities, and in-depth learning were important to survey respondents who made it clear they wanted more of this. A majority of survey respondents stated the main reasons they attended the conference were to connect with other women in the agriculture industry and to grow their own leadership skills and opportunities. This bears out the planning team’s experiences as well as the research shared earlier in this paper.
One of the top takeaway messages from the conference was how much women are influencing the agricultural industry through leadership roles. The number of survey respondents that were surprised by the changing and powerful roles of women indicates a need for more visibility of women in the industry.
The online evaluation surveys allowed respondents time to provide thoughtful written responses. This was important in giving the leadership conference advisory committee a broad view of what was successful and what needs to be improved. The mixture of questions based on learning objectives, overall conference experiences, and open-ended feedback gave the committee insights into several different areas of the conference.
A report of the survey results was shared with the advisory committee in December of 2017. These stakeholders found the evaluation outcomes useful in making informed decisions. Combined with their own personal experiences of the conference, the survey results gave the committee building blocks to begin planning conference improvements.
Some of the key recommendations for the next conference and research-based educational experience for women in agriculture in Iowa are summarized below.
Recommendations for Needs Assessment
- Continue gathering audience needs through an advisory committee.
- Maintain a women-centered learning environment focused on leadership.
- Offer education around farm transition planning, communication, and career planning.
Recommendations for Educational Design
- Identify the next conference theme around communication. Include farm family transition. planning, conflict resolution, career advice and becoming a leader in agricultural organizations.
- Ask all speakers to consider preparing take-home handouts, teaching a specific skill, and making a strong call to action with practical next steps.
Recommendations for Scheduling
- On Day 1, continue to offer travel and attendance flexibility for participation in optional tours and intensive workshops. Begin and end earlier.
- On Day 2, eliminate 30-minute breakout sessions. Lengthen all breakout sessions to 75 minutes to allow for more in-depth discussion on topics of important to the audience.
- On Day 2, shorten general sessions from 120 minutes to 75 minutes.
Recommendations for Program Delivery
- Continue to implement a comprehensive online survey after the conference that includes knowledge changes around the learning objectives.
- Offer a planned reception area to greet Women Impacting Agriculture honorees.
- Plan for both guided and informal (on your own) networking activities to facilitate new friendships.
This article provides an example of how program evaluation can be incorporated into the program development processes of needs assessment, educational design, scheduling and program delivery. Extension educators may find it useful to adapt some of these ideas for using outcome based evaluation to guide their own program planning.
This article provides an example of how program evaluation can be incorporated into the program development processes of needs assessment, educational design, scheduling and program delivery. Extension educators may find it useful to adapt some of these ideas for using evaluation outcomes to guide their own program planning.
Arbuckle J.G. (2017). 2017 Iowa farm and rural life poll summary report. Iowa State University Extension and Outreach PM 3075. Ames, IA. Available at: https://store.extension.iastate.edu/product/Iowa-Farm-and-Rural-Life-Poll-2017-Summary-Report
Alston, M. (1998). Women the silent partners of agriculture. Proceedings of the 9th Australian Agronomy Conference, Australia (pp. 31-36). Wagga, Wagga, Australia: Australian Society of Agronomy, Inc.
AgCareers.com. (2015). Gender Roles & Equality in Agribusiness. Retrieved August 9, 2018 from https://www.agcareers.com/track-report-downloads.cfm?ID=64.
Barbercheck, M., Brasier, K. J., Kiernan, N. E., Sachs, C., Trauger, A., and Findeis, J. (2009). Meeting the extension needs of women farmers: A perspective from Pennsylvania. Journal of Extension, 47(3), 1-11.
Chayal, K., and Dhaka, B. (2010). Analysis of role performance of women in farm activities. Age, 30(60), 30.
Fitzpatrick, J.L., Sanders, J.R., and Worthen, B.R. (2011). Program evaluation: Alternative approaches and practical guidelines. New Jersey: Pearson Education.
Iowa State University Institutional Research. (2017). Iowa State University Fact Book. Retrieved August 9, 2018 from https://www.ir.iastate.edu/factbook/2017.
Schlake, M., and Vogt, R. (2017). Women in agriculture survey – 2017 report. Retrieved August 9, 2018 from https://wia.unl.edu/documents/2017-wia-report.pdf.
USDA National Agricultural Statistics Service. (2014). 2012 Census of Agriculture. Retrieved August 9, 2018 from https://www.agcensus.usda.gov/Publications/2012.
Zhang, W., Plastina, A.., and Sawadgo, W. (2018). Iowa farmland ownership and tenure survey 1982–2017: A thirty-five year perspective. Retrieved August 9, 2018 from https://www.extension.iastate.edu/agdm/articles/zhang/ZhaJul18.html.