Brief Summary of Minutes of Annual Meeting

The meeting was called to order at 8:00AM MST by Rose Judd-Murray, chair.

Note: General information and objectives for this Agricultural Literacy Multistate Research Project (W3006) are available at https://www.agliteracy.org/research/multistate.cfm

The official site for this project is  https://www.nimss.org/ where project details including participants and minutes are on file. Use “3006” in search.

Please make sure to double-check that your name is on the participant list along with your selected objectives.

Objectives:

1. Assess the agricultural knowledge of diverse population segments related to agriculture, including consumers, students, and producers. Specifically, explore and/or measure: a. points of acquisition of agricultural knowledge; b. decisions made based on assessed knowledge.
2. Assess attitudes, perceptions, and motivations of diverse population segments related to agriculture, such as consumers, students, and producers. Specifically, explore and/or measure: a. how perceptions, attitudes and motivations are developed; b. decisions made based on assessed attitudes, perceptions, and motivations; c. behavior changes that have occurred due to changes in attitude, perceptions, and/or motivation.
3. Evaluate agricultural literacy programs to measure program impact. Specifically:
   a. measure impacts of agricultural literacy programs related to critical thinking and problem solving; b. explore and evaluate peer and participant-centered agricultural literacy programming methods to determine their effectiveness in addressing defined agricultural literacy outcomes.

• Minutes from the prior meeting
  • Spielmaker moves to accept the minutes, with the edit of changing the date to 2019 in Anchorage and not in 2020.
  • Second by Denise Stewardson
  • Motion to accept, accepted
    • Brian or Rose will amend the minutes from the previous meeting (done)

Updates from current W3006 projects

Rose Judd-Murray-USU

• Will building a new common IRB to measure multistate ag literacy help ease workload and collaboration. The old IRB we have been using lapsed in the spring.
  • Question if it was helpful/ worth it to have a common IRB
  • Question if it is more accessible to all who have solo IRBs
    • Spielmaker- question about additional demographic data collection, let people know what questions you are asking to increase consistency and reliability with the instrument
  • Rose will update the common IRB, taking suggestions for additions/ changes to IRB
• Multistate research project with Kansas state and NMSU to look at Extension professionals using the JMALI assessments to measure ag literacy in extension professionals and primary volunteers. Common IRB was created and collected lots of data – USU extension funded incentives. In all three states, there was consistency in levels of ag literacy. They were in the literacy in proficiency category. 75% of individuals surveyed were women and primarily in early careers. Rose would like to continue to explore this research with other states

Josh Stewart-OSU

• Introductions of additions to OSU team, recent IRB submission- population being all incoming College of ag students. Partnership with ag in the classroom for specialty crop grants and virtual field trips. They need to follow up with their partner, Jessica. COVID created lots of challenges regarding the development of this project. Rose asks the ideal number of students they would like to survey- Proposed 800. She is expecting to have at least 300-500 incoming.

Jonathan Valez- OSU

• Virtual field trips for Ag in the classroom programming took off. Surprised by how many students (300,000) participated. Summer ag institute, trying to do regional spin-offs of the program- trying to get some other programs started that are regionally affiliated to work with smaller institutions such as community colleges. They are starting an Ag Comms program up at OSU, new hires, etc. Currently a minor but to make a major in a few years. Ag Sciences and Natural Resources Communications.

Kellie Enns & Jenny Bennett- CSU

• Three significant initiatives in ag literacy- Grad students who took a yearlong ag lit curriculum and administered LMALI- showed significant growth in ag literacy. The second piece that was explored could we improve a student's desire to be involved in agriculture (affinity)—an exploration into affinity and Ag Lit. Second space, CSU put 3 buildings in Denver (food, animal health, water)- CSU Ag Ed has a learning lab, from a gift of 3 million dollars to provide more accessible ag literacy education. PBL will be administered to center to secondary ag programs to address critical issues in agriculture. CSU team has developed and facilitated a badging program for intensive ag literacy experiences at Ag events such as fairs and shows to become Junior Ag Ambassadors, a great way to include FFA members and other students from the College.
  • Rose asks about evaluation at this learning lab, and Kellie is unsure of the methods at this point.
• The first PBL will launch in January, along with two hires. State Officers live in this space and will be doing a lot of the interpretive work of the SPUR center. CAMs on the road, working with USU and CSU on various projects
• Kellie suggested cross-state internships to allow non-formal students to have some internship experience.

Deb Spielmaker-USU

• A reminder that NCAL has lots of data sets and resources for collecting data. JMALI will be a Kahoot at FFA National Convention this year (about 23,000 FFA members). A unique wristband will be awarded, Indiana person will be operating the booth for Kahoot. The Ag literacy certification project is a big project Spielmaker is working on right now
• Michelle Burrows- USU
• Worked on growing a nation c3 framework alignment, working with Jenny B and Amelia M (grad students in CTE Ph.D. program) on state data from 2021 Ag in the Classroom state program reports. The USU team is Interested in surveying FFA members using the JMALI to measure their agricultural literacy. There is an interest in looking at what works on social media to get people to go to the matrix and download the lessons. Also, working with Texas ag in the classroom and measuring various data points

Nellie Hill & Gaea Hock- K State

• November, December 2022 JAE magazine theme is ag literacy- the editor is Kellie Enns. Gaea developed a cultivating ag literacy class at K state. They worked with Rose to collect data from undergrad students and extension professionals/ certified volunteers. Clarkson farms research- Amazon has a show called Clarkson farms. K State is looking at how this serves as a gateway to agriculture.

Quincy Clark- OSU

• New member excited to collaborate on creating a curriculum for underrepresented populations to teach problem-solving within an Ag context. The curriculum is for grades 4-6.

Whitney Stone- OSU

• Also, new member and excited to collaborate

Amelia Miller-USU

• Finished major project measuring Ag Literacy assessment in Iowa. She is also working with Jenny B on state reports of NCAL programs in 2021 and looking at a trend analysis of NCAL programs over the last 10 years. Brian Warnick and Deb Spielmaker conducted one from 2006-2010, so this would pick up after that. Dissertation, working on ag literacy certification program

Carl Igo- MSU

• At MSU, continuing to work on the USDA NIFA grant for a retrospective approach and looking for collaborators, ready to roll out the second phase of the project- IDing K-12 teachers who are effectively using the Ag literacy standards and integrating them. He would like to make 8–12-minute videos to engage students to be used as an instructional strategy video
• Two math teachers, two science, two SS, and two ELA- a total of 8 teachers. Videographers will come to them, OR they can be flown out to next summer's conference (perhaps).
• Preference for teachers who have been national award winners from the NCAL conference. But want to concentrate on teachers that integrate Al Lit outcomes to NGSS standards. Spielmaker says that she can help push that out through the listserv
• Reporting on Behalf of Shannon Arnold on agritourism, going across the western part of the US to help connect smaller production operations with the idea of agritourism, Western SAR grant (Deb S adds that this applies to W3006 objective 1)

Kevin Curry- Penn State

• Interested in using the JMALI instrument with preservice students and more general education students to measure ag literacy. Developed a similar ag literacy assessment for K-2^nd grade level and have partnered with Elementary schools' challenges because COVID has postponed data collection

Denise Stewardson- USU

• Specialty crop block grant, paired with nutrition education. Got an extension grant also- paired nutrition resource boxes focused on grades 3-5 to put together nutrition resources rooted in agricultural literacy. Had flipped data collection to virtual- Now working on evaluation to investigate how nutrition education impacts agricultural literacy. "Crops in a box."

Jamie Greig- UT Knoxville

• Reporting for Shelly, the research area is CIS technology and communications in agriculture. Working with Rose to develop ag literacy extension programming and evaluate instrument reliability. They are looking to evaluate adult consumers and their ag literacy. Wants to work with Soybean board and evaluate consumer understanding of soybeans. Work with commodity groups to develop yearly campaigns to understand ag literacy related to that specific commodity and develop a tool kit to evaluate understanding. They are also working on a project that involves rural libraries and 4-h clubs and how they can work together to improve CIS knowledge. NIFA PDAL grant submission, working with Nebraska Lincoln- to recruit high school ag teachers and show them how to use VR to create agricultural experiences.

Brian Warnick- USU

• changed positions to Department head of technical education.

Closing remarks

• Brian Warnick- Reminded the group that an annual report needs to be submitted. It would be helpful to include in the report is how projects align with the W3006 objectives. Jenny needs to include how projects align with the report's objectives.
• Carl Igo: has anyone done any work evaluating retention in agricultural colleges using the JMALI survey? (CSU has a 5-year study) Curious about the possibility of the upcoming round of USDA NIFA grants to measure ag literacy amongst indigenous populations? Using the instrument with international graduate students? – Rose to work with individuals in Australia
  • Deb S- Rose JM should send a survey to ask people to report their activities under each objective
• The movement to adjourn by Deb S
• Accepted
• Meeting concludes

The W3006 has completed its third year working on agricultural literacy research. The committee has focused this year on efforts that strengthen programs and research studies through multi-state collaborative efforts. Several members of the committee have worked together to measure the agricultural literacy levels of Cooperative Extension professionals and their primary volunteers across multiple states; others have worked together to evaluate groups of college-aged students. The progress on these efforts is noted in the minutes from the fall 2022 meeting. Accomplishments and limited impacts are reported under each of the W3006 objectives below. The committee is optimistic that we can continue collaboratively to increase the reach of impact-based evidence. 

Accomplishments by Objective

Objective 1: Assess the agricultural knowledge of diverse population segments related to agriculture, including consumers, students, and producers. Specifically, explore and/or measure: a. points of acquisition of agricultural knowledge; b. decisions made based on assessed knowledge.

Accomplishments: Outputs & Milestones

1a. Using the Judd-Murray Agricultural Literacy Instrument (JMALI), data was collected and/or analyzed during the spring of 2022 for three states, Utah, Kansas, and New Mexico. Utah State University (Rose Judd-Murray), Kansas State University (Gaea Hock & Nellie Hill), and New Mexico State University (Shannon Norris-Parish) worked collaboratively to determine how Cooperative Extension professionals and primary volunteers performed on the agricultural literacy assessment. Arizona (Jeremy Elliott-Engel, University of Arizona) was slated to participate in this multi-state effort, but their process was delayed.

In the full study, participants had a foundational awareness of agricultural production as noted by their agricultural literacy equaling factually literate (≥ 8 correct answers) or higher. In New Mexico (N = 41), 51% of professionals (n = 21) scored an applicable proficiency answered ≥ 12 questions correctly, which was the highest percentage among the states. The volunteer data from New Mexico was an exception, however, as the low response rate (n = 5) for this population is not representative of the population. The low response rate was due in part to a new system that used a non-centralized volunteer database. Interestingly, most employees in all career stages and states worked for Extension less than three years. These trends could also represent how long professionals have served in a particular role for the organization, as most reported participating for five years or less. More notable, however, is there were no significant relationships in any career stage or years of service related to the agricultural literacy proficiency scores. Comparatively, the length of time multi-state volunteers served had no significant effect on their proficiency scores. While these results are somewhat reassuring to know literacy levels were consistent, we question why there is not a difference among career stages. Because many of the participants had been in the profession for three years or less, we wonder if their agricultural literacy levels could be higher due to the proximity of completing a degree or relearning material because of relocating to a new position.

The JMALI was also slated to be used by Oregon State University to collect data on their college students but was delayed. They are in the process of obtaining ethics approval to conduct it in spring 2023.

Kevin Curry, Penn State University, developed a similar agricultural literacy assessment for K-2^nd grade students and have partnered with elementary schools. Study has been delayed due to continued COVID challenges.

 CQ University researchers (Sydney location) reached out to Dr. Judd-Murray regarding the use of JMALI to measure the agricultural literacy of their college student populations. They sought collaboration for using the instrument and will partner with Utah State University to modify the instrument for use in Australian agricultural literacy research. The goal is to move the effort forward to a study publication and greater international development in agricultural literacy.

1b. Debra Spielmaker, Utah State University, worked with a graduate student on a case study to understand the baseline proficiency of students at Tri-Center Elementary school in Neola, Iowa. The research was guided by two objectives: 1. To determine a baseline of the agricultural literacy proficiency knowledge of kindergarten through fifth grade students at Tri-Center Elementary, 2. To make recommendation to Iowa program leaders concerning the development of programming to increase agricultural literacy. A paper was published and presented at AAAE in May 2022.

1c. Josh Stewart, Oregon State University, reported hosting two Summer Ag Institutes (SAI). Geographically located East and West, they were designed for local and regional elementary and secondary teachers. The institutes each draw approximately 25 teachers/year for a week-long, immersive experience. The SAI is a partnership with Oregon Farm Bureau, Oregon Ag in the Classroom (AITC), and local agricultural producers.

1d. Josh Stewart, Oregon State University, reported the co-authorship of a successful Specialty Crop Block grant with Oregon AITC. The purpose of the grant is four-fold: to record "coffee talks with specialty crop producers and scientists at Oregon State for use in secondary classrooms, to create lesson plans and activities for teachers to use, to provide professional development for teachers around using the videos and lesson plans, and to conduct at least five live, virtual farm tours where students have the chance to ask producers questions.

1e. Rose Judd-Murray, Utah State University, along with colleagues Debra Spielmaker and Michelle Burrows, has completed the validation of a 6-8^th grade agricultural literacy assessment based on the benchmarks of the National Agricultural Literacy Outcomes. The 15-question assessment measures the proficiency levels of youth in a sliding scale of measurement (exposure, factual literacy, and applied proficiency). The completion of this instrument means that there is now a validated agricultural literacy assessment for K-12 through post-high school assessment (re: adults). The instrument will be finalized and made available for free on the National Center for Agricultural Literacy (NCAL) website.

1f. Kellie Enns and Jenny Bennett, Colorado State University, utilized the Longhurst-Murray Agricultural Literacy (LMALI) assessment with elementary school students and measured student affinity for desire to be involved in agricultural careers.

1g. Denise Stewardson, Utah State University, obtained a Specialty Crop Block grant and paired agricultural education with nutrition education. She used this grant and an additional smaller grant to develop nutrition resource boxes for 3-5^th grade students (i.e., Crops in a Box). She is currently working on crafting an evaluation to investigate how nutrition education impacts students’ agricultural literacy.

1h. Shelly Rampold, University of Tennessee at Knoxville, continues to work on a collaborative project with Nebraska (PDAL grant) to recruit high school agricultural teachers for training that improves their understanding and use of virtual reality (VR) technology. The effort is designed to improve virtual agricultural experiences and agricultural exposure.

Impacts

1a. Debra Spielmaker, Utah State University, notes that the research conducted this year found some short-term impacts, however the “no impact findings’ of these studies did provide information for changes that could lead to greater impacts. She conducted research and presented a peer-reviewed paper on "A Case Study: Agricultural Literacy Proficiency in an Iowa Elementary School." This research meeting poster has also been accepted as an article in the Journal of Agricultural Education (in press). This study sought to measure the impacts of an agricultural literacy intervention. While this research was not structured as a pretest and posttest, the study was conducted to understand the history of interventions at Tri-Center Elementary as a part of establishing a baseline for agricultural proficiency at this school. The results demonstrated that 3rd and 4th grade students performed at a exposure and factual literacy levels, as predicted. However, the majority of fifth grade students also performed at the factual literacy level. Researchers expected these students to score at a proficient level based on the intervention provided. This demonstrated the intervention had an impact on the majority of 3rd and 4th graders, but not 5th graders. This research provides insight for program planners to make decisions about next steps with Iowa agricultural literacy outreach at this school.

The peer-reviewed research poster "Agriculture in the Classroom Preservice Teacher Seminar Evaluation" found that the Utah AITC preservice program had a short-term impact where 92% of the preservice teachers found the seminar informative and they planned to use Utah AITC resources in the future. However, mid-term survey of a low response rate (12%) indicated participants did not follow through in their own classrooms. We (the researchers) recommended to the Utah AITC program leaders that they find ways to follow-up or engage with these teachers in professional learning communities during their first few years of teaching to remind the new educators about the resources, and how to use them to address curricular standards. Utah AITC staff should also consider the timing of the seminar within the undergraduates’ course sequence to ensure most effective placement for more immediate implementation of the resources. With these changes, this study should be replicated.

1b. Oregon State University and their block grant partners have filmed three virtual farm tours for the project. Those videos have reached 6,766 students in 427 separate classrooms. The outputs associated with this project have yet to show intermediate to long-term impact, but they can be associated with the potential for future impact due to student and teacher exposure to agricultural content.

1c. Rose Judd-Murray, Nellie Hill, Gaea Hock, and Shannon Norris (USU, KSU, NMSU) noted that the multi-state research effort related to JMALI assessment of Extension professionals and volunteers showed the potential for more impact due to the study highlighting the need for state-level preparedness efforts to support agricultural literacy advancement across various career stages and suggested state staff and faculty seek opportunities to help Extension personnel and volunteers refine acquired knowledge. Additionally, indirect connection with Extension leaders increased opportunities for collaboration. Due to this connection, the Department of Agricultural and Extension Education (New Mexico) has increased direct contact with representatives in all 33 counties who now have awareness of the JMALI instrument.

Objective 2: Assess attitudes, perceptions, and motivations of diverse population segments related to agriculture, such as consumers, students, and producers. Specifically, explore and/or measure: a. how perceptions, attitudes and motivations are developed; b. decisions made based on assessed attitudes, perceptions, and motivations; c. behavior changes that have occurred due to changes in attitude, perceptions, and/or motivation.

Accomplishments: Outputs & Milestones

2a. Kansas State University conducted a content analysis of tweets related to Clarkson’s Farm (agri-tourism entity). The results of their study were presented at the North Central AAAE 2022 regional meeting.

2b. Josh Stewart, Oregon State University, reported hosting two Summer Ag Institutes (SAI). Geographically located East and West, they were designed for local and regional elementary and secondary teachers. The institutes each draw approximately 25 teachers/year for a week-long, immersive experience. The SAI is a partnership with Oregon Farm Bureau, Oregon Ag in the Classroom (AITC), and local agricultural producers. The SAI programs ask teachers to explore in-depth agricultural production through in-person visits to production farms and facilities, then create lesson plans for use in their classrooms.

2c. Kellie Enns and Jenny Bennett, Colorado State University, utilized the Longhurst-Murray Agricultural Literacy (LMALI) assessment with elementary school students and measured student affinity for desire to be involved in agricultural careers.

Impacts

To be determined.

Objective 3: Evaluate agricultural literacy programs to measure program impact. Specifically:

a. measure impacts of agricultural literacy programs related to critical thinking and problem solving; b. explore and evaluate peer and participant-centered agricultural literacy programming methods to determine their effectiveness in addressing defined agricultural literacy outcomes.

Accomplishments: Outputs & Milestones

3a. Debra Spielmaker, Utah State University, worked on two projects in this area. One was a synthesis of the literature on agricultural literacy to determine what recommendations had been made in regard to measuring the impacts of agricultural literacy interventions, and a second research project to determine the efficacy of an agricultural literacy workshop with K-6 preservice teachers. Both studies were presented as research posters at the Western Regional AAAE meetings in Fall 2022.

Impacts

      To be determined.

Planned Projects for 2023

Multi-state efforts: Continued data analysis of agricultural literacy levels of undergraduate student and Extension professionals/volunteers. Arizona has obtained ethics approval to complete the assessment of their professionals and volunteers. Plans to work as a team to expand the WAAAE paper into a full manuscript for the Journal of Extension.

 Continue to explore how Clarkson’s Farm (Kansas) influences attitudes regarding agriculture.

 Debra Spielmaker and Rose Judd-Murray, Utah State University, are currently working on the Agricultural Literacy Certification Program and will guide a graduate student through determining its effectiveness in 2023.

 Oregon State University will continue to host the SAI workshops, work to produce impact in the Specialty Crop Block grant project through 2023, and complete the JMALI assessments with their college student population. New faculty member, Quincy Clark, will be crafting curriculum for underrepresented populations to teach problem-solving within an agricultural context (4-6^th grade).

 New Mexico State University will seek to investigate agricultural knowledge of diverse population segments, particularly through college students and members of support affiliations to soldiers working in food and agriculture, related to defense efforts. Additional plans to extend previous studies related to water and agricultural education awareness into the Journal of Applied Communications.

 Colorado State University has a new agricultural learning laboratory in Denver. Project-based learning will be the focus for secondary agricultural education programs to address critical issues in agriculture. The CSU team has developed and will facilitate a badging program for intensive ag literacy experience at events (e.g., fairs, stock shows) in 2023.

 Michelle Burrows, Utah State University, plans to work with Dr. Judd-Murray to assess FFA members with the JMALI to determine proficiency of agricultural literacy. She is also assisting Texas AITC to collect, monitor, and analyze data points related to their social media accounts.

Carl Igo, Montana State University, set to develop 8–12-minute videos to engage students in agricultural education. Videos will integrate with national core curriculum standards to encourage the use of them directly in the classroom.

Kevin Curry, Penn State University, plans to use the JMALI with PSU preservice students and “general education” students to form a comparative study related to their agricultural literacy proficiency scores.

Shelly Rampold, University of Tennessee at Knoxville, seeking to use the JMAL to assess adult consumer’s knowledge of agriculture. Will continue to work with commodity groups to develop yearly campaigns to understand agriculture related to that specific commodity and develop a tool kit to assist them in evaluation of consumer knowledge. Soybean-specific resources may be prioritized.

Bennett, J., Spielmaker, D. M., Burrows, M. S. (2022, May 17-19). A synthesis of recommendations within agricultural literacy intervention research [Poster presentation]. American Association for Agriculture Education, Las Cruces, NM, United States.

Hill, N., Claflin, K., Specht, A., & Hock, G. (2022, October 6-8). Edutainment on the farm: a content analysis of tweets about Clarkson's Farm [Paper presentation]. North Central Region American Association for Agricultural Education Research Conference, Columbia, MO, United States.

Judd-Murray, R., Hill, N., Norris, S., & Hock, G. (2022, September 19-21). A multi-state agricultural literacy assessment of extension professionals and volunteers [Paper & Oral presentation]. Western Region American Association for Agricultural Education (AAAE) Research Conference, Las Cruces, NM, United States.

Miller, A. J., Warnick, B. K., Spielmaker, D.M. (2022, May 18-21). A case study: Agricultural literacy proficiency in an Iowa elementary school [Paper presentation]. American Association for Agriculture Education, Oklahoma City, OK, United States.

Miller, A.J., Konakis, Z.A., Yoshikawa-Ruesch, E., Spielmaker, D.M. (2022, September 19-21). Agriculture in the Classroom preservice teacher seminar evaluation [Poster presentation]. American Association for Agriculture Education, Las Cruces, NM, United States.

*W3006 committee members are bolded in the citations.