Brief Summary of Minutes of Annual Meeting

The meeting was called to order at 8:00 AM MST by Debra Spielmaker, chair.

Note: General information and objectives for this Agricultural Literacy Multistate Research Project (W3006) are available at https://www.agliteracy.org/research/multistate.cfm
The official site for this project is https://www.nimss.org/ where project details including participants and minutes are on file. Use “3006” in search.

Please make sure to double-check that your name is on the participant list along with your selected objectives.

Objectives:
1. Assess the agricultural knowledge of diverse population segments related to agriculture, including consumers, students, and producers. Specifically, explore and/or measure: a. points of acquisition of agricultural knowledge; b. decisions made based on assessed knowledge.
2. Assess attitudes, perceptions, and motivations of diverse population segments related to agriculture, such as consumers, students, and producers. Specifically, explore and/or measure: a. how perceptions, attitudes and motivations are developed; b. decisions made based on assessed attitudes, perceptions, and motivations; c. behavior changes that have occurred due to changes in attitude, perceptions, and/or motivation.
3. Evaluate agricultural literacy programs to measure program impact. Specifically:
a. measure impacts of agricultural literacy programs related to critical thinking and problem solving; b. explore and evaluate peer and participant-centered agricultural literacy programming methods to determine their effectiveness in addressing defined agricultural literacy outcomes.
Debra opened discussion from the meeting in Anchorage in 2019 regarding the committee’s change in future leadership. Debra has led this committee for the past 12 years; consequently, Rose Judd-Murray agreed to take the committee chair position for the next two years.
Gaea Hock moved to elect Rose Judd-Murray (USU) as chair of the W3006 committee for the next two years (2021-2023); seconded by Haley Traini. Unanimous vote to approve.

Denise Stewardson has served as secretary of this multi-state committee for nearly 10 years but has decided to resign this position.
Gaea Hock moved to elect Jenny Bennett (CSU) as secretary by acclamation. Motion passed.

Brian Warnick, committee administrative advisor, provided an update. He commended the committee on gaining more traction in addressing the objectives of the proposed project. Members are encouraged to complete reports for formal submission to NIMMS. The committee has three years to address the objectives; impacts are important for reporting to validate the project. Members can review other multistate projects to see those documented objectives and impacts.

For a new project in 2024: The next project for study must be determined in 2023.

Brian thanked Debra and Denise for their service.

Debra reminded committee to search activity accomplishment reports at https://www.nimss.org/
The National Research Agenda for AAAE is approximately two years from being published. W3006 can address some of those priorities.

Gaea asked if there were a professional development need at National AAAE to inform others working on multistate projects, especially graduate students and new faculty. Can fellows or administrative-level faculty provide information? Brian Warnick will reach out to AAAE Professional Development chair to put this on the agenda for the national meeting. Debra is willing to assist this effort; it is important to help faculty navigate their experiment stations and understand that this committee is open to any region.

Debra has created a Google Doc with member’s responses to their progress on the W3006 objectives. Committee members reported on their research progress:

Gaea Hock & Nellie Hill
• Nellie joined W3006 in August 2021 and got approval from IRB to use the Judd-Murray Agricultural Literacy Instrument (JMALI). Data was collected from students on JMALI (survey is live and getting responses). Gaea and Nellie met with agriculture agents in Kansas and are attending the Kansas state conference to collect JMALI data in October 2021. Rose: Are you collecting data from all Extension agents (e.g., Home and Community, Agriculture and Natural Resources)? Gaea: Yes; we intend to collect data from all agents. Gaea is also collecting data from university courses in agricultural literacy. Gaea thanked Rose for her assistance; Gaea added a couple questions to JMALI for her research purposes. Josh Stewart and Haley Traini requested Gaea’s survey: Rose will share that survey.

Rose asked if there were a way to use National Center for Agricultural Literacy resources to incentivize responses to the JMALI survey. Debra said it may be possible; Brian encouraged members to look for other funding sources to provide incentives.

Kelli Enns
• Kellie created a Learning Lab in urban Denver (CSU lab in food and agriculture building, museum-like open to public): Elementary badging, MS bridging, HS career pathways. 11,000 contacts in two pop-up experiences; badged over 350 students (classroom experience and adventure/scavenger hunt). Lab is open 365 days/year. Summer internships are available; serves a K-12 audience. Lab educators being hired have no agricultural literacy experience. State officers will also have training. Second program: Agricultural literacy curriculum for Colorado to be used by AmeriCorps program; pilot-tested and picked up by two elementary schools (500 students). Kelli is using the Longhurst-Murray Agricultural Literacy Instrument (LMALI), badging for Grades 3-5; Third program: All college freshmen take course “Experiences in Agriculture.” Kelli is collecting JMALI data. Gaea Hock asked for course syllabus; Kelli Enns can provide that.
• CSU Spur https://nationalwesterncenter.com/csu-spur/ Curriculum is presented to all Extension agents. Agricultural teachers are presenting after-school programming.

Josh Stewart & Haley Traini
• Josh secured funding from a USDA Specialty Crop Block Grant for a virtual fieldtrip series; funding begins in October 2021 (working with Oregon AITC). OSU Extension will facilitate these fieldtrips in spring 2022. He is also working with OSU’s IRB office to complete approval for survey distribution.

Kevin Curry
• Spring 2021: Kevin surveyed junior and senior candidates in agricultural education teacher education. Students enrolled in a science literacy general education course also took the survey. Kevin will send date to Rose.

Jihyeong Son
• Received USDA grant in Extension Risk Management Education Program for small-scale farmers using social media for their marketing needs. Met with local farmers using social media for their marketing; collected data in 2020. This project meets project objectives for diverse populations.

Katie Stofer
• Transitioned her work into health extension (medical school/pharmacy school at UF working in preventative health) on bringing trust to rural populations. Hoping to get data on health side that may be relevant for agricultural literacy. Will collect data fall 2021 on JMALI from agriculture students.

Debra Spielmaker
• Bekka Israelsen, Utah AITC, is using preservice students to take 5th grade assessment to collect pre-knowledge and HS assessment as wrap-up with these students. Debra is working on middle school agricultural literacy assessment: teachers across the country provided populations to take survey (~500 students). Tango rewards were used as incentives; a graduate student is now needed to analyze data. NCAL is working on digital agricultural literacy badging for grades 3-5; will use submissions for data collection on agricultural literacy.

Amelia Miller

• Amelia is a second-year Ph.D. student conducting an independent study looking at data from NCAL Matrix users of the MyBinder function (virtual storage of lessons). As an innovative project for AAAE: Farm Crates via Michigan AITC subscription boxes for teachers (2 four-month subscriptions—$100 per four months—based on seasonal commodities tied to core educational standards). Content is created in NearPod and Pear Deck for teachers. Amelia is also analyzing date from a K-5 JMALI assessment in Iowa, using a case study on a particular school having success in using assessment.

Debra commented that several state AITC programs are using the JMALI literacy assessment which supports the objectives for W3006.

Michelle Burrows
• Michelle is working on data collection with NCAL—Google analytics of Matrix users and how they use the lessons. Looking at what is popular in specific regions (26 state programs that use the Matrix). She is also placing icons in lessons in the National Agricultural Literacy Curriculum Matrix to recognize socioscientific issues; Utah teachers are addressing socioscientific content in their curriculum.
Rose Judd-Murray
• Rose commented that in coming years, there is a need for Extension personnel due to retirements. Early career and mid-career agents need agricultural literacy—also needed by volunteers. AZ, NM, KS, CO, OR, UT are assisting her in surveying Extension agents. In spring 2021 she conducted assessment at Utah State University College of Agriculture & Applied Sciences general education courses. She has a master’s student using Google analytics looking at who is downloading the JMALI/LMALI assessments and how they’re using them: will they share the scores; what were barriers? Received USDA PDAL grant for agricultural literacy certification program: 50 teachers/volunteers/early career agricultural education professionals using 8-10 training modules (digital badging with 12 hours of instruction), pre- and post-assessments for pilot test. 500 teachers will roll it out. Katie Stofer is the evaluator for this grant. Michelle Burrows is helping with experts for Delphi panel.

Gaea is editor for The Agricultural Education Magazine. She asked for a theme editor for 2022. Kelli Enns volunteered to serve as theme editor for articles.

Meeting adjourned at 10:04 AM.

The W3006 just completed its second year working on agricultural literacy research. While COVID stalled some research goals, the committee has been working this past year to get programs and research studies in place to continue our research agenda. Several members secured additional grant funds this past year to initiate programming and research. These grants are noted in the minutes from the fall 2021 meeting. Accomplishments and limited impacts are reported under each of the W3006 objectives below. The committee is optimistic that we can return to our research agenda timeline to report greater impacts. Annual minutes are posted on the NIMMS website.

Participants: https://www.nimss.org/meetings/project/18611

Objectives:

1. Assess the agricultural knowledge of diverse population segments related to agriculture, including consumers, students, and producers. Specifically, explore and/or measure: a. points of acquisition of agricultural knowledge; b. decisions made based on assessed knowledge.

Accomplishments – Outputs & Milestones

1a. Using the Judd-Murray Agricultural Literacy Instrument (JMALI), data was collected during the fall of 2021 at Kansas State University, Utah State University, Colorado State University, and Penn State University to determine how college students performed on a high school agricultural literacy assessment. Oregon State plans to collect data in the spring of 2022. The data will be analyzed and reported in May of 2022. The JMALI was also administered to Extension personnel in Utah to determine their performance on the assessment which will help to determine what, if any, professional development training may be needed.

1b. Utah State University researchers completed data collection on a grade 6-8 (middle school) National Agricultural Literacy Outcome assessment. The data will be analyzed, and a final instrument will be completed in April of 2022. The data collection used an innovative approach to gain access to students nationwide. Eighth grade teachers were recruited via social networks and served as "marketers" for the research and data collection. Participants were recruited via their teachers who used a video presentation to introduce students and their parents to the research. (https://docs.google.com/presentation/d/15vRhwfsb5W1E8bgKrwnaIeZ7PhRCs7HtIzT_-OiycjY/edit?usp=sharing). Responses were collected from 494 students.

Impacts

1a. Full impact is to be determined. This is the first-time research has been conducted to measure agricultural literacy among a college population with the same instrument. This approach allows us to conduct more valid analysis to measure significant differences related to knowledge, conduct correlations, and to develop predictive models (using regression analysis). The findings will help to construct programming at both the secondary and post-secondary levels to increase agricultural literacy.

1b. Full impact is to be determined. The development of the middle school (grade 6-8) agricultural literacy assessment will complete the package of assessments for K-12 education.

2. Assess attitudes, perceptions, and motivations of diverse population segments related to agriculture, such as consumers, students, and producers. Specifically, explore and/or measure: a. how perceptions, attitudes and motivations are developed; b. decisions made based on assessed attitudes, perceptions, and motivations; c. behavior changes that have occurred due to changes in attitude, perceptions, and/or motivation.

Accomplishments – Outputs & Milestones

None to report. Future projects are noted in the planned projects for 2022 (e.g., the Agricultural Literacy Certification program).

Impacts

To be determined.

3. Evaluate agricultural literacy programs to measure program impact. Specifically:
   measure impacts of agricultural literacy programs related to critical thinking and problem solving; b. explore and evaluate peer and participant-centered agricultural literacy programming methods to determine their effectiveness in addressing defined agricultural literacy outcomes.

Accomplishments – Outputs & Milestones

3a. Debra Spielmaker, Utah State University, completed data collection from state and territory Agriculture in the Classroom program leaders to evaluate program effectiveness. As the data collection frame was 2020, COVID impacted the results. Additional program data will be collected early in 2022 on the 2021 calendar year to determine the use of and impacts of critical thinking and problem-solving instruction within teacher professional development. As COVID impacted teacher professional development in 2021, this research question may be re-evaluated in 2022.

3b. Dr. Spielmaker completed research with a graduate student, Amelia Miller, using descriptive data to determine the characteristics and instructional resource saving patterns among users of the National Agricultural Literacy Curriculum Matrix. National analysis of teacher website behavior indicates nearly all teachers are sourcing lessons online (Scholastic, 2013). The findings were shared in a peer-reviewed research poster (noted in publications) titled “Characteristics of Agricultural Literacy Database Users.”

3c.  Kellie Enns, Colorado State University, led the development of a Learning Lab focusing on food and agriculture. This lab is museum-like and open to the public. A new program implementing elementary student agricultural badging, middle school agricultural knowledge bridging, and a high school career pathways was launched. Over 11,000 contacts were made, and the program badged over 350 students (classroom experience and adventure/scavenger hunt). The lab is open 365 days/year. Summer internships have been made available to serve a K-12 audience.

3d. Dr. Enns also worked with others to develop an agricultural literacy curriculum for Colorado. This program will be used by the AmeriCorps program.

3e. CSU researchers also developed a program, “Experiences in Agriculture,” that will engage all College of Agricultural Sciences freshmen. As part of the assessment in the course, students will complete the JMALI to determine agricultural literacy gains.

Impacts

3a. The national descriptive data has been used to secure additional funding for the National Agriculture in the Classroom (NAITC) Organization. The mission of this organization is to "increase agricultural literacy through preK-12 education."

3b. The research demonstrated a limited success of NAITC’s Agricultural Literacy Curriculum Matrix MyBinder lesson plan saving feature. The target audience, number of users, diversity of states represented, grade/content affiliation, and top lessons downloaded all support the MyBinder to achieve the mission to increase agricultural literacy by providing educators with standards-based lessons. Based on the patterns observed, educators indicating specific grade level needs are saving those types of lessons. However, the extremely high percentage of one-day only logins cause for concern and signifies the need for greater promotion of the feature. The impact of this research highlights how time and funding should be spent to improve engagement among educators with vetted agricultural literacy resources. Findings suggested MyBinder, as a platform for saving instructional resources, requires greater professional development among agricultural literacy program leaders to share this resource with preK-12 educators. Additional research needs to be conducted to determine why MyBinder users are not returning to their binders. Future research using the Theory of Planned Behavior could address this question and answer more questions about the actual use of the instruction in classrooms, along with student agricultural literacy gains.

3c. To be determined. Dr. Enns is collecting data using the Longhurst-Murray Agricultural Literacy Instrument (LMALI) to determine the level of agricultural literacy among agricultural badging completers. The results will help determine the impact of this program.

3d. To be determined. The program will be evaluated, and participants will complete one of the National Agricultural Literacy Outcome assessments to determine the program’s impacts.

3e. To be determined. The program will be evaluated, and participants will complete the JMALI assessment to determine the program’s impacts.

Planned agricultural literacy research projects for 2022

Use the JMALI to assess Kansas Extension agents and volunteers. (Kansas)

Conduct a study on using a retrospective approach to enhance agricultural literacy efforts. (Montana)

Complete the data analysis of the 6-8 National Agricultural Literacy Outcome assessment data. Complete the development of an Agricultural Literacy Certification Program (ALCP) for the final development of an ALCP where we can conduct evaluation research and use the National Agricultural Literacy Outcome assessments to measure agricultural literacy among participants and the audiences/students they work with. (Utah)

Conduct an analysis of the National Agriculture in the Classroom Facebook page to determine the type of posts that result in teachers/user taking action. (Utah)

Complete an analysis of the agricultural literacy assessment downloads and use. (Utah)

Analyze assessment results from 400 elementary students in Iowa on data collected in the spring of 2021. (Utah)

Analyze data currently being collected on elementary preservice students’ pre-posttest knowledge using the National Agricultural Literacy Outcome assessments. (Utah)

Collect data on students who have been part of the DIGS workshops and used the DIGS curriculum, measure the impact of badging on agricultural literacy achievement. Researchers developed an Agriculture Academy engaging 1,100 participants. Of this group, 165 students became Junior Ag Ambassadors at Larimer County Fair. The program will be formally evaluated in 2022. (Colorado)

Appel, S. & Enns, K. (2021, September 27-29). Laughter in literacy: Influencing youth agricultural literacy through a summer day camp [Poster presentation]. American Association for Agricultural Education Western Region Conference.

Flood, K., & Curry Jr., K. W. (2021, June 21-24). Assessing the agricultural literacy needs of preservice agriculture teachers. [Poster presentation]. North American Colleges and Teachers of Agriculture.

Jarvis, S. (2020). The influence of 4-H instructor beliefs on the teaching of animal food production to youth populations [Doctoral dissertation, The University of Arizona]. University of Arizona Digital Archive https://repository.arizona.edu/handle/10150/648610. [Chair Rice, A.]

McFadden, B. R., Rumble, J. N., Stofer, K. A., Folta, K. M., Turner, S., & Pollack, A. (2021). Gene editing isn’t just about food: Comments from US focus groups. GM Crops & Food, 1-11.

Miller, A. J., Spielmaker, D. M., Blodgett, M. L., Ritter, T. S. (2021, September 27-29). Agricultural literacy education for remote learning [Poster presentation]. American Association for Agricultural Education Western Region Conference.

Miller, A. J. & Spielmaker, D. (2021, May 25). Characteristics of agricultural literacy database users [Poster presentation]. American Association for Agricultural Education National Conference. http://aaaeonline.org/resources/Documents/National/2021meeting/PosterSession2021.pdf

Spielmaker, D. M. (2021, June 28-30). 2020 Agricultural literacy program reports [Conference session]. National Agriculture in the Classroom Conference, Des Moines, IA, United States. https://www.jotform.com/report/21165465943605915

*W3006 committee members are bolded in the citations.