Brief Summary of Minutes of Annual Meeting

Multistate Agricultural Literacy Research Committee (W3006) Fall Meeting

September 18, 2020, via Zoom

Members Present:

Kellie Enns, Colorado State University Kellie.enns@colostate.edu

Gaea Hock, Kansas State University ghock@ksu.edu

Rose Judd-Murray, Utah State University rose.juddmurray@usu.edu

Michael Martin, Colorado State University Michael.j.martin@colostate.edu

Debra Spielmaker, Utah State University debra.spielmaker@usu.edu

Denise Stewardson, Utah State University denise.stewardson@usu.edu

Josh Stewart, Oregon State University josh.stewart@oregonstate.edu

Kathryn Stofer, University of Florida stofer@ufl.edu

Jonathan Velez, Oregon State University jonathan.velez@oregonstate.edu

Thomas Dormody, New Mexico State University tdormody@nmsu.edu

Kevin Curry, Pennsylvania State University kxc554@psu.edu

Carl Igo, Montana State University cigo@montana.edu

Gigette Webb, University of Arizona gigettewebb@email.arizona.edu

Hailey Clement Traini, Oregon State University haley.clement@oregonstate.edu

Amber Rice, University of Arizona amrice@email.arizona.edu

Brian Warnick, Administrative Advisor, Utah State University brian.warnick@usu.edu

Members Absent:

Cary Trexler, University of California—Davis cjtrexler@ucdavis.edu

Kimberly Bellah, Murray State University kbellah@murraystate.edu

Nellie Hill, New Mexico State University nhill@nmsu.edu

Tiffany Rogers-Randolph, Kansas State University trogersrandolph@ksu.edu

Jihyeong Son, Washington State University jihyeong.son@wsu.edu

The meeting was called to order at 9:00 a.m. MST by Debra Spielmaker, chair.

Gaea Hock moved to accept the meeting minutes of April 24, 2000; seconded by Jonathan Velez. Motion carried.

Note: General information and objectives for this Agricultural Literacy Multistate Research Project (W3006) are available at https://www.agliteracy.org/research/multistate.cfm

The official site for this project is  https://www.nimss.org/ where project details including participants and minutes are on file. Use “3006” in search.

Please make sure to double-check that your name is on the participant list along with your selected objectives. This is a five-year project, so please select objectives accordingly.

Objectives:

1. Assess the agricultural knowledge of diverse population segments related to agriculture, including consumers, students, and producers. Specifically, explore and/or measure: a. points of acquisition of agricultural knowledge; b. decisions made based on assessed knowledge.
2. Assess attitudes, perceptions, and motivations of diverse population segments related to agriculture, such as consumers, students, and producers. Specifically, explore and/or measure: a. how perceptions, attitudes and motivations are developed; b. decisions made based on assessed attitudes, perceptions, and motivations; c. behavior changes that have occurred due to changes in attitude, perceptions, and/or motivation.
3. Evaluate agricultural literacy programs to measure program impact. Specifically:
   a. measure impacts of agricultural literacy programs related to critical thinking and problem solving; b. explore and evaluate peer and participant-centered agricultural literacy programming methods to determine their effectiveness in addressing defined agricultural literacy outcomes.

Members and guests of the committee introduced themselves.

Members provided an overview of their work on W3006:

• Josh Stewart (OSU): Submitted grant on agricultural literacy with AITC for self-paced curriculum for teachers/students; ongoing conversation with AITC. Working towards JMALI use; reaching out to academic programs office to get emails for incoming freshmen
• Jonathan Velez (OSU): Developing Agricultural Science and Natural Resources Research minor; developed six agricultural communication courses; involved in Summer Ag Institute: 1000 teachers over the years with Oregon Farm Bureau (K-12 teachers), one-week intro to agriculture—captive audience provides opportunity for research
• Haley Traini (OSU): Oregon AITC working with middle school clubs on virtual fieldtrips; OSU Leadership Academy undergraduates—how agricultural leaders as undergraduates facilitate agricultural literacy
• Tom Dormody (NMSU): Looking at keyhole gardens built at university research center: agricultural science students teach 2^nd graders + K-3 teachers on gardens; surveyed them on K-2 NALOS: were keyhole gardens effective in teaching the NALOS? Rated it high for teaching agricultural literacy; Tom submitted to Journal of Extension, considering K-12 agricultural education journal (Debra: Revising NALOS; encouraging participants to let her know of publications so she can curate publications on agliteracy.org)
• Kevin Curry (Penn State): Science literacy research, working with 2^nd grade elementary Master Gardener project—raised beds, planted seeds for salad party, pollinators; affective measures regarding agricultural literacy; pilot data of quantitate and qualitative data (attitudes, behaviors); hoping to be in gardens in spring
• Gaea Hock (Kansas State): Youth Water Advocates Conference (AES project tied to multistate project)—hired undergraduate to move project forward; collect data with this group; reminder: Look at how your university assigns/funds/administers AES projects; plans to send out JMALI to university students (university very careful about emails to students); editor of The Agricultural Education Magazine, theme in May/June 2021—Tell Me Something Good https://www.naae.org/profdevelopment/magazine/themes.cfm. Virtual AgriLand at Kansas State Fair: https://ksagclassroom.org/virtual-agriland/
• Carl Igo (Montana State University): Still working on revisions to state AITC program, hiring new director; current COVID protocols limit access to university classroom populations thereby limiting use of JMALI; NIFA/PDAL proposal accepted, food/fiber project previously administered looking at long-term follow-up, collecting focus group data in Oklahoma and Montana cohorts (possible Oregon and Utah); retrospective piece will create videos of teaching these agricultural literacy lessons, shared via NAITC
• Katie Stofer (University of Florida): Looking at using JMALI with two populations—spring semester freshmen (Innovation Academy) and transfer students. Working on teacher education agricultural literacy programs using qualitative research; working with Debra and Rose on a proposal to implement volunteer base for administering agricultural literacy (Katie is evaluator)
• Amber Rice (University of Arizona): Works with Gigette Webb (AITCO and teacher education preparation; online master’s students interested in agricultural literacy, working on ways to capitalize on that population/interest; hoping to distribute Qualitrics survey to U of A students
• Gigette Webb (Arizona AITC state contact): Looking forward to working with Amber Rice on research opportunities; conducted successful summer virtual academy with teachers, can use that population for future research
• Michael Martin (CSU): College-aged and adult populations—how people think about agricultural literacy via a political perspective (idealogy): how do they define an agriculturally literate person? Hoping to administer the JMALI survey in the future; Western AAAE poster on how to incorporate diversity into an agricultural education course
• Kelli Enns (CSU): College of Agriculture hired an instructor to promote Ag Adventure programming and experiences for freshmen in college of ag, how to build community in that college; targeted that class for JMALI but inconsistencies in class formats too great for good data collection; National Western Stock Show renovated; CSU will have three buildings to provide year-round engagement daily beginning January 2022; working with AmeriCorps volunteers using agricultural literacy curriculum in afterschool programs; agricultural literacy picked up as fundamental goal in college’s strategic plan
• Rose Judd-Murray (USU): Agricultural literacy certification grant via USDA with Debra and Katie; working to finish grades 6-8 version of JMALI but delayed due to COVID; hoping to collect data in spring from USU student populations; reminder that universities can use USU’s IRB process to administer surveys; nonformal and community-based USU students looking for internships—perhaps looking at CSU opportunities in the future
• Denise Stewardson (Utah AITC): Can offer populations via state AITC program and university general education agricultural literacy course; interested in replicating afterschool agricultural literacy program from CSU
• Debra Spielmaker (USU): NALO grades 6-8 assessment using crowd sourcing via social media (completed application online but COVID halted the opportunity), hoping to follow up with teachers in spring; eLearning resources created via NCAL; co-PI on USDA Rapid Response grant application; Amelia Miller conducting research on descriptive study on teachers using MyBinder feature of NAITC Agricultural Literacy Curriculum Matrix; Rose’s master’s graduate student examining teachers who downloaded JMALI instruments and sharing proficiency levels of those students; Amelia Miller also looking on data collected related to mobile agricultural literacy labs regarding attitudes, perceptions; looking at teachers’ use of resources: credibility, science-based

Review of JMALI assessment (via Qualtrics) for end of 12^th grade (9-12) to be used for university students: This link was shared with all participants to view the survey/assessment. As this is a public document the link is not shared here.

• Time commitment = 20 minutes noted on IRB, but that was high end
• Instrument was validated ages 18-23, so useful beyond freshman year
• Useful for agriculture and non-agriculture students; ideal population is first semester freshmen students (summative assessment of end-of-12^th grade students)
• Ideal population is all freshmen at a particular institution
• Can this be used spring semester 2021?
• Suggestion: Break question #6 into formal and nonformal experiences

This discussion led to requests for another meeting to revise the Qualtrics survey, adapt it for a specific institution, make a plan to administer the survey (population and timeline).

Debra will share Qualtrics link for revisions; a meeting will be scheduled within the next month to revise the survey. The plan is to administer the survey in spring 2021.The minutes of April 24, 2020, have the IRB protocol.

Meeting adjourned at 11:03 a.m.

Multistate Agricultural Literacy Research Committee (W3006) Meeting

April 24, 2020, via WebEx

Members Present:

Kellie Enns, Colorado State University Kellie.enns@colostate.edu

Gaea Hock, Kansas State University ghock@ksu.edu

Rose Judd-Murray, Utah State University rose.juddmurray@usu.edu

Michael Martin, Colorado State University Michael.j.martin@colostate.edu

Debra Spielmaker, Utah State University debra.spielmaker@usu.edu

Denise Stewardson, Utah State University denise.stewardson@usu.edu

Josh Stewart, Oregon State University josh.stewart@oregonstate.edu

Cary Trexler, University of California—Davis cjtrexler@ucdavis.edu

Brian Warnick, Administrative Advisor, Utah State University brian.warnick@usu.edu

Members Absent:

Kimberly Bellah, Murray State University kbellah@murraystate.edu

Haley Clement, Oregon State University haley.clements@oregonstate.edu

Kevin Curry, Pennsylvania State University kxc554@psu.edu

Thomas Dormody, New Mexico State University tdormody@nmsu.edu

Carl Igo, Montana State University cigo@montana.edu

Amber Rice, University of Arizona amrice@email.arizona.edu

Kathryn Stofer, University of Florida stofer@ufl.edu

Jonathan Velez, Oregon State University jonathan.velez@oregonstate.edu

Gigette Webb, University of Arizona gigettewebb@email.arizona.edu

Guests Present:

Jenny Bennett, Colorado State University Jennifer.bennett@colostate.edu

Sasha Broadstone, Utah State University sasha.broadstone@usu.edu

The meeting was called to order at 1:00 p.m. MST by Debra Spielmaker, chair.

Members and guests of the committee introduced themselves.

Sasha Broadstone, Utah State University, introduced herself and explained her role with the Institutional Review Board (IRB). Sasha will provide information and answer questions relating to the committee’s research and the use of IRB.

Note: General information and objectives for this Agricultural Literacy Multistate Research Project (W3006) is available at https://www.agliteracy.org/research/multistate.cfm

The official site for this project is  https://www.nimss.org/ where project details including participants and minutes are on file. Use “3006” in search.

Spielmaker explained the project’s objectives:

1. Assess the agricultural knowledge of diverse population segments related to agriculture, including consumers, students, and producers. Specifically, explore and/or measure: a. points of acquisition of agricultural knowledge; b. decisions made based on assessed knowledge.
2. Assess attitudes, perceptions, and motivations of diverse population segments related to agriculture, such as consumers, students, and producers. Specifically, explore and/or measure: a. how perceptions, attitudes and motivations are developed; b. decisions made based on assessed attitudes, perceptions, and motivations; c. behavior changes that have occurred due to changes in attitude, perceptions, and/or motivation.
3. Evaluate agricultural literacy programs to measure program impact. Specifically:
   a. measure impacts of agricultural literacy programs related to critical thinking and problem solving; b. explore and evaluate peer and participant-centered agricultural literacy programming methods to determine their effectiveness in addressing defined agricultural literacy outcomes.

Please make sure to double-check that your name is on the participant list along with your selected objectives. This is a five-year project, so please select objectives accordingly.

Rose Judd-Murray gave a presentation on her doctoral research on the development of grade 9-12 assessments for measuring proficiency levels in agricultural literacy—the Judd-Murray Agricultural Literacy Instrument (JMALI). Details are available at https://www.agliteracy.org/research/assessment.cfm This can be used as a summative assessment and is currently being conducted in higher education populations.

Rose then explained how others could use the USU approved IRB protocol within their own institution. She explained that to conduct research with this assessment, institutions who are members of the Smart IRB system could easily use Smart IRB for protocol approval, https://smartirb.org/reliance/. Rose provided a link to a Google Drive folder where her presentation and the forms could be found. Institutions wanting to participate in research with eth JMALI this fall as part of this Multistate Research should go through these steps (these are outlined in the PowerPoint presentation in the Google Drive folder): 

1. If you plan to get involved, email Judd-Murray (juddmurray@usu.edu) with intent to participate and provide your email associated with Google Drive.
2. Access the SMART IRB: Online Reliance System (if your institution is a member, https://smartirb.org/reliance/) or use the Reliance Agreement Template in the Google folder.
3. File a reliance agreement via https://smartirb.org/reliance/ you will receive approval, or submit the Agreement template and Rose will add you to USU’s IRB agreement.
4. This is an exempt protocol; some universities don’t provide the exempt certificate, so you may need to reach out to your institution about their procedures. These materials have already been developed which you can use for your protocol.
5. Anonymity: Most institutions have an anonymity requirement when students are used as a population in on-campus research. USU uses the SONA system to ensure no personal information collected. Your institution may have a similar system. The SONA system, and others like this one, are used to distribute surveys and maintain anonymity. Researchers will need to describe the method of distribution that will be used to maintain anonymity withing their institution. Compensation options are built into SONA, and researchers may want to consider compensation for students who opt into participating (e.g., extra credit for a particular class). The JAMLI Assessment takes approximately 20 minutes.
6. Once notified that an institution wants to participate, Judd-Murray will copy the Qualtrics survey and provide each institutional lead with access to link the Qualtrics Survey with the SONA (or other) institutional program for distribution.

The intended timeline for this project: Fall semester 2020 (first 2 weeks in) and Spring semester 2021 (first 2 weeks in). This means the IRB agreements need to get in place this summer. Judd-Murray encouraged participants to reach out to other colleagues nationwide to participate.

Brian Warnick, administrative advisor for this project, stated that this type of research has been discussed for more than 15 years; this is a true multistate effort and is very exciting! He encouraged as much participation as possible from all project participants.

Meeting adjourned at 2:10 MST.