Brief Summary of Minutes of Annual Meeting

10/26/2023

Attending: Geb Bastian, RJ Gibbs, Mike Puglisi, Kylie Pybus, Kate Yerxa, Annie Roe, Lola Adedokun, Annie Roe, Sarah Amin, Maya Azar Atallah, Helen Chipman, Andrea Leschewski, Stacy Onofrietti, Lexi MacMillan Uribe, Karina Diaz Rios, Diana Romano, Josh Phelps, Tomisin Mayaki, Susan Baker, Nurgul Fitzgerald, Amanda Missimer, Luisa Santos, Katelin Alfaro Hudak

Welcome, Introductions - Geb Bastian and R.J. Gibbs

• Reminders to review the membership roster and update information if needed.

Welcoming Remarks - Dr. Helen Chipman

• History and purpose
  • Started in NC1169 in 2008 to advance EFNEP-related research, program evaluation and outreach, and the work that is done is incorporated into programming.
  • EFNEP is funding 100% programming and research is not an allowable cost, but the work of this group helps ground EFNEP in research.
  • Interface between program and research
  • Please communicate and loop Dr. Chipman in with research ideas as soon as possible in your planning process.
• EFNEP Overview and Program Priorities
• Reviewed current EFNEP and NIFA staffing and vacant position (NIFA EFNEP coordinator is currently vacant).
• EFNEP is currently funded at $70 million through cooperative Extension programs and has been funded for 55 years.
• National goals (FY 2020 – 2024)
  1. Program reach - Increase program reach despite flat funding.
  2. Program reporting - Improve balance between youth and children for program reporting.
  3. Program quality - Maintain important levels of program quality.
  4. Program advancement
     • Use technology in teaching, partnership and coordination, program reporting embraces advances in nutrition education, and program monitoring incorporates advances in technology.

• Current and emerging priorities
  • Increase reach without sacrificing effectiveness.
  • Audience – additional focus on older youth and young adults.
  • Dosage – engaged vs. non-engaged learners; how to get more active participation.
    • Effectiveness of hybrid uses of technology
    • Development and implementation of resources/methods for older youth/young adults with limited financial resources
    • Utility and effectiveness of WebNEERS direct data app
    • Data collection especially 24-hour recall
  • USDA-NIFA overview and research opportunities
  • Research Education and Economics is the mission area that includes NIFA and EFNEP
  • Encourage everyone to think about serving as a grant reviewer to learn about how the process works and strengthen your own proposals.
  • WebNEERS discussion
  • Reminder that if researchers want to use WebNEERS for a study, you do have to get permission to use it. Connect with people at your university who administer EFNEP to make sure that what you are doing works for the program side of EFNEP.
  • WebNEERS cannot include control group data—all data entered into WebNEERS needs to be for actual participants.
  • Potential to develop another WebNEERS that could include control data if there is a need. This system would be a mirror of EFNEP data so it could not include a lot of additional variables not currently being collected in EFNEP.
  • Several members of CBA and others discussed the need for long-term tracking of both EFNEP graduates and controls for longer than a program year.
  • Will explore with the developers about moving forward with a WebNEERS system for controls and other data.

Workgroup Updates:

Spanish FPAQ – Drs. Lexi MacMillan Uribe and Karina Diaz Rios

• Phase 2 – cognitive interviews
• Recruit 60 individuals through CA, NJ, OK, TX
• Adults, Spanish-speakers, qualify for a qualified federal assistance program, cannot have participated in a nutrition education program.
• Waiting for IRB approval to get started.

Youth Evaluation – Dr. Geb Bastian and Tomisin Mayaki

• 10/23 National rollout of EFNEP 6^th – 12^th grade questionnaire
• Submitted to JNEB in February 2023
• Moving forward with test-retest and confirmatory factor analysis
• Criterion validity has not been assessed using “gold standard” measurements of dietary behavior and physical activity such as dietary recall and accelerometer – will be conducting 3 studies to measure these concepts.
• Need help from other states with recruitment.
• Funding ideas to help with participant incentives.

Behavioral CBA – Dr. Annie Roe

• Discussion about including obesity related to HEI
• Difficult to utilize caloric intake
• Reminder that EFNEP is not a weight reduction program
• Obesity is a mediator of the relationship between food and the other diseases we are looking at – obesity many times mediates the relationship between hypertension
• Favor keeping the 24HDR in order to capture the HEI

Biomarker Cost Benefit Analysis Update – Dr. Andrea Leschewski

• Funded by AFRI grant: assess economic value generated in EFNEP through improvement in chronic disease biomarkers
• Use biomarkers in place of behavioral data, since behavioral data is self-reported
  • BMI, blood pressure, HgbAIC
  • CO, FL, MD, WA
• In Year 2
  • Currently collecting data
  • 2 publications: JNEB abstract, 1 JOE paper on pilot data
  • 1 paper under review in JNEB
  • 3 presentations
• Want to create a tool that programs can used to assess cost-benefit of their program
  • Can use behavioral data and/or biomarker data (if you have it); can get data based on just one biomarker
  • Would it be best to have the tool as a downloadable Excel tool or a web-based tool?
    • Do we know someone who knows R or has made this kind of tool? Discussed options within our Extension programs
    • Data is only entered in aggregate, no concern about individual health data
    • Where should this be housed?

24-Hour Diet Recall Pen and Paper – Dr. Karen Franck

• Team has completed data collection (coordinator survey, peer educator interviews and form review)
• 2 manuscripts published in Nutrients and one to be submitted to JNEB
• Need to meet with Dr. Chipman to consider next steps for informing training

Physical Activity Barriers – Dr. Sarah Amin

• Interviews with peer educators to identify facilitators and barriers for successful implementation of physical activity content in EFNEP lessons
• Conducted 15 interviews with peer educators
• Submitted EFNEP Coordinators conference proposal and plan to present at SNEB
• Looking to recruit team members to help expand the project nationally over the coming year

Quality of Life (QoL) – Dr. Annie Roe

Project purpose:

1. To substantiate anecdotal findings with research and provide a valid method to quantify effects of EFNEP programming on Quality of Life (QoL).
2. To investigate the impact of EFNEP education on participants’ and educators’ quality of life.

FY19-FY23 Accomplishments:

• Cognitive interviews to establish face validity
• Test/retest protocol to establish temporal reliability
• Compare results from E-QoL to other established QoL tools to establish construct validity

FY24-FY28 Methods:

• National longitudinal study to determine impact of EFNEP on participants’ QoL
  • Free living, non-pregnant, English speaking
  • E-QoL administered electronically (baseline, post-EFNEP, 6-mth follow up)
• EFNEP participants’ perceptions survey
  • Social support, engagement, experiential learning, and other educational resources or interventions outside of EFNEP
• Class delivery survey
  • Instructors’ number of years of service, the number of lessons, number of teaching hours, instructor’s perceptions of class engagement
• Program Delivery and Perceived Social Support
  • Recruit EFNEP participants to complete E-QoL pre/post EFNEP classes, surveys on participant perception and class delivery administered at post-program.

Impact Statements – Kylie Pybus

Purpose:

1. Explain the development of Impact Statements for the Expanded Food and Nutrition Education Program.
2. Identify data sources in WebNEERS for behavior changes in the core content areas of:
   • Diet Quality, Physical Activity, Food Resource Management, and Food Safety
3. Discuss how to use EFNEP Impact Statements in your program.

Identify specific outcome categories for impact statements and the EFNEP outcomes related to the defined categories.

Summary:

• Outcomes from Adult Questionnaire: These statements are intended to report the behavior changes of adult EFNEP participants and connect those changes to the economic, societal, and environmental outcomes as reported in the literature.
• Legislative Educational Visits: As EFNEP staff, we do not engage in lobbying activities. However, we can meet with legislative staff through educational visits.
• Other Elements in Telling Our Story: Impact Statements are meant to be a starting point or supplement in telling our EFNEP story. EFNEP Coordinators may report other data such as the Program Impact stories and 24-hour dietary recall data.

Retrospective Pre-Test – Nurgul Fitzgerald

• Project was completed, Qing defended her dissertation.
• Side-by-side format was preferred by participants.
• Internal reliability was good for all domains except for food safety.
• Retrospective pre/post seemed to improve evaluation by limiting response shift.
• More exposure (overall time) found better outcomes.
• This is not feasible for the way WebNEERS is currently set up, but we will plan to keep working on it, though it will need to be outside of EFNEP.

CBA Working Group

• New members: Lola Adedokun, Lexi MacMillan Uribe
• Timeline for biomarker study: End date for grant is 12/31/26, so biomarker end point is set
  • Data collection this upcoming year, analysis years 4 and 5
• Timeline for behavioral CBA
  • Finalize matrix for optimal nutrition behaviors this fall
  • Request FY23 data when available
  • Data analysis fall 2024
  • Publish paper in 2025
• Funding to make the tool
  • Reach out to Extension directors, IT, evaluation personnel, etc.
  • Will likely look for small internal funds to start
• Behavioral CBA: determining optimal behavior for each disease, then identifying what the participant has to achieve to reduce risk
  • Discussed importance of reaching a benchmark versus making improvement but not reaching the benchmark
  • Need to assign criteria for improvement for each variable and how many of the criteria have to be improved/met to count as having lowered risk
    • Geb: Foodborne illness
    • Lola: Diabetes and heart disease
    • Nurgul: cancer
    • Lexi: stroke and hypertension
    • Aim to work on these and meet again on December 6^th at 2 PM EST
      • Annie will send a message with tasks and an example
    • Also discussed having individual criteria for each disease or combined disease risk

October 27, 2023

QoL Working Group

TIMELINE

Year 1: October 2023- September 2024

Fall 2023

Cognitive interviews to establish face validity.

• Targeted recruitment
• Call it at the end of 2023; either have saturation or acknowledge limitation

Spring/Summer 2024

Cognitive interviews to establish face validity

• (January/February) Analyze interview data and finalize E-QoL tool
• Annie, Josh Phelps, Michelle Krehbiel

Test/retest protocol to establish temporal reliability & Compare results from E-QoL to other established QoL tools to establish construct validity

• (January/February) Identify comparison QoL tool
  • Perhaps identify a “set” of tools that focus on individual domains/constructs
  • Geb, Sarah, Kate
• (April) Protocol submitted to IRB
• Spring/summer recruit and test
  • Pending funding to support participant compensation (200 subjects x $25 each = $5,000)
• (January) Annie look into internal funding

EFNEP participants’ perceptions survey

• (Spring) Develop survey in conjunction with DAB group to assess social support, engagement, experiential learning, and other educational resources or interventions outside of EFNEP
  • Nurgul, Mike, Annie

Year 2: October 2024- September 2025

Fall 2024

Class delivery survey

• Develop survey in conjunction with DAB group to assess instructors’ number of years of service, the number of lessons, number of teaching hours, instructor’s perceptions of class engagement (more may be included by the DAB group)
  • Nurgul, Mike, Annie

Spring/Summer 2025

National longitudinal study to determine impact of EFNEP on participants’ QoL and influence of program delivery and perceived social support on outcomes

• (April) Protocol submitted to IRB
  • Free living, non-pregnant, English speaking EFNEP participants
  • E-QoL administered electronically (baseline, post-EFNEP, 6-mth follow up)
  • Perceived social support administered electronically at post-EFNEP
• EFNEP peer-educators
  • Class delivery survey completed at post-EFNEP
• Recruitment and data collection

Year 3: October 2025 – September 2026

• Continue data collection

Year 4: October 2026 – September 2027

• Finish data collection
• Data analysis

Year 5: October 2027 – September 2028

• Data analysis
• Data dissemination

DAB Working Group

DAB Update/EFNEP Curricula Assessment

• Previous research has assessed how curricula align with Dietary Guidelines and core areas required by EFNEP.
  • Changes in national guidelines, EFNEP’s lesson content, evaluation methods, physical activity and the introduction of technology has brought the need for this assessment.
• This workgroup is focused on developing best practices.

1. First step: develop content analysis methodology and adjust.
2. Second step: Apply methodology to FY23 data.
3. Then surveying implementation characteristics with coordinators, supervisors, and educators

Content Analysis: assessing modality, dosage, deviation from original content; also, alignment with Dietary Guidelines, Physical Activity guidelines and national evaluation instruments.

• Will assess associations between content/implementation and behavioral change outcomes.
  • Develop study protocol: start with literature review.
    • Susan, Mike, Sarah, RJ, Nurgul, Maya, Karina, Diana
  • Survey of EFNEP supervisors, coordinators, educators: Annie, Karina, RJ, Sarah
  • Started a systematic review of information seeking behavior in the literature.
    • Annie, Nurgul, Maya, Sarah, Kylie, Lexie

Next Meeting: November 28^th at 3 PM; December 19^th at 3 PM

Physical activity: educator interviews

• Expanding nationally to other regions
• Will make sure that we represent all tiers and regions.
  • Annie, RJ, Susan, Geb will pass on to South Dakota.
• Will circulate current interview template.
• Codebook should be finished soon, will have initial themes.

October 28, 2023

Annual Accomplishments Report – Appendix D

• Share your accomplishments to subgroup chairs by December 1, 2023, and the subgroup chairs will send it to Geb and RJ.
  • These accomplishments include journal articles, presentations (oral or poster presentations), grants, fact sheets, book chapters, master’s thesis, dissertations, graduate students

Subgroup Meeting Schedule:

DAB – 4^th Thursday of each month @ 3pm ET for 90 minutes

Workgroup Leadership:

• DAB Leadership: Co-chairs Mike Puglisi and Lexi MacMillan Uribe

• QoL – Leadership: Co-chairs Annie Roe and Stacy Onofrietti
• CBA
  • Behavioral chair – Annie Roe
  • AFRI chair – Andrea Leshewski

NC3169:

• Co-chairs: Geb Bastian and R.J. Gibbs

Mid-year Meeting: (HOLD if we don’t have an in-person meeting in the spring)

• May 16 and 17

Short-term accomplishments

• Approval of a new 5-year plan of work for NC3169
• Investigating an alternative means of EFNEP program evaluation (retrospective test) that may be more valid and reduce participant burden and staff time
• Compiling current practices in 24-hour diet recall administration which will lead to the development of recommendations for best practice
• Estimating the cost benefit of EFNEP programming based on actual changes in participants’ biomarkers

Outputs

          Presentations

1. A Leschewski, S Pierce, and A Roe. 2023. Economic Evaluation of Nutrition Education Interventions – A Biomarker-Based Approach. Agricultural and Applied Economics Association 2023 Annual Meeting, July 23-25, 2023, Washington DC.
2. A Leschewski, S Baker, S Pierce, K Pybus, M Udahogora, K Shelnutt, and A Roe. 2023. A Cost-Benefit Analysis of EFNEP Utilizing Biomarkers of Chronic Disease Risk: Year One. 2023 AFRI Diet, Nutrition, and Prevention of Chronic Disease Program (A1344) Project Director’s Meeting, July 20-23, 2023, Washington DC. 
3. A Leschewski, S Baker, A Roe, K Pybus, M Udahogora, K Shelnutt, and S Pierce. 2023. A Cost-Benefir Analysis of EFNEP Utilizing Biomarkers of Chronic Disease Risk. National Institutes of Health Prevention Research Coordinating Committee (PRCC) Meeting. October 10th, 2023.
4. Chen Q, Bastian GE, Palmer-Keenan D, Fitzgerald N. Eliciting Temporal Perceptions of a Retrospective Food and Physical Activity Behaviors Survey Among Low Income Adults. J Nutr Ed Behav. 55(7):S70-S71. DOI: 10.1016/j.jneb.2023.05.155 (oral abstract presentation at the Society for Nutrition Education and Behavior meeting)
5. Doctoral Dissertation Defense (oral presentation): Chen Q. (September 25, 2023). Examining the Use of Retrospective Pretest-Posttest Design to Evaluate the Behavioral Outcomes of Adult Participants in the Expanded Food and Nutrition Education Program. New Brunswick, NJ.

Publications

1. A Leschewski, MC Aragon, D Weatherspoon, K Barale, G Auld, R Acquah-Sarpong*, and S Baker. Expanded Food and Nutrition Education Program Generates Economic Value Through Body Mass Index Improvement: A Cost-Benefit Analysis. Journal of Extension. (Accepted November 2022 – Currently in-press).
2. A Leschewski, S Baker, S Pierce, K Pybus, M Udahogora, K Shelnutt, and A Roe. 2023. A Cost-Benefit Analysis of EFNEP Utilizing Biomarkers of Chronic Disease Risk: Year One. Journal of Nutrition Education and Behavior, 55(7), 105.  
3. A Leschewski, S Pierce, MC Aragon, SS Baker, M Udahogora, K Pybus, NO Duffy, A Roe, and K Sankavaram. 2023. A Cost-Benefit Analysis of Adult EFNEP Utilizing Biomarkers of Chronic Disease Risk: Intervention Research Methods. Journal of Nutrition Education and Behavior. (Revise and resubmit).
4. Fuller, S. M., Phelps, J. A., Baker, S., & Walsh, J. (2023). Qualitative Analysis of the Expanded Food and Nutrition Education Program’s 24-hour Dietary Recall. The Journal of Extension, 61(1), Article 5. https://doi.org/10.34068/joe.61.01.05
5. Pybus, K., Gibbs, R. L., Franck, K., & Aragón, M. (2023). Development and Evaluation of Impact Statements for the Expanded Food and Nutrition Education Program (EFNEP). The Journal of Extension, 61(2), Article 19. https://doi.org/10.34068/joe.61.02.19
6. Roe, A., Sankavaram, K., Baker, S., Frank, K., Puglisi, M., Earnesty, D., Henson, T., 2023, 24-Hour Dietary Recall in the Expanded Food and Nutrition Education Program: Perspective of the Program Coordinator. Nutrients, 15(19).  https://doi.org/10.3390/nu15194147
7. Franck, K., Puglisi, M., Roe, A., Baker, S., Henson, T., Earnesty, D., Sankavaram, K., 2023, Conducting 24-Hour Dietary Recalls in Group Settings with Adults Having Low-Income: Perspectives of EFNEP Peer Educators. Nutrients, 15(18). https://doi.org/10.3390/nu15184020
8. Earnesty, D., Franck, K., Baker, S., Roe, A.J., Puglisi, M., Sankavaram, K. Dietary Recall Standardization Needed for Expanded Food and Nutrition Education Program. JNEB. ( in review)
   
   Graduate Students Trained/In training
9. Qing Chen, Ph.D., Rutgers University, Advisor: Nurgul Fitzgerald (graduated Oct 2023)
10. Maya Azar Atallah, Ph.D., Rutgers University, Advisor: Nurgul Fitzgerald
11. Tomisin Mayaki, Ph.D., South Dakota State University, Advisor: Geb Bastian
12. Luisa Santos, M.S., University of Rhode Island, Advisor: Sarah Amin
    
    Awards
13. Excellence in Extension Award for a Team (accepted on behalf of the AES Multistate Research Group NC3169: EFNEP-Related Research, Program Evaluation, and Outreach), USDA National Institute of Food and Agriculture & APLU Extension Committee on Organization and Policy, 2022
    1. The award monies ($5000) were used to support NC3169 research expenses through competitive internal mini-grants, which were awarded to Nurgul Fitzgerald, Andrea Leschewski, and Lexi MacMillan Uribe.

Activities

FPAQ Retrospective Testing

• Cognitive interviews for testing the retrospective pretest-posttest format were completed with 45 adult EFNEP graduates in CO, NJ, GU, MD, and TN.
• Retrospective pre-posttest was piloted among 284 adult EFNEP participants in a 3-group design. Results indicated that there was response shift bias among specific questions, such as regarding preparing meals at home.
• Follow-up semi-structured interviews with some of the participants that completed the pilot test confirmed that response shift bias was at play.
• Youth Evaluation and Spanish FPAQProtocols for both projects have been developed and are being submitted/reviewed by IRB

24-Hour Diet Recall

• Data collection is completed for all 3 phases: survey of EFNEP coordinators, interviews with peer educators, and a form review of diet recall methods
• 2 manuscripts have been published and 1 is submitted.

Physical Activity Barriers

• 15 semi-structured interviews with peer educators in the Northeast region have been conducted, with plans to expand to a national sample.

Cost-Benefit Analysis

• AFRI Biomarker project: data collection is underway
• Protocol is being developed for a cost-benefit analysis using participant responses to the FPAQ

Quality of Life

• 4 new cognitive interviews were conducted
• Data was analyzed to identify which questions need to further testing and which geographic areas/demographic characteristics need representation 
• A temporal reliability protocol has been developed

Impact Statement Infographics

In FY22, four impact statements for the Expanded Food and Nutrition Education Program (EFNEP) were developed and evaluated. The statements were drafted after conducting literature reviews for core content areas of EFNEP that include diet quality, food resource management, physical activity, and food safety. This work was done based on the previous work completed in NC3169, to develop the valid and reliable 30-item adult questionnaire. 

A manuscript was submitted in FY22 and accepted in the Journal of Extension in FY23.  Along with this manuscript, a 2-page infographic was developed and made available to EFNEP implementing agencies to provide a framework for promoting adult behavior change as a result of their participation in nutrition education programming. This document will soon be made available on the EFNEP digital resources page for future use.

Milestones

• Targeted recruitment strategies will be explored for qualitative EFNEP studies (e.g., QoL and Spanish FPAQ cognitive interviews). One such proposed idea was the development of a database of all 76 EFNEP programs, contact information, and relationships with members of NC3169 to facilitate asks for study recruitment.
• Three new graduate students joined the project: Maya Azar Atallah, Tomisin Mayaki, and Luisa Santos. Four new faculty and Extension professionals have been recruited to the new project cycle: Dr. Katelin Alfaro Hudak, Dr. Stacy Onofrietti, Ricardo Kairios, and Dr. Krystal Hodge. Dr. Lola Adedokun rejoined the group after a brief absence due to changing Extension responsibilities.
• Data collection for Spanish FPAQ cognitive interviews, Youth Evaluation and QoL validity/reliability, and national Physical Activity barrier interviews will begin in 2024.
• A systematic review of nutrition information seeking behavior will begin in 2024.
• A new update to WebNEERS to collect data for non-EFNEP participants (control data) will be explored further in 2024.
• Internal grants will be written for research expenses, namely subject payment, in 2024.

1. A Leschewski, MC Aragon, D Weatherspoon, K Barale, G Auld, R Acquah-Sarpong*, and S Baker. Expanded Food and Nutrition Education Program Generates Economic Value Through Body Mass Index Improvement: A Cost-Benefit Analysis. Journal of Extension. (Accepted November 2022 – Currently in-press).
2. A Leschewski, S Baker, S Pierce, K Pybus, M Udahogora, K Shelnutt, and A Roe. 2023. A Cost-Benefit Analysis of EFNEP Utilizing Biomarkers of Chronic Disease Risk: Year One. Journal of Nutrition Education and Behavior, 55(7), 105.  
3. A Leschewski, S Pierce, MC Aragon, SS Baker, M Udahogora, K Pybus, NO Duffy, A Roe, and K Sankavaram. 2023. A Cost-Benefit Analysis of Adult EFNEP Utilizing Biomarkers of Chronic Disease Risk: Intervention Research Methods. Journal of Nutrition Education and Behavior. (Revise and resubmit).
4. Fuller, S. M., Phelps, J. A., Baker, S., & Walsh, J. (2023). Qualitative Analysis of the Expanded Food and Nutrition Education Program’s 24-hour Dietary Recall. The Journal of Extension, 61(1), Article 5. https://doi.org/10.34068/joe.61.01.05
5. Pybus, K., Gibbs, R. L., Franck, K., & Aragón, M. (2023). Development and Evaluation of Impact Statements for the Expanded Food and Nutrition Education Program (EFNEP). The Journal of Extension, 61(2), Article 19. https://doi.org/10.34068/joe.61.02.19
6. Roe, A., Sankavaram, K., Baker, S., Frank, K., Puglisi, M., Earnesty, D., Henson, T., 2023, 24-Hour Dietary Recall in the Expanded Food and Nutrition Education Program: Perspective of the Program Coordinator. Nutrients, 15(19).  https://doi.org/10.3390/nu15194147
7. Franck, K., Puglisi, M., Roe, A., Baker, S., Henson, T., Earnesty, D., Sankavaram, K., 2023, Conducting 24-Hour Dietary Recalls in Group Settings with Adults Having Low-Income: Perspectives of EFNEP Peer Educators. Nutrients, 15(18). https://doi.org/10.3390/nu15184020
8. Earnesty, D., Franck, K., Baker, S., Roe, A.J., Puglisi, M., Sankavaram, K. Dietary Recall Standardization Needed for Expanded Food and Nutrition Education Program. JNEB. ( in review)