Duration: 10/01/2013 to 09/30/2018

Administrative Advisor(s):

NIFA Reps:

Non-Technical Summary

Statement of Issues and Justification

Statement of Issues and Justification A crisis now exists for obtaining data to enable us to understand the human capital characteristics of rural regions of the United States and the development challenges they face. The Decennial Census Long Form, used throughout the 20th century to collect essential data on population characteristics from 1 in every 6 households throughout the United States, was discontinued after the 2000 Census. As a result, there is no longer a regular source of reliable data on the characteristics of people who live in each county and community of the United States. Its replacement, the American Community Survey, collects data from about two million U.S. households each year, thus making it possible to produce acceptable city and urban county estimates (for population characteristics such as age, education, income, occupation, commute time to work, and other essential indicators of human capital and well-being). Data from this survey can also be accumulated across rural counties over multiple years so that acceptable estimates for rural regions containing a number of counties may also be obtained. However, if ones interest is in a specific rural county or even small clusters of rural counties, no reliable data now exist. This is, in particular, a problem for sparsely population regions of Western United States. The absence of such data becomes a major concern when professionals and businesses interested in economic development activities are trying to assess and use key information on the economic development potential of such areas. The void produced from loss of the long form Census data is not filled by other national surveys. The sample sizes for most nationwide surveys are too small to reliably measure attributes of specific rural areas since the variability is so large, and, in many cases, sample sizes are too small for use at the state level. If data are going to be available for rural places and people on their human capital characteristics and other relevant information such as farm and agricultural group activities and interests, it is essential that data on these populations be collected in other ways. Understanding attitudes, behaviors and demographic characteristics of the general public is only part of the data problem that now prevails among agricultural and rural populations. Sample surveys, many of which have been conducted by professionals in Agricultural Research and Extension programs, have long been used to provide specific data of interest to local municipalities as a needed supplement to the questions formerly asked in the Decennial Census. In addition, sample surveys have been used to regularly and efficiently obtain information from agricultural production groups, rural interest groups and others to describe problems and identify solutions for which no official statistics are available. The capability of sample surveys that sets them apart from other methods of collecting data is that only a few hundred or thousand questionnaires collected randomly in a carefully designed sample of a specific population (e.g. a rural community) allows one to estimate characteristics (from attitudes to behaviors) within a few percentage points of the actual population parameters at a high level of statistical confidence for the population being studied (Dillman, Smythe, and Christian, 2009). No other social science data collection method has this capability. However, this capability is in danger of not being realized because of rapid technological and other changes that make traditional data collection methods less effective. Finding ways to improve sample survey methods so they can be used to meet rural and agricultural data needs is the proposed purpose for this Coordinating Committee. Background In the last quarter of the 20th century, telephone surveys became the main method of conducting general public surveys needed for estimating a wide range of behaviors from employment rates and the effects of rural development initiatives on entrepreneurial efforts to the consumption of food products. During this time, the telephone also became the dominant mode for surveying opinions, e.g. community and employment satisfaction, desire for new products and services, and satisfaction with rural and urban development activities. Face-to-face interviews continue to be used for the nation's most critical national surveys (e.g. the monthly Currently Population Surveys estimate of employment rates and the USDA's Agricultural Resource Management Annual Survey of Farm operators) and mail surveys are used for surveys of list populations (e.g., participants in Extension programs whose names and addresses were available.) However, telephone survey methods are no longer working well for producing valid statistical estimates because of low response rates (often less than 10%), and reduced household coverage. Fewer than 70% of households now have landlines, the traditional sample frame for drawing household samples via random-digit dialing. Cell phones can now be included in random sample frames, but because of the loss of geographythe area code being a valid indicator of where one livessub-national (state, county and community) surveys by telephone cannot be counted on to accurately represent the populations they purport to describe with precision. In response to this trend, some surveyors have tried to turn to the web-only surveys as an alternative data collection strategy. The difficulty with this proposed solution is that only about 75% of the U.S. households have access to the web, and only about 67% of households have access that is used more than once every two weeks. Some who do have web access also seem unwilling to respond to web surveys. In addition there is no usable sample frame for contacting the general public by email, confounding the problem of gaining household responses over the web. It is also evident that those without web access have significantly less education, lower incomes, are less likely to be married, have less stable employment and differ in other significant ways, as shown by ( Brenner & Raine, 2012; Zickhur & Smith, 2012). Web-only surveys are not yet able to produce valid results in household surveys. The Work of WERA 1010 and Its Predecessors WERA 1010, now in its fifth and final year, has a long history of studying ways of improving survey methodologies for rural and agricultural surveys that can provide data which are impossible to obtain for much of Rural America other than through the use of sample survey methods. Research has been aimed at improving mail, web, and telephone methods, including use in mixed-mode surveys, whereby some people are surveyed via one mode (e.g. mail) and others by a second mode (e.g. web or telephone). Research and its application to survey practice has been undertaken cooperatively for more than 20 years under Regional Project W-183, followed by Western Region Coordinating Committees (WERA 1001 and WERA 1010), with the aim of improving survey methods for use in rural and agricultural surveys. During the last five years, research under WERA 1010 has been conducted with increased urgency, hoping to find solutions to current sample survey design problems. It is important to put the work of WERA 1010 into the context of work conducted during the previous five years under WERA 1001 (2002-2007). Historically, that Committee attempted to focus its interactions and work around the most pressing issues facing survey methodology. Thus the focus of WERA 1001 was predominately on measurement issues that affected the use of mixed-mode surveys whereby data were collected by aural (interview) and visual (self-administered) questionnaires, considered then to be the major problem being experienced in survey methodology. A significant proportion of the publications produced by that committee reported measurement experiments aimed at learning the extent and reasons that visual questionnaires often produced different answers to the same question (Christian and Dillman, 2004). In 2007 it was becoming apparent that the prevalence of landline telephones was destined to continue its decline. Because of the reduced coverage as well as dismal response rates, a new approach to surveying was needed. It was also becoming apparent that the best sample frame for reaching rural households was the U.S. Postal Services residential Delivery Sequence File which became available for survey sampling. WERA participants began researching the possibilities this list provides as a replacement for telephone methods because of pioneering work by Todd Rockwood, a professor at the University of Minnesota who began attending WERA meetings at that time, prior to becoming a member of WERA 1010. When WERA 1010 was formed there was a significant shift in research conducted and disseminated by its members. At that time the predominant thinking among survey researchers was that contacting people by mail and asking them to respond over the web, if they were able, would result in very low response rates. WERA members began researching several aspects of this problem. A head-to-head comparison of telephone-only with mail-only and mail with a push to web was begun by Lesser (2011a). That work clearly showed that response rates to a mail-only approach was superior to the response rates that could be obtained by telephone only. Work by Dillman and a research team at Washington State University, provided strong evidence that it was possible to push a significant proportion (43%) of households to the web. However, this work also showed that following up the early web response with a mail follow-up obtained responses from people (older, less educated, with lower incomes) not able or willing to respond over the web. The resulting publication, Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21 century, (Smyth, Dillman, Christian and ONeill, 2010) provided initial evidence that pursuing a web+mail methodology could provide an effective alternative to the telephone-only survey approach that had dominated survey methodology from the 1970s through the 1990s. It also showed that the method could be especially effective in rural communities and regions, where American Community Survey demographic estimates are least likely to be available. This work on general public surveys was greatly expanded during the life of WERA 1010. Statewide general public survey experiments in Oregon (Lesser et.al, 2007a; Lesser et.al.2008a; Lesser et.al, 2008b; Lesser & Yang, 2009; Lesser et.al, 2010; Lesser et.al. 2011a; Lesser et.al, 2011b;) and Washington (Messer & Dillman, 2011; Messer, 2012) confirmed that the web+mail methodology could be effectively used to push many respondents to the web. It also showed that that the mail follow-up was essential for achieving better representation of the population, as measured by American Community Survey results (which Smyth et al. (2010) could not evaluate because of such data not being adequate for estimating true population characteristics of the rural region in Washington and Idaho that was studied). Studies in these states have used different methodologies, survey topics and populations, but all find that there is much potential for combining mail and web data collection. Finally, Israel (2009a; 2009b; 2010b) showed this was true for list populations too. WERA 1010 pursued many different dimensions of this challenge. Work was also begun on how one might leverage the survey situation of having postal addresses and email addresses for particular clientele groups. Several studies by Israel and his colleagues tested multiple ways of combining mail contact and email follow-ups in order to bolster responses and improve data quality for evaluation surveys of Extension clients (Israel, 2011; 2012a). This work found that email augmentationsending quick emails to make responding easier for individuals who had just been contacted by mail also tested on students by Millar and Dillman (2011), was effective for client surveys. Other work by WERA 1010 members was aimed at advancing research on visual design and layout effects on measurement studied by WERA 1001. These included open-ended question effects (Israel, 2010a; Israel & Galindo-Gonzalez, 2010; Lesser & Newton, 2007b; Smyth & Dillman, 2007; Swinford, 2007), response option format (Smyth et al., 2006a) and general visual layout effects (Christian & Dillman, 2004; Dillman et al., 2005; Israel, 2006; Mahon-Haft and Dillman, 2010; Rookey & Dillman, 2008; Smyth et al., 2006b; Toepoel & Dillman, 2011). Work also was conducted regarding effects of personalization and incentives on response behavior (Beebe et al., 2005; Dillman et al., 2007; Wilcox et al., 2010). In addition, considerable research involving Fred Lorenz (Conger et al, 2011; Lorenz et.al, 2010) were focused on applying data collected based on WERA research experiments to the design and analysis of longitudinal surveys. One of the challenging issues faced by WERA 1010 was a finding that permeated the work of all members,  as a general rule, the use of a mail only data collection strategy consistently produced higher response rates than the web+mail strategy developed by committee members. For example, 10 comparisons of mail-only vs. web+mail response rate comparisons conducted from 2007 to 2012 have shown that in every case the mail-only response rates are higher than the web+mail treatments, with a difference of 2-15 percentage points (mean = 10 percentage points) (Smyth et al., 2010, Messer and Dillman, 2011, Messer, 2012, Messer et al., 2012). However, mail is considered by many surveyors to be old-fashioned, and to suffer from many problems, e.g. higher item nonresponse than web, the difficulty of using extensive numbers of branching questions, higher costs for printing, postage, and labor, and more time to collect the data. Yet, it became apparent to members of WERA 1010 that mail-only surveys had become our highest response rate method for many surveys, and for the general public, had superior coverage to other modes. Thus, part of our work shifted to identifying and jointly addressing problem issues associated with mail and how to maximize its effectiveness in surveying the general public. A related question for which we sought and are still currently seeking answers was to find the optimal mix of web+mail for achieving cost and data quality objectives. For example, its been argued that higher item nonresponse rates to mail questionnaires diminish the effectiveness of obtaining higher unit response rates. WERA members organized a special session at the 2011 meetings of the American Association for Public Opinion Research to report findings from our research. The papers presented there were published in 2012 by Survey Practice, an electronic journal created by AAPOR to bring important research findings to practitioners in a format that they could use. The consistent finding of these articles (Messer, Edwards & Dillman, 2012; Lesser, Newton & Yang, 2012; Israel & Lamm, 2012; Millar & Dillman, 2012) is that the item nonresponse differences while higher for mail in most studies, are only insignificantly greater, thus suggesting this problem is not an important barrier to the use of a web+mail methodology. One possible reason for the small difference between mail and web item nonresponse pertains to the construction methods used for both web and mail questionnaires by WERA members is the use of the visual design techniques developed and tested by WERA 1001 members between 2002-2007. One of the other means by which results from WERAs work has been disseminated to practitioners is through the book, Internet, Mail and Mixed-Mode Surveys: The Tailored Design Method, 3rd edition (Dillman, Smyth, and Christian, 2009). That book on data collection methods included most of the major findings and recommendations developed through work under WERA 1001. This book, in its combined editions appears to be the most cited reference for designing sample survey data collection procedures now in print. It will continue to provide a major outlet for transforming the work by WERA members into survey practice recommendations inasmuch as work has commenced on the 4th edition of this work is now beginning. Renewal of WERA 1010 will allow current members, as well as new members, to continue and initiate new coordinated scientific work across states on the design and conduct of sample surveys. It is apparent as this proposal is written that the work of this committee will focus in part on refining the web+mail methodology invented by members of this committee, for example, testing the limits of holding back on mail follow-up until later in the implementation process, testing the ability to collect data from lesser known sponsors (e.g. universities in one state trying to collect data in another), the judicious use of incentives more than once during the data collection process. We will work to identify the most cost-efficient design using both web and mail that will optimize response rates for the lowest cost per completed questionnaire. We will also investigate how social media can be utilized in our survey methods, such as in recruitment to surveys. In addition, we will explore the use of smartphones and tablets to better determine whether surveys can be successfully completed using these devices. The advent of integrative technologies, such as QR codes, might also provide opportunities to advance survey methodologies.

Objectives

Procedures and Activities

WERA 1001 meets annually for researchers to share their activities and plans for the coming year. All participants are encouraged to present tentative plans on their future studies in order to obtain advice and comment from other participants. One of the typical outcomes of these discussions is to encourage other participants to develop a new test and/or repeat a test conducted by other colleagues on the committee in their own survey work. Previous work on WERA and its W-183 predecessor has shown that members are particularly effective in developing new tests because of their roles in experiment station, extension and other work locations in helping design surveys. WERA members are often consulted by other Agricultural Experiment Station and Extension employees for improving their survey designs. Opportunities come-up each year to do experiments by convincing these professionals that inclusion of an experiment will help them design a better survey. The joint work now underway on address-based sampling and the use of mail with web data collection methods are examples of how committee members actively influence others to conduct new experiments, not previously planned. Because survey methodology is changing so rapidly with the decline of telephone, and the desire to be cost-effective through use of the web, it is difficult to anticipate the exact experiments that members of the committee will conduct during the life of the committee. The typical time-lag between development of a new idea and getting it tested by WERA members in this area of research is several months to a year. Committee members report at the annual meeting, typically held in February of each year. When the results of a new idea tested during the year appear promising, another committee member will find a way to provide a further test (and sometimes an exact replication) the same year. Thus, the committee is quite dynamic in its operation. We expect this philosophy of operation to continue under renewal of the coordinating committee.

Expected Outcomes and Impacts

Projected Participation

View Appendix E: Participation

Educational Plan

Educational outreach to professionals and graduate students involved in agricultural and rural research. WERA committee members have conducted presentations, workshops, and short courses at conferences attended by agricultural and rural researchers, including that of the Rural Sociological Society, the Southern Association of Agricultural Scientists, the American Association for Public Opinion Research, and the American Statistical Association to inform participants about the latest methodological findings of the committee and their application. We plan to continue our outreach efforts with these and other relevant groups. Outreach to Extension professionals working in the field. County agents and state specialists frequently conduct surveys to assess needs and evaluate programs while dealing with the constraints of limited resources and access. Committee members, including Glenn Israel, will conduct in-service training workshops which incorporate WERA research findings into practical steps for participants to conduct cost-effective, credible surveys. In addition, WERA members will develop brief fact sheets on selected topics to provide user-friendly advice about survey procedures to county agents and specialists. Finally, WERA committee members, including Israel (Florida) and Lesser (Oregon), will assist, consult, and collaborate with others to conduct surveys of extension audiences (e.g., Singletary & Smith, 2006). In these instances, methods developed by the committee are embedded into the survey design and implementation. Outreach to survey methodologists, evaluators, and other relevant professional groups. WERA members regularly conduct presentations and workshops for survey methodologists and professional conferences (e.g., American Association of Public Opinion Research). In addition, members periodically present findings to other relevant groups, such as participants at the American Evaluation Association conference (e.g., Israel, 2012b). This facilitates the diffusion of research findings to a broad array of practioners involved in surveys, including those working in the agricultural and rural development fields. Publication of research findings in peer-reviewed journals. WERA committee members will publish study results in leading journals in the survey methodology field as well as applied journals used by the committees primary stakeholders. The committee has established a long-standing record of productivity in well-respected journals, including Public Opinion Quarterly, Rural Sociology, Journal of Official Statistics, and numerous other journals and this work is frequently cited by other survey scholars. The committee plans to continue publishing in these venues.

Organization/Governance

A chair and secretary will be elected annually. The chair will be responsible for developing an agenda for the annual meeting, and facilitating communication among participants throughout the year. The secretary will be responsible for taking minutes and mailing them to the Administrative Advisor and members.

Literature Cited

(References include those cited plus recent additional work by participants that provides selective background for the proposed coordinating committee activities completed under WERA-1001 and WERA-1010). Beebe, T.J., M.E. Davern, D.D. McAlpine, K.T. Call, and T.H. Rockwood. 2005. Increasing Response Rates in a Survey of Medicaid Enrollees: The Effect of a Prepaid Monetary Incentive and Mixed Modes (Mail and Telephone). Medical Care. 3(4):411-4. Brenner, Joanna and Lee Rainie. (2012). Pew Internet: Broadband. Pew Internet & American Life Project, Pew Research Center. Accessed June 3, 2012 at: http://pewinternet.org/Commentary/2012/May/Pew-Internet-Broadband.aspx Christian, L. and D.A. Dillman. 2004. The Influence of Symbolic and Graphical Language Manipulations on Answers to Paper Self-Administered Questionnaires. Public Opinion Quarterly. 68(1):57-80. Conger, R. D., Stockdale, G. D., Song, H., Robins, R. W., & Widaman, K. F. (2011). Predicting change in substance use and substance use cognitions of Mexican origin youth during the transition from childhood to early adolescence. In Y. F. Thomas, L. N. Price, & A. V. Lybrand (Eds.), Drug use trajectories among African American and Hispanic youth. New York: Springer. Dillman, D.A., V. Lesser, R. Mason, J. Carlson, F. Willits, R. Robertson, and B. Burke. (2007). Personalization of Mail Surveys for General Public and Populations with a Group Identity: Results from Nine Studies. Rural sociology, 72(4), 632-646. Dillman, D.A. 2006. Why Choice of Survey Mode Makes a Difference. Public Health Reports. 121(1):11-13. Dillman, D.A., A. Gertseva, and T. Mahon-Haft. 2005. Achieving Usability in Establishment Surveys Through the Application of Visual Design Principles. Journal of Official Statistics. 21(2):183-214. Dillman, D.A. and L.M. Christian. 2005. Survey Mode as a Source of Instability Across Surveys. Field Methods. 17(1):30-52. Dillman, D. A., Smyth, J. D., & Christian, L. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method. (3rd ed.). Hoboken, NJ: John Wiley and Sons. Israel, G. D. 2006. Visual Cues and Response Format Effects in Mail Surveys. Paper presented at Southern Rural Sociological Association, Orlando, FL, February. Israel, G. D. 2009a. Obtaining Responses by Mail or Web: Response Rates and Data Consequences. Survey practice, June. Available at: http://surveypractice.org/2009/06/29/mail-vs-web/. Israel, G. D. 2009b. Obtaining Responses by Mail or Web: Response Rates and Data Consequences. JSM Proceedings, Survey Research Methods Section. 5940-5954. Available at: http://www.amstat.org/Sections/Srms/Proceedings/. Israel, G. D., & Galindo-Gonzalez, S. 2010. Getting Optimal Answers to Open-ended Questions: An Experiment with Verbal Prompts and Visual Cues. Paper presented at the annual meeting of the Rural Sociological Society, Atlanta, GA, August. Israel, G. D. 2010a. Effects of Answer Space Size on Responses to Open-ended Questions in Mail Surveys. Journal of official statistics, 26(2), 271-285. Israel, G. D. 2010b. Using Web Surveys to Obtain Responses from Extension Clients: A Cautionary Tale. Journal of extension, 48(4), available at: http://www.joe.org/joe/2010august/a8.php. Israel, G. D. 2011. Strategies for Obtaining Survey Responses from Extension Clients: Exploring the Role of E-mail Requests. Journal of extension, 49(3), available at: http://www.joe.org/joe/2011june/a7.php. Israel, G. D., & Lamm, A. J. 2012. Item Non-response in a Client Survey of the General Public. Survey Practice, April. Available at: http://surveypractice.wordpress.com/2012/04/17/item-nonresponse-in-a-client-survey-of-the-general-public/#more-6070 Israel, G. D. 2012a. Combining Mail and E-mail Contacts to Facilitate Participation in Mixed-Mode Surveys. Social Science Computer Review. Published online November 28, 2012 at http://ssc.sagepub.com/content/early/2012/11/26/0894439312464942. doi: 10.1177/0894439312464942 Israel, G. D. 2012b. Mixed-Mode Methods for Follow-up Surveys of Program Participants. Demonstration presented at the Annual Conference of the American Evaluation Association, Minneapolis, MN. October. Lesser, V.M., & L. Newton. 2007a. Comparison of Delivery Methods in a Survey Distributed by Internet, Mail, and Telephone. Proceedings of the International Statistics Institute Meetings. Lisbon, Portugal, August. Lesser, V.M., & L. Newton. 2007b. Effects of Mail Questionnaire formats on answers to Open-Ended Questions. Unpublished paper presented at annual meetings of the American Statistical Association, Salt Lake City, Utah. August 3, 2007. Lesser, V.M., K. Hunter-Zaworski, L. Newton and D. Yang. Using Multiple Survey Modes in a Study of Individuals with Disabilities. Presented at the American Association for Public Opinion Research, New Orleans, Louisiana, May, 2008a. Lesser, V.M., L. Newton, and D. Yang. Evaluating Frames and Modes of Contact in a Study of Individuals with Disabilities. Presented at the American Statistical Association Meetings, Denver, Colorado, August, 2008b. Lesser, V.M. and D. Yang. Alternatives to Phone Surveys: a study comparing Random Digit Dialing with Mail and Web using the Postal Delivery Sequence File. Presented at the American Association for Public Opinion Research, Hollywood, Florida, May, 2009. Lesser, V.M., L. Newton, and D. Yang. Does Providing a Choice of Survey Modes Influence Response? Presented at the American Association for Public Opinion Research, Chicago, Illinois, May, 2010. Lesser, V.M., L. Newton, and D. Yang. Evaluating Methodologies to Increase Internet Responses in Mixed-Mode Surveys. Proceedings of the International Statistics Institute Meetings, Dublin, Ireland, August, 2011a. Lesser, V.M., A. Olstad, D. Yang, L. Newton. Comparing Item Nonresponse and Responses Across Modes in General Population Surveys. Presented at the American Association for Public Opinion Research, Phoenix, Arizona, May, 2011b. Lesser, V. M., Newton, L. A., & Yang, D. (2012). Comparing item nonresponse across different delivery modes in general population surveys. Survey Practice, April. Available at: http://surveypractice.wordpress.com/2012/04/17/comparing-item-nonresponse-across-different-delivery-modes-in-general-population-surveys-2/#more-6026 Lorenz, F., L. Hildreth, V.M. Lesser, & U. Genshel. (2010). General-specific questions in survey research: a confirmatory factor analysis approach. Presented at the Annual Meeting of the Rural Sociology Society, August. Mahon-Haft, T. A., & Dillman, D. A. (2010). Does Visual Appeal Matter? Effects of Web Survey Aesthetics on Survey Quality. Survey research methods, 4(1), 43-59. Messer, B. L., & Dillman, D. A. (2011). Using address-based sampling to survey the general public by mail vs. Web plus mail. Public Opinion Quarterly, 75(3), 429-457. doi: 10.1093/poq/nfr021 Messer, B. L., Edwards, M. L., & Dillman, D. A. (2012). Determinants of item nonresponse to web and mail respondents in three address-based mixed-mode surveys of the general public. Survey practice, April. Available at: http://surveypractice.wordpress.com/2012/04/17/determinants-of-item-nonresponse-to-web-and-mail-respondents-in-three-address-based-mixed-mode-surveys-of-the-general-public/#more-5998 Messer, Benjamin L. (2012). Pushing households to the web: Experiments of a web+mail methodology for conducting general public surveys. Dissertation. Pullman, WA: Washington State University. Millar, M. M., & Dillman, D. A. (2011). Improving response to Web and mixed-mode surveys. Public Opinion Quarterly,75,(2), 249-269. doi: 10.1093/poq/nfr003. Millar, M. M., & Dillman, D. A. (2012). Do Mail and Internet Surveys Produce Different Item Nonresponse Rates? An Experiment Using Random Mode Assignment. Survey practice, April. Available at: http://surveypractice.wordpress.com/2012/04/17/do-mail-and-internet-surveys-produce-different-item-nonresponse-rates-an-experiment-using-random-mode-assignment/ Redline, C.D., D.A. Dillman, A. Dajani, and M.A. Scaggs. 2003. Improving Navigational Performance in U.S. Census 2000 By Altering the Visual Languages of Branching Instructions. Journal of Official Statistics. 19(4):403-420. Rookey, Brian, & Dillman, Don A. 2008. Do Web and Mail Respondents Give Different Answers in Panel Surveys. Unpublished paper prepared for Annual Conference of the American Association for Public Opinion Research. New Orleans, LA. Singletary L. and M. Smith. 2006. Nevada Agriculture Producer Research and Education Needs: Results of 2006 Statewide Needs Assessment. University of Nevada Cooperative Extension, EB-06-02. pp. 118. Smyth, J.D., D.A. Dillman, L.M. Christian, & M.J. Stern. (2006a). Comparing Check-All and Forced-Choice Question Formats in Web Surveys. Public Opinion Quarterly. 70(1):66-77. Smyth, J.D., D.A. Dillman, L.M. Christian, & M.J. Stern. (2006b). Effects of Using Visual Design Principles to Group Response Options in Web Surveys. International Journal of Internet Science. 1(1):5-15. Smyth, J.D., & Dillman, D.A. (2007). Open-ended Questions in Mail, Web and Telephone Surveys. Unpublished paper presented at annual meeting of the American Statistical Association, Salt Lake City, Utah. August. Smyth, J. D., Dillman, D. A., Christian, L. M., & ONeill, A. C. (2010). Using the Internet to survey small towns and communities: Limitations and possibilities in the early 21st century. American Behavioral Scientist 53(9):325-37. Swinford, Stephen. 2007. How Answer Spaces Affect Answers to Open-Ended Questions in Mail Surveys; Results from Multiple Experiments. Unpublished paper presented at Annual Meeting of the American Statistical Association, Salt Lake City, Utah. August. Toepoel, V., & Dillman, D. A. (2011). Words, Numbers, and Visula Heuristics in Web Surveys: Is There a Hierarchy of Importance? Social Science Computer Review, 29(2), 193-207. Wilcox, A. S., Giuliano, W. M., & Israel, G. D. 2010. Response Rate, Nonresponse Error, and Item Nonresponse Effects When Using Financial Incentives in Wildlife Questionnaire Surveys. Human dimensions of wildlife, 15(4), 288-295. Zickhur, Kathryn & Aaron Smith. (2012). Digital differences. Pew Internet & American Life Project, Pew Research Center. Accessed June 4 2012 at: http://pewinternet.org/~/media//Files/Reports/2012/PIP_Digital_differences_041312.pdf

Attachments

Land Grant Participating States/Institutions

AZ, FL, GA, IA, IL, MO, MT, OR, TX, UT, WA

Non Land Grant Participating States/Institutions

University of Idaho