Advantages and Drawbacks of Various Data Collection Procedures
This KnowledgeBase archive includes content and external links that were accurate and relevant as of September 30, 2019.
|Self-administered questionnaire||Inexpensive. Can be administered quickly if distributed to a group. Well suited for simple and short questionnaires.||No control for misunderstood questions, missing data, or untruthful responses. Not suited for exploration of complex issues.|
|Interviewer-administered questionnaires||Interviewer can probe to ensure question is understood. With good rapport, may obtain useful open-ended comments, including evidence to support response.||Confidentiality is an issue. May require hiring interviewers. Training is needed to establish consistency, nature, and use of probing questions|
|Open-ended interviews||Usually yields richest data, details, new insights. Best if in-depth information is wanted.||Same as above (interviewer-administered questionnaires); also, often difficult to analyze|
|Focus Groups||Useful for gathering ideas and different viewpoints, discovering new insights, & improving question design.||Not suited for generalizations about population being studied.|
|Tests||Provide "hard" data, which administrators and funding agencies often prefer. Relatively easy to administer. Good instruments may be available from vendors.||Available instruments may be unsuitable. Developing and validating new project-specific tests may be expensive and time consuming. Objections may be raised because of test unfairness or bias.|
|Observations||If well executed, best for obtaining data about behavior of individuals and groups||Usually expensive. Needs well-qualified staff. Observation may affect behavior being studied.|
|Documents, records, and student work||Existing material can be used to develop data at a convenient time.||Checklists or rubrics for generating data from the written material must be created. Careful definitions must be established to ensure consistency.|
Adapted from User-Friendly Handbook for Project Evaluation (p. 44), by F. Stevens, F. Lawrenz, and L. Sharp (n.d.), Directorate for Education and Human Resources, National Science Foundation.
The contents of this website were developed under a grant from the U.S. Department of Education and are intended for general reference purposes only. However, those contents do not necessarily represent the policy of the U.S. Department of Education or the Center, and you should not assume endorsement by the Federal Government. Some resources on this site require Adobe Acrobat Reader. This website archive includes content and external links that were accurate and relevant as of September 30, 2019.