Choosing Appropriate Quantitative Research Design

Choosing Appropriate Quantitative Research Design Quantitative research is designed to provide quantitative (numerical) data that answers questions related to trends, attitudes, opinions, or the impact of treatment on a population. Before quantitative research begins, it is necessary to identify the purpose of the study, the population to be studied, the variables involved in the study, and the type of data that would be most useful as an end product. After identifying these components, researchers can next hone in on the design of the research, the design of the research collection tool, sampling procedures, the survey or measurement instrument, data collection methods, and data analyses methods (data organization, data interpretation, scaling, etc.) and issues (validity, reliability, threats to validity, etc.).The research is based on theory or hypotheses and is usually tests for the impact of an intervention on a population. The impact of an intervention or treatment may be measured using traditional experimental methods and proc esses or by survey methods. Survey methods include personal interviews, telephone interviews, mailed questionnaires, group administered questionnaires, or dropped-off household surveys (Trochim, 2006). Though the process is presented in simple terms in this paper, the actual design process is a very complex set of decisions related to the methodologies and procedures of quantitative research. This paper seeks to outline the strengths and limitations of the most widely used research design models to determine the appropriate research design for quantitatively researching the use of technology web 2.0 innovations to determine its effect on learning and test performance in the elementary classroom. The quantitative study of the social science is especially challenging because of the fact that the independent variables cannot be manipulated and that the research must often be performed in real time in a natural setting. The research is characterized by the manipulation of a variable but extreme care must be taken not to harm the study participants in any way throughout the study. Research is usually centered around the determination of a property-disposition relationship (attitude-focused) or a stimulus-response relationship (behaviorally-focused). Quantitative researchers must determine which type of relationship is best suited for their specific study. The following are critical factors in making this critical determination: Time Interval: the period of time between introduction of the independent variable and the response to the variable Degree of specificity: isolation of the independent variable to determine its effect Nature of comparison groups: comparison of before/after groups or experimental/control groups for statistical analysis Time sequence of events: determining the timeline for the relationship between cause and effect These elements of quantitative research drive the decisions regarding research methodologies and procedures related to choosing an appropriate research design. The following is a brief overview of the research designs used in social sciences. Quantitative Research Design Comparison Experimental Design Experimental design is usually associated with the life and physical sciences where independent variables are easily manipulated. Experimental design compares the results of an experimental group (that receives exposure to an independent variable) with a control group (that does not receive exposure to an independent variable). This design often uses a pretest and posttest measurement to analyze the differences between groups. The advantages of this type of research design for studying social science include the ability of the researchers to introduce and control extrinsic and intrinsic (independent) variables as well as the easy identification of causal inferences that strengthen the validity of the research. Disadvantages of experimental design for social science includes the inability to replicate the experiment in a real-life social setting resulting in weak external validity and the reliance on volunteers or self-selected participants who may not represent the actual population. As a result, generalizability is decreased due to the small sample of participants selected for the study. Cross-Sectional and Quasi-Experimental Designs Cross-sectional design is recognized by its utilization of surveys to determine study participants backgrounds, past experiences, and attitude to determine the relationship between research variables. This type of research is not conducive to experimental design because of the difficulties in manipulating the independent variable during the study. Cross-sectional design relies on statistical analysis to approximate the relationship between variables and may not produce accurate causal inferences. Internal validity is weak as a result. Quasi-experimental design is identified by random selection of study participants without the requirement of random selection of participants to a comparison group, study of more than one population sample, and studies conducted over time. It is difficult to disaggregate the data produced by this type of study since the population sample will consist of a mixture of subjects with various traits and characteristics. Causal inferences are difficult to determine with this design. Performing the study over time and the analysis of data by (similar) categoric or contrasted (different) groups are strategies used to increase the validity of this design for social science research. Planned variation design, panels, time-series designs, and control-series designs are alternative quasi-experimental social science research designs that attempt to increase internal and external validity by controlling stimuli introduction, research methods, cause-and-effect identification, and causal inference determination respectively. Combined designs employ two or more of the designs mentioned above in effort to assess the causal effects of variables using a multi-method, multi-design approach to studying social science. The advantages offered by these designs include increased internal and external validity as a result of the ability to perform research in real-life, natural settings with a representative population. Since there is no assignment of participants to treatment or comparison groups, researchers are able to perform studies that could be considered unethical or impossible using traditional experimental designs. The disadvantages associated with these designs include difficulty determining causal inferences (due to a wide variety of differences inherent in the study population) and the inability of researchers to manipulate the independent variable. Pre-Experimental Design Pre-experimental design is appropriate when no other design is able to study a population due to limitations in time, population, or a specified event. If there is a single event that occurs at a specified time for a specified group of people, there is only one opportunity to study the impact of a treatment. For this reason, pre-experimental design is considered to be the weakest type of research design with a high risk of causal inference error. There is usually not an assignment of subjects to an experimental or control group and this design usually does not include a comparison group. A one-shot case study is often used with this type of design and does not offer high validity due to the limited ability to generalize study results to a wider population. An advantage of this design includes allowing researchers the ability to scientifically show that more research is needed to explore a particular hypothesis. Weak internal and external validity and the inability to make causal infe rences are considered to be disadvantages for this research design. Determining Which Design is Most Appropriate My research problem studies the relationship between the use of web 2.0 innovative technologies (such as Skype, Second Life, etc.) and depth of research, test performance, and self- motivated learning for grade 3 through 8 students. Important factors that are necessary to consider are the identification of the independent variable, identification of dependent variables, availability of a control or contrast group, ethical implications of this study, and availability of the study (treatment) population. The independent variable in the study is the use of web 2.0 technologies for research. Dependent variables are research depth, test performance, and student self-motivation to learn. The grade 3 through 8 students for the control group and experimental group are available at my current place of employment. Since the treatment involves using technology to learn, there is no presumed risk or ethical issue since using technology is an ordinary part of the students day of learning. The tre atment of using technology for communication over the Internet is a manipulation of the use of technology in the classroom. Special care will be taken to ensure that students adhere to Internet safety rules during communication sessions. Experimental design is the most appropriate research design for this study for the following reasons: Availability and randomization of control and experimental groups Variables can be easily manipulated Pre-test and post-test measurements are possible Causal inferences will be easily identified Johnson and Christensen (2007) state that quantitative research is appropriate for describing what is seen and generating new hypotheses and theories. Since the measurements of the dependent variables reflect behavioral rather than cognitive outcomes, the tools used to measure the study outcomes will not include surveys but rather observational logs. This further supports the use of the experimental design for this study. Other Considerations Other research designs are not considered appropriate and are detailed for the following reasons. Cross-sectional design is not appropriate for this study because surveys are not necessary to determine the participants backgrounds, past experiences, or attitudes. Also, since the independent variable can be manipulated, statistical analysis will not be necessary to approximate causal inferences. Quasi-experimental design should not be employed since the assignment of participants to a control or comparison group is possible and there is no need for an extended period of time for this study. Also, there is no need to systematically introduce stimuli, use panels, or take measurements over a number of time periods. Data is not expected to change for individuals due to history, maturation, or test-retest effects. Furthermore, there are no ethical considerations present in the experimental design for the study. Lastly, the pre-experimental design is not appropriate for this study since thi s is not an event-based or time-sensitive study.