《(精品)Lecture6_ResMeth_Ozer.ppt》由会员分享，可在线阅读，更多相关《(精品)Lecture6_ResMeth_Ozer.ppt（42页珍藏版）》请在taowenge.com淘文阁网|工程机械CAD图纸|机械工程制图|CAD装配图下载|SolidWorks_CaTia_CAD_UG_PROE_设计图分享下载上搜索。
1、Research in Educational ServicesResearch in Educational ServicesEVAL 65511 002Ipek OZERLecture 6Wednesday,October 3,2012Wednesday,October 3,20121TodayChapter 7Factorial DesignsRepeated Measures DesignsTime Series DesignsRandomness and RepresentativenessChapter 8Quasi-Experimental Designs ValidityPos
2、ttest Only,Nonequivalent Control Group DesignsPretest-Posttest,Nonequivalent Control Group DesignsTime Series DesignsAction Research and Quasi-Experimental DesignsInterpreting ResultsQuiz 22What are the types of experimental designs?Between Group DesignsTrue experiments(pre-and posttest,posttest onl
3、y)Quasi-experiments(pre-and posttest,posttest only)Factorial DesignsWithin Group or Individual DesignsTime series experimentsRepeated measures experimentsSingle subject experiments3Between subjects designs have different groups of subjects that are assigned to the treatment conditions.2 IVs:Teaching
4、 Methods and Teacher.3 different teaching methods and 2 different teachers.6 different groups with different students.4Group 1Group 2Group 3Group 4Group 5Group 6PretestTreatment APosttestPretestTreatment APosttestPretestTreatment BPosttestPretestTreatment BPosttestPretestControlPosttestPretestContro
5、lPosttest5Subject 1Subject 2Subject 3Subject 4Subject 5Subject 6PretestCtrlTestPretestCtrlTestPretestCtrlTestPretestCtrlTestPretestCtrlTestPretestCtrlTestATestATestATestATestATestATestBPosttestBPosttestBPosttestBPosttestBPosttestBPosttestWithin subjects designs(repeated measures)consist of the same
6、group of subjects receiving more than one treatment.(Ctrl)The traditional method,and take an in-class test.(A)The online course online and then have the exam.(B)Small group discussions and then have the posttest.Chapter 7(cont.)Experimental Design6Common Research DesignsExperimental ResearchPosttest
7、-only Control Group DesignPretest-Posttest Control Group DesignSolomon Four-Group DesignFactorial DesignsRepeated Measures DesignsTimes series designs7Solomon Four-Group Design(last week)This is a combination of the pre-post control and the posttest only control group designs.There are two control(i
8、.e.,2 and 4)and two experimental groups(i.e.,1 and 3).(1)R G1XO2(2)R G2O4O1O3(3)R G3XO5(4)R G4O68Solomon Four-Group DesignRandom Selection of Subjects from PopulationRandom Assignment of Subjects to ConditionsExperimental TreatmentControl GroupPosttest MeasurementPretest MeasurementExperimental Trea
9、tmentControl Grouphttp:/ DesignsIt is not enough to know the effect of a single treatment on an outcome;several treatments may,in fact,provide a better explanation for outcome.Between-group design in which the researcher studies two or more categorical,IVs,each examined at two or more levels.Purpose
10、:to study the independent and simultaneous effects of two or more independent treatment variables on an outcome.Each independent variable is represented by a separate number which indicates the number of levels for that variable10Factorial Designs(2X2)Random Selection of Subjects from PopulationRand
11、om Assignment of Subjects to ConditionsFactor 2 Level 1(X1)Factor 2 Level 2(X2)Posttest MeasurementFactor 1 Level 1(Z1)Factor 1 Level 2(Z2)Factor 2 Level 1(X1)Factor 2 Level 2(X2)11Conducting a factorial designRQ:“Do rates of smoking vary under different combinations of type of instruction and level
12、s of depression?”Levels of each factor or IV:Types of instruction:A health-hazards lectureA standard lectureLevels of depression:LowMediumHighhttp:/doctor.co.uk/interactive/interactivetests/goldberg.php122 LEVELS of instruction3 LEVELS of depressionThe design is called a two by three factorial desig
13、n.(2X3)Factorial Design Groups assigned to 2 conditions13GroupExtent of Depression(depression scores)Type of InstructionDependent Variable(scores on instrument measuring smoking)Group 1LowHealth-hazards lecturePosttestGroup 2MediumHealth-hazards lecturePosttestGroup 3HighHealth-hazards lecturePostte
14、stGroup 4LowStandard lecturePosttestGroup 5MediumStandard lecturePosttestGroup 6HighStandard lecturePosttestFactorial DesignsThere are 3 possible outcomes from a factorial design:No significanceMain effects-the influence of each IV on the outcome.Interactions-exist when the influence on one IV depen
15、ds on the other IV.14Main&Interaction effects15Means and Main effects of the Six Groups in the Factorial DesignDepressionLowMediumHighType of InstructionHealthMean rate of smokingMean rate of smokingMean rate of smokingStandardMean rate of smokingMean rate of smokingMean rate of smokingMain effects
16、of depressionMain effects of type of instruction16The results of scores on the posttest(i.e.,rate of smoking)and the three factors of depression.The graph displays the scores for both groups(standard and health lecture).The extent of smoking for both groups increases with the level of depression.17I
17、nteraction Effects(Crossed)The extent of smoking for the groups receiving the standard lecture increase as depression increases.Smoking rates for students who experience the health lecture are constant for each level of depression.Interaction Effects(Not Parallel)The extent of smoking for the groups
18、 receiving the standard lecture increase as depression increases.The extent of smoking for the groups receiving the health lecture decrease as depression increases.Factorial DesignsFactorial Designsuses two or more IVs with at least two levels of each variable;All levels of each IV are taken in comb
19、ination with the levels of the other Ivs.Factorial Designs commonly use numerical designations such as 2 X 2 or 2 X 4.This indicates how many IVs and the levels of each IV.18Factorial DesignsAdvantages1.One design for multiple IVs,instead of separate designs for each IV.2.Can investigate interaction
20、s.3.Random/Error Variance is reduced.4.Can enhance control.Disadvantages1.Not economically/practically feasible(e.g.,larger sample size).2.Interpretation of complex interactions.2 X 3 X 2 X 4 design19Repeated Measures DesignsRepeated Measurestaking additional observations in order to check on a dela
21、yed effect or duration of effect.This is an extension of the posttest-only control group design.Observations are taken on both groups after the experimental treatment has been administered.At specified intervals,more observations are taken on each group,but there are no more additional treatments.20
22、21Subject 1Subject 2Subject 3Subject 4Subject 5Subject 6PretestCtrlTestPretestCtrlTestPretestCtrlTestPretestCtrlTestPretestCtrlTestPretestCtrlTestATestATestATestATestATestATestBPosttestBPosttestBPosttestBPosttestBPosttestBPosttestWithin subjects designs(repeated measures)consist of the same group of
23、 subjects receiving more than one treatment.Repeated Measures DesignRandom Selection of Subjects from PopulationRandom Assignment of Subjects to ConditionsMeasurement of DVExperimental ConditionControl GroupMeasurement of DVMeasurement of DVMeasurement of DVMeasurement of DVMeasurement of DV22Repeat
24、ed Measures DesignsAdvantages(1)Sometimes in education and behavioral sciences,it takes a while for an effect to manifest itself,and many times the duration of the effect is unknown.This design specifically targets these situations.Disadvantages(1)Multiple Observation Effectearlier observations may
25、have an effect on subsequent ones.(2)Susceptible to history and maturation internal validity concerns.(1)R G1XO2(2)R G2O4O1O3O5O623Experimental Time Series DesignTime Seriesinvolves repeated measurements,usually some measures/observations taken prior to the experimental treatment,and some measures/o
26、bservations taken after the experimental treatment.(1)R G1XO2(2)R G2O4O1O3O5O6O9O11O8O10O12O13O7O1424Experimental Time-Series DesignRandom Selection of Subjects from PopulationRandom Assignment of Subjects to ConditionsMeasurement of DVExperimental ConditionControl GroupMeasurement of DVMeasurement
27、of DVMeasurement of DVMeasurement of DVMeasurement of DVMeasurement of DVMeasurement of DV25Experimental Time Series DesignAdvantages(1)Same as with repeated measuressometimes in education and behavioral sciences,it takes a while for an effect to manifest itself,and many times the duration of the ef
28、fect is unknown.(2)Can examine if a trend is present before a treatment/intervention(i.e.,unlike repeated measures).Disadvantages(1)Same as with repeated measuresthe multiple observation effect.(2)Susceptible to history and maturation internal validity concerns.26Randomness and RepresentativenessRan
29、dom Selectionhow you draw the sample for your study from a population;related to sampling(i.e.,external validity).using an equal probability selection methodRandom Assignmenthow you assign the sample that you draw to different groups or treatments in your study;related to design(i.e.,internal validi
30、ty).It is possible to have both in a study.It is possible to have one or the other.It is possibly to have neither.27Chapter 8/QUASI-EXPERIMENTAL RESEARCH28Without randomization,there are no true experiments.Quasi-Experimental DesignQuasi-Experimental Designinvolves the use of intact groups of subjec
31、ts in an experiment,rather than assigning subjects at random to experimental treatments.Validity ConcernsInternal Validity(e.g.,Differential Selection of Subjects)External Validity(e.g.,Generalizability)How to fix?Attempt to establish(i.e.,logically argue)the degree of equivalence between groups,and
32、 the applicability of the results to other populations.29Posttest Only,Nonequivalent Control Group DesignPosttest Only,Nonequivalentcontains as many groups as there are experimental treatments,plus a control group;intact groups are used,and subjects are measured only once after the experimental trea
33、tments have been applied.Nonequivalent Groupsno random assignment used;intact groups;groups are not similar,but can be argued logically as such.Mostly Disadvantages(1)Selection Bias(2)No Pretests(1)G1X(2)G2O2O1No“R”!30Pretest-Posttest,Nonequivalentsimilar to the posttest only design,but the subjects
34、 are pretested.Multiple Group design No control group,just treatment groupsAdvantages(1)Aids in checking for Selection Bias due to pretest.Pretest-Posttest,Nonequivalent Multiple Group DesignG1:O1X1O2G2:O3X2O4G3:O5X3O631Pretest-Posttest,Nonequivalent Control Group Design(eg.)G1:O1X1O2G2:O3-O4Intact
35、Group of ParticipantsProgram groupPosttest MeasurementPretest MeasurementIntact Group of ParticipantsControl groupPosttest MeasurementPretest Measurement32Selection-Maturation?Probably not because comparison didnt mature at all.Selection-History?Possibility of an event(i.e.,other than treatment)that
36、 occurred with the program group but not the comparison group?Regression?Probably not because the comparison group didnt move.The“Program Group”starts out higher(i.e.,5pts),but can this explain the 15 differential point hike?Pretest-Posttest,Nonequivalent Control Group DesignSometimes,you have to lo
37、ok at your results to examine threats(i.e.,post hoc)!33Selection-Maturation?The two groups differed to begin with,so they may already be maturing at different rates.Selection-History?Are the two groups both reacting differently to an outside event?Selection-Testing?The two groups started out differe
38、ntly,so perhaps they responded differently to the pretest.Selection-Mortality?Did some of the low scoring participants drop out?Pretest-Posttest,Nonequivalent Control Group Design34Regression to the Mean?Program starts out high then decreases.Program starts out low and then increases.Pretest-Posttes
39、t,Nonequivalent Control Group Design35Quasi-Time Series DesignQuasi-Time Seriesused with one or more intact groups;involves repeated measures before and after an experimental treatment.Single-Group Time SeriesDisadvantages(1)Plausible alternative explanations.(2)No control group.(1)G1XO2O4O1O3O536Mu
40、ltiple-Group Time Seriesuses two or more intact groups;similar to the single-group time series,but with a control group.Advantages(1)Can strengthen internal validity(e.g.,History)(2)Selection bias check from observations before experimental manipulation.VariationsInterrupted Time Series(i.e.,random
41、and systematic/persistent).(1)G1XO2(2)G2O4O1O3O5O6O9O11O8O10O12O13O7O14Quasi-Time Series Design37Time Series DesignA:Intervention did work.B:Intervention did work.C:Short-term impact of intervention.D:Not stable pre-intervention;not stable post-intervention;no way to determine effect.38Action Resear
42、ch and Quasi-Experimental DesignsAction Researchresearch conducted at the local level,typically by a practitioner who is focused on solving a specific problem.Most likely will be a quasi-experimental design.Can be collaborative in nature(e.g.,local schools and universities).Can take place in a non-s
43、chool setting.Can involve just one teacher conducting research in a classroom,or research on a larger scale involving teams of people.39Interpreting ResultsIf an experiment is designed properly,the researcher should be able to draw some conclusions about the experimental effect.We want to correctly
44、interpret the Os in the diagrams(i.e.,the patterns in the data),and what these patterns mean about the experimental treatment.=Significant Differences(i.e.,Groups are NOT Equal)No Significant Differences(i.e.,Groups are Equal)40Interpreting ResultsResults:O1 O2,O3 O4,O5 O6O2=O4O2 and O4 O6O1=O3=O5=O
45、7=O8(1)R G1X1(2)R G2O4O2(3)R G3X3(4)R G4O8O6X2O3O1O7O5Interpretation:lThere are effects for all experimental treatments.lThe effects of X1 and X2 are the same,but they are different from X3.(1)Lecture(2)Discussion(3)Online(4)Regular Instruction41Quiz ReviewLectures 3,4,and 5Sections of Research proposalSections of Dissertation proposalControlling varianceThreats to Internal and External validityCharacteristics of good quantitative research designResearch designs-Diagramming42