Analysis of a group: After collecting the data, the researcher will proceed to reiteratively read through the data in order to identify recurring cultural ideas or themes. These cultural themes will be identified as important and supported with details such as quotations, passages, and field notes taken from interviews, documents, and observations (Creswell, 1998).
Analysis of themes: After collecting the data, the researcher will proceed to reiteratively read through the data in order to identify recurring specific ideas or themes. These themes will be identified as important and supported with details such as quotations, passages, and field notes taken from interviews, documents, and observations (Creswell, 1998).
ANCOVA: Analysis of Covariance or ANCOVA is a type of inferential statistical analysis. It is most commonly used in experimental and quasi-experimental research design to compare the average (mean) scores of two or more groups of students. Analysis of variance (ANOVA) and analysis of covariance (ANCOVA) are similar, but have one very important difference. While both procedures determine the amount and significance of mean group differences, the ANCOVA uses a covariate to statistically reduce or correct for pre-existing differences between the groups. The ANOVA does not take this extra step. Some of you may be asking, ‘what is a covariate?’ A covariate is any variable that is measured in both the treatment and control groups prior to an intervention. A covariate could be a pretest score, prior achievement, socioeconomic status (SES), ethnicity, age, etc. Basically, anything that has an impact on the dependent variable and that cannot be controlled for by design can be a covariate. The whole point of using this type of analysis is to statistically reduce the amount of variance between the two groups that is due to pre-existing difference and not the intervention. Let’s look at an example.
An educational technology director is trying to determine the impact of an elementary math computing program on two school districts in her state. District A receives the computing program and district B does not receive the program. At the end of the year, the director runs an ANOVA on the two districts’ achievement scores to compare means. She finds a significant difference between the districts. However, the director also knows district A is much more affluent than district B. Due to the fact that the director knows SES is highly correlated with achievement and she does not have a pretest, she runs an ANCOVA using SES as her covariate in order to statistically reduce pre-existing differences between the two districts’ posttest scores. The results are no longer significant and the program needs further research to validate its efficacy.
The explanation and example above use broad strokes to outline a complex procedure.
ANOVA: Analysis of variance or ANOVA is an inferential statistical procedure that examines the difference or variance between a treatment and control group on some type of posttest.
Artifacts: Artifacts are the objects that people make or use while they function within their environment. In an educational technology context, artifacts could be smartboards, laptops, or calculators (Creswell, 1998).
Assertions: Assertions are a subset of the findings of a case study. These assertions are made during the last stage of the data analysis and are analogous to the researchers conclusions based upon his or her reading of the data in light of his or her theory or conceptual framework (Creswell, 1998).
Axial coding: "Axial coding is a process that follows open coding. It consists primarily of “relating categories of information to the central phenomenon category" (Creswell, 1998, p. 239).
Biography: A biography is a narrative study that focuses on the recording and relating of another person's life. Data collection can include document analysis, interviews, and photograph analysis.
Bounded system: Whatever ‘case’ is being studied is bounded by certain parameters such as location, time, political structure, etc. These parameters define or ‘bound’ the system or ‘case’ that is being studied (Creswell, 1998).
Case: The case is the actual event, location, program, process, or group that is being studied. It is the ‘bounded system’ (Creswell, 1998).
Case study: A case study is an in-depth exploration of a particular context, such as a classroom or group of individuals that involves the collection of extensive qualitative data usually via interview, observation, and document analysis.
Categorical aggregation: Categorical aggregation is the collection of instances or occurrences within the data that represent a specific category, theme, or idea that the researcher determines to be important as a result of it emerging from the analysis of the data (Creswell, 1998).
Causal relationship: A causal relationship is a relationship in which one thing causes another thing to happen or change. For example, if program x, causes achievement scores to increase, it is a causal relationship. The goal of the experimental design is to determine or isolate a causal relationship. A cause-and-effect relationship.
Collective case study: A collective case study is a case study that consists of several different cases (Creswell, 1998).
Context of the case: The context of the case is the setting within which the researcher places his or her case. The context description is vitally important, as it will have influence on the overall research as well as a reader’s interpretation of the research. Context can include historical, political, social issues as well as family dynamics or physical setting of the case (Creswell, 1998).
Control group: The control group is the group that is used as a comparison against which an intervention group is compared.
Correlation coefficient: A correlation coefficient (r): is “a decimal number between .00 and + or –1.00 that indicates the degree to which two quantitative variables are related” (Fraenkel & Wallen, 1993, p. 549).
Covariate: A covariate is any variable that is measured in both the treatment and control groups prior to an intervention. A covariate could be a pretest score, prior achievement, socioeconomic status (SES), ethnicity, age, etc. Basically, anything that has an impact on the dependent variable and that cannot be controlled for by design can be a covariate.
Criterion variable: The criterion variable is the outcome variable that is predicted in correlational research.
Criterion-referenced test: A criterion-referenced test is a test or instrument that determines the score of an individual by the amount of material mastered in a certain content area such as mathematics. It differs from norm-referenced tests in that it does not compare the individual score to a group score.
Cronbach's alpha: Cronbach's alpha is a coefficient (a number between 0 and 1) that is used to rate the reliability of an instrument. It is obtained through a statistical procedure run on a computer program such as SPSS or SAS. It is also referred to as an ‘alpha coefficient.’
Descriptive statistics: Descriptive statistics is a field that focuses on simply describing different characteristics of the data rather than trying to infer something from the data.
Emergent design: Emergent design is the process by which data collection decisions are made based upon the previous data collection's results.
Ethnography: An ethnography is a type of case study that focuses upon the cultural patterns that develop within a group, e.g. district, school, classroom, etc.
Experimental research: Experimental research is also called randomized controlled research or randomized controlled trials. Experimental or randomized studies "randomly assign individuals to an intervention group or to a control group, in order to measure the effects of the intervention" (U.S. Department of Education, 2003, p. 1).
External validity: External validity refers to the extent that the results from a study can be generalized to other people at other times in various conditions. See Study DIAD.
Gatekeeper: A gatekeeper is any person who regulates access to a research site. For example, if an ethnographer would like to do research in a school setting, one of the gatekeepers would be the school building principal (Creswell, 1998).
In vivo codes: In vivo codes use the interviewee’s exact words to name codes or categories. The intent in using such codes is to draw the attention of the reader to the category (Creswell, 1998).
Inferential statistics: Inferential statistics focuses on trying to infer or reach conclusions about the data that reach beyond the data itself.
Internal consistency: Internal consistency consists of reliability procedures that include split-half, Kuder-Richardson, and Cronbach’s Alpha. All of them consist of running internal statistical procedures in order to determine the instrument’s reliability.
Internal validity: Internal validity refers to the degree to which inferences concerning causal relationships can be said to be true. In other words, have you controlled for every other possible confounding or extraneous variable? Can you establish that program x causes y? This answer to this question revolves around internal validity.
The threats to internal validity are history, maturation, regression, selection, mortality, interaction with selection, diffusion of treatments, compensatory equalization, compensatory rivalry, resentful demoralization, testing, and instrumentation. See Study DIAD.
Intervention/treatment: The intervention or treatment variable is the independent variable that the researcher manipulates to gauge the effect or impact on a dependent or outcome variable. For example, a researcher wants to determine the effect of handheld computers in the classroom. The researcher would implement the handhelds in her treatment group and compare certain outcomes measure such as achievement and engagement to a control group that did not have the handhelds. In this case, the handheld computers would be the intervention or treatment.
Key informants: Key informants are individuals who targeted for data collection because their information will provide an accurate and relatively large perspective on the setting as well as lead to other sources of information (Creswell, 1998).
Kuder Richardson: Kuder Richardson is a reliability procedure that results in a coefficient corresponding to the reliability of the instrument. It is obtained through a statistical procedure run on a computer program such as SPSS or SAS.
Likert scale: Likert scale is a method of scaling answers to correspond to varying degrees of measurement. A typical Likert scale would provide a statement such as ‘I feel confident using computers’ and then ask the subject to answer whether they, 1) strongly agree, 2) agree, 3) disagree, 4) strongly disagree. This is an example of a Likert scale.
Longitudinal study: Longitudinal study is a study in which a researcher or evaluator collects survey data in order to track changes or trends in a cohort or group over time. The same group would be surveyed at multiple points in time with a survey.
Matching: Matching is a process used to control for pre-existing differences that may have an effect on the outcome measure or dependent variable between the two groups being compared.
Examples of some matching characteristics typically used include gender, SES, race, pretest score, and age. The more variables controlled through matching and other procedures, the stronger our inferences will be from the data.
Mean: The mean is the average of a set of numbers or scores in a distribution. To get the mean, add up all the values and divide this sum by the total number of all the values.
Measures of central tendency: Measures of central tendency are single numbers that are used to summarize a larger set of data in a distribution of scores. The three measures of central tendency are mean, median, and mode.
Measures of variability: Measures of variability are numbers that indicate how spread out a set of scores is along a distribution. Scores can be bunched up around the mean or spread out significantly along the distribution. The three measures of variability are range, standard deviation, and variance.
Median: The median is the score or number, which falls directly in the middle of a distribution of numbers.
Mode: The mode is the number or score, which occurs most frequently in a distribution of numbers.
Negatively skewed: Counter-intuitively, a negatively skewed distribution has most of its scores bunched up at the higher end of the distribution.
Norm-referenced test: A norm-referenced test is a test or instrument that allows a researcher or evaluator to compare an individual’s score to a group score on the same test or instrument.
Normal distribution: A normal distribution is another name for the bell curve; this distribution is a very important concept in statistics and research. It has certain distinct characteristics, which make it a very useful tool for descriptive and inferential statistics.
Participant observation: Participant observation is the primary approach to data collection in ethnography. The researcher immerses herself in the culture-sharing group and becomes a participant within the setting (Creswell, 1998).
Positively skewed: Counter-intuitively, a positively skewed distribution has most of its scores bunched up at the lower end of the distribution.
Predictor variable: The predictor variables are “the variables from which projections are made in a prediction study” (Fraenkel & Wallen, 1993, p. 555).
Probabilistic equivalence: Probabilistic equivalence is the idea that two groups are similar enough to compare results and draw inferences from the same measurements after an intervention is implemented.
Properties: Properties are a sublevel of a category. Properties make up the defining dimensions of a category (Creswell, 1998).
Propositions: Propositions are hypotheses relating categories in a study (Creswell, 1998).
Random assignment: Random assignment is the process by which units or subjects in your sample (i.e., students, classrooms, schools, districts) are assigned to treatment and control groups by chance. This theoretically controls for pre-existing differences between the two groups that may confound your findings. If you use randomization to create your groups and you have enough units or subjects in each group, the groups will have probabilistic equivalence.
Range: The range is the distance from the highest score to the lowest score on a distribution. To find the range, subtract the lowest value from the highest value.
Reliability: Reliability is the consistency of the scores across time obtained from an instrument used to measure something. For example, if we gave a group of 2nd grade students a test on fractions and the classroom average or mean is 85 and we gave the same test to the same children one week later, we would expect the classroom average or mean to be around 85. This is assuming fractions were not significantly addressed in class during that week. In other words, holding everything else constant, will we get the same results across time with the same instrument? If so, the instrument is said to have reliability. If not, the instrument is not reliable and this is a serious threat to the internal validity of your research.
The most commonly used reliability tests include Cronbach’s alpha, Kuder Richardson, and split-half procedure.
Scatterplot: A scatterplot is a "plot of points determined by the cross-tabulation of scores on coordinate axes; used to represent and illustrate the relationships between two quantitative variables” (Fraenkel & Wallen, 1993, p. 556).
Skewed: A distribution is skewed if the majority of the scores are bunched up at one end or the other.
Split-half: A split-half is an internal consistency or reliability procedure that consists of scoring the two halves of an individual’s test and then computing a correlation coefficient for the two halves. The most common procedure for this is called the Spearman-Brown prophecy formula.
Standard deviation: The standard deviation is the typical or average deviation or difference between the set of scores and the mean or average of those scores. It is not literally the ‘average’ deviation, but for our purposes this is a good way to think about it.
Strategies: "in axial coding, these are the specific actions or interactions that occur as a result of the central phenomenon" (Creswell, 1998, p. 242).
Stratified random sampling: Stratified random sampling is a procedure which first categorizes a population into subgroups (e.g. African American, Asian, Hispanic, Caucasian, etc.) and then randomly selects from each subgroup until a desired number is reached. In this way, researchers are able to obtain large enough samples for each subgroup for statistical analysis.
T-test: A T-test is merely another way to look at the differences between two groups. The means of both groups are compared to each other in order to conduct this statistical analysis.
Test-retest: A test-retest is a reliability test procedure where the exact same instrument is given to the same group of people two times and the two results are compared for consistency. A correlation or relationship between the two scores is represented with a coefficient.
Treatment group: The treatment group is also called the intervention or experimental group. It is the group that receives the intervention or treatment that is being studied. For example, if group A receives a computer-delivered math program and group B receives the regular math program, then group A is the treatment group and group B is the control group with the computer-delivered math program being the treatment variable.
True-score theory: The true-score theory states that each observed score is actually composed of two parts: the true ability of the subject and random error that is found in any measurement situation.
Validity: Validity is the degree to which accurate inferences can be made from the results of an instrument. Is the instrument measuring what it is supposed to measure and can we make inferences from it? If so, the instrument is said to be valid. Evidence of validity includes content, construct, concurrent, and predictive validity.
Variance: Variance is the average amount of dispersion or spread in a distribution of scores. The variance is squared to find the standard deviation.
Z-score: A z-score is a conversion of a raw score on a test or instrument to a standardized score represented in units of standard deviations. This is a commonly used statistical procedure that is used to compare scores of tests that might not be measured on the same scale.
