Drilling Down into the Data on Educational Outcomes

Xitao Fan
Photo by Tom Cogill

Analyzing educational data is no easy task, as Professor Xitao Fan can attest. An educational psychologist who specializes in educational measurement and applied quantitative methods, he addresses such technical questions as ensuring that tests actually measure the issues they are supposed to measure. He cites the example of an algebra test, with its questions about travelers arriving at locations at different times. “Does it measure reading comprehension or math?” he asks. “We need to construct a test by using sound psychometric principles so we can be confident that the score reflects what the test is designed to measure.”

In addition to his work on many technical issues in educational measurement and quantitative methods, Fan also addresses the practical challenges that arise when handling the vast quantities of data in education. Here again, he focuses on ways to apply quantitative methods so that they produce more definitive answers to the issues that educational researchers confront. At the same time, Fan applies his technical expertise to specific educational questions. For example, Fan and his colleague Robert Tai are looking at a large national data set to determine the factors that influence a student’s decision to become a scientist or mathematician. “There is a growing body of evidence that suggests that the United States is losing its lead in science and math,” he notes. “We want to find out where we can be most effective at reversing this trend.” They used data from a 12-year national study that tracked students beginning in the eighth grade and that included data on family background and performance.  They found that students who expressed a desire to become a scientist or mathematician in their early teens were almost twice as likely as other children to enter these fields, while controlling for other relevant variables. “The pattern was not obvious from the data,” he says. “It only became apparent with the application of specific quantitative tools.”

Another question he studied is the extent to which parental involvement influences student learning. The conventional wisdom is that it does, but there has been little empirical evidence.  Fan found that not all forms of parental involvement are equally effective. Parental involvement with students, such as setting high expectations, is a good indicator of student performance, while parental involvement with schools, such as serving on the PTO, is not an important factor. “There are a lot of generalizations that people make about education,” Fan says. “The key is to use quantitative methods to identify the generalizations that matter.”