Applied Microeconomics

Economics of Education, Labor Economics, Applied Microeconomics, and Big Data Analytics

• Risk of Workforce Exit Due to Disability: State Differences in 2003-2016
(with Yonatan Ben-Shalom, and Mariel Finucane)
Click for abstract

Abstract: A better understanding of trends in workforce exit due to disability and how these trends vary across states and subgroups can help federal and state policymakers identify both individual-level and state-level factors associated with increased risk of workforce exit due to disability. Using national survey data and Bayesian multilevel modeling techniques, we produce yearly estimates of trends in the risk of workforce exit due to disability for states and demographic subgroups. These estimates are more stable and have narrower uncertainty intervals than estimates produced using classical statistical methods. We identify Current Population Survey respondents as being "newly at-risk" of exiting the workforce due to disability if they reported being employed in one month and out of the labor force because of a disability in the next month, and we refer to their share of the working-age population as the "at-risk rate." We find that age, education, race, and gender are important factors for the at-risk rate, in decreasing order. Holding demographics constant across states and time reduces the cross-state variation in the at-risk rate but does little to reduce variability over time. State at-risk rates are typically higher than application rates for the Social Security Administration's disability programs, but the relationship between these rates varies considerably by state. Our preliminary exploration of the reasons for cross-state variation in this relationship suggests that differences across states may be due to differences in their industrial composition, job opportunities, and safety net structure.


• Speaking on Data's Behalf: What Researchers Say and How Audiences Choose
(with Jesse Chandler, Mariel Finucane, Jeffrey G. Terziev, and Alexandra M. Resch)
Click for abstract

Abstract: Bayesian statistics have become popular in the social sciences, in part because they are thought to present more useful information than traditional frequentist statistics. Unfortunately, little is known about whether or how interpretations of frequentist and Bayesian results differ. In this paper, we test whether presenting Bayesian or frequentist results based on the same underlying data influences the decisions people made.


• What Works for Whom? A Bayesian Approach to Channeling Big Data Streams for Public Program Evaluation
(with Mariel Finucane, and Scott Cody)
Click for abstract

Abstract: In the coming years, public programs will capture even more and richer data than they do now, including data from web-based tools used by participants in employment services, from tablet-based educational curricula, and from electronic health records for Medicaid beneficiaries. Program evaluators seeking to take full advantage of these data streams will require novel statistical methods, such as Bayesian approach. A Bayesian approach to randomized program evaluations efficiently identifies what works for whom. The Bayesian approach design adapts to accumulating evidence: Over the course of an evaluation, more study subjects are allocated to treatment arms that are more promising, given the specific subgroup from which each subject comes. We identify conditions under which there is more than a 90% chance that inference from the Bayesian adaptive design is superior to inference from a standard design, using less than one third the sample size.


• Understanding Types of Evidence: A Guide for Educators
(with Greg Chojnacki, Alex Resch, Alma Vigil, Steve Bates)
Click for abstract

Abstract: This document was created as a resource for districts seeking to evaluate the effectiveness of educational technologies used in their districts. Although the examples focus on educational technologies, the concepts apply to other programmatic decisions undertaken by school districts or other organizations, such as adopting a curriculum or case management approach.


• MOOCs as a Massive Research Laboratory: Opportunities and Challenges
(with Paul Diver)
Click for abstract

Abstract: This paper explores the opportunities and challenges that Massive Open Online Course (MOOCs) are generating for research. A wide variety of topics related to pedagogical methods and student incentives lend themselves to research using MOOCs, and throughout we discuss lessons that can be gained both from observational comparisons as well as from the opportunity to run experiments on randomly chosen groups of students. We start by discussing dropout rates and study how students who decide to drop out are different from those who continue in the course. We then discuss class forums and video lectures and how interacting with this material is correlated with achievement. After that, we explore the strong correlation between procrastination and achievement and the implications for course design. We also examine the role of certificates in MOOCs and how they can affect choices and outcomes. Finally, we examine the potential of linking data across courses and the opportunities and challenges of working with data that originates in surveys of MOOC participants. All of these research opportunities offer Big Data challenges as well which have to be addressed with parallel computing.


• The Productivity of Pell Grant Spending: Enrollment Versus Attainment
(with Sarah Turner)
Click for abstract

Abstract: Does the share of students enrolling in a college receiving federal Pell grants correspond to a college’s effectiveness in equipping students with economically meaningful postsecondary credentials? Focusing on four-year institutions, we propose an alternative measure that estimates the expenditures in Pell grants needed to produce one baccalaureate degree recipient at an institution ("Pell cost"). Our estimates of "Pell cost" tell a compelling story that contrasts sharply with public pronouncements made by organizations such as the New York Times and Education Trust, which have chastised institutions for low representations of students receiving Pell grants. There is wide variation in the Pell cost among four-year colleges and institutions with the same Pell shares vary considerably in their efficiency in using federal dollars. The measures presented in this paper are a "proof of concept" which rely on a number of assumptions necessitated by the limitations of available aggregate data. As such, they are intended to underscore the need for a fuller use of detailed administrative data to assess how well colleges and universities are helping low-income students to achieve goals of economic prosperity.

• Never put off till tomorrow?
Click for abstract

Abstract: This paper uses two randomized control trials to show that a very low-cost intervention can increase student achievement in a massive online open course (MOOC). First, an email (directive nudge) sent to a randomly selected group of students encouraged the students to procrastinate less. Students assigned to the treatment group were 16.85 percent more likely to complete the course. Second, another randomized control trial demonstrated that the effect on the completion rate cannot be attributed to the Hawthorne effect.


• The effects of informational nudges on students' effort and performance: Lessons from a MOOC
Click for abstract

Abstract: I evaluate the impact of giving students information that compares their performance to that of their classmates. I run a randomized experiment in the context of a Coursera massive open online course (MOOC), assigning students to one of two possible treatments. In the first, framed positively, students are told how many of their classmates they outperformed on a quiz. In the second, framed negatively, students are told how many of their classmates outperformed them. I find evidence that students respond to this informational nudge, and that framing matters. Students who are doing relatively poorly respond to the negative treatment by exerting more effort, and this effort translates, in some cases, into higher achievement.


• Massive Open Online Courses (MOOCs) as a Brick-and-Mortar Complement
(with Louis A. Bloomfield & Sarah Turner)
Click for abstract

Abstract: We show that a Massive Open Online Course (MOOC) can serve as a complement to a brick-and-mortar introductory physics course. In two different randomized control trials we use small monetary incentives and informational nudges to encourage students into enrolling in a MOOC. Using these experiments as instruments we show that MOOC enrollment significantly improves performance in the brick-and-mortar classroom. For example, a student who chooses to enroll in the MOOC will score, on average, 1.64 standard deviations higher in the Quiz that has the most MOOC-related.