An improved estimate of self-employment hours for quarterly labor productivity

Quarterly labor productivity is defined as the ratio of output to hours worked. Data on employee hours come from the BLS Current Employment Statistics (CES) survey, which is an establishment survey.¹ The CES sample is drawn from the longitudinal database of employer records for business establishments covered by the Unemployment Insurance program. Therefore, the sample does not cover workers who work for themselves in their own unincorporated businesses or as independent contractors (referred to as unincorporated self-employed workers or proprietors), nor does it cover those who work as unpaid family workers.² To include hours of work for these classes of workers in the total hours measure, the BLS productivity program supplements the CES hours with hours worked by the unincorporated self-employed and unpaid family workers from the BLS Current Population Survey (CPS), which is a household survey.³ For simplicity, we henceforth use “self-employed” as shorthand for unincorporated self-employed and unpaid family workers. Similarly, we use “self-employment” as shorthand for unincorporated self-employment and unpaid family work.

Hours worked by the self-employed can be volatile for two broad reasons. First, these hours can increase or decrease from one period to the next, as people start businesses and shut them down or as they use self-employment as a bridge between wage-and-salary job spells.⁴ Second, because the self-employed constitute such a small share of the labor force, it is harder to measure their hours by using the CPS.

Chart 1 shows the percent change in quarterly self-employment hours from 2000 to 2019.⁵ Self-employment was procyclical over this period, with substantial decreases in self-employment hours during recessions.⁶ In this chart, and in most of the analyses that follow, we begin the series in 2000 because productivity measures are consistently estimated back to 2000 and the current seasonal adjustment process uses data starting in 2000. We end the series in 2019 because the scale of changes in self-employment during the COVID-19 pandemic dwarfed any earlier visible volatility.⁷ Although the series is considerably volatile, the source of that volatility is not obvious—no economic theory beyond business cycle theories predicts sudden quarterly fluctuations in self-employment.

A concern with productivity measurement is that sometimes the volatility in self-employment hours is so outsized that it noticeably affects the measure of total hours, even though self-employment hours make up only a small share of total hours (about 7.3 percent in the fourth quarter of 2022). Chart 2 shows hours growth for employees, the self-employed, and all workers from the second quarter of 2000 through the fourth quarter of 2019. In several quarters, spikes in the self-employment hours series substantially affect the hours for all workers, creating a divergence between the hours growth rate for employees and all workers. For example, in the third quarter of 2005, employee hours grew by 2.5 percent, but hours for all workers grew by only 1.3 percent once a 10.0-percent fall in self-employment hours was factored in. Similarly, in the fourth quarter of 2014, employee hours grew by 3.4 percent, but hours for all workers grew by 4.6 percent because of a 19.3-percent increase in self-employment hours.

Chart 3 illustrates another way of looking at the impact of growth in self-employment hours on growth in total hours, comparing the relative percentage-point contributions of employees and the self-employed to the percent change in hours for all workers. Here, for illustrative purposes, we restrict the data to the first quarter of 2010 through the fourth quarter of 2019. Because self-employment hours account for such a small share of total hours, we would expect the dots representing hours growth for all workers to be close to the top of the bars measuring the contribution of employees, and the size of the bars measuring the contribution of the self-employed to be small relative to the bars measuring the contribution of employees. For some data points, however, the dots are quite far from the end of the employee-contribution bars. For example, there are substantial gaps in the first three quarters of 2019.

Sources of volatility

In a companion working paper (forthcoming), Cindy Michelle Cunningham and Sabrina Wulff Pabilonia investigate the sources of volatility in self-employment, focusing on sources relating to the CPS sample design that might be addressed to create a smoother series.⁸ The authors identify three sources of volatility in the CPS estimate of self-employment, which is measured by a respondent’s reported class of worker for the job that he or she worked at last week, for which adjustments could feasibly be made to reduce volatility. These sources of volatility include proxy responses, imputations, and sample rotation.

In the CPS, one household member age 15 years and older often provides labor force information on behalf of other household members. This member is a proxy reporter for other household members. For example, a mother may answer for her college-age children, one spouse for another, and so on. Cunningham and Pabilonia look at how proxy responses in the CPS might affect the volatility of self-employment hours—whether differences between how a worker describes their class of worker and the way another member of the household responding on their behalf does might cause self-employment measures to change over time. In general, the authors find that while the self-employment trends are similar whether measured by proxy reporters or self-reporters, in all periods proxy reporters report less self-employment (for the respondents on whose behalf they are answering) than do self-reporters. In chart 4, we find somewhat more volatility in the annualized quarter-to-quarter growth of self-employment hours when that information is collected by a proxy.⁹

A second source of volatility examined by Cunningham and Pabilonia results from the imputation of class of worker when it is missing in the data because the respondent answered “don’t know” or “refused” to the class-of-worker question (in other words, a case of item nonresponse as opposed to survey nonresponse). The number of imputed instances of self-employment in the CPS has been rising slowly since 2000, although the weighted count of the imputed self-employed is still less than a tenth of the weighted count of the nonimputed self-employed.¹⁰ Chart 5 compares the volatility of self-employment hours from imputed responses with that from nonimputed responses. The effect of imputed responses is occasionally large enough to exacerbate an already large change in the nonimputed responses. For example, in the second quarter of 2010, hours grew by 31.7 percent for the nonimputed self-employed and by 176.1 percent for the imputed self-employed, resulting in an overall growth in self-employment hours of 34.9 percent; in the second quarter of 2017, hours grew by 22.5 percent for the nonimputed self-employed and by 68.9 percent for the imputed self-employed, resulting in an overall growth in self-employment hours of 30.0 percent. And in a few quarters, the growth rate of the imputed values moves in the opposite direction of the nonimputed values and is large enough to slightly offset what would otherwise be a large increase in self-employment hours. For example, in the third quarter of 2006, hours grew by 9.5 percent for the nonimputed self-employed but fell by 33.8 percent for the imputed self-employed, resulting in an overall growth of 5.4 percent in self-employment hours.

Following CPS respondents across consecutive months, Cunningham and Pabilonia examine respondents’ transitions into and out of self-employment that coincide with changes both between self- and proxy responses and between imputed and nonimputed responses. They find that these transitions from one month to the next occur at lower rates when the reporter type stays the same (either a self-reporter in both months or a proxy reporter in both months) than when the reporter type changes from self to proxy, as well as when the responses are nonimputed in both consecutive months compared with imputed in both months or changes in imputed status. In particular, the likelihood of transitioning is largest between self- and proxy responses for those switching between unincorporated self-employed and not employed, suggesting that household respondents do not always know about the self-employment work another household member does.

Cunningham and Pabilonia investigate methods to correct for this source of volatility by directly editing the underlying CPS data. For example, they replace one month’s proxy response with an adjoining month’s self-response. This smoothing strategy requires subjective decision making regarding which response is more likely to be accurate, how many months forward or backward to look for edits, and whether observed transitions might be genuine. Ultimately, the authors conclude that the edits to the underlying CPS data have minimal impact on the volatility of self-employment hours. Given the subjective nature of these edits and the complexity of implementing them into official statistics, we do not pursue this smoothing strategy. However, researchers interested in studying the dynamics of self-employment may choose to test whether making such edits affects their results.

Finally, Cunningham and Pabilonia investigate how the CPS sample rotation design—respondents moving into and out of the sample—might lead to increased volatility in measures of self-employment. In the CPS, respondents answer questions for 4 consecutive months (months in sample (MIS) 1–4), then are out of the sample for the next 8 consecutive months, and finally return to the sample for another 4 consecutive months (MIS 5–8). Thus, from one calendar month to the next, one-quarter of respondents exit the sample and are replaced with entering (or, in the case of MIS 5, reentering) respondents. This sample rotation creates the potential for two sources of volatility. First, the fraction of individuals who are self-employed may differ between entering and exiting rotations. This difference can arise by chance because of sampling error.¹¹ The second source, known as rotation-group bias, is due to systematic differences by MIS in how respondents report self-employment status, with workers in earlier MIS being more likely to report self-employment.¹² For the 2000–22 period, the average percentage of self-employed workers is about 7.0 percent in MIS 1, 6.8 percent in MIS 2–4, 6.7 percent in MIS 5, and 6.6 percent in MIS 6–8. These differences can arise for various reasons, including respondent fatigue, differences in unit nonresponse by class of worker, and in-person versus telephone interviewing.

Chart 6 shows the hours growth in self-employment, in total and broken into two sources—the growth that comes from the difference between those who entered the sample in a quarter and those who exited it in the previous quarter and the growth that comes from those who were surveyed in both quarters (continuers). We find evidence of both sample rotation effects and rotation-group bias. Sample rotation effects are evident in the considerably greater volatility in the quarterly growth rates for respondents who were not present in both quarters relative to continuers. The average growth rate is 8.8 percent for those who were not present in both quarters and −4.1 percent for continuers.¹³ The impact of rotation-group bias can be seen in chart 6 by looking at the self-employment growth rate for respondents present in both quarters. Because respondents are more likely to report self-employment in MIS 1 than in subsequent months, the quarter-to-quarter changes in the number of self-employed workers in the sample tend to be negative. These fluctuations, together with rotation-group sampling errors that affect the relative distribution of self-employment in incoming and outgoing rotation groups, suggest that the sample rotation design plays an important role in the volatility of self-employment hours.

Given these findings, we describe below how we apply two adjustments to the self-employment hours series to reduce excess volatility. The first adjustment involves directly compositing self-employment hours as is done for national employment and unemployment statistics also calculated from the CPS. The second adjustment involves removing a component of seasonal adjustment—the final irregular component adjusted for extreme values—which we show is primarily picking up sampling error.

Directly compositing self-employment hours

In 1954, BLS began using a composite estimate for reporting employment and unemployment measures. Morris H. Hansen, William N. Hurwitz, Harold Nisselson, and Joseph Steinberg describe the composite estimate as a weighted average of two different level estimates: one based solely on the current month and one that accounts for information from the previous month for the three-quarters of the sample that is surveyed in both months.¹⁴ The composite estimates reduce the variance of estimates of levels and changes, with the largest reductions occurring for estimates of changes.

Expanding on Hansen et al.’s description of the two weighted components of the composite estimate, Margaret Gurney and Joseph F. Daly, as well as Elizabeth T. Huang and Lawrence R. Ernst, show that this estimate can be improved by adding a bias-correction term that gives slightly more weight to data from respondents in MIS 1 and 5—months when unemployment responses have been found to be much higher.¹⁵ BLS uses these adjusted composite estimates, referred to as AK-composite estimates, for reporting national employment, unemployment, and those not in labor force, but not for reporting self-employment hours. We use this method to directly composite self-employment hours for the quarterly labor productivity series.

The composite estimate of self-employment hours in calendar month t, Ŷ_t^C, is measured by

where Ŷ_i,t is an estimate of self-employment hours for the population in month t using only responses in rotation group i, and A and K are parameters. The first bracketed term is the direct estimate of self-employment, an average of the estimate over all eight rotation groups i, ignoring any possible differences in responses across rotation groups. The second bracketed term adds to the previous month’s composite estimate, Ŷ_t−1^C, the change in estimated self-employment among those continuing in the survey from the previous month (i ∈ S (= 2, 3, 4, 6, 7, 8)). The final bracketed term is the bias correction that reweights the estimates from the incoming (i ∉ S) and continuing (i ∈ S) parts of the current month’s sample.

In an AK-composite estimate, the parameters A and K are selected to minimize the variance of the AK estimator relative to the direct estimate. How closely a set of parameters approximates the best linear unbiased estimate for a labor force characteristic depends both on the pattern of responses across rotation groups and on the correlation over time in the labor force estimates. This means that the optimal values for estimating the level of employment may not be optimal for estimating the level of unemployment, or the month-to-month change in unemployment. Janice Lent, Stephen Miller, and Patrick Cantwell find that A = 0.3 and K = 0.4 are optimal values for estimating unemployment levels and close to optimal for month-to-month changes, whereas A = 0.4 and K = 0.7 perform the best for estimating employment levels and changes.¹⁶ For our composite estimate of self-employment hours, we use A = 0.4 and K = 0.7, assuming that self-employment most closely resembles employment in terms of the correlation over time. We directly composite only self-employment on main jobs because class-of-worker information for second jobs is only collected in MIS 4 and 8. To the directly composited estimates, we add hours on secondary jobs, which have been weighted with CPS outgoing rotation weights.

Chart 7 compares the quarterly growth rate for the composited self-employment hours series with the growth rate for the published series from the second quarter of 2000 to the fourth quarter of 2019. Compositing reduces the size of most spikes in the growth rate. For example, in the second quarter of 2014, the growth rate of self-employment hours was −12.1 percent without compositing and only −6.0 percent with compositing. In the third quarter of 2019, the rate was 19.0 percent without compositing and 8.5 percent with compositing. Thus, it appears that compositing substantially reduces volatility in self-employment hours.

Seasonal adjustment and removing irregulars

To obtain quarterly self-employment hours for productivity measures, BLS seasonally adjusts the basic monthly CPS self-employment hours series and then averages these monthly data to obtain a quarterly series. Although the primary purpose of seasonal adjustment is to provide a clearer picture of underlying trends and cyclical movements distinct from regular seasonal movements, this adjustment can also be used to dampen unusual movements in the data. The X-13 ARIMA-SEATS program first models the time-series properties of the data, then uses that model to adjust and extrapolate forward the series, and finally decomposes the adjusted series into a trend-cycle component, a seasonal component, and an irregular component.¹⁷ Removing the seasonal component leaves the seasonally adjusted series, which consists of the trend-cycle component plus the irregular component.

The irregular component consists of ordinary noise in the series that can be due to both sampling and nonsampling error, as well as extreme abnormal events such as unseasonable weather, natural disasters, pandemics, or strikes. Outliers in the original data series are typically identified in the time-series model—either a priori or by the program’s automatic outlier-detection feature. In this way, the seasonal component can be estimated without being distorted by the outliers, but the seasonally adjusted data series will continue to include their impact.

We might expect greater sampling error in estimates of self-employment hours compared with estimates of hours for all workers, mainly because the sample sizes for the self-employed are relatively smaller. Response error also can arise because differences between self-employment and contract work and between incorporated and unincorporated self-employment are subtle and may be subject to greater differences in responses between the worker and a proxy respondent, and even from one interview to the next. Because our objective is to reduce the volatility in our estimate of self-employment hours, we want to remove the part of the irregular component that may be due to sampling error. The X-13 ARIMA-SEATS program used for seasonal adjustment produces a final irregular component series that excludes the impact of extreme values—which in most cases are due to abnormal events. Thus, we can remove this irregular component (hereafter referred to as the extreme-value-adjusted (EVA) irregular), along with the seasonal component, from the original series in order to obtain a smoother seasonally adjusted self-employment hours series.

To check whether the EVA irregular series is a reasonable estimate of the sampling error, we compare the size of the series’ irregulars with the standard errors of our estimates of self-employment obtained from generalized variance functions (GVF).¹⁸ Because the CPS publishes GVF model parameters for employment only, we restrict our analysis to the number of self-employed workers. We assume that the conclusions we draw from this analysis can be applied to self-employment hours.

Chart 8 shows the EVA irregular series obtained from a seasonal adjustment decomposition, standardized by dividing the EVA irregulars by the estimated standard errors of our composited self-employment estimates.¹⁹ Nearly all the values of the irregular component fall within 2 standard errors of our self-employment estimate, suggesting that the EVA irregular is due to sampling error and not economic outliers. This implies that removing this irregular component from the self-employment series can potentially further smooth the series without removing real movements in self-employment hours.

Charts 9a and 9b show the combined effects of compositing as well as removing the EVA irregular on the quarterly growth in self-employment hours. While chart 9a focuses on the second quarter 2000 through the fourth quarter of 2019, chart 9b extends the series through the fourth quarter of 2022 to show that the new method of calculating self-employment hours does not diminish the effects of the COVID-19 pandemic on the series. Removing the EVA irregular further reduces the volatility of the series beyond the reductions obtained from compositing the series alone (chart 7), without oversmoothing and distorting important economic outliers. If we were to instead remove the entire irregular series, we would remove most of the effects of the pandemic and potentially oversmooth the series during the 2000–01 recession and the 2007–09 Great Recession. (See chart 10.)

Effects of adjustments to self-employment hours

The outsized effect of self-employment hours volatility on total hours growth is the primary motivation behind Cunningham and Pabilonia’s working paper and the new method implemented here. An important check, then, is to see how this method of estimating self-employment hours affects the volatility in total hours growth. Chart 11 shows the published quarterly total hours growth and our estimate of quarterly total hours growth that uses the adjusted self-employment hours series. For most of the large spikes in the series, the adjustment reduces the volatility of total hours growth, especially toward the end of the series. In two noticeable instances, however, the adjusted series has larger spikes than the published series—in the second quarter of 2014 and in the third quarter of 2016. Looking back to those data points in chart 9a, we see that, in the second quarter of 2014, self-employment hours growth was −12.1 percent before the adjustments and −2.7 percent after; in the third quarter of 2016, that growth was −10.5 percent before the adjustments and −0.5 percent after. These large negative spikes in the unadjusted self-employment hours series were distorting underlying positive spikes in employee hours growth that now are evident after the self-employment hours series has been smoothed.

To see the reduction in volatility better, in chart 12, we reexamine the contributions to total hours growth that we presented in chart 3. The bars showing the contribution of the self-employed to hours growth are much smaller than they were in chart 3. For example, the contribution of the self-employed is substantially reduced in the third quarter of 2019. Under this new method of estimating self-employment hours, the self-employed contributed 0.2 percentage point to a 0.9-percent growth in total hours, whereas before the adjustments they contributed 1.4 percentage points to a 1.9-percent growth in total hours. Similarly, in the second quarter of 2016, the self-employed contributed 0.8 percentage point to a 1.2-percent growth in total hours under the old method, but they contributed 0.5 percentage point to a 1.0-percent growth in total hours under the new method.

Finally, in chart 13, we examine the effects of smoothing the self-employment hours series on annualized quarter-to-quarter growth in labor productivity. Over the 2000–22 period, the absolute change in productivity was greater than 0.5 percentage point in 21 of the period’s 91 quarters. In 8 of the 21 quarters, the productivity rate under the new method was larger in magnitude than the rate under the old method. This difference occurred in quarters that, prior to adjustment, had unusually large spikes in self-employment hours relative to employee hours. For example, in the third quarter of 2019, productivity grew at a 2.4-percent annual rate under the old method, but at a 3.4-percent rate under the new method. In two instances (the fourth quarter of 2002 and the second quarter of 2011), productivity growth changed direction from a small decrease to a small increase. Although we see changes in measured productivity using the new method, adopting the method does not affect long-run productivity growth, which averaged 1.9 percent annually over the 2000–22 period.

Conclusion

We find that self-employment hours are volatile because of survey-error-related volatility, which sometimes has outsized impacts on quarterly labor productivity. In 2024, to improve its measure of productivity, BLS will implement two adjustments to self-employment hours estimates: (1) directly compositing the estimates and (2) removing the EVA irregular component from the series. Because these adjustments will reduce survey-related volatility without oversmoothing the data, we will still be able to capture cyclical changes in self-employment. Consequently, the adjustments will improve our measure of labor productivity.