Pneumonia remains a major cause of child death globally, and improving antibiotic treatment rates is a key control strategy. Progress in improving the global coverage of antibiotic treatment is monitored through large household surveys such as the Demographic and Health Surveys (DHS) and the Multiple Indicator Cluster Surveys (MICS), which estimate antibiotic treatment rates of pneumonia based on two-week recall of pneumonia by caregivers. However, these survey tools identify children with reported symptoms of pneumonia, and because the prevalence of pneumonia over a two-week period in community settings is low, the majority of these children do not have true pneumonia and so do not provide an accurate denominator of pneumonia cases for monitoring antibiotic treatment rates. In this review, we show that the performance of survey tools could be improved by increasing the survey recall period or by improving either overall discriminative power or specificity. However, even at a test specificity of 95% (and a test sensitivity of 80%), the proportion of children with reported symptoms of pneumonia who truly have pneumonia is only 22% (the positive predictive value of the survey tool). Thus, although DHS and MICS survey data on rates of care seeking for children with reported symptoms of pneumonia and other childhood illnesses remain valid and important, DHS and MICS data are not able to give valid estimates of antibiotic treatment rates in children with pneumonia.