Priority setting remains a big challenge for health managers and planners, yet there is paucity of literature on evaluating priority setting. The purpose of this paper is to present a framework for evaluating priority setting in low and middle income countries. We conducted a qualitative study involving a review of literature and Delphi interviews with respondents knowledgeable of priority setting in low and middle income countries. Respondents were asked to identify the measures of successful priority setting in low and middle income countries. Responses were grouped as: immediate internal or external/delayed internal or external. We also identified some pre-requisites for successful priority setting. The immediate internal measures included increased efficiency in decision making, improved quality of decisions and fairer priority setting. Immediate External measures included-improved public understanding and acceptance of decisions, increased public participation, increased trust. Delayed Internal measures included increased satisfaction, understanding, compliance, balanced budget, achievement of organization goals, and improved internal accountability. Delayed External measures include impact on policy and practice, improved population health and reduction of health inequalities, achievement of health system goals and strengthening of health care systems. Identified pre-requisites for successful priority setting included; the presence of credible priority setting institutions, incentives for participation and implementation and resources, capacity and political will to implement. These would be augmented in a conducive political, social and economic context. This framework, although not exhaustive, provides a practical basis for planning and evaluating priority setting in low and middle income countries.