Navigating from the heights of technical rigour to the swampy reality: Lessons from New Zealand in population health outcomes monitoring (Italian Journal of Public Health, 2008)
Background: The objective of this paper is to describe and discuss two documents produced by the NewZealand Ministry of Health concerning the monitoring of outcomes of public health programs.The New Zealand Government is increasingly expecting planners and managers of publicly funded services toshift their focus from the delivery of ‘outputs’ towards achievement of ‘outcomes’. Intervention logic modelsand outcomes monitoring are promoted by central government agencies as suitable management methodsfor implementing the change. [1-3]Methods: To help managers design and implement comprehensive, effective and measurable populationhealth programmes the Ministry of Health recently published two guidance documents. The first documentprovided guidance about how to plan programmes using a generic logic model approach.[4] The second setout in detail a process on how to monitor population health programmes.[5] The intent of the documents wasto help managers navigate between the heights of technical rigour and the swamps of reality in the deliveryof population health programmes.[6]Results: A number of issues and implications for how population health programmes are planned monitoredand performance assessed have been identified by the guidance documents. Issues include the problem ofsmall numbers, understanding the difference between outcomes monitoring and traditional forms ofevaluation, and outcomes monitoring being seen as a tool for punitive performance management rather than‘continuous programme improvement’. Implications include more time spent on the design of programmes.Planners will need to focus upon better sequencing of activities, setting more specific and time limited goals,and to be more informed about how to use research to inform the selection of interventions.Conclusions: The guidance documents promoted by the Ministry of Health have provided useful advice abouthow to develop and use logic models and outcomes monitoring in the planning of population healthprogrammes. Practitioners particularly welcomed the inclusion of a glossary that included explanatorycomments and examples. Two key issues have been identified with the application of outcomes monitoringthat need to be managed. The first is the issue of statistically small numbers associated with relatively rarehealth events. The second is to promote outcomes monitoring as a tool for continuous programmeimprovement, rather than as a potentially punitive.
Keywords: outcomes monitoring; logic model; planning; management; public health programmes