AURDA Presentation/Meeting Feb 10, 2004

Background: This meeting was called to get user feedback on the AURDA tool before final release. It was decided to call the meeting in early February so that the AURDA project could be completed prior to Professor Kathy Eagar's visit in late February. In spite of the relatively late notice, at least two representatives were able to come from each of the Auckland DHBs.

Present:

Discussion: Discussion ranged from technical queries about the NZ-CAOS episode model through to broader debate about the lessons learned from the project.

Key Issues Raised:

1. The importance of data quality in projects such as CAOS, MHINC, and MH-SMART. There were some concerns about growing data collection requirements in spite of substantial data quality issues with existing data collection exercises. The KISS principle was mentioned. It was also noted that if data is not considered reliable it becomes completely worthless. A final point was that genuine DHB representation was important to keep projects “real”.

2. The possible benefits of extending AURDA beyond DHB comparisons e.g. looking at ethnic or gender differences for all DHBs combined. It was considered useful to continue with the existing interactive interface. It was noted that the additional costs were marginal given the work which had already been completed.

3. The critical importance of convergence in DHB IT systems. Too much resource is wasted with each DHB reinventing the wheel. As the CAOS project demonstrated, this can result in major data quality problems even in an environment of focused data quality auditing.

4. The value of cleaning data at source as it is entered – not after the event. It was also suggested that DHBs should collaborate on common standards for data collection e.g. for MHINC. At present, there are substantial differences in approach between DHBs. Some approaches are likely to produce better quality data than others and some degree of consolidation could be beneficial.

5. The dangers of simplistic comparisons using complex data such as that in casemix. Looking at AURDA it soon became apparent that the particular episode model underlying NZ-CAOS – i.e. ongoing and completed episodes in a 3 month framework – had some significant counterintuitive aspects. It was demonstrated, for example, that a DHB could have cheaper adult inpatient episodes overall, and yet appear more expensive in both the complete and ongoing branches. It was also observed that small changes in filtering e.g. looking at episode costs for female clients only, could reverse the position of DHBs and significantly change overall relativities. Finally, the risk of possible gaming of a casemix system was raised.

6. The evolutionary nature of casemix. In spite of the limits of the existing casemix classification – e.g. the minimal cost variation in the important Adult Inpatient Ongoing branch – it is possible that a future system will fulfil the promise of casemix. The improvements in NZ-CAOS vis-à-vis MH-CASC were acknowledged.

7. The importance of the Question or the hypothesis in research projects. It was suggested that there could still be a lot of additional value to be gained analysing the CAOS data to answer clearly-defined questions.

8. The possibility of collaborative development of benchmarks was also discussed. The counter-argument was presented that it would be better to focus on getting data used internally first.

Next Steps

1. Disseminate the results of the meeting and get feedback from the other involved DHBs

2. Investigate the opportunities for greater DHB involvement in decision-making about the development of additional data collection requirements

3. Finalise AURDA and distribute

4. See if there is interest in a smaller project (e.g. $1,000 per DHB) to extend AURDA so that it covers DHB-combined data