Australia’s Department of Health and Aged Care has been criticized for not having enough confidence in the quality of the data it uses to monitor and report immunization coverage against COVID-19.
A recently published report by the Australian National Audit Office (ANAO) reviewed the department’s conduct of the rollout of a COVID-19 vaccine, which it said was “one of the largest health logistics exercises in Australian history”.
WHY IT MATTER
The Department of Health has established four key systems to manage and monitor vaccine distribution, namely the Australian Immunization Register (AIR), the Vaccine Administration System (VAS), the COVID-19 Vaccine Administration System (CVAS) and the Decision vaccine data. Data collection and IT management for these systems were outsourced from IT services firms including Services Australia (for AIR), Salesforce (for VAS and CVAS), Accenture (for dashboards) and Amazon Web Services (for CVAS).
According to the audit report, the department cannot assign its responsibilities under various privacy, confidentiality and security laws to the data collected through these systems.
“There is no health [the] assurance that third parties have IT controls in place to ensure the confidentiality, integrity and availability of data,” the release said.
The report also found that the department “did not formally review the data” entered into these systems, resulting in “undetected and undisclosed inaccuracies” in the data, particularly in the AIR and CVAS systems.
It was said that the department had not clarified its responsibility for data quality in AIR, where all vaccination records must be reported. The national registry was noted to have a 14 percent error rate, based on a 2018 study by the National Center for Immunization Research and Surveillance. Although mandatory reporting of vaccinations through AIR may have improved error rates, “there is no available evidence to support this,” the report said.
In addition, the audit highlighted that there are data gaps related to the vaccination status of some priority and target groups. The department was flagged as not having accurate data on the vaccination rate of critical and high-risk workers – a priority group under the first phase of the vaccine rollout – “as occupation data is not collected in AIR”.
Meanwhile, the ANAO still acknowledges the department’s use of available data to plan and support vaccine rollout despite inaccuracies in data collection. “However, [the Health Department] did not quantify inaccuracies in its internal or external data processes during the audited period.”
In response to the Auditor General’s findings and recommendation, the Department of Health agreed to establish processes to ensure it “regularly receives and reviews” data quality assurance and IT controls in place in externally managed risk-based systems, including IT security, change management and batch processing. “The department will undertake an independent review of its IT controls and the implementation of its internal quality assurance framework.”
THE WIDER CONTEXT
The ANAO said it carried out the audit to provide independent assurance to Parliament that the country’s COVID-19 vaccine rollout was “planned and executed effectively”.
In essence, it found that while the Ministry of Health’s final management arrangements in managing the roll-out of the vaccine were “largely effective”, its implementation was “partially effective”, with vaccine delivery to priority groups and the general population ” does not achieve the objectives’.
The auditor general then recommended improving data quality and IT controls and conducting a comprehensive review of vaccine distribution. “The [Health] The department is committed to effectively implementing the recommendations and has already initiated steps to address the issues identified in this audit,” it said in response.
Since administering the first COVID-19 vaccines in February last year, the department has administered about 63 million doses nationwide by August 2022.