Large Scale Data Collection and Validation



Executive Summary

The Australian Taxation Office (ATO) collects data from numerous sources to support its compliance activities and to enable streamlined reporting of circumstances for Australian Businesses and Citizens.

The timely collection and processing of data to produce a high quality, trusted information asset is critical to ensuring participation in the taxation system and a level playing field for Australians.

Over the past decade the number of data sources, the frequency of data collection and variety of data formats has grown substantially. The ATO identified a need to enhance its technology infrastructure to include a flexible, robust and performant data collection and validation platform, known as the Bulk Data Exchange (BDE).

The team from Capability Wise and eBlueprint collectively led the $13 millon project to implement an enterprise-class Bulk Data Exchange. This included all facets of the delivery lifecycle including, but not limited to project management, scoping, technical design, build, implementation and the hyper-support period.

The team liaised with agency staff and external stakeholders including multi-national software vendors, banks, insurance companies and other government departments to ensure a fit-for-purpose and reliable solution was implemented within the project period.

The Bulk Data Exchange is now a cornerstone component of the ATO’s data ecosystem, collecting the bulk of ATO’s inbound data (by volume) and enabling secure external data sharing between all levels of Australian government and select international government bodies.

The Challenges

The ATO has been progressively increasing the volume of data it collects over the past 30 years to improve its understanding of each Business’ and Citizen’s taxation circumstances. This information is often collected from third-party organisations in whatever format it is available and using whatever data transfer protocol minimised the barriers to data collection. This approach whilst maximising the available data sources resulted in a disparate suite of connections, formats and platforms that were time consuming and difficult to effectively manage.

The ATO sought to improve this situation through the creation of a single bulk data exchange platform that was capable of collecting and validating third-party data in various formats, supporting common connection protocols and providing a single administrative and technology control tower to manage key data transfer activities.

How Our Consultants Helped

To support the ATO to achieve their target outcome the teams from Capability Wise and eBlueprint jointly delivered the enterprise-class BDE Platform.

Our expert consultants provided the following capabilities:

  • Project Leadership – project initiation and technology leadership of the $13m investment to establish the Bulk Data Exchange.

  • Technology Project Management – day-to-day project management of technology components of the project.

  • Bridging User Experience and Technology – working closely with the ATO teams to translate the desired user experience into pragmatic, robust technology features, and translating technology constraints into viable business process outcomes.

  • Solution Architecture Leadership – to describe the critical technology components and capabilities required for a successfulsolution.

  • Secure Data Exchange Expertise – to design a data transfer solution that balanced IT Security Controls, Authentication Techniques and Usability to ensure confidential data assets remain protected.

  • Technical Delivery Excellence – to build, validate and document a reliable, scalable solution that met all ATO and Australian Government requirements, including the Information Security Manual.

  • Information Modelling and Validation Expertise – to analyse, document, develop and quality assure data collection specifications, validation rules, error handling and data transformations for downstream utilisation.

  • Application Support and Management – to support the ATO to introduce the solution into its production environment, training agency support staff, and provide hyper-support during the warranty period.

Additionally, due our expertise and experience several members of our project team were subsequently re-engaged to design and deliver the ATO’s other major data collection platform, the eCommerce Platform, which focuses on direct interaction with small and medium business systems.

The Results

The Bulk Data Exchange Project has been a major success for the Australian Taxation Office. Within the first week of operation the ATO received several positive unsolicited feedback items from Australian Businesses, including a statement that the solution was “the best new IT System the Australian Taxation Office has introduced in years”.

The solution has been identified as one of two long-term strategic data collection channels that will support the ATO’s ongoing requirements between now and 2024. BDE continues to be extended by the business to enable more and more data interconnections across federal, state and territory agencies, local governments and international government partners.

BDE also supports the ATO to connect securely with major private sector organisations including banks, private health funds and share registries through direct machine-to-machine technologies. Additionally, it integrates directly behind the ATO’s Web Channels to provide Australian Businesses and Tax Agents with a low barrier entry point for digital data transfer.


Tags: Digital Data EDI Bulk