Governments large and small spend considerable amounts of public money to pay for health facilities, public safety, social aid and public works, and capital improvement projects. This money is usually derived from taxes that are allocated to federal programs by Congress, but they can also come directly from agency fines, fees, or settlement collections. This makes reporting Federal public spending data including agency financial information somewhat problematic and just plain difficult.


Public spending and budgets are absolutely essential to publish as Open Data. This post will describe some of the challenges and opportunities in helping the public to understand where Federal public spending is going and to whom; especially the difference between the use of standards in regulatory versus financial data reporting. Several pieces of legislation are about to take effect at the federal level in 2017. Congress should take action immediately to expand existing law to include federal revenue derived from fees and fines. This would give the public a holistic view of government appropriation and expenditure. This could also be an opportunity for local and state governments to emulate the 2014 federal Digital Accountability and Transparency Act, or DATA Act (PL 113-101) with their own localized public appropriations and expenditures.


Two years ago Congress unanimously approved, and President Obama signed, the legal reform necessary to express federal financial and spending information as a single data set. The DATA Act directs Treasury and OMB to create a single, government-wide data structure for all federal financial information.

The Washington DC-based Data Coalition, founded by Hudson Hollister, noted that the Committee and Subcommittees mentioned above were particularly interested in the revenue that agencies receive as fees, fines, penalties, and settlements outside the Congressional appropriations process. So far, the data structure that Treasury and OMB are enforcing under the DATA Act focuses on expenditures, not on receipts, so a single electronic picture of all spending won’t be able to provide full details on how fees, fines, penalties, and settlements are spent.

By requiring Treasury and OMB to set up a single government-wide structure for Federal public spending data on appropriations, grants, and contracts, the DATA Act gives us a single electronic picture of all that information. This information would be in one consolidated mass that allows Congress and the public to comprehend and consequently, to control spending; an almost verbatim quote from Thomas Jefferson in a letter to then-Treasury Secretary Albert Gallitin.

While data enrichment with the Data Coalition’s recommendations to Congress will certainly help give us a more holistic view of Federal spending, it is not the only challenge.


“Commercially available software will use this single data set to portray an electronic picture of federal spending. Several of our Data Coalition members are working on software that will provide agencies, and Congress, and the public with new ways to comprehend, and control spending.”


Credit: DAIMS Model Courtesy of the Data Coalition and Hudson Hollister

Some of the issues the Data Coalition supports, echoed by OpenDataSoft, and this author are the development of “gold standards” around financial data reporting. In general these standards should seek to emphasize the following strategies.


Creating so-called “Gold Standards” in financial reporting will require the type of ongoing engagement at the public sector (policy) and private sector (technology) levels. The public and private sectors should keep a dialogue open as to how to best implement these standards within data publishing platforms to allow more ease of use in local and federal agencies to comply with new policy.


We have started to see the emergence of standards within Open Data. These standards emerge out of an inconvenience not to do so. LIVES is a great example of standardization regarding food inspection data used in the United States. Is XBRL the “Gold Standard” we seek? By engaging with Open Data and open government advocates at both the public and private sector we can exert some pressure on agencies to publish data on company registers and other corporate financial data. Certainly the last 8 years of efforts both in Congress and the White House have seen a huge shift in thinking from the early days of “pay for play” access to SEC data and the culture we see today of “open by default.”


Financial reporting data has quite a number of competing standards and champions both for and against each standard. Rather than list all of them we will focus on the one that is at the center of the DATA Act: XBRL.

While the DATA Act is responsible for budget and expenditure data from Federal agencies, regulatory bodies such as the SEC also use XBRL.

XBRL is not a flat file in the sense that Open Data publishing platforms can easily reuse all data from an XBRL filing. The SEC introduced XBRL in 2005 as a way of providing enriched data to corporate filings. While using the term “filed,” the SEC considers XBRL documents as “furnished not filed,” so there is a clear distinction between these supplemental documents and the official filed documents for which registrants are subject of many more legal requirements.1-2

An XBRL instance document is a computer file containing the computer-readable tags that are representations of financial information contained in a company’s financial statements. The instance document is intended for input to computer software rather than to be read directly by people. The instance document consists of six interrelated computer files: a taxonomy of the company’s financial statement elements and five files known as “Link Bases” containing the labels, calculations, references, definitions, and financial statement structure associated with the financial statements.


OpenGov and Socrata have both declined to support XBRL, a financial reporting standard. XBRL allows business and financial systems to work together using a single standard for the reporting of financial data. Studies have varied the efficacy of this standard. In 2009 North Carolina State University published an analysis of the phase-in of the SEC XBRL standard and found serious errors and miscategorization of the data. Here’s a quote from the abstract:

“The SEC phase-in of XBRL financial statement filings began June 2009, and by 2011, all public registrants will be required to file XBRL disclosures. While the SEC expects the interactivity of XBRL-tagged data to add value to financial reports, this benefit will materialize only if the XBRL statements are accurate and reliable. If inaccuracies or other significant problems occur in initial XBRL filings, registrants stand to lose credibility and users will lose confidence in the data, potentially forcing the abandonment of the XBRL reporting initiative. This study evaluates the accuracy of early voluntary filings and develops an expectation about the accuracy of mandated filings. While improvements in the XBRL standard and related technology will mitigate certain errors, other errors, related to inexperience, will persist.”3

Later publications cite a steep learning curve and lack of automation as the source of the inaccuracies.

Here’s an additional quote from the abstract:

“Since the mandate by the U.S. Securities and Exchange Commission (SEC) to begin interactive data reporting in June 2009, according to XBRL Cloud, an XBRL product and service provider, more than 4,000 filing errors have been identified. We examine the overall changing pattern of the errors to understand whether the large number of errors may hamper the transition to interactive data reporting. Using a sample of 4,532 filings that contain 4,260 errors, we document a significant learning curve exhibited by the XBRL filers. Specifically, we find that the number of errors per filing is significantly decreasing when a company files more times, suggesting that the company filers or the filing agents many companies use learn from their experiences and therefore the future filings are improved. Our findings provide evidence to encourage the regulatory body, the filers, and the XBRL technology supporting community to embrace the new disclosure requirement in financial reporting. The significantly decreased error pattern also helps address the information users’ concerns regarding the data quality of XBRL filings.”4


“The increasing global adoption of XBRL and its potential to replace traditional formats for business reporting create a need for quality assurance for XBRL-tagged data. Although prior studies have addressed assurance issues on XBRL-related documents (i.e., instance documents and extension taxonomy) and related audit objectives, they primarily focus on the U.S. and, thus, may not be comprehensive enough for use in other countries. Furthermore, no prior literature discusses what and how computer-assisted audit functions can help auditors while they are performing assurance on XBRL-related documents. The main goal of this paper is to introduce computer-assisted audit functions that can be used by auditors to perform audit tasks to attain identified audit objectives. Based on professional guidelines and prior academic studies, this study introduces a set of audit objectives and related audit tasks that auditors might confront if they are asked to provide assurance on XBRL-related documents. The study then demonstrates a set of related computer-assisted audit functions for conducting the audit tasks and discuss how the identified audit objectives could be achieved using these functions.”5


According to “Letting the Public In Opportunities & Standards for Open Data – Financial Transparency Coalition.” 11 of the 20 most secretive societies in the world use XBRL.6 This is not a technical flaw in XBRL but more of data governance and policy flaw or “feature” used by these societies to hide who actually benefits from company trading.

Having the SEC adopt XBRL as part of the DATA Act has some far-reaching consequences especially in country-to-country financial disclosures. XBRL can expose what the Financial Transparency Coalition calls Beneficial Ownership (BO) data of companies reporting to the SEC and the SEC’s global counterparts.7 This is part of what the Data Coalition is asking of congress.

The Data Coalition is asking Congress to amend the company reporting requirements to allow the identifiers of individual BOs to be published and to amend the regulations to be published under a single entity. Finally the Data Coalition seeks to have data standards brought over State, Local and Tribal financial reporting.


Having the Federal Government expand the DATA act according to the DAIMS model would solve local problems as well. This would go a long way in addressing some of the common obstacles faced in many of the local and state government Open Data efforts regarding financial and regulatory Open Data.


Undoubtedly a sincere sentiment but one not borne out by the work the World Bank has done in studying financial reporting, accountability, and accessibility. In Open Financial Data Demand, Capacity, and Citizen Engagement in Online and Offline Communities published by the World Bank in 2014 respondents in a survey overwhelmingly cited a desire to understand and read public financial disclosure data. In the US, almost everyone has access to some form of the Internet.8 59% of the rest of the world is offline.9 Most respondents of the World Bank Survey indicate a desire for access to public expenditure and budget data over 60% could understand these data.

The widely held misconception that the public is not interested in nor can they understand public sector financial data is a boondoggle.


1. SEC. (2005). XBRL Voluntary Financial Reporting Program on the EDGAR System. Securities and Exchange Commission. Washington D.C.: Federal Register, February 8, 2005. https://www.sec.gov/rules/final/33-8529.htm.

2. Bartley, Jon W., and Chen, Y. Al and Taylor, Eileen Zalkin, A Comparison of XBRL Filings to Corporate 10-Ks – Evidence from the Voluntary Filing Program (February 18, 2010). Available at SSRN: https://ssrn.com/abstract=1397658 or http://dx.doi.org/10.2139/ssrn.1397658

3. Bartley, Chen, Taylor (2010), p. 2.

4. Hui Du, Miklos A. Vasarhelyi, and Xiaochuan Zheng (2013) XBRL Mandate: Thousands of Filing Errors and So What?. Journal of Information Systems: Spring 2013, Vol. 27, No. 1, pp. 61-78. http://aaajournals.org/doi/10.2308/isys-50399

5. J. Efrim Boritz and Won Gyun No (2016) Computer-Assisted Functions for Auditing XBRL-Related Documents. Journal of Emerging Technologies in Accounting: Spring 2016, Vol. 13, No. 1, pp. 53-83. http://aaajournals.org/doi/abs/10.2308/jeta-51436

6. Davies, Tim, and Robin Gower. “Letting the Public In: Opportunities & Standards for Open Data – Financial Transparency Coalition.” Financial Transparency Coalition. Open Data Services, n.d. Web. 04 Dec. 2016. https://financialtransparency.org/reports/letting-the-public-in.

7. Idem.

8. Lee, Open Financial Data Demand, Capacity, and Citizen Engagement in Online and Offline Communities 2014; World Bank

9. Idem.


Popular posts from this blog

Podcast: Open Data Discussions with Anthony Fung

Open Data Licensing