Background

In 2007 and early 2008, New York State issued a ​Request For Public Comment concerning electronic record policy for government documents. This is an attempt to craft a response to that call for comments. The public comment period runs until 2008-01-18. If you have comments for the state to read, you should mail them to erecords-study@oft.state.ny.us. You should also publish them so others can read the arguments.

If you feel that these arguments resonate with you, feel free to copy or cite them in your responses to the State. And of course, this is the web: you should always feel free to link to this page.

Freedom as a Fundamental Goal

I believe that New York State (NYS) should explicitly require the use of free and open standards for all electronic/digital records, and these standards should apply to all communications entered into by the state, including:

• document formats (UTF-8 text, OpenDocument Format, W3C-compliant XHTML, etc)
• communications protocols (IEEE 802 Ethernet, IP, TCP, HTTP, TLS, etc)
• certification mechanisms (OpenPGP, X.509, etc)
• operating systems and software (GNU/Linux, OpenOffice, Firefox, etc)

When i say free, i should be clear that i'm referring to liberty, not price. The standards and tools promoted/required by the public sector must not be held under the control of a private organization. Anyone should be able to make use of the technology in question, including modification and redistribution of any tools required to access the data.

There should be no proprietary hurdles to jump in order to have access to government data or records. This should be an explicit state mandate.

Below, i detail some of the specifics about why choosing free standards is important.

Encouraging public access

2. What mechanisms and processes should the State of New York

establish for accessing and reading its electronic records in order to encourage public access to those records?

Requiring free and open mechanisms for accessing public data will encourage public access because there are no proprietary requirements to be met in order to view the data.

For example, no NYS World Wide Web site should ever require the use of Microsoft Internet Explorer. This would cause problems for citizens who have not purchased a Microsoft operating system, since IE only runs on Microsoft's Windows (IE has not been supported on Mac OS for years, and has never been available on GNU/Linux or any other operating system). Instead, NYS websites should adhere to the W3C's XHTML 1.0 standard. Users of all modern computer systems have access to free tools which can render this public standard, so no one is disenfranchised.

A more subtle example: no NYS electronic records should be stored or published in Adobe's Flash format (a.k.a. swf). While Adobe makes players for this format available without charge for most modern operating systems, users are not allowed to modify or redistribute these players, and the license of the format specification itself prohibits use of the specification to make another player. This lack of liberty (despite the absence of cost for those platforms provided) means that the public is required to make some sort of arrangement with Adobe (a specific private company) in order to access public data. Since Adobe has no requirement to interact with the public in an evenhanded way, and citizens have no legal recourse to accessing the data themselves, they are effectively discouraged from accessing it.

Encouraging interoperability

3. What mechanisms and processes should the State of New York

establish for accessing and reading its electronic records to encourage interoperability and data sharing with citizens, business partners and other jurisdictions?

Free formats, standards, and tools encourage interoperability and data sharing because there is no restriction on adoption for the other parties in communication.

If NYS was to choose a proprietary format for electronic records, it would need to pay the proprietor of that format a fee for its use. If a neighboring jurisdiction was to choose a proprietary format for electronic records, it would also need to pay the proprietor a fee for its use. If the two jurisdictions happened to choose different proprietary formats, then both jurisdictions will need to pay fees to both vendors if interoperability is desired.

While this would be a windfall for the vendors, the costs to each jurisdiction scale exponentially as the number of jurisdictions desiring mutual interoperability grows. Better to choose free interchange standards so that there are no additional per-jurisdiction per-vendor costs due to the proprietary nature of the records.

Encouraging appropriate government control

4. What mechanisms and processes should the State of New York

implement to encourage appropriate government control of its electronic records?

Free formats, standards, and tools allow governments (and other entities) to retain control over their own data.

As a fictional example, consider sanitation district boundary records stored in GBG format in 1995. GBG was (fictionally) owned at the time by Dispatch Service Co., which specialized in sanitation logistics. Fast forward to 2008: DSCo has decided that it does not want to support the format any longer (or has been sold to a parent company uninterested in sanitation logistics, or has simply collapsed). The State is now in a weak position to have any access to tools needed for modifying garbage truck routes.

Choosing free formats and free tools from the beginning would allow the government better control over the data because it would be insulated from the fate of any particular instance of DSCo.

Encouraging choice and vendor neutrality

5. What mechanisms and processes should the State of New York

consider for encouraging choice and vendor neutrality when creating, maintaining, exchanging and preserving its electronic records?

Free formats, standards, and tools avoid vendor-lock-in and promote competitive industry.

As in the above example with sanitation logistics, choosing free formats and free tools would let NYS negotiate among any vendor to meet its sanitation dispatch needs. Even if DSCo was still a thriving, healthy corporation, if the formats and tools used were all free, it would be forced to compete with any other entity willing to do the work needed by the city. The underlying freedom translates into a better negotiating position for the State, and encourages healthy competition and interoperability among the potential vendors.

Electronic record life cycle

6. Are there mechanisms and processes the State of New York should

establish that are specific to the management of its electronic records in its various life cycle stages (creation, maintenance, exchange, preservation and disposal)?

Free formats, standards, and tools preserve the integrity of electronic records, and enable verifiable disposal.

When the underlying technologies are unencumbered by any restrictive vendor-applied rules, it becomes much easier to keep data up-to-date, and to be certain that destroyed data is actually destroyed.

If you want to ensure that records can be kept up-to-date, again choosing free formats, standards, and tools helps you to make changes and negotiate between vendors.

Similarly with record disposal. Only unfettered, transparent access to your systems can verify that no traces of the record remain. While i don't expect complete destruction of records to be a high priority for most government tasks, it is extremely important in some cases. For example, court-ordered destruction of data gathered by unconstitutional methods should be verifiable.

Data preservation

7. How should the State address the long term preservation of its

electronic records? What should the State consider regarding public access to such archived content?

Free formats, standards, and tools make it easier to transfer any electronic records to new media, and ensure that the government and the public of the future will be able to interpret the data. Can you read a Lotus 123 spreadsheet on your computer today? Can you read an old ASCII text file? The free ASCII format is much more accessible in the future. Similarly, choosing UTF-8 and the Open Document Format today will give a better shot for the readers of tomorrow.

Preserving data for your own future consumption is very similar to sending data to a relatively-sophisticated party with whom you only have one-way communications. Remember that you don't have any way of knowing if the particular operating system, interface, or libraries are available to your future self. If you want that remote party to be able to understand your data, you are better off using well-documented, freely-available tools and formats.

Specific statute changes

8. What changes, if any, should be made to the government records

management provisions in New York Statutes? (Please reference those laws which are cited here: ​http://www.archives.nysed.gov/a/records/mr_laws.shtml).

Implementation costs

9. What constraints and benefits should the State of New York

consider regarding the costs of implementing a comprehensive plan for managing its electronic records?

Highly-specialized data formats

10. What should the State of New York consider regarding the

management of highly specialized data formats such as CAD, digital imaging, Geographic Information Systems and multimedia?

Ongoing costs and savings

11. What constraints and benefits should the State of New York

consider regarding potential savings or additional costs associated with the management of defined electronic record formats?

Existing policies as precedent

12. What existing policies and procedures in the private or public

sector for the management of electronic records would be appropriate for the State of New York to examine? Please cite specific examples.

Adequacy of existing policy

13. Are New York State's existing standards, regulations and

guidelines regarding records management adequate to meet the challenges of electronic records retention? How should these standards, regulations and guidelines be changed?

Miscellaneous

14. What else should the State of New York consider about this

subject?