Abstract
The need for all archives to move from the paper based to the digital world is very apparent. However, governments and funding entities are unwilling to invest in the back capture of archival materials. State Records NSW overcame this impasse by entering into commercial agreements with 3rd party providers of genealogical data. This article outlines some of the reasons for entering into such agreements and the positive outcomes for State Records NSW and users of digital archival material.
The Archives Authority of NSW and the Church of Jesus Christ of Latter Day Saints
State Records New South Wales (SRNSW) and its predecessor, the Archives Authority of New South Wales, have, by extreme economic necessity, had a long history of partnerships with external service providers, now called third party providers. Christine Yeats, the former Manager, Public Access, at SRNSW, was the key individual involved in the process of establishing third party agreements. It is only with the assistance of these partnerships that SRNSW has been able to undertake, first, the preservation of some of the most heavily used and, therefore, most vulnerable items from the state archives collection and, second, the improvement of the accessibility of highly used items from its collection via microform surrogates.
The Archives Authority of NSW came into existence with the passage of archives legislation in New South Wales (NSW) in 1960. In the first 20 years of its existence, the Authority relied to a great extent on the work done by representatives of the Church of Jesus Christ of Latter Day Saints, in particular, the Genealogical Society of Utah, which is known today as Family Search. Without the efforts of the Latter Day Saints, or Family Search, it would have been impossible to microfilm such a vast quantity of material during the Authority’s formative period of operation.
The Latter Day Saints prepared and filmed hundreds of rolls of microfilm at no cost to the Authority and provided the Authority with viewing copies, as well as a copy master for further duplication. This extraordinary resource – made up of thousands of rolls of microfilm – was the foundation of the Authority’s later initiatives to radically improve the accessibility of the New South Wales state archives by taking the records to the people.
The Genealogical Research Kit: the first step in taking the records to the people
The genealogical boom of the 1980s – leading up to Australia’s Bicentenary in 1988 – saw record levels of records usage at the state archives. It was not unusual to have 100 researchers per day coming into the Authority’s reading room at 2 Globe Street, The Rocks in central Sydney. With this level of patronage and limited resources, something had to give. The then-principal archivist, John Cross, devised a system to offer microform copies of the key colonial and genealogical records for sale to libraries, societies, historical associations and family history groups – some or all of the components of which became known as the Genealogical Research Kit (GRK).
The take-up was very good, and, before long, copies of those all-important shipping records, convict indents, registers of land grants and so on were widely available in Sydney and New South Wales libraries and historical societies, interstate around Australia and even in New Zealand.
The Archives Resources Kit: the second step in taking the records to the people
In late 1999, the then-Director of SRNSW, David Roberts – my predecessor – decided to revamp and re-badge the Genealogical Research Kit with the addition of the Colonial Secretary’s Papers, from 1788 to 1825, along with some additional microform material and published guides. Out of this initiative, the Archives Resources Kit (ARK) was born. It was a resource for all researchers, not just family historians.
To set this initiative in motion, 39 (and later 40) Community Access Points (CAPs), comprising the six regional repositories, together with 33 (and later 34) central libraries in rural and regional New South Wales, were selected. The lucky 40 were offered the opportunity to become part of the CAP network. Those that did not have a copy of the old GRK were provided with the ARK. Those with existing GRK holdings were topped up with the additional ARK material. All this was provided free of charge in what was planned as a project to deliver the records to the people relying on SRNSW resources for their research needs, first as analogue surrogates and then in digital format.
NamesPlus!
In response to the genealogical boom of the 1980s, which swept through most libraries and archives, the Archives Authority published a number of core indexes for its records. This initiative was reactivated in the early 2000s with the launch of NamesPlus! – a plan to capture (or document, where they could be found) the names of everyone mentioned in the official records of the nineteenth century. This was a very ambitious project that was very well supported by a pool of more than 50 volunteers at SRNSW. The initiative also supported the Archives Resources Kit, since it provided indexes to many records in the ARK. So NamesPlus! was born and, with it, the development of an information-rich website through more than 80 indexes and lists of the thousands of name-based records in Archives Investigator – the main archival finding aid of SRNSW. In all, over one million names are now searchable on SRNSW’s website, making it one of the most popular government websites in New South Wales.
SRNSW and the digital age
As we went into the 2000s, it became clear to us at SRNSW that we would be challenged to achieve a viable alternative to the traditional reading room in the digital environment. Small digitisation projects were feasible and were being undertaken; however, digitisation, on the scale being requested by the research community, could not be achieved without a significant budget increase or assistance from outside. Funding bids and business cases to the New South Wales State Government had been made on this issue for many years without result. To have done nothing in relation to large-scale digitisation in the face of increasing customer demands would have been disastrous for SRNSW and would have consigned the organisation to increasing irrelevance in a digital world. The small cottage-industry digitisation projects we had could hardly keep pace with the industrial requirements of our users.
Genealogy is good business: enter the third party provider
Looking back at the pre-digital modes of delivering services to our researchers now seems like looking back at another age. The world of the family historian has also changed dramatically in the past few years. Today, genealogists expect and demand the digital experience. They are not content with indexes and microfilms at an archive, library, reading room or regional repository. They want access to the records when it suits them, not when the reading room is open – a common scenario that was being experienced across the world. It is not surprising, therefore, that entrepreneurs took genealogy from a being hobby to big business.
It is also not surprising that about six or so years ago, SRNSW was approached by what we now refer to as a third party provider – the organisation that was originally known as My Family, but today known as Ancestry. They approached SRNSW to see if we would be interested in entering into a commercial arrangement. Basically, the third party provider was talking about the possibility of the following combination of options:
| • | giving us money, | ||||
| • | digitising core records by drawing on the 3,000 or so reels of microfilm – thanks largely to the work of the Latter Day Saints all those years ago, | ||||
| • | indexing the records, | ||||
| • | making the records and the indexes available on the provider’s website, | ||||
| • | giving SRNSW free reading room access (so researchers could view the material free in reading rooms), | ||||
| • | charging people to view the records, and | ||||
| • | giving us a copy of the digitised images. | ||||
Given our history of parlous funding since our inception in 1960, this was an offer that was too good to refuse.
Hurdles to be overcome
Before any of this could come to fruition, there were a number of significant challenges, which, at the time, appeared to be insuperable. The first challenge related to the issue of entering into an arrangement where researchers would have to pay for access to the New South Wales state archives. Up until this time, archives had operated on the basis of free access to the records of government, which were, ultimately, the records of the people and a cornerstone of our democratic system. Researchers would now have to pay for access to records through Ancestry; however, much of the material that was the focus of our original approach was already widely available throughout New South Wales. In addition, SRNSW had indexed a sizeable portion of the collection, thereby rendering it accessible to web visitors, at least to the point of being able to identify the relevant record item. Furthermore, along the way, we had successfully experimented with what we call ‘bulk digitisation’ – a browseable experience – when we digitised some of the ever-popular shipping records.
We agreed to the approach, but, unfortunately, it was not going to be that easy. In discussion with our legal advisors from the New South Wales Crown Solicitors’ Office, we were given the advice that SRNSW did not have the power to enter into such an agreement with a third party provider. Naturally, all kinds of legal arguments were offered. The only solution was to amend the State Records Act 1998.
To most of us, this beggared belief, as we had been selling microfilm copies of the archives for nearly 40 years. Personally, I am still mystified by this advice to change our legislation, but, then again, I am not a lawyer. Not to be deterred, we set about having the required changes made to the Act.
In due course – admittedly, quite a long time, given the minor nature of the required change – the State Records Act was amended in 2005. We had expected the amendment to appear in part 8 of our Act, which deals with publishing, copying, copyright and the handling of copies of state archives. However, it was placed in part 4 of the Act under the heading ‘Authority entitled to control of State archive not currently in use’.
At this point, we naturally concluded that we could now go ahead with our agreement with Ancestry. Not so, according to the lawyers from the Crown Solicitor’s Office. The proposed agreement was analysed, discussed, dissected, held up to the light and – after many, many meetings – agreement between Ancestry and SRNSW was reached. In the meantime, years had gone by since the first approach. This experience is an example of a nineteenth-century legal attitude in direct conflict with the technological requirements of the twenty-first century.
Once the agreement was signed and the contractual arrangements agreed to, the process began. At last, we were able to provide Ancestry with the microfilms, and, in due course, the material began to appear on Ancestry’s website.
Benefits of the third party partnership
Was the appearance of the first SRNSW archives on the Ancestry website the ‘eureka moment’ we were expecting? Yes, from the perspective of SRNSW, it was an achievement, thanks largely to the work of both Christine Yeats of SRNSW and Brad Argent from Ancestry and their dogged perseverance and patience in light of the attitude of our Crown Solicitor’s Office.
There is no doubt that, for SRNSW and its clients, the partnerships with third party providers have delivered enormous benefits. Through these partnerships, SRNSW have been able to realise the plan of making the state’s most popular records available ‘with a click of the mouse’ and enabled us to respond to the demands of researchers, many of which are the people of New South Wales.
Thanks to our good working relationship with Ancestry, important records series, such as the Colonial Secretary’s papers (1788–1825), are now also available free of charge on the Ancestry website. Other records, such as the Convict Indents, will be available to view at no charge in approximately two years. Other series, such as the Industrial School records, will also be free when they are added to the Ancestry site.
Research is no longer confined to the traditional reading room. Researchers are, of course, still welcome there, and we hope this never changes. Now they are free to also wander into cyberspace and choose between service providers, because there are two third party providers doing business with SRNSW.
Ours have been financial arrangements, which are always welcome, but, more importantly, SRNSW is now better able to focus attention on other areas in the management of the 70 kilometers of state archives stored at the Western Sydney Records Centre at Kingswood.
As at 30 June 2012, Ancestry had indexed and digitised one-quarter of the microfilm stock in the reading rooms, which amounts to just over 1,000 reels, including the Archives Resources Kit. This means that 150 record series – or 1% of the collection – has now been digitised. Considering that only around 4% to 5% of the collection gets regular use, these are incredible statistics.
As at 30 June 2012, there have been 35 million hits on SRNSW material on the Ancestry website – again, something which SRNSW could only dream of achieving on its own.
Public reaction and impact on reading rooms
There was some negative feedback from the research community, which was surprising, since we were responding to user demands. Some accused us of having given the archives away, to which we countered that the archives are still with us, we are simply offering another research option. Others asked why they now have to pay for the records, to which we, again, countered that anyone can still view records in many places free of charge. These comments have largely dissipated, as family historians have increasingly embraced the digital world.
Has there been a change in what happens in the reading rooms? Yes, there has. We are a long way from the days when 100 readers queued at the reference desk to get issued with microfilm reels. Today, we count the 35 million readers who have accessed the state archives via Ancestry in the virtual reading room of cyberspace.
Closer to home, there are fewer readers in the reading room. The ones who visit often want records that are not, and will probably never be, accessible online. Others have exhausted what they can do online and now need to go to the uncopied material.
Archives and other information providers continue to face the dilemma faced by going so far, but not all the way, into the digital world. In a world which lives by the motto ‘if it’s not online, it doesn’t exist’, archives are challenged by the need to keep reminding the research community that it is only 1% of the collection that is available online, albeit an impressive 1%.
Conclusion
Was it worth it? Our response would be a resounding ‘yes’. If SRNSW had its time over, would it do things differently? Perhaps just a little. From SRNSW’s perspective, the agreements with its current third party providers are just a continuation of the long history of working with external providers and organisations – or whatever tag you want to apply to them – that have helped it achieve its objectives and, more importantly, take the records to the people.
In hindsight, the decision to consult the Crown Solicitor was probably unnecessary. Other, similar jurisdictions have forged ahead without legal advice and encountered no penalty or complications, so perhaps we, at SRNSW, were overly cautious. For agencies and organisations with little or no funding for the digitisation of collections, there is little choice but to embrace the third party provider. The advantages are quickly evident, and the rewards for doing so are not only financial, but they provide a service to our researchers, which, otherwise, would not be given if it were left to the vagaries of government funding. Future projects are already in the pipeline and are only constrained by the capacity to do the massive amounts of work that are required on behalf of the provider. Would we do it again? Most definitely.