ArchivesDirect hosted service

ArchivesDirect website. February 18, 2015.
ArchivesDirect is a web based hosted service of Archivematica offered by DuraSpace for creating OAIS-based digital preservation workflows with content packages that are archived with DuraCloud and Amazon Glacier. It includes open source preservation tools, and generates archival packets using microservices, PREMIS, and mets xml files. ArchivesDirect is intended for small to mid sized institutions. Duraspace is a partnership with DSpace, Fedora, and Vivo.

Pricing and subscription plans include:
ArchivesDirect Standard (System, training, 1 TB): $11,900
ArchivesDirect Digital Preservation Assessment: $4,500
Additional Storage in Amazon S3 and Glacier: $1,000/TB/year

Digital Preservation Matters.

Digital Preservation Coalition publishes ‘OAIS Introductory Guide (2nd Edition)’ Technology Watch Report

Digital Preservation Coalition publishes ‘OAIS Introductory Guide (2nd Edition)’ Technology Watch Report. Brian Lavoie.  Digital Preservation Coalition. Watch Report. October, 2014. [PDF]

The report describes the OAIS, its core principles and functional elements, as well as the information model which support long-term preservation, access and understandability of data. The OAIS reference model was approved in 2002 and revised and updated in 2012. Perhaps “the most important achievement of the OAIS is that it has become almost universally accepted as the lingua franca of digital preservation”.

The central concept in the reference model is that of an open archival information system. An OAIS-type archive must meet a set of six minimum responsibilities to do with the ingest, preservation, and dissemination of archived materials: Ingest, Archival Storage, Data Management, Preservation Planning, Access, and Administration. There are also Common Services, which consist of basic computing and networking resources.

An OAIS-type archive references three types of entities: Management, Producer, and Consumer, which includes the Designated Community: consumers expected to independently understand the archived information in the form in which it is preserved and made available by the OAIS. This is a  framework to encourage dialogue and collaboration among participants in standards-building activities, as well as identifying areas most likely to benefit from standards development.

An OAIS-type archive is expected to:

• Negotiate for and accept appropriate information from information producers;
• Obtain sufficient control of the information in order to meet long-term preservation objectives;
• Determine the scope of the archive’s user community;
• Ensure the preserved information is independently understandable to the user community
• Follow documented policies and procedures to ensure the information is preserved against all reasonable contingencies
• Make the preserved information available to the user community, and enable dissemination of authenticated

An OAIS should be committed to making the contents of its archival store available to its intended user community, through access mechanisms and services which support users’ needs and requirements. Such requirements may include preferred medium, access channels, and any access restrictions should be clearly documented.

 The OAIS information model is built around the concept of an information package, which includes: the Submission Information Package, the Archival Information Package, and the Dissemination Information Package. Preservation requires metadata to support and document the OAIS’s preservation processes, called Preservation Description Information, which ‘is specifically focused on describing the past and present states of the Content Information, ensuring that it is uniquely identifiable, and ensuring it has not been unknowingly altered’. The information consists of:

• Reference Information (identifiers)
• Context Information (describes relationships among information and objects)
• Provenance Information (history of the content over time)
• Fixity Information (verifying authenticity)
• Access Rights Information (conditions or restrictions)

OAIS is a model and not an implementation. It does not address system architectures, storage or processing technologies, database design, computing platforms, or other technical details of setting up a functioning archival system. But it has been used as a foundation or starting point. Efforts, such as TRAC, have been made to put the attributes of a trusted digital archive into a ‘checklist’ that could be used to support a certification process. PREMIS is a preservation metadata initiative that has emerged as the de facto standard. METS, and XML based  document form, has become widely used for encoding OAIS archival information packages.

The ‘OAIS reference model provides a solid theoretical basis for digital preservation efforts, though theory and practice can sometimes have an uneasy fit.’

Digital Preservation Matters.

ARMA 2014: The Convergence of Records Management and Digital Preservation

ARMA 2014: The Convergence of Records Management and Digital Preservation. Howard Loos, Chris Erickson. October 2014. [PDF]
Presentation on records management and digital preservation given at the ARMA 2014 conference.
Notes:

• Records Management mission: To assist departments in fulfilling their responsibility to identify and manage records and information in accordance with legal, regulatory, and operational requirements
• RIM Life Cycle to DP Life Cycle
• Challenges and successful approaches
• Storing records permanently with M-Discs
• Introduction to Digital Preservation, challenges, format sustainability, media obsolescence, metadata, organizational challenges,
• Life of digital media
• Best practices and processes
• OAIS model
• Rosetta Digital Preservation System
• Library of Congress Digital Preservation Outreach & Education (DPOE) Network

Digital Preservation Matters.

National Library of Australia’s Digital Preservation Policy

Digital Preservation Policy 4th Edition (2013). National Library of Australia.  May 26, 2013.
This site outlines the National Library of Australia’s policy on preserving its digital collections, and collaborating with others to preserve digital information resources. The primary objective of their digital preservation activities is maintaining the ability to meaningfully access digital collection content over time. The primary concern is preserving the ability to access the Preservation Master File from which derivatives files may be created or re-created over time. To this end, preservation of digital library material includes:

•     Bit-level preservation of all digital objects, ie. keeping the original files intact;
•     Ensuring that authenticity and provenance is maintained;
•     Ensuring that appropriate preservation information is maintained;
•     Understanding and reporting on risks which affect ongoing access;
•     Performing appropriate actions to ensure that objects remain accessible;
•     Periodic review of preferred formats and digital metadata standards

Preservation of the Library's digital collections involves four main goals:

1.     Maintaining access to reliable data at bit-stream level;
2.     Maintaining access to content encoded in the bit streams;
3.     Maintaining access to the intended content; and
4.     Maintaining the stated preservation intent for all digital material over time.

While specific preservation activities may focus on one or more of these goals, the Library’s preservation responsibility is only fulfilled when all four goals have been adequately addressed.

The Library uses the concepts in the Open Archival Information Systems (OAIS) Reference Model and other international standards and best practices, such as PREMIS and Open Planets Foundation.

Digital Preservation Matters.

Light, Dark and Dim Archives: What are they?

Light, Dark and Dim Archives: What are they?  May 2013.

 The following is a compilation of a few definitions or examples of Light, Dark and Dim archives to better understand what they are.

 The notion of "dark archives", supporting little or no access to archived materials, has met with scant enthusiasm in the library community. This suggests that digital repositories will function not just as guarantors of the long-term viability of materials in their custody, but also as access gateways. Lavoie

A secure digital repository sometimes referred to as a "dark archive" Kirsch

Dark Archive: An archive that does not grant public access and only preserves the information it contains. This can refer to a digital archive or repository as well as brick & mortar archive. Michigan

Dark archive: The purpose of a dark archive is to function as a repository for information that can be used as a failsafe during disaster recovery. UCPress

The Dark Archive is a secret place for storing archival material with restricted user access. Tufts

We chose to create a “dark” archive to focus our efforts on securing and preserving large volumes of content important to libraries and their users; however, it is not exclusively dark. Participating libraries experience the archive as a “light” or accessible archive in two ways: auditing the archive to ensure we are prepared to support eventual use and accessing of content that has been made available as the result of a “trigger event” or post-cancellation access claim. Portico

Dark archives are certainly misunderstood both inside and outside the industry.  So, what is a dark archive?  It is, simply put, an archive of information that is not used for public access.  Most often it serves as a failsafe copy of a light archive, i.e. a publicly available version of the information, for use in disaster recovery operations.  Dark archives need not be a fully operational copy of an information system, rather just the content behind the information system.  This is an important distinction because maintaining an exact operational copy of an information system is a much more complex and expensive undertaking than maintaining only the content the information system operates on.  Metaphorically, at its base definition, a dark archive will require more than a flip of the switch to make a light archive. Osti Bog

Dark Archive: An archive that is inaccessible to the public. It is typically used for the preservation of content that is accessible elsewhere. See also dim archive, light archive.

Dim Archive: An archive that is inaccessible to the public, but that can easily be made accessible if required. It's typically used for the preservation of content that is accessible elsewhere. See also dark archive, light archive.

Light Archive: An archive that is accessible to the public. See also dim archive and dark archive.  CDL

A DDP network may be an open archive, or it may reside somewhere on the spectrum from dim to dark archive. That is, it may be open to only the contributors’ servers for ingesting (dark archive); it may be open to specified users, such as the contributing institutions’ communities (dim archive); or it may provide unrestricted access (open archive). This status will determine whether contributors will focus solely on long-term preservation issues, or some combination of preservation and public access issues. MetaArchive

Dark Archive: Digital archive for which access to content is limited to organizational custodians.

Dim Archive: Digital archive that incorporates elements of both the Dark and Open Archive models. Access for some materials is restricted to organizational custodians, while access for others may be open to a broad user community.

Open Archive:  A digital archive that is publicly accessible. MetaArchive

 Below are two figures from the OAIS Model (2012) showing the Access functions.

Digital Preservation Matters.

OAIS 2012 update.

OAIS 2012 update. Barbara Sierman. juli 30, 2012.

Reference Model for an Open Archival Information System (OAIS). Magenta Book. Issue 2. June 2012. 

The main changes are the following:

·         Access Rights information is added as an element to the Preservation Description Information.

·         emulation as a strategy to preserve access services or the original look and feel and explains how different varieties of emulation will fit in the OAIS model.

·         There is more interaction between the Administration Functional Entity and the Preservation Planning Functional Entity. The Preservation Planning Functional entity will create preservation plans (and not only “migration plans” as mentioned in the previous version), based on its monitoring activity,  and will send these to the Administration functional entity to be performed. But the Administration Functional Entity (cq.  the function The Establish Standards and Policies) will also receive periodic risk analyses created by Preservation Planning to act upon, which gives the Preservation Planning Functional Entity a more active role in monitoring not only the outside world but also the OAIS itself.  Interaction also takes place when Preservation Planning sends recommendations on AIP updates, and The Administration Functional Entity replies with preservation requirements (added to the already exisiting “migration goals and approved standards”). So for creating new migration packages not only the preservation requirements resulting from monitoring the Designated Community are input, but also the preservation requirements from Administration. In general some loose ends seems to be united here.

·         The confusing use of the word “authentication” has now changed and the term “authenticity” is defined (adding the much used definition of “The degree to which a person (or system) regards an object as what it is purported to be.” With an important addition: “ Authenticity is judged on the basis of evidence.”

·         The term Information Package is redefined to : “ A logical container composed of optional Content Information and optional associated Preservation Description Information. Associated with this Information Package is Packaging Information used to delimit and identify the Content Information and Package Description information used to facilitate searches for the Content Information.”

·         A new definition is introduced “Other Representation Information” and described as “Representation Information which cannot easily be classified as Semantic or Structural. For example software, algorithms, encryption, written instructions and many other things may be needed to understand the Content Data Object, all of which therefore would be, by definition, Representation Information, yet would not obviously be either Structure or Semantics. Information defining how the Structure and the Semantic Information relate to each other, or software needed to process a database file would also be regarded as Other Representation Information.”

·         This chapter is adapted with the above mentioned changes and refines the definitions for an AIP version (An AIP whose Content Information or Preservation Description Information has undergone a Transformation on a source AIP and is a candidate to replace the source AIP. An AIP version is considered to be the result of a Digital Migration.) versus an AIP edition (An AIP whose Content Information or Preservation Description Information has been upgraded or improved with the intent not to preserve information, but to increase or improve it. An AIP edition is not considered to be the result of a Migration.)

 

Digital Preservation Matters.

Guidelines on the Production and Preservation of Digital Audio.

Guidelines on the Production and Preservation of Digital Audio. The International Association of Sound and Audiovisual Archives. ed. by Kevin Bradley. Second edition 2009.
A web version of guidelines on creating and preserving digital audio objects.  Areas of interest covered include:

• Metadata, including preservation metadata
• Important to keep all aspects of preservation and transfer relating to audio files, including all technical parameters 
• Persistent Identifiers
• Naming conventions
• Digital preservation planning
• OAIS: the services and functions for monitoring the environment and providing recommendations to ensure that the information stored remains accessible to the Designated User Community over the long term, even if the original computing environment becomes obsolete
• Preservation planning is the process of knowing the technical issues in the repository, identifying the future preservation direction (pathways), and determining when a preservation action, such as format migration, will need to be made.
• Small scale digital archiving
• Digital preservation is as much an economic issue as a technical one.
• Any proposal to build and manage an archive of digital audio objects should have a strategy which includes plans for the funding of ongoing maintenance and replacement
• Risks
• The only way to know the condition of a digital collection is constant and comprehensive testing.
• A preservation strategy requires a listing of the risks associated with the loss of technical expertise and how that will be addressed.   
• An archive can distribute risk in a number of ways,  such as:
• form local partnerships so that content is distributed
• establish a relationship with a stable well funded archive; 
• engage a commercial supplier of storage services
• Recording formats
• It is not recommended that audio streams be recorded for long term storage.
• Recommend recording as a digital data file
• Recommend recording as .wav or preferably BWF .wav files 
• The large investment in a single format will also help support the continuance of that format for the longest period, as the industry will not change an entrenched format without significant benefits.
  
  Digital Preservation Matters.

OAIS / TDR presentation at FDLP.

OAIS / TDR presentation at FDLP.  James A. Jacobs. Federal Depository Library Conference. Free Government Information. October 2011. [PDF]
A presentation giving an introduction to the "Reference Model for an Open Archival Information System" (OAIS) and the "Audit And Certification Of Trustworthy Digital Repositories" (TDR).  This includes slides with speaker notes and a nice handout about related information with links. Every library decision should assess the impact of digital issues.  Notes:

OAIS
1. It defines the functional concepts of a long-term archive with consistent, unambiguous terminology.
2. It gives us a functional framework for designing archives, and a functional model.
3. It gives us a standard for “conformance.”
4. It is a “Reference Model” that describes functions; it is not an “implementation”
5. Some key OAIS concepts are:
   - Designated Community: An identified group of potential Consumers who should
      be able to understand a particular set of information.
   - Description of roles and functions in the information life cycle.
   - The Long Term: Long enough for there to be concern about changing technologies,
      new media and data formats, and a changing user community.
   - Preserved content must be usable according to the designated community

TDR
Documents what is being done and how well it is being done.
Provides 109 “metrics” for measuring conformance to OAIS in three areas: 
1. Organizational Infrastructure
2. Sustainability and succession plan
3. Digital Object Management
4. Technical Infrastructure And Security Risk Management