Investing in Curation. A Shared Path to Sustainability. Final RoadMap.

Investing in Curation. A Shared Path to Sustainability. Paul Stokes. The 4C project. March 9, 2015.
Digital curation involves managing, preserving and adding value to digital assets over their entire life cycle. Actively managing digital assets maximizes their value and reduces the risk of obsolescence. The costs of curation is a concern to stakeholders. The final version of the road map is now available; it starts with a focus on the costs of digital curation, but the ultimate goal is to change the way that all organizations manage their digital assets.

The vision: Cost modeling will be a part of the planning and management activities of all digital repositories.

• Identify the value of digital assets and make choices
• Value is an indirect economic determinant on the cost of curating an asset. The perception of value will affect the methods chosen and how much investment is required.
• Content owners should have clear policies regarding the scope of their collections, the type of assets sought, the preferred file formats.
• Establish value criteria for assets as a component of curation, understanding that certain types of assets can be re-generated or re-captured relatively easily, thereby avoiding curation costs
• Demand and choose more efficient systems
• Requirements for curation services should be specified according to accepted standards and best practices.
• More knowledgeable customers demanding better specified and standard functionality means that products can mature more quickly.
• Develop scalable services and infrastructure
• Organizations should aim to work smarter and be able to demonstrate the impact of their investments.
• Design digital curation as a sustainable service
• Effective digital curation requires active management throughout the whole lifecycle of a digital object.
• Curation should be undertaken with a stated purpose.
• Making curation a service further embeds the activity into the organization's normal business function.
• Make funding dependent on costing digital assets across the whole lifecycle
• Digital curation activity requires a flow of sufficient resources for the activity to proceed.
• Some digital assets may need to be preserved in perpetuity but others will have a much more predictable and shorter life-span.
• All stakeholders involved at any point in the curation lifecycle will need to understand their fiscal responsibilities for managing and curating the asset until such time that the asset is transferred to another steward in the lifecycle chain.
• Be collaborative and transparent to drive down costs
• Each organization is looking to realize a return on their investment.
• If those who provide digital curation services can be descriptive about their products and transparent about their pricing structures, this will enhance possible comparisons, drive competitiveness and lead the market to maturity.

Digital Preservation Matters.

Ending the Invisible Library | Linked Data

Ending the Invisible Library | Linked Data. Matt Enis. Library Journal. February 24, 2015.
The World Wide Web began as a collection of web pages that were navigated with links. Now, and going forward, the web is increasingly about data and relationships among data objects. The use of MARC is "becoming an anachronism in an increasingly networked world". The site schema.org, is a collection of structured data ­schemas that help web designers specify entities and relationships among entities, but these tools were not designed with libraries in mind. MARC lacks the ability to encode this information or make it accessible on the web. Libraries need to start formatting their data so it can be accessed from internet search tools.

The W3C Schema Bib Extend Community Group (librarians, vendors, and organizations) have been working to expand schema.org to better represent library bibliographic information for search engines. The Library of congress has been working with the BIBFRAME project; “a major focus of the project is to translate the MARC 21 format to a Linked Data model while retaining as much as possible the robust and beneficial aspects of the historical format.” This will structure library records so that search engines can “extract meaningful information" and make it available. Ultimately, LC plans for BIBFRAME to replace MARC; there is a tool to convert MARC records to BIBFRAME.

The Libhub Initiative is a proof-of-concept project to build a network of libraries using BIBFRAME standards to link data between institutions and show how this can make library resources more visible on the internet.

Digital Preservation Matters.

Office Opens up with OOXML

Office Opens up with OOXML. Carl Fleischhauer, Kate Murray. The Signal. February 3, 2015.
Nine new format descriptions have been added to the Library’s Format Sustainability Web site. These closely related formats relate to the Office Open XML (OOXML) family, which are the formats of the Microsoft family of “Office” desktop applications, including Word, PowerPoint and Excel. Formerly, these applications produced files in proprietary, binary formats with the extensions doc, ppt, and xls. The current versions employ an XML structure for the data and an x has been added to the extensions: docx, pptx, and xlsx.

"In addition to giving the formats an XML expression, Microsoft also decided to move the formats out of proprietary status and into a standardized form (now focus on the word Open in the name.) Three international organizations cooperated to standardize OOXML."

The list of the nine:

• OOXML_Family, OOXML Format Family, ISO/IEC 29500 and ECMA 376
• OPC/OOXML_2012, Open Packaging Conventions (Office Open XML), ISO 29500-2:2008-2012
• DOCX/OOXML_2012, DOCX Transitional (Office Open XML), ISO 29500:2008-2012; ECMA-376, Editions 1-4
• DOCX/OOXML_Strict_2012, DOCX Strict (Office Open XML), ISO 29500:2008-2012; ECMA-376, Editions 2-4
• PPTX/OOXML_2012, PPTX Transitional (Office Open XML), ISO 29500:2008-2012; ECMA-376, Editions 1-4
• PPTX/OOXML_Strict_2012, PPTX Strict (Office Open XML), ISO 29500:2008-2012; ECMA-376, Editions 2-4
• XLSX/OOXML_2012, XLSX Transitional (Office Open XML), ISO 29500:2008-2012; ECMA-376, Editions 1-4
• XLSX/OOXML_Strict_2012, XLSX Strict (Office Open XML), ISO 29500:2008-2012; ECMA-376, Editions 2-4
• MCE/OOXML_2012, Markup Compatibility and Extensibility (Office Open XML), ISO 29500-3:2008-2012, ECMA-376, Editions 1-4

 "Meanwhile, readers should remember that the Format Sustainability Web site is not limited to formats that we consider desirable. We list as many formats (and subformats) as we can, as objectively as we can, so that others can choose the ones they prefer for a particular body of content and for particular use cases."

Digital Preservation Matters.

ARMA 2014: The Convergence of Records Management and Digital Preservation

ARMA 2014: The Convergence of Records Management and Digital Preservation. Howard Loos, Chris Erickson. October 2014. [PDF]
Presentation on records management and digital preservation given at the ARMA 2014 conference.
Notes:

• Records Management mission: To assist departments in fulfilling their responsibility to identify and manage records and information in accordance with legal, regulatory, and operational requirements
• RIM Life Cycle to DP Life Cycle
• Challenges and successful approaches
• Storing records permanently with M-Discs
• Introduction to Digital Preservation, challenges, format sustainability, media obsolescence, metadata, organizational challenges,
• Life of digital media
• Best practices and processes
• OAIS model
• Rosetta Digital Preservation System
• Library of Congress Digital Preservation Outreach & Education (DPOE) Network

Digital Preservation Matters.

Millenniata Announces Results of ISO/IEC 10995 Standard Tests – Storage Newsletter

Millenniata Announces Results of ISO/IEC 10995 Standard Tests – Storage Newsletter. Press Release. StorageNewsletter.com. 2013.06.10.
Millenniata, Inc. announced the results of its longevity test program based on the ISO/IEC 10995 standard. The results showed the median expected life of the M-DISC DVD was 1,332 years.  The same tests showed that other archival DVDs have an expected life of only 2.7 to 3.0 years.

Digital Preservation Matters.

NDSA Storage Report: Reflections on National Digital Stewardship Alliance Member Approaches to Preservation Storage Technologies

NDSA Storage Report: Reflections on National Digital Stewardship Alliance Member Approaches to Preservation Storage Technologies. Micah Altman, et al. D-Lib Magazine. June 2013.

The structure and design of digital storage systems is a cornerstone of digital preservation.  To better understand ongoing storage practices of organizations committed to digital preservation, the National Digital Stewardship Alliance conducted a survey of member organizations. This article reports on the findings of the survey. 

Key Findings

The key findings from the survey were:

• 90% of respondents are distributing copies of at least part of their content geographically.
• 88% of respondents are responsible for their content for an indefinite period of time.
• 80% of respondents use some form of fixity checking for their content.
• 75% of respondents report a strong preference to host and control their own technical infrastructure for preservation storage.
• 69% of respondents are considering, or currently participating in, a distributed storage cooperative or system (ex. LOCKSS alliance, MetaArchive, Data-PASS).
• 64% of respondents are planning to make significant technological changes in their preservation storage architecture in the next three years.
• 51% of respondents are considering or already using a cloud storage provider to keep one or more copies of their content.
• 48% of respondents are considering, or currently contracting out, storage services to be managed by another organization or company.

Digital Preservation Matters.

The Digital-Surrogate Seal of Approval: a Consumer-oriented Standard

The Digital-Surrogate Seal of Approval: a Consumer-oriented Standard.  James A. Jacobs, James R. Jacobs. D-Lib Magazine. March/April 2013.
"Digital-Surrogate Seal of Approval" (DSSOA) is a proposed way to describe the accuracy and completeness of digital objects that were created from printed books and other non-digital originals. It indicates that the original has been digitized completely and with 100% accuracy. This seal of approval may be applied to a digitized version of an analog original when it accurately replicates the original. To do this, two criteria must be met and verified:

1. Completeness. All pages of the original are fully and completely reproduced.
2. Accuracy. The original layout and appearance are preserved. All text is legible and there is no visual degradation when compared to the original.

The seal of approval, with a Statement of Verification,  can be applied by those responsible for the digital objects at any stage of the life cycle. The Statement of Verification must describe the methodology used and confirm 100% compliance. A "digital surrogate" in this context is a complete, accurate, digital replica of a bibliographically-identified, analog original item. The seal of approval  may be applied to a specific item or to a collection when all items in the collection meet the criteria.  The criteria may be used by organizations that create, manage, preserve, or deliver digital content. They may apply it to items that they curate as a way of communicating to their user-communities the completeness and accuracy of digital surrogates. The article also  addresses metadata elements, requirements and exceptions. This may not be appropriate or necessary for all digitization projects or collections.

Digital Preservation Matters.

Training in Digital Preservation - Alliance for Permanent Access

Training in Digital Preservation - Alliance forPermanent Access. William Kilbride, Chiara Cirinnà, Sharon McMeekiny. 21 February 2013. [PDF] 
This paper summarizes the current digital preservation needs based on the APARSEN project.
The need for training is great and the resources available are relatively meagre: so there is an
opportunity to collaborate in order to maximise impact.  In the training courses that have taken place there were four themes consistently expressed in feedback:

1. There is a great demand for training from staff already engaged in library and archive settings, especially for introductory material.
2. Audiences welcomed practical, case-study based training that matched their needs over theoretical knowledge. Tools and services beyond their level of knowledge or which lacked practical application were also less popular.
3. The audiences wanted practical interaction with preservation processes, including trying out the tools for themselves.
4. Audiences did not feel the need  to have a complete overview of preservation before they got started, and were less interested in the theoretical which they saw as a hindrance.

 However, training could be popular but still leave significant gaps so training should not just be based on the feedback.  The report gives many recommendations for training for Operational Staff,
Operational Managers, and Senior Managers regarding standards, object life-cycles, practical experience, legal and policy frameworks, ingest, provenance, metadata, financial planning, user communities, and succession planning.  They point to a very large unmet training need and a long list of topics which training providers can actually provide.

By developing training that meets proven needs we can provide a strong foundation to
an ever larger and ever more diverse community.
 

Digital Preservation Matters.

How big is the sound of music?

How big is the sound of music?  Lucas Mearian. Computerworld. March 21, 2013. 

Recently audiophiles and musicians have been moving toward master-quality music that's playable from a hard drive. That has led to greater use of lossless file formats. A popular file format, or codec, is the MP3.  This is a compressed file format; it is referred to as "lossy," meaning data is lost in the translation from the original master to the compressed format. Analog audio is recorded by sampling it 44,100 times per second, and then the samples are used to reconstruct the audio signal when playing it back digitally. An uncompressed file on a CD for example, uses 44.1KHz or a 1,411Kbits of data per second (Kbps) while a compressed file may only offer a bit rate of up to 256Kbps, which results in lower quality.  Within file formats, there are many sampling rates or frequencies; the higher the sampling rate, the higher the sound quality.

Uncompressed files are often stored as WAV files (Waveform Audio File Format), which can typically be 10 - 40 times larger than MP3 files.Other lossless formats include FLAC (Free Lossless Audio Codec), AIFF (Audio Interchange File Format) and Apple's ALAC (Apple Lossless Audio Codec). These lossless file formats require less storage space than WAV files. These new lossless formats can save disk space and offer high-fidelity music playback. For example, an album in the WAV format make take up 640 MB of space; the same album would take up about 300 MB in the lossless FLAC format. And lossy MP3 file would take up about 60 MB. It is estimated that CDs only offer about 15% of the data that was in a master sound track; when that CD is further compressed into a lossy MP, even more depth and quality of a recording is lost.

A relatively new high-definition file format, called Direct-Stream Digital (DSD), was created by Sony and Philips.  DSD uses a sample rate of 2.8224MHz or 64 times that of a CD's 41.1KHz.
 

Digital Preservation Matters.

OAIS 2012 update.

OAIS 2012 update. Barbara Sierman. juli 30, 2012.

Reference Model for an Open Archival Information System (OAIS). Magenta Book. Issue 2. June 2012. 

The main changes are the following:

·         Access Rights information is added as an element to the Preservation Description Information.

·         emulation as a strategy to preserve access services or the original look and feel and explains how different varieties of emulation will fit in the OAIS model.

·         There is more interaction between the Administration Functional Entity and the Preservation Planning Functional Entity. The Preservation Planning Functional entity will create preservation plans (and not only “migration plans” as mentioned in the previous version), based on its monitoring activity,  and will send these to the Administration functional entity to be performed. But the Administration Functional Entity (cq.  the function The Establish Standards and Policies) will also receive periodic risk analyses created by Preservation Planning to act upon, which gives the Preservation Planning Functional Entity a more active role in monitoring not only the outside world but also the OAIS itself.  Interaction also takes place when Preservation Planning sends recommendations on AIP updates, and The Administration Functional Entity replies with preservation requirements (added to the already exisiting “migration goals and approved standards”). So for creating new migration packages not only the preservation requirements resulting from monitoring the Designated Community are input, but also the preservation requirements from Administration. In general some loose ends seems to be united here.

·         The confusing use of the word “authentication” has now changed and the term “authenticity” is defined (adding the much used definition of “The degree to which a person (or system) regards an object as what it is purported to be.” With an important addition: “ Authenticity is judged on the basis of evidence.”

·         The term Information Package is redefined to : “ A logical container composed of optional Content Information and optional associated Preservation Description Information. Associated with this Information Package is Packaging Information used to delimit and identify the Content Information and Package Description information used to facilitate searches for the Content Information.”

·         A new definition is introduced “Other Representation Information” and described as “Representation Information which cannot easily be classified as Semantic or Structural. For example software, algorithms, encryption, written instructions and many other things may be needed to understand the Content Data Object, all of which therefore would be, by definition, Representation Information, yet would not obviously be either Structure or Semantics. Information defining how the Structure and the Semantic Information relate to each other, or software needed to process a database file would also be regarded as Other Representation Information.”

·         This chapter is adapted with the above mentioned changes and refines the definitions for an AIP version (An AIP whose Content Information or Preservation Description Information has undergone a Transformation on a source AIP and is a candidate to replace the source AIP. An AIP version is considered to be the result of a Digital Migration.) versus an AIP edition (An AIP whose Content Information or Preservation Description Information has been upgraded or improved with the intent not to preserve information, but to increase or improve it. An AIP edition is not considered to be the result of a Migration.)

 

Digital Preservation Matters.

Geospatial Data Preservation.

Geospatial Data Preservation. Website. November 2011.
The Geospatial Resource Center is being developed as a finding tool for freely available web-based resources about the preservation of geospatial information. A variety of selected resources are being added, including reports, presentations, standards, and information about tools for preparing geospatial assets for long-term access and use. The resources are indexed to enable searching of titles and are categorized to facilitate discovery by choosing among topics, resource types, or both. The website contains many valuable resources.  A few resources from these three categories:

Education & Training

• Appraisal and selection of geospatial data for preservation
• Best Practices for Geospatial Programs
• Copyright Quickguide

Tools & Software

• Cost Estimation Toolkit
• Conversion tools for geospatial data
• Geospatial metadata tools

Policies & Benefits

• Collection policies,
• Content standards,
• Policies on Open geospatial data access and preservation

OAIS / TDR presentation at FDLP.

OAIS / TDR presentation at FDLP.  James A. Jacobs. Federal Depository Library Conference. Free Government Information. October 2011. [PDF]
A presentation giving an introduction to the "Reference Model for an Open Archival Information System" (OAIS) and the "Audit And Certification Of Trustworthy Digital Repositories" (TDR).  This includes slides with speaker notes and a nice handout about related information with links. Every library decision should assess the impact of digital issues.  Notes:

OAIS
1. It defines the functional concepts of a long-term archive with consistent, unambiguous terminology.
2. It gives us a functional framework for designing archives, and a functional model.
3. It gives us a standard for “conformance.”
4. It is a “Reference Model” that describes functions; it is not an “implementation”
5. Some key OAIS concepts are:
   - Designated Community: An identified group of potential Consumers who should
      be able to understand a particular set of information.
   - Description of roles and functions in the information life cycle.
   - The Long Term: Long enough for there to be concern about changing technologies,
      new media and data formats, and a changing user community.
   - Preserved content must be usable according to the designated community

TDR
Documents what is being done and how well it is being done.
Provides 109 “metrics” for measuring conformance to OAIS in three areas: 
1. Organizational Infrastructure
2. Sustainability and succession plan
3. Digital Object Management
4. Technical Infrastructure And Security Risk Management

Audit And Certification Of Trustworthy Digital Repositories.

Audit And Certification Of Trustworthy Digital Repositories. CCSDS. September 2011. [PDF, 77pp.]

The Management Council of the Consultative Committee for Space Data Systems (CCSDS) has published this manual of recommended practices. It is based on the 2003 version from RLG. “The purpose of this document is to define a CCSDS Recommended Practice on which to base an audit and certification process for assessing the trustworthiness of digital repositories. The scope of application of this document is the entire range of digital repositories.”

The document addresses audit and certification criteria, organizational infrastructure, digital object management, and risk management.  It is a standard for those who audit repositories; and, for those who are responsible for the repositories, it is an objective tool they can use to evaluate the trustworthiness of the repository.