Why we should all think about data preservation

Why we should all think about data preservation. Stephanie Taylor. School of Advanced Study. February 19, 2015.
The SHARD project, which ended in 2012, identified  four basic principles of digital preservation for researchers:

1. Start early: The sooner you start thinking about what to preserve, how to do it, and when, the greater the chance of avoiding problems. Early planning means involving everyone in a research project in the discussion to help identify additional issues.
2. Explain it: Context provides meaning and is vital in digital preservation. There is little point in preserving material and data without context.
3. Store it safely: Backups are not preservation. It needs multiple copies in different locations. Use open source file formats and be careful how you and others handle and access files. Select carefully the files to be preserved.
4. Share it: Sharing your research material and data is beneficial.  In one way or another, the main reason to carry out preservation at all, on any level, is to be able to share your work with others, now and in the future.

Many things are being lost or threatened because no one saw a good reason to preserve them until it was almost too late.  The Data Preservation Online Training resource guides students through the reasons to preserve and share data and challenges that they might face.

Digital Preservation Matters.

The Dataverse Network

The Dataverse Network. Harvard Dataverse Network. 2014.
The Dataverse Network is an open source application to publish, share, reference, extract and analyze research data. It facilitates making data available to others and to replicate work of other researchers. The network hosts multiple studies or collections of studies, and each study contains cataloging information that describes the data plus the actual data and complementary files.

The Dataverse Network project develops software, protocols, and community connections for creating research data repositories that automate professional archival practices, guarantee long term preservation, and enable researchers to share, retain control of, and receive web visibility and formal academic citations for their data contributions.

Digital Preservation Matters.

Are libraries sustainable in a world of free, networked,digital information?

Are libraries sustainable in a world of free, networked,digital information? Lluís Anglada. El profesional de la información. 7 November 2014. [PDF]

Interesting article looking at libraries through the stages of modernization, automation and digitization, and at a formula for evaluating the importance of libraries to society. The article concludes that "if the current generation of librarians does not introduce radical changes in the role of libraries, their future is seriously threatened."

The formula proposed is the sustainability is equal to the value divided by the cost, and the value is the use minus the dysfunctions and modified by the perceptions of the library "S= (U - D + 2P) / C".

Libraries are changing because of technology and needs, but there is a danger that people will perceive them as unable to provide the information that users demand. If this continues, those funding the libraries will provide less support. The perceptions must change in order for libraries to be sustainable.

Some thoughts from the article:

• Libraries are changing from being a space to store, locate and use books to places where people interact and socialize. This should transform the perception that citizens have of their libraries, seeing them as places to ‘change lives by giving people the tools they need to succeed’.
• Libraries depend on public funding, and their future depends on the perception or mental image of libraries held by administrators and policy makers who allocate budgets  
• Libraries used to show statistical data on resources; they must now show their value to those who support them financially 
• The emergence of new roles for libraries does not mean that all library services have evolved over time. In the new environment, some traditional strengths of libraries are weakening.
• Library catalogues and automated systems were innovative in the ’80s, but have been stuck in outmoded practices. Users have adapted quickly to the ‘googlization’ of information and do not understand why they should have to look in different places to get a unique solution to an information need. 
• Two key elements for future library sustainability: perception and adaptation to a new paradigm
• The perception of libraries remains increasingly attached to the printed book, from 69% of Americans in 2005, to 75% in 2010.
• Libraries may end up being seen as useful only to preserve the past (i.e. the printed book), and consequently of little use to handle digital information.
• The library has been steadily declining in importance in university budgets.
• People sustain libraries because of a positive perception and a feeling that the libraries are important. We believe that society still needs the functions performed by libraries and librarians, but the feeling alone does not make them immediately sustainable. 
• We must soon establish a new stereotype of ‘library’ in people’s minds, one that is not based on the physicality of the buildings or books, but focuses on the role of support and assistance in the difficult process of using information and transforming it into knowledge. 
• The creation of perceptions of a library and librarian that are associated with assistance regarding information is a contribution that has not yet been made. 
• This is the challenge and responsibility for young librarians: to create a new perception of our profession. We must establish a new stereotype of ‘library’ in peoples’ minds, one that
  is not based on the physicality of the buildings and books, but on the role of support and assistance in the difficult process of using information and transforming it into knowledge.

Digital Preservation Matters.

Five steps to decide what data to keep

Five steps to decide what data to keep. Angus Whyte. Digital Curation Centre. 31 October 2014.
 This guide aims to help UK Higher Education Institutions aid their researchers in making informed choices about what research data to keep. 

It will be relevant to researchers making decisions on a project-by-project basis, or formulating departmental guidelines. It assumes that decisions on particular datasets will normally be made by researchers with advice from the appropriate staff (e.g. academic liaison librarians) and taking into account any institutional policy on Research Data Management (RDM) and guidance available within their own domain.

Step 1. Identify purposes that the data could fulfill
Step 2. Identify data that must be kept
Step 3. Identify data that should be kept
Step 4. Weigh up the costs
Step 5. Complete the data appraisal 

The final step is to weigh the value of the data and any costs still to be incurred, "considering the long-terms aims, the qualities you identified, the time and money already invested in it and the risks of being unable to prepare any ‘must keep’ data for preservation."

Digital Preservation Matters

Step 1. Identify purposes that the data could fulfill

Step 2. Identify data that must be kept

Step 3. Identify data that should be kept

Step 4. Weigh up the costs

Step 5. Complete the data appraisal

- See more at: http://www.dcc.ac.uk/resources/how-guides/five-steps-decide-what-data-keep#sthash.bSZLU9IQ.dpuf

Angus Whyte, Published: 31 October 2014

Angus Whyte, Published: 31 October 2014

Angus Whyte, Published: 31 October 2014

Angus Whyte, Published: 31 October 2014

Angus Whyte, Published: 31 October 2014

4 Ways Academic Libraries Are Adapting For The Future

4 Ways Academic Libraries Are Adapting For The Future. Brad Lukanic. Fast Company. October 24, 2014.
Instead of becoming obsolete, academic libraries are changing and becoming more relevant.

• Libraries have become the heart of the spirit of collaboration and innovation--going beyond being places to merely access knowledge to become hubs to truly explore and create.
• Universities around the country are evolving their libraries into catalysts for discovery, learning, collaboration, and scholarly breakthroughs.
• Each institution requires a unique vision for creating an intellectual center for 21st-century teaching, learning and collaboration. That vision must reflect the institution’s own voice and decide what it means to support the creation of academic scholarship. 
• Libraries must offer learning opportunities and access to local and global networks of information, with the ability to easily create and share knowledge. 
• Libraries must offer the settings and tools to foster discovery and learning in a dynamic academic framework.

Four key areas critical to the changing landscape for academic libraries:

1. Respond to strategic campus and business needs
2. Infuse technology infused in every aspect of service 
3. Embrace flexibility
4. Create places to engage

Academic libraries are changing from a repository of collections to a catalyst for discovery and creation, where people, knowledge, and research intersect to resolve our world’s greatest challenges.

Supporting the Changing Research Practices of Chemists.

Supporting the Changing Research Practices of Chemists.  February 25, 2013. Matthew P. Long, Roger C. Schonfeld. Ithaka S+R. February 26, 2013. [PDF]

This report, intended for those who support chemists, including librarians, is about the latest research methods, practices, and information services needs of academics chemists. Chemists need services to make their lives easier and their research groups more productive; this includes minimizing paperwork and administrative tasks. They value academic libraries primarily for the access that they provide to electronic journals and other online resources. Researchers are often frustrated by an inability to share large amounts of data with a collaborator. Few chemists visit the physical library, but they use the library digital collections heavily.

In the survey, fewer than 10% reported a research consultation with a librarian, asked for help with a data management, or asked for assistance on an issue related to publishing in the past year; they rarely reach out to the library to discuss issues or request support. The main search sites for chemists are Web of Knowledge/Web of  Science, SciFinder, and PubMed. It would be helpful to have tools to help process all of this information,  a pre-scan of announcements from journals, and organize their materials. Electronic Lab Notebooks (ELNs) make it easy to share, archive, and search through past lab notes, but are at risk in the lab. Labs generally do not have good data management infrastructure or proper external support for developing it, especially in sharing and preserving files.

It is difficult for academic chemists to coordinate the recording and preservation of data after the completion of a project. When data are saved, they are often held in unstable or at-risk formats  or in formats where no one else can access or interpret them. Sometimes a large amount of potentially useful data is not shared or preserved in any durable way. One chemist invited the library to come and speak to the department about preservation and access. Chemists have a general lack of awareness of  effective data curation and preservation. Data management and preservation is time-consuming and rarely straightforward; it requires expert advice and constant monitoring.

The findings:

1. Chemists need better support in data management, sharing and preservation.
2.  Many researchers remain anxious about keeping up with the newest literature.
3. They need new tools to stay aware of new research and also serendipitous discovery.
4. Chemists  require greater support in disseminating their research, including articles, data, and other materials.

Other areas of concern for academic chemists : laboratory management, gaining access to industrial funding, and teaching support.

We see some real potential for the academic library to stretch the definition of the services it offers to the academic chemist. The library may also have a role in working with other service providers and ensuring that academics are aware of the latest research tools. It is clear from this project that libraries must think strategically about whether and how to invest in services for chemists.
 

Digital Preservation Matters.

Sustaining Our Digital Future Institutional Strategies for Digital Content

Sustaining Our Digital Future: Institutional Strategies for Digital Content. Nancy L Maron, Jason Yun and Sarah Pickle. Strategic Content Alliance. January 29, 2013. (PDF, 91 pp.)
The shift with digital media in scholarly communications is transformative; data sets, dynamic digital resources, websites, digital collections,  crowd sourced or born digital content: there are challenges and opportunities, along with questions about who is responsible for maintaining them, and how to maximize the value of the content.

Some findings:

• project have received support from the host institution, but few have plans for ongoing support.
• There are potential partners on campus, but project leaders do not seek them early
  enough when critical decisions are being made.
• Digital projects across campuses may be hosted by many groups, which poses challenges for discovery. There is often no single place for users to find digital projects and some projects can too easily slip from view.
• Current funding styles do not support ongoing operation
• „Campus-wide solutions are beginning to emerge, but even these tend to address just the basic “maintenance” issues of storage, preservation and access.
• „Focus is often on creating new content, with little thought about ongoing efforts to enhance the content or update user interfaces.

Recommendations:

•  Perform an early and honest appraisal to find which projects are likely to require support after completion:
1. Digital content requiring just “maintenance”: plan that the content will be deposited and integrated into some other site, database, or repository.
2. „Digital resources requiring ongoing growth and investment: These require early sustainability planning, including identifying institutional or other partners and careful consideration of the full range of costs and activities needed to keep the resource vibrant.
•  Be realistic in assessing the future needs of the resource at its outset and in continuing support.
•  „Identify campus partners early on.
•  „Consider how central your project is to the overall mission of the institution.
• „Consider if projects could be drawn together to create a deeper network of support, both for “maintenance” projects and those with the potential to really grow.
• Develop ways to help users find decentralized content and to reach out to content users. These could start as an inventory of all of the digital holdings or common catalogs.
• „Determine where scale solutions pay off, where experts are best placed to champion a project, or create common storage, usage and preservation systems for an organization.
• Continue to identify and support ongoing development of the “front-end”, including user needs, interface development, and content enhancement. Pay attention to the changing needs of users and determine what enhancements the digital resource will require.

Libraries, museums, technology departments, and digital humanities centres are among the players that have begun to emerge as potential leaders of greater coordinated digital support on university campuses. Libraries have begun to consider the support of digital resources to be a critical part of their missions. Some universities provide advisory support to project leaders and libraries provide help for digital projects, from help to understand grant requirements, co-developing projects, providing hosting, curation, and preservation expertise.

 Sustainability and Use: 

•  Research data platforms: At some institutions major initiatives are underway to develop research data platforms. The goal of the platform is not just preservation and storage but access and reuse. The first step is to have a platform. From there they can test and refine the service for researchers depositing data sets, library  curated collections, and university departments.

• "A coherent digital policy from early review, guidelines on costings and deposit standards, to forecasting what ongoing activities will be needed and who will carry them out, would ideally remove much of the risk of “digital time bombs” while obliging both project leaders and university leaders to take a moment to envision the ongoing impact they want these resources to have, and how to best achieve that."

• Unlike universities (who often play the role of reluctant, passive, or simply unaware host, to a great deal of digital content created by their scholars) museums and libraries tend to be the ones initiating this work and are eager to build and maintain these collections.

• Despite the benefits of centralisation, the mere presence of a catalogue and centralised
  repository does not ensure greater usage of or engagement with its holdings.

• Many institutions devote considerable attention to the upfront creation of content, but not nearly as much to its ongoing enhancement or reuse, resulting in collections that are certainly present in the main catalogue, but otherwise exist only as capsules of content, frozen in time.

•  Once the project is finished, management of the digital resource is not always clear.

Digital Preservation Matters.

The Problem of Data

The Problem of Data. Lori Jahnke, Andrew Asherpub, Spencer D. C. Keralis. CLIR Report. Council on Library and Information Resources. August 12, 2012.
Excellent report on data storage, use, and curation.  A section contains a snapshot of the current digital data curation education landscape.  Below are some long notes and excerpts from the PDF article:

Key Findings

• None of the researchers interviewed for this study have received formal training in data management practices, nor do they expresssatisfaction with their level of expertise.
• Few researchers, especially among those who are early in their career, think about long-term preservation of their data.
• The demands of publication output overwhelm long-term considerations of data curation. Metadata and documentation are of interest only if they help a researcher complete his or her work.
• There is a great need for more effective collaboration tools, as well as online spaces that support the volume of data generated and provide appropriate privacy and access controls.
• Few researchers are aware of the data services that the library might be able to provide and seem to regard the library as a dispensary of goods (e.g., books, articles) rather than a place for research/professional support.

Recommendations

• There is unlikely to be a single out-of-the-box solution that can be applied to the problem of data curation. Instead, an approach is needed that emphasizes working with researchers to identify or build appropriate tools.
• Researchers must have access to adequate networked storage.
• Universities should revise access policies to support multi - institutional research projects.
• Programs should begin early in the researcher career path for the greatest long-term benefit.
• Data curation systems should be integrated with the active research phase (i.e., as a backup, etc).
• Privacy and data access control tools should be developed to manage confidential data. Policies must be developed that support researchers in using these technologies.

Other notes:

• Data curation, a term generally defined as a set of activities that includes the preserving, maintaining, archiving, and depositing of data to keep it secure, intact, and accessible for reuse.
• Many researchers expressed concerns surrounding the ethical reuse of research data. Additional work is needed to establish best practices in this area, particularly for qualitative data sets.
• Most participants reported feeling adrift when establishing protocols for managing their data and added that they lacked the resources to determine best practices, let alone to implement them. Almost none of the scholars reported that data curation training was part of their graduate curriculum.
• Perhaps one of the more complicated issues for data curation is the complex life cycle of research data and projects. Data collection may occur throughout the project and change from before it is completed.
• Scholars may collect data on a phenomenon unrelated to their current project with no clear idea of the potential usefulness of those data. Such data might be integrated with a later project, given away to an interested colleague, or never used at all.
• It would be helpful to have a way to collect data into a collection space that could be used throughout the project.
• The researchers held contradictory views about the value of their data. Some wanted to associate their data with publications or to have it available for use in the classroom
• Few of the researchers thought about long-term preservation of their data, especially those who were early in their career.
• The academic system offers little or no career reward for preserving one’s data.
• Data preservation strategies must take into account varied, proprietary, and non-standard data formats, and provide a real-time benefit for the scholar in meeting research goals.
• Given the lack of infrastructure for sharing and storing data, the social sciences may face similar problems of data loss in documenting social phenomena as researchers begin to work within larger collaborative groups and with larger data sets. Data stored on personal media devices are especially vulnerable to this type of loss, as few scholars have the skills necessary to maintain data over time and across hardware and software platforms. Several of the scholars interviewed reported storing data on legacy systems that may become inaccessible
• University policies that appropriately address the ethical considerations relating to data sharing and preservation would benefit researchers, administrators, and technologists alike.
• Researchers hold tremendous amounts of data on personal computers and hard drives, many of which are not backed up adequately. Among the participants, the research data ranged from under 1 GB to multiple terabytes. Data types included various formats of images, video, audio files, data sets, documents, etc.
• Managing large files presents significant challenges for researchers in that university infrastructures typically do not provide adequate storage space or sufficient bandwidth for data access.  The data may be lost when researchers upgrade their computers or software. Few researchers put more than minimal effort into organizing non-active data or ensuring its continued compatibility with new software or hardware.
• There is a clear need for libraries to move beyond passively providing technology to embrace the changes in scholarly production that emerging technologies have brought.  
• The data preservation step must be fully integrated into a scholar’s research workflow. Not only are necessary metadata and other materials much more easily captured while research is in progress, but also there is a real opportunity to streamline research workflows and to provide much needed support. Scholars need help with the technical aspects of managing and preserving data, as well as with basic curation issues (e.g., what to keep and what to delete), and the ethical implications of sharing their data (e.g., what is an appropriate latency period for the data and how does one balance the need to provide meaningful access with the risk of inadvertently exposing confidential participant information).
• Although some researchers acknowledge that their data could be useful to other researchers, there is little incentive to invest time in archiving or repackaging data sets.
  
• Extensive outreach to scholars is necessary to build the relationships that will facilitate data preservation. This is likely to be a slow process initially. Researchers are unlikely to engage with those they do not view as peers.
  
• Researchers need additional tools to manage preserved data on their own, and they would benefit from access to professionals who can offer advice on management strategies.
• Researchers typically align themselves with their disciplines rather than with their institutions; therefore, support models that extend beyond the university are likely to be especially beneficial.
• Reaching the level of collaboration among universities and the technical interoperability required to capture and preserve a career’s worth of data in the current environment is a challenge.
• Current data management systems must be fundamentally improved so that they can meet the capacity demand for secure storage and transmission of research data. Integrating the data preservation system with the active research cycle is essential to encourage researcher investment.
• Researchers are not well positioned to meet the technical and policy challenges without the coordinated support of libraries, information technology units, and professionals who possess both technical and research expertise.

•  One example concerning the PETRA e+e collider project in Hamburg, Germany; In the more than 25 years since, theoretical insights and computing advancements have made the data valuable once again. However, much of the data have been irrevocably lost to corrupt storage media, lost computer code, and deactivated personal accounts. These early particle physics experiments are unique, as modern colliders operate at higher energy levels and cannot replicate the particle interactions.

Digital Preservation Matters.

The Digital Preservation Network.

The Digital Preservation Network. George Eberhart. American Libraries Magazine. May 4, 2012.
The ARL membership meeting discussed the recently established Digital Preservation Network, a federation of universities intent on securing the long-term preservation of the digital scholarly record.  James Hilton said that only universities, not private industry, can solve the problem of preserving born-digital data and making it accessible to future generations. “Universities last for centuries. Companies do not.”
Other notes from the report:

• The problem is too large for just libraries to handle. The presidents of universities must be on board with this.
• The data and metadata of research and scholarship are susceptible to multiple single points of technical, political, and funding failure.
• The program asks for a $20,000 initial commitment to help fund project software.

Digital Preservation Matters.

Library of Congress Digital Preservation Newsletter.

Library of Congress Digital Preservation Newsletter. May 2012. [PDF]
Items from the Newsletter include:

• Key outcomes of the NDIIPP program are to identify priorities for born digital collections and engage organizations committed to preserving digital content.
• Viewshare  is being used for the collections
• Floppy Disks are Dead, Long Live Floppy Disks
• Floppy disks are fragile constructions that were never designed for permanence.
• Difficult to determine what is on the floppy and to recover
• A floppy disk controller called Catweasel allows computers to access a wide variety of older disk formats (must have the floppy drive).
• Web archiving.  
• Because of the scope of the web sites, consider partnering with other institutions.
•  Preservation of and Access to Federally Funded Scientific Data
• Research data produced by federally funded scientific projects should be freely available to the wider research community and the public
• Public data should be a public resource, and data sharing supports core scientific values like openness, transparency, and replication. 
• Lack of resources for curating scientific data and a lingering tradition of data hoarding create resistance to open access to research data.

 Digital Preservation Matters.

Research Librarians Consider the Risks and Rewards of Collaboration.

Research Librarians Consider the Risks and Rewards of Collaboration. Jennifer Howard. The Chronicle of Higher Education. October 16, 2011.

Association of Research Libraries’ meeting discussed research and preservation projects like the HathiTrust digital repository and the proposed Digital Public Library of America, plans for which are moving ahead. Concerning the Digital Public Library of America: “Library” is a misnomer in this case, which is more of a federation of existing objects. It wouldn’t own anything. The main contribution would be to set standards and link resources.  “The user has to drive this.”

They said that it’s almost three times more expensive to store materials locally than it is to store them with HathiTrust. Researchers now also create and share digital resources themselves via social-publishing sites such as Scribd. There is a need for collection-level tools that allow scholars and curators to see beyond catalog records.

Discussed Recollection, a free platform built by NDIIPP and a company named Zepheira to give a better “collection-level view” of libraries’ holdings. The platform can be used to build interactive maps, timelines, and other interfaces from descriptive metadata and other information in library catalogs. So, for instance, plain-text place names on a spreadsheet can be turned into points of latitude and longitude and plotted on a map.

“Rebalancing the Investment in Collections,” discussed that libraries had painted themselves into a corner by focusing too much on their collection budgets. Investing in the right skills and partnerships is most critical now. “The comprehensive and well-crafted collection is no longer an end in itself.”

On person told librarians that they shouldn’t rush to be the first to digitize everything and invest in every new technology. “Everybody underestimates the cost of innovation,” he said. “Instead of rushing in and participating in a game where you don’t have the muscle, you want to stand back” and wait for the right moment.

Digital Preservation Matters.

More (digital) wake-up calls for academic libraries

More (digital) wake-up calls for academic libraries. Rick Luce. LIBER 2011. Duurzame toegang blog. June 2011.
The topic was the core business of academic libraries: serving researchers and the scientific research process. There are many changes taking place in the sciences: "zetabytes of data; dynamic, complex data objects that require management; communities and data flows becoming much more important than static library collections, etc." The warning to academic libraries was that if libraries do not develop those services the new researcher needs, someone else will, and then there is no future for the research library. We need a "fundamental transformation process that will affect every aspect of the ‘library’ business."  The library needs to provide a repository between the scientific process and IT infrastructure that supports and preserves workflows.

Tomorrow will be too late – (born) digital in library special collections.

Tomorrow will be too late – (born) digital in library special collections. 28 June 2011.

Report from the annual conference of LIBER, the Association of European Research Libraries: It appears that according to recent studies, research libraries are still functioning within an analogue paradigm. Many libraries have digitized collections and provide online access, but their digitization efforts “mostly lack strategic planning, access is still mostly provided in a controlled way (for a limited group of users), preservation issues are still not being addressed adequately, and born-digital material (including audiovisual content) is blatently missing from collections and collection plans.”  If libraries stay in their comfort zone of digital access in a controlled network, their information role will become insignificant.  They need:

• A knowledge of the user groups and their behaviours
• Content, access, and preservation strategies
• Strategic alliances
• Permanent innovation
• An open mind

 There are opportunities for libraries but they need to transition their methods.

Meeting the Challenge of Media Preservation: Strategies and Solutions.

Meeting the Challenge of Media Preservation: Strategies and Solutions." Indiana University Bloomington. September 2011. (128 page pdf.)
This excellent study is the result of a year of research and planning to address the problems an earlier report in 2009.  It looks at the preservation and conservation of audio, video, and film, including: guiding preservation principles, facility planning, prioritization, digitization methodologies, strategies for film, principles for access, technological infrastructure needs, and engagement with campus units and priorities. It is specifically for the university, but the information and recommendations are of interest to others.  Their mission is to preserve the time-based media holdings of Indiana University so that they may be accessible. They estimate their media holdings at more than 560,000 audio, video, and film objects, and nearly all on obsolete formats. And they estimate they only have a  fifteen- to twenty-year window of opportunity to digitally preserve audio and video holdings. They propose to collect rich descriptive and technical metadata to support digitization and future interpretation and management of digital content.

• The media preservation crisis impacts every institution with media collections.
• Because campus holdings are very large and time pressures great, even high-efficiency workflows may not preserve everything in time. 
• Not every recording is an appropriate candidate for long-term preservation.
• Research, instruction, curation, and public availability are core university missions supported by media preservation efforts.
• Access is the end goal of any preservation work, and it must be developed in tandem with
  media preservation efforts.
• Access to preserved holdings is critical to the success of the project and to the realization of its value to the campus.
• The vision is an era characterized by a wealth of media content preserved long term and made accessible and integrated into campus research and instruction.
• We live in a watershed moment in which acute challenges demand a coordinated effort to address dramatic technological and cultural changes in the way users access time-based media.
• Target: high resolution audio preservation and production masters—24 bit, 96 kHz sample rate.

Some recommendations are that a 10,000-square-foot Indiana Media Preservation and Access Center is built employing 25 staff: administrators, audio and video engineers, film specialists, processing technicians, and IT support. The annual output is projected at 2-3PB of data per year with a total fifteen-year target of 39PB of data storage.  The first year of work will be focused on developing solutions to the challenges posed by legacy media. The second year will begin developing management strategies and workflows for file-based born digital recordings.

Their guiding principles include: Curatorial Responsibility; Standards and Best Practices; Online Accessibility; Description Services; usability of metadata; copyright strategies; Access Digitizing and Preservation. Their efforts need to be combined into a trusted digital repository.  "Preservation metadata requirements need to be defined, and tools need to be developed to support audio and video preservation package validation, technical metadata capture, and repository ingest."

This is a long study but is well worth the time to read all the way through it.   This university link also includes other related materials, such as the media preservation survey, and a brochure "Our History is at Risk".

Digital Preservation Matters - May 7, 2010

The Library of Congress Unlocks The Ultimate Archive System. Ken Weissman. Creative COW Magazine.  7 May 2010.

Article about the Library of Congress' Film Preservation Laboratory, where they are working on the  ultimate archive system, starting with the restoration of films first printed on paper instead of film.  One of the Library's main missions is to preserve America's memory for future generations of Americans, with no end point. They have a collection of 6.3 million audio-visual and film materials.  They have a pilot project digitizing their paper print collection. They are designing the workflow and they use the concept of Preservation Index (preservation quality of a storage environment).  The plan for now is to "scan the images, restore or preserve them as needed, then run them back to film, and put the film away at 25 degrees, 30% relative humidity, for practically forever. For most people, in practice, somewhere between 600 and 2000 years is beyond forever."

---

Zettabytes overtake petabytes as largest unit of digital measurement. Heidi Blake. Telegraph.co.uk.  04 May 2010.

IDC, the technology consultancy released in the annual survey of the world's digital output.  Humanity's total digital output currently stands at 8,000,000 petabytes but is expected to pass 1.2 zettabytes this year. The rapid growth of the "digital universe" has been caused by the explosion of social networking, online video, digital photography and mobile phones. Around 70 % of the world's digital content is created by individuals, but it is stored on content-sharing websites such as Flickr and YouTube.  In 2007, they estimated that the digital universe was equivalent to 161,000 petabytes.  [The term exabyte was used previously, it is larger than a petabyte.] They estimate the digital universe over the next decade will expand by a factor of 44.  Read more about Sortabytes, Peptabytes, and Lumabytes. 

---

The Sun-Times Preserves Its Photo Archive by Selling It. Michael Miner. Chicago Reader. May 6, 2010.

John Rogers claims to have the world's largest private collection of vintage photos, about 30 million images. The Sun-Times has sold its archive of more than a million photos and negatives to Rogers, though they retain the intellectual rights.  And Rogers is obliged to re-create the "entire library in digital searchable form".  He is doing for the Sun-Times what they wanted to do, but couldn't afford.  They estimated the processing would have taken many years and millions of dollars.  Once the photo archive is digitized the Sun-Times  will be able to tap a growing "aftermarket" for copies of old news photos.  Rogers can digitize 200,000 images a month and hoping to go to 400,000.  The creation of metadata is the expensive part. 

The Rocky Mountain News is a good example of what can happen when a newspaper folds.  The paper went out of business in February 2009.  "All those photos were given to the Denver Public Library and are sitting in a basement in storage. The library can't sell them to me, and they don't have the money to digitize them. So they'll stay in the basement. I spoke to a very nice lady at the library. I said, 'Can they be accessed by the public?' She said, 'Not at this time.' 'Will they ever be digitized?' 'We don't have the funds to do it." Instead, he bought the Denver Post, so he considers the Rocky Mountain News pictures redundant.

---

New program helps secure data.  Science Alert. 28 April 2010. 

Researchers at Monash University have developed the  MyTARDIS/TARDIS program to give researchers a place to securely store research information. It also has the ability  to share the most complex of scientific data through the internet. "The program records the data generated from an experiment, catalogues it, making it searchable, and transfers it back to the home institution, where the researcher can analyse the data using MyTARDIS, then make it publicly available on the TARDIS system alongside publication of the results in a scientific journal."  [What better name?]  The software has created a central place where researchers can exchange information rapidly and securely.

---

"Link Rot" & Legal Resources on the Web: A 2010 Analysis.  Sarah Rhodes. Legal Information Archive. May 2010.

The Legal Information Archive site has information about the Chesapeake Project, which contains government, policy, and legal information archived by several law libraries. 

This particular article their third annual analysis of link rot among the original URLs for archived materials.  The term "link rot" refers to a URL that no longer points to the resource that it originally did.  The link may return a not found message, or may point to a different resource. The results of their evaluation of 1,266 born-digital online titles that were harvested, link rot was found to be present in:

• 2008:   48 of 579 URLs,  8.3 %
• 2009:   83 of 579 URLs, 14.3 %
• 2010: 160 of 579 URLs, 27.9 %

A results table shows that over 90 % of the top-level domains in the sample were state-government, .org, and .gov URLs. 

---

Imation launches broad line of secure removable storage devices.  Lucas Mearian. ComputerWorld.  May 3, 2010.

Imation announced a new line of products, four flash drives, two hard drives and an optical line of Blu-ray discs and removable tape cartridges, all with a range of encryption and security management tools.

---

shortDOI™ Service.  Web site. International DOI Foundation.  May 6, 2010.

A new service is available to transform long DOI names , which are often very long strings. The service creates short handles of the name.  A DOI (Digital Object Identifier) is a persistent name that can be give to internet resources, instead of a url which can change.  The short DOI service returns a short cut that will resolve to the same object as the log DOI form. 

---