Rosetta and Amazon Storage

Rosetta and Amazon Storage. Chris Erickson. February 2015.
In the search for more file storage, as well as more affordable file storage, we tried Amazon Simple Storage Service (Amazon S3). The plan was to connect the Rosetta Digital Preservation System to the Amazon cloud storage, and evaluate it as a possible storage solution. There is a free trial. The Free Tier includes 5GB storage, 20,000 Get Requests, and 2,000 Put Request.

Setup:
I tried various configurations, but decided on a single bucket for the files. I setup buckets for the IEs and metadata, but after trying it, decided to only keep the files on Amazon. That would keep the metadata local. I had tried nested folders, but couldn't figure out how to designate that in the storage rules and definition. So I create the folders by time period.

In the Rosetta Admin interface I create a File storage group, using the S3 storage plugin, and then entered the Bucket name, Secret Access Key, Access Key ID, and left the Maximum waiting time at the default. For the test, I set up a retention code for Amazon, and the storage rule used that code to determine what went to the Amazon storage. In a real storage instance, it would be better to use something that would not change, like the producer, etc.

It took a few tests to get everything in sync. The result was that Rosetta stored the content in Amazon just fine. I also tried adding content with a one day retention period, and the content was removed from Amazon after the day. A fixity check task was also able to work without a problem.

This gives us another storage option, though we decided to not use it at present.

Pricing at the time of this comparison, was:

	1 TB		50 TBs
Digital Storage Costs	Annual  Cost	20 Year Projected	Yearly  Charge	10 Year Projected	20 Year Projected	50 Year Projected
Cloud Storage
Amazon S3 - Regular	$360	$7,200	$17,706	$177,060	$354,120	$885,300
Amazon S3 - Copy / Glacier	$480	$9,600	$23,706	$237,060	$474,120	$1,185,300
Amazon S3 - Reduced & Glacier	$288	$5,760	$14,165	$141,648	$283,296	$708,240
DuraSpace - Preservation	$1,800	$36,000	$36,100	$361,000	$722,000	$1,805,000
DuraSpace - Dark copy/Glacier	$1,925	$38,500	$42,350	$423,500	$847,000	$2,117,500
DuraSpace - Enterprise Plus	$5,625	$112,500	$64,425	$644,250	$1,288,500	$3,221,250

More storage options will be considered. 

Digital Preservation Matters.

Preserving progress for future generations

Preserving progress for future generations. Rebecca Pool. Research Information. February/March 2015.
Digital preservation remains one of the most critical challenges facing scholarly communities today. From e-journals and e-books to emails, blogs and more, electronic content is proliferating and organizations worldwide are trying to preserve information before the electronic information is lost. Some of the organizations include: Portico (which preserves content on behalf of participating publishers; the number of open access journals it includes is rising, ); CLOCKSS (still grappling with the cost models of providing preservation service).

There is a rising demand for the preservation of dynamic content. No one is able to "capture dynamic content and [preserve] a day-to-day, or even, minute-to-minute feed of this content." There are only snapshots. CLOCKSS is developing the ‘how to’ process to preserve these ‘snapshots’ across multiple locations, validating each against the other, and is also exploring the best pricing structures to preserve such content.

Other organizations include LOCKSS, The Digital Preservation Network, HathiTrust, Preservica, Archivematica, and Rosetta, whose recent clients are the State Library of New South Wales and the State Library of Queensland.

The digital preservation development is clearly gaining momentum, growing in both size and complexity. "Clearly progress is being made and you can measure that by the maturity of solutions on offer." But for most organizations, the urgency of digital preservation has yet to hit home.

"Trying to sell the idea of digital preservation on the basis of return on investment has been very hard. By its nature, it’s a long-term activity and you’re really hedging your bets against future risks. I think we are still in the very early days of genuinely understanding the value of digital assets... and transferring this understanding over to financial assets doesn’t yet work very well." The European consortium 4C (Collaboration to Clarify the Costs of Curation) has been investigating this problem. Their road map helps organisations appraise digital assets, adopt a strategy to grow preservation assets and develop costing processes. In addition they have developed a model for curation costs. The only way to understand the costs of preservation is though sharing, through openness and collaboration.

Digital Preservation Matters.

Ex Libris plugins for Rosetta on Github

Ex Libris plugins for Rosetta on Github. January 2015.
The Github site for Rosetta plugins. Includes:

• Rosetta.Jpylyzer Jpylyzer technical metadata plugin
• rosetta.JWPlayer
• rosetta.IABookReader
• rosetta.Jpylyzer-MDExtractorPlugin
• rosetta.Drmlint-RiskExtractorPlugin
• rosetta.Split2JpgMigrationToolPlugin Migration Tool Plug-in to split multi-page files to multiple jpg files
• rosetta.IIPImageVPP Rosetta IIPImage Viewer Pre-Processor Plugin

Digital Preservation Matters.

Memorial University of Newfoundland selects Ex Libris Solutions, including Rosetta

Memorial University of Newfoundland selects Ex Libris Solutions, including Rosetta. Press Release. Ex Libris. January 28, 2015.
The Memorial University in Newfoundland and Labrador, Canada has adopted a suite of Ex Libris solutions comprised of the Alma library management solution, the Primo discovery and delivery solution, and the Rosetta digital asset management and preservation system. These solutions replace multiple disparate legacy systems used by the Library.

Rosetta will enable Memorial University to manage and preserve its important collections of Newfoundland's history, including a huge collection of digitized newspapers. Using the Primo search interface for physical collections, digital and digitized assets, and electronic resources, Memorial will provide a seamless discovery experience to users, whatever their learning and teaching needs. 

Digital Preservation Matters.

University of Arizona selects Ex Libris Rosetta.

University of Arizona selects Ex Libris Rosetta. Press Release. Ex Libris. January 27, 2015.
The University of Arizona has adopted the Rosetta digital management and preservation solution. Rosetta will help the university provide sustained access to scholarly digital content and research to both university members and the broader academic community.

"After evaluating a number of commercial digital preservation systems, we found that Rosetta had the unique capabilities that Arizona requires. Our priorities for 2015 led us to seek a preservation solution that could be used collaboratively by a number of campuses. Rosetta's ability to provide end-to-end digital asset management and preservation for the vast array of assets and research data that the university possesses, its consortial architecture that allows participating institutions to maintain a degree of autonomy, and its ability to act as a transitional component between multiple display layers, made it the clear choice for Arizona."

Digital Preservation Matters.

The State Library of NSW selects Ex Libris and Axiell Solutions

The State Library of NSW selects Ex Libris and Axiell Solutions. Press release. 4 December 2014.
The State Library has announced it will adopt a suite of Ex Libris solutions which includes Alma, a  library management solution, the Primo discovery and delivery solution, and the Rosetta digital asset management and preservation system. Also the Adlib Archive software to manage its archival collections. These solutions replace multiple legacy systems used by the Library. "Rosetta’s comprehensive management and preservation capabilities for digital and digitized collections will ensure that the vast collections held in trust for the people of NSW will be preserved and available in the future."

Digital Preservation Matters.