LOCKSS: Format Migration - Revision history

Ntay: restoring post-2016 edits

2019-08-14T23:40:05Z

restoring post-2016 edits

Dshr: /* Format Migration in the CLOCKSS Archive */

2014-04-06T21:50:58Z

‎Format Migration in the CLOCKSS Archive

Dshr: /* Obsolescence of Web Formats */

2013-12-13T18:27:16Z

‎Obsolescence of Web Formats

Dshr: /* Relevant Documents */ Add reference

2013-12-13T03:13:56Z

‎Relevant Documents: Add reference

Dshr: /* Obsolescence of Web Formats */ Chrome emulates the Amiga 500

2013-12-13T03:10:38Z

‎Obsolescence of Web Formats: Chrome emulates the Amiga 500

Dshr: /* Format Migration in the CLOCKSS Archive */ Add sample FITS output

2013-12-09T17:24:55Z

‎Format Migration in the CLOCKSS Archive: Add sample FITS output

Dshr: /* Format Migration in the CLOCKSS Archive */ Add stubs

2013-12-09T16:56:16Z

‎Format Migration in the CLOCKSS Archive: Add stubs

Dshr: /* Relevant Documents */ Response to 11/12/13 request

2013-12-07T04:02:37Z

‎Relevant Documents: Response to 11/12/13 request

Dshr: /* Availability of Format Converters */ Response to 11/12/13 request

2013-12-07T04:00:33Z

‎Availability of Format Converters: Response to 11/12/13 request

Dshr: /* LOCKSS Strategy for Format Obsolescence */ Response to 11/12/13 request

2013-12-07T03:58:44Z

‎LOCKSS Strategy for Format Obsolescence: Response to 11/12/13 request

@@ Line 75: / Line 75: @@
 Changes to this document require:
 * Review by LOCKSS Engineering Staff
-* Approval by LOCKSS Chief Scientist
+* Approval by LOCKSS Technical Manager
 == Relevant Documents ==

@@ Line 48: / Line 48: @@
 The CLOCKSS archive is a dark archive. No readers (Consumers in the OAIS terminology) ever "interact with [CLOCKSS] services to find preserved information of interest and to access that information in detail". If content is ever triggered from the archive, readers access it from one of a number of re-publishing systems. Dissemination of triggered content is a transaction between the archive and one or more of these republishing systems which involves construction of a [[Definition of DIP|Dissemination Information Package]] and its transmission to the re-publishing system(s). If a subsequent reader's browser is unable to render the format in which the digital object was represented in the DIP, and is thus stored in the re-publishing system the reader is accessing, the technique described above can be applied.
-The LOCKSS software used by the CLOCKSS archive integrates the [https://code.google.com/p/fits/ File Identification Tool Set (FITS)], which includes  [http://jhove.sourceforge.net/ JHOVE], [http://sourceforge.net/projects/droid/ DROID] and other tools. FITS is currently configured as follows:
+The LOCKSS software used by the CLOCKSS archive can integrate the [https://code.google.com/p/fits/ File Identification Tool Set (FITS)], which includes  [http://jhove.sourceforge.net/ JHOVE], [http://sourceforge.net/projects/droid/ DROID] and other tools. FITS is currently configured as follows:
 * Priority: Put DROID at top of list, then JHOVE.
 * Disable NLNZ Metadata Extractor, which gives Exceptions.
@@ Line 54: / Line 54: @@
 * Configure DROID to exclude html, xhtml & pdf because it performs badly on these file types.
 * Configure JHOVE to exclude js  because it performs badly on files of this type.
-Various outputs from FITS are available through the LOCKSS daemon's GUI for every URL in an AU. Samples of FITS output are available:
+Various outputs from FITS are available through the [[LOCKSS: Basic Concepts#LOCKSS Daemon|LOCKSS daemon's GUI]] for every URL in an AU. Samples of FITS output are available:
 * [[Media:ABER6 FITS.pdf|A harvest AU]] (80-page PDF).
 * [[Media:LA24 FITS.pdf|A file transfer AU]] (188 page PDF).

@@ Line 15: / Line 15: @@
 ** The Internet Archive uses JMESS to [http://blog.archive.org/2013/10/25/microcomputer-software-lives-again-this-time-in-your-browser/ deliver emulations of ancient software].
 ** The [https://github.com/s-macke/jor1k jor1k project] has the [http://linux.slashdot.org/story/13/11/12/161227/linux-kernel-running-in-javascript-emulator-with-graphics-and-network-support Linux kernel running] in an emulated computer implemented in Javascript.
-** Google's Chrome can now [http://news.cnet.com/8301-1023_3-57615373-93/google-emulates-1980s-era-amiga-computer-in-chrome/|emulate the Amiga 500] via the Native Client interface.
+** Google's Chrome can now [http://news.cnet.com/8301-1023_3-57615373-93/google-emulates-1980s-era-amiga-computer-in-chrome/ emulate the Amiga 500] via the Native Client interface.
 Thus it is far from clear that, even if Web formats eventually suffer obsolescence, format migration would be necessary. By the time obsolescence might happen, it might well be that delivering a transparent emulation to the reader's browser would be the preferred method.

← Older revision		Revision as of 03:13, 13 December 2013
Line 97:		Line 97:
	# Sebastian Macke "jor1k project" https://github.com/s-macke/jor1k		# Sebastian Macke "jor1k project" https://github.com/s-macke/jor1k
	# warmflatsprite "Linux Kernel Running In JavaScript Emulator With Graphics and Network Support" http://linux.slashdot.org/story/13/11/12/161227/linux-kernel-running-in-javascript-emulator-with-graphics-and-network-support		# warmflatsprite "Linux Kernel Running In JavaScript Emulator With Graphics and Network Support" http://linux.slashdot.org/story/13/11/12/161227/linux-kernel-running-in-javascript-emulator-with-graphics-and-network-support
		+	# Steven Shankland "Google emulates 1980s-era Amiga computer in Chrome" http://news.cnet.com/8301-1023_3-57615373-93/google-emulates-1980s-era-amiga-computer-in-chrome/

@@ Line 15: / Line 15: @@
 ** The Internet Archive uses JMESS to [http://blog.archive.org/2013/10/25/microcomputer-software-lives-again-this-time-in-your-browser/ deliver emulations of ancient software].
 ** The [https://github.com/s-macke/jor1k jor1k project] has the [http://linux.slashdot.org/story/13/11/12/161227/linux-kernel-running-in-javascript-emulator-with-graphics-and-network-support Linux kernel running] in an emulated computer implemented in Javascript.
 Thus it is far from clear that, even if Web formats eventually suffer obsolescence, format migration would be necessary. By the time obsolescence might happen, it might well be that delivering a transparent emulation to the reader's browser would be the preferred method.

← Older revision		Revision as of 04:02, 7 December 2013
Line 88:		Line 88:
	# David S. H. Rosenthal. "Formats Through Time" October 9, 2012 http://blog.dshr.org/2012/10/formats-through-time.html		# David S. H. Rosenthal. "Formats Through Time" October 9, 2012 http://blog.dshr.org/2012/10/formats-through-time.html
	# David S. H. Rosenthal. "Are Format Specifications Important For Preservation?" January 4, 2009 http://blog.dshr.org/2009/01/are-format-specifications-important-for.html		# David S. H. Rosenthal. "Are Format Specifications Important For Preservation?" January 4, 2009 http://blog.dshr.org/2009/01/are-format-specifications-important-for.html
		+	# Dirk von Suchodoletz, Klaus Rechert, Isgandar Valizada "Towards Emulation-as-a-Service: Cloud Services for Versatile Digital Object Access" http://dx.doi.org/10.2218/ijdc.v8i1.250
		+	# I. Valizada, K. Rechert, K. Meier, D. Wehrle, D. v. Suchodoletz and L. Sabel "Cloudy Emulation – Efficient and Scalable Emulation-based Services" http://purl.pt/24107/1/iPres2013_PDF/Cloudy%20Emulation%20%E2%80%93%20Efficient%20and%20Scaleable%20Emulation-based%20Services.pdf
		+	# JPC project "JPC: The Pure Java x86 PC Emulator" http://jpc.sourceforge.net/home_home.html
		+	# Jason Scott "Microcomputer Software Lives Again, This Time in Your Browser" http://blog.archive.org/2013/10/25/microcomputer-software-lives-again-this-time-in-your-browser/
		+	# Sebastian Macke "jor1k project" https://github.com/s-macke/jor1k
		+	# warmflatsprite "Linux Kernel Running In JavaScript Emulator With Graphics and Network Support" http://linux.slashdot.org/story/13/11/12/161227/linux-kernel-running-in-javascript-emulator-with-graphics-and-network-support

@@ Line 65: / Line 65: @@
 * makes it [http://blog.dshr.org/2009/01/are-format-specifications-important-for.html easy to implement a converter], since the necessary syntax and semantics are available in source code form.
 Note that, since in the LOCKSS approach format migration takes place at access time rather than at some earlier pre-emptive migration time, any criticism of the approach on the basis that converters might not be available applies ''a fortiori'' to the pre-emptive approach.
 == Change Process ==

@@ Line 22: / Line 22: @@
 * The probability of a format going obsolete is low.
 * If a format does go obsolete, it will be a long time after its introduction.
 Further, studies have shown that:
 * Digital objects in archives are [http://www.ssrc.ucsc.edu/pub/adams-ssrctr-11-01.html infrequently accessed by readers].
@@ Line 32: / Line 33: @@
 * Use this pipeline to generate a temporary access copy of the original in a format suitable for the reader's browser.
 * Discard this access copy when it is no longer needed.
-A framework to support this strategy was implemented in the LOCKSS software and [http://dx.doi.org/10.1045/january2005-rosenthal demonstrated in 2005].
+A framework to support this strategy was implemented in the LOCKSS software and [http://dx.doi.org/10.1045/january2005-rosenthal demonstrated in 2005]. To avoid wasting resources implementing capabilities which have no realistic prospect of being needed in the foreseeable future, work in this area is on hold. When there is evidence that some format of content under preservation is facing obsolescence, a decision will be taken as to whether a production version of this migration strategy is the appropriate path to take, or whether (for example) an in-browser emulation strategy would be more effective.
 This strategy has a number of significant advantages:
@@ Line 39: / Line 40: @@
 * It performs any format migration that is actually necessary as late as possible, when the technology for performing it is likely to be better.
 * It expends resources as late as possible, exploiting the time value of money to the maximum extent.
 == Format Migration in the CLOCKSS Archive ==