CLOCKSS: Extracting Triggered Content - Revision history

Ntay: restoring post-2016 edits

2019-08-15T00:47:27Z

restoring post-2016 edits

Dshr: Previous change and this one approved by Tom Lipkis

2014-06-25T01:43:11Z

Previous change and this one approved by Tom Lipkis

Dshr: /* CLOCKSS: Extracting Triggered Content */ Clarification resulting from discussion of draft certification report

2014-06-20T14:16:37Z

‎CLOCKSS: Extracting Triggered Content: Clarification resulting from discussion of draft certification report

Dshr: Changes after auditor visit: approved by Tom Lipkis

2014-04-10T03:36:42Z

Changes after auditor visit: approved by Tom Lipkis

Dshr: Add step of verifying content - approved by Tom Lipkis in meeting 2/12/14

2014-02-13T17:41:34Z

Add step of verifying content - approved by Tom Lipkis in meeting 2/12/14

Dshr: Undo revision 260 by Dshr (talk) Forgot to record approval

2014-02-13T17:40:20Z

Undo revision 260 by Dshr (talk) Forgot to record approval

Dshr: /* Preparing File Transfer Content for Dissemination */ Add step of verifying content

2014-02-13T00:15:20Z

‎Preparing File Transfer Content for Dissemination: Add step of verifying content

Dshr: /* Re-registering Triggered Article DOIs */ refer to DOI link

2014-02-12T20:57:19Z

‎Re-registering Triggered Article DOIs: refer to DOI link

Dshr: /* Re-publishing the Triggered Content */ visual check

2014-02-07T17:27:54Z

‎Re-publishing the Triggered Content: visual check

Dshr: /* Overview of the Trigger Process */ responsibility

2014-02-07T17:26:33Z

‎Overview of the Trigger Process: responsibility

@@ Line 5: / Line 5: @@
 * A publisher who has acquired a title deciding not to re-host a run of back issues.
 * Disaster.
-The [https://www.clockss.org/clocksswiki/files/CLOCKSS_Participating_Publisher_Agreement.pdf CLOCKSS Participating Publisher Agreement] paragraph 4.E states:
+The [https://clockss.org/wp-content/uploads/2018/10/2019-CLOCKSS-Participating-Publisher-Agreement.pdf CLOCKSS Participating Publisher Agreement] paragraph 4.E states:
 <blockquote>
 Released Content use terms and restrictions will be determined by an accompanying Creative Commons license (or equivalent license) chosen either by Publisher or, if Publisher fails to respond within thirty (30) days following receipt by it of the notice described above, by the CLOCKSS Board.
@@ Line 11: / Line 11: @@
 At present, EDINA and Stanford have volunteered to re-publish triggered content, but it is important to note that the Creative Commons license means that anyone can re-publish such content, and that re-publishing is not a core function of the CLOCKSS archive.
-Once the CLOCKSS board notifies the LOCKSS Executive Director that a trigger event has occurred, the CLOCKSS Metadata Lead and assigned LOCKSS staff run a process that delivers the triggered content to the re-publishing sites, and updates a [https://www.clockss.org/clockss/Triggered_Content section of the CLOCKSS website] with links pointing to the re-published content at both sites. In OAIS terminology, the package delivered to the re-publishing sites is a [[Definition of DIP|Distribution Information Package (DIP)]].
+Once the CLOCKSS board notifies the LOCKSS Executive Director that a trigger event has occurred, the CLOCKSS Metadata Lead and assigned LOCKSS staff run a process that delivers the triggered content to the re-publishing sites, and updates a [https://clockss.org/triggered-content/ section of the CLOCKSS website] with links pointing to the re-published content at both sites. In OAIS terminology, the package delivered to the re-publishing sites is a [[Definition of DIP|Distribution Information Package (DIP)]].
 Additionally, a publisher with content in CLOCKSS can request to be supplied with a copy. In such a case, the process described below is followed but the content is delivered to the requesting publisher not to a re-publishing site.
-The [https://www.clockss.org/clocksswiki/files/CLOCKSS_Participating_Publisher_Agreement.pdf CLOCKSS: Publisher Agreement] specifies that content must be unavailable for 6 months before it is triggered. It is important to observe that this delay is mandatory only when content is being triggered ''without the consent of the publisher''. In all cases so far content has been triggered ''at the request of the publisher'', and the technical process described below has taken 2-4 weeks.
+The [https://clockss.org/wp-content/uploads/2018/10/2019-CLOCKSS-Participating-Publisher-Agreement.pdf CLOCKSS: Publisher Agreement] specifies that content must be unavailable for 6 months before it is triggered. It is important to observe that this delay is mandatory only when content is being triggered ''without the consent of the publisher''. In all cases so far content has been triggered ''at the request of the publisher'', and the technical process described below has taken 2-4 weeks.
 == Overview of the Trigger Process ==
@@ Line 229: / Line 229: @@
 ** CLOCKSS Content Lead
 ** CLOCKSS Network Administrator
-* Approval by CLOCKSS Technical Lead
+* Approval by CLOCKSS Technical Manager
 == Relevant Documents ==
 # [[Definition of AIP]]
 # [[Definition of DIP]]
-# [https://www.clockss.org/clockss/Triggered_Content CLOCKSS Triggered Content]
+# [https://clockss.org/triggered-content/ CLOCKSS Triggered Content]
 # [[LOCKSS: Format Migration]]
 # [[LOCKSS: Metadata Database]]

@@ Line 15: / Line 15: @@
 Additionally, a publisher with content in CLOCKSS can request to be supplied with a copy. In such a case, the process described below is followed but the content is delivered to the requesting publisher not to a re-publishing site.
-The [https://www.clockss.org/clocksswiki/files/CLOCKSS_Participating_Publisher_Agreement.pdf CLOCKSS: Publisher Agreement] specifies that content must be unavailable for 6 months before it is triggered. It is important to observe that this delay is mandatory when content is being triggered ''without the consent of the publisher''. In all cases so far content has been triggered ''at the request of the publisher'', and the technical process described below has taken 2-4 weeks.
+The [https://www.clockss.org/clocksswiki/files/CLOCKSS_Participating_Publisher_Agreement.pdf CLOCKSS: Publisher Agreement] specifies that content must be unavailable for 6 months before it is triggered. It is important to observe that this delay is mandatory only when content is being triggered ''without the consent of the publisher''. In all cases so far content has been triggered ''at the request of the publisher'', and the technical process described below has taken 2-4 weeks.
 == Overview of the Trigger Process ==

← Older revision		Revision as of 14:16, 20 June 2014
Line 14:		Line 14:

	Additionally, a publisher with content in CLOCKSS can request to be supplied with a copy. In such a case, the process described below is followed but the content is delivered to the requesting publisher not to a re-publishing site.		Additionally, a publisher with content in CLOCKSS can request to be supplied with a copy. In such a case, the process described below is followed but the content is delivered to the requesting publisher not to a re-publishing site.
		+
		+	The [https://www.clockss.org/clocksswiki/files/CLOCKSS_Participating_Publisher_Agreement.pdf CLOCKSS: Publisher Agreement] specifies that content must be unavailable for 6 months before it is triggered. It is important to observe that this delay is mandatory when content is being triggered ''without the consent of the publisher''. In all cases so far content has been triggered ''at the request of the publisher'', and the technical process described below has taken 2-4 weeks.

	== Overview of the Trigger Process ==		== Overview of the Trigger Process ==

@@ Line 5: / Line 5: @@
 * A publisher who has acquired a title deciding not to re-host a run of back issues.
 * Disaster.
-The contract under which publishers deposit content in the CLOCKSS archive states that triggered content will be re-published under a Creative Commons license. At present, EDINA and Stanford have volunteered to re-publish triggered content, but it is important to note that the Creative Commons license means that anyone can re-publish such content, and that re-publishing is not a core function of the CLOCKSS archive.
+The [https://www.clockss.org/clocksswiki/files/CLOCKSS_Participating_Publisher_Agreement.pdf CLOCKSS Participating Publisher Agreement] paragraph 4.E states:
 Once the CLOCKSS board notifies the LOCKSS Executive Director that a trigger event has occurred, the CLOCKSS Metadata Lead and assigned LOCKSS staff run a process that delivers the triggered content to the re-publishing sites, and updates a [https://www.clockss.org/clockss/Triggered_Content section of the CLOCKSS website] with links pointing to the re-published content at both sites. In OAIS terminology, the package delivered to the re-publishing sites is a [[Definition of DIP|Distribution Information Package (DIP)]].
 == Overview of the Trigger Process ==
@@ Line 23: / Line 29: @@
 == Identifying the Triggered Content ==
-The triggering process begins by identifying the triggered content within the CLOCKSS preservation network. First, the Archival Units (AUs) that contain the title are identified by querying the [[LOCKSS: Metadata Database|database of bibliographic metadata]] that is compiled and maintained by each CLOCKSS box. This database contains a record of articles published for each preserved title, indexed by year, volume, and issue.
+The triggering process begins by identifying the triggered content within the CLOCKSS preservation network. First, the [[LOCKSS: Basic Concepts#Archival Units|Archival Units (AUs)]] that contain the title are identified by querying the [[LOCKSS: Metadata Database|database of bibliographic metadata]] that is compiled and maintained by each CLOCKSS box. This database contains a record of articles published for each preserved title, indexed by year, volume, and issue.
-A query to this database on a production CLOCKSS box yields a list of AU identifiers (AUIDs) that identify the associated AUs (AIPs in OAIS terminology) being preserved in the CLOCKSS network. A similar query to an ingest machine identifies any triggered AUs that have yet to emerge from the ingest pipeline (see [[CLOCKSS: Ingest Pipeline]]). The AUIDs are the same on every CLOCKSS box and, as described in [[Definition of AIP]], uniquely identify the location on each box of that AU (AIP). For content that was harvested from a publishers website, a single AU will normally contain all the files for a given volume or a year. For content obtained via file transfer from publishers, content is organized into AUs that contain titles, volumes, and issues from the same publisher that were ingested in a given calendar year.
+A query to this database on a production CLOCKSS box yields a list of [[LOCKSS: Basic Concepts#AUID|AU identifiers (AUIDs)]] that identify the associated AUs ([[Definition of AIP|AIPs]] in OAIS terminology) being preserved in the CLOCKSS network. A similar query to an ingest machine identifies any triggered AUs that have yet to emerge from the ingest pipeline (see [[CLOCKSS: Ingest Pipeline]]). The AUIDs are the same on every CLOCKSS box and, as described in [[Definition of AIP]], uniquely identify the location on each box of that AU (AIP). For content that was harvested from a publishers website, a single AU will normally contain all the files for a given volume or a year. For content obtained via file transfer from publishers, content is organized into AUs that contain titles, volumes, and issues from the same publisher that were ingested in a given calendar year.
 == Extracting the Triggered Content ==
@@ Line 34: / Line 40: @@
 For file transfer content, the extraction process involves locating the directories in the CLOCKSS box that correspond to the AUs that contain the triggered content. Within these directories are either sub-directories or archive files that contain the content. These directories or archive files are copied from the repository to a server where the triggered content will be prepared for publication.
 == Preparing File Transfer Content for Dissemination ==
@@ Line 81: / Line 89: @@
 The technique for re-publishing newly triggered content delivered as this type of [[Definition of DIP|DIP]] used by the current re-publishing sites is as follows. On each of the re-publishing machines:
-* Configure the appropriate plugin and Title Database (TDB) entries, by updating the triggered-content configuration and plugin repositories on <tt>props.lockss.org</tt> (see [[LOCKSS: Property Server Operations]].
+* Configure the appropriate [[LOCKSS: Basic Concepts#Plugins|plugin]] and [[LOCKSS: Basic Concepts#Title Database|Title Database (TDB)]] entries, by updating the triggered-content configuration and plugin repositories on <tt>props.lockss.org</tt> (see [[LOCKSS: Property Server Operations]]).
 * Force the LOCKSS daemon to reload its configuration and plugins from the repository.
 * Unpack the AUs into the repository hierarchy of the LOCKSS daemon.
@@ Line 98: / Line 106: @@
 == Re-registering Triggered Article DOIs ==
-Most publishers register individual article Digital Object Identifiers (DOIs) with a registrar sponsored by DOI International, such as [http://www.crossref.org CrossRef] Once a title is no longer available from the publisher, the registration records for the articles should be updated to refer to the content hosted at Edina and Stanford. This is done by preparing a tab-separated file with the DOI for each article and the corresponding new URL. A separate file is required for the articles hosted at Edina and Stanford. The registrar uses the data in this file to update the records for the corresponding DOIs.
+Most publishers register individual article Digital Object Identifiers (DOIs) with a registrar sponsored by DOI International, such as [http://www.crossref.org CrossRef]. Once a title is no longer available from the publisher, the registration records for the articles should be updated to refer to the content hosted at Edina and Stanford. This is done by preparing a tab-separated file with the DOI for each article and the corresponding new URL. A separate file is required for the articles hosted at Edina and Stanford. The registrar uses the data in this file to update the records for the corresponding DOIs.
 For content that is being served by the two re-publishing servers, this task is simple because their LOCKSS daemon provides an OpenURL resolver, allowing access to articles via their DOIs. The DOI information is available from the article metadata, via the DOI link in the daemon UI. Here is a portion of the file for a title being hosted at Edina that can be sent to the DOI registrar.

@@ Line 38: / Line 38: @@
 File transfer content is typically a collection of files that include PDFs of articles, XML formatted full-text files, supporting files such as images and multi-media, and metadata files that contain bibliographic information. These files are input to the processes of the publishing platform that displays the publisher's content to readers. The exact content of the collection varies between publishers.  Typically, further processing involving the following steps is required to extract the content from the directories or archive files that were copied from the CLOCKSS box repository, and generate from them a web site:
 * Unpacking any archive files that contain the content into their own directories and isolate the files that correspond to the triggered content. Publishers tend to group files for individual issues into their own directories, so isolating the files involves retaining only those directories that correspond to the content being triggered and discarding directories for other content.
 * Adding any full-text PDF files unmodified to the web site.

@@ Line 18: / Line 18: @@
 # re-registering triggered article DOIs.
 The following sections describe the process as it is currently implemented. It is possible that content triggered in the future might contain files whose format has become obsolete. The additional processes that would be needed in this situation are described in [[LOCKSS: Format Migration#Format Migration in the CLOCKSS Archive|LOCKSS: Format Migration]].
 == Identifying the Triggered Content ==