Fix Type of Item in Hoboken Historical Photographs collection

Project:RUcore Jobs & Reports
Component:Job - production
Category:task
Priority:normal
Assigned:ananthan
Status:closed
Description

In approximately 1,110 resources in the collection, Hoboken Historical Photographs, the Type of Item is misspelled. It is spelled "Stillimage" (lower case "image"), and it should be "StillImage" (upper case "Image"). This causes the DOI not to resolve.

1. Change Stillimage to StillImage
2. Update the metadata for the DOI
3. Change the DOI status from reserved to public

In the course of diagnosing this issue, I fixed the record rutgers-lib:13315. In my investigation, I did not find any other resources in this collection that have a correct Type of Item.

Comments

#1

I'll start looking into this as a fixds and then an update DOI project.

#2

Assigned to:triggs» dhoover

This will require two stages - 1) to fix the MODS sections and 2) to update the DOI metadata and make the DOIs public. The following refers to the fixds project in stage 1.
The following files are ready in /mellon/cvsroot:
-rwxr-xr-x 1 triggs developers 1282 Feb 15 14:23 runfixdsupdatehhpmods.sh
-rw-r--r-- 1 triggs developers 29970 Feb 15 14:23 putupdatehhpmodslist-prod
-rw-r--r-- 1 triggs developers 4324 Feb 15 14:23 fixds-updatehhpmods-readme.txt
-rwxr-xr-x 1 triggs developers 13618 Aug 11 2015 /mellon/cvsroot/fixds.pl
-rwxr-xr-x 1 triggs developers 13618 Aug 11 2015 /mellon/cvsroot/fixds-10.pl
The fixds-updatehhpmods-readme.txt file contains the following information about dryrun and realrun of the script.

JAT - February 15, 2017

README for running the fixds.pl script on prod for the set of MODS sections that need "mods:typeOfResource"
changed from "Stillimage" to "StillImage" so that their DOIs can be updated and made public as requested in
<a href="http://software.libraries.rutgers.edu/node/3623" title="http://software.libraries.rutgers.edu/node/3623">http://software.libraries.rutgers.edu/node/3623</a>

Copy these file from /mellon/cvsroot:
fixds.pl (the current version is fixds-10.pl)
putupdatehhpmodslist-prod
fixds-updatehhpmods-readme.txt
runfixdsupdatehhpmods.sh

To view the fixds.pl manpage:
perldoc /path/to/fixds.pl

To run fixds for the "updatehhpmods" MODS sections on prod use the following commands:
cd to the directory with fixds-10.pl (the script calls this version by name). Put the filelist,
putupdatehhpmodslist-prod, in the same directory as well as the runfixdsupdatehhpmods.sh script.

Then type:
./runfixdsupdatehhpmods.sh -dryrun 2>/dev/null
If the script reports any failures, you may want to rerun the above command without redirecting the standard error (see examples below).
This will test the availability and wellformedness of the datastreams without actually ingesting them into
the repository. You should look for output like the following:

Running dryrun test...5 /mellon/htdocs/dlr/EDIT/TESTOBJECTS/DEVELMODS/BAD/targlist objects
5 retrieved and successfully parsed...
0 retrieved but unsuccessfully parsed...
0 retrieved for objects that do not exist...
Log file is "testfixdslog-140505-134751"

When you are satisfied that the run will be clean, type the following:
./runfixdsupdatehhpmods.sh
This will actually run the fedora ingest commends and ingest the datastreams into the repository. You should look for output like the following:

Running fixds with ingest...5 /mellon/htdocs/dlr/EDIT/TESTOBJECTS/DEVELMODS/BAD/targlist objects
5 successfully ingested...
0 not successfully ingested...
Log file is "fixdslog-140505-140144"

Note: There are 1110 corrected MODS sections listed in putupdatehhpmodslist-prod.

#3

Stage 2 is also ready in /mellon/cvsroot. This is a script to update the DOI metadata and make the DOIs public for the HHP collection. It is the same script used in May 2015 to perform a similar task. The new file wit the new listofpids for HHP is:
-rw-r--r-- 1 triggs developers 30720 Feb 16 11:28 /mellon/cvsroot/rundoidopublic-hhp.tar
triggs@rep-devel:/mellon/htdocs/dlr/EDIT/TESTOBJECTS/HOBOKEN> tar tvf /mellon/cvsroot/rundoidopublic-hhp.tar
-rw-r--r-- triggs/developers 1373 2017-02-16 11:27 rundoidopublic-readme.txt
-rwxr-xr-x triggs/developers 1713 2015-03-02 16:33 rundoidopublic.sh
-rw-r--r-- triggs/developers 19980 2017-02-16 11:25 listofpids

The readme is basically the same as before:

This is a readme file for using rundoidopublic.sh.

This script will be delivered as a tar file called rundoidopublic-hhp.tar which will include the script, the listofpids file, and this readme.
Untar the file and type:

./rundoidopublic.sh -dryrun
for the dry run test and
./rundoidopublic.sh
for the actual run.

The following is the sample output of the final test on devel:

triggs@rep-devel:/mellon/htdocs/dlr/EDIT> ./rundoidopublic.sh dryrun
Running dryrun test...1 ./listofpids objects
1 retrieved and DOIs successfully parsed...
0 have bad DOIs that cannot be managed...
0 objects do not yet have DOIs...
Log file is "testrundoilog-150302-162014"
triggs@rep-devel:/mellon/htdocs/dlr/EDIT> ./rundoidopublic.sh
Running rundoi for real...1 ./listofpids objects
1 metadata successfully updated...
1 retrieved and DOIs successfully changed to public...
0 have bad DOIs that cannot be managed...
0 objects do not yet have DOIs...
Log file is "rundoilog-150302-162023"
triggs@rep-devel:/mellon/htdocs/dlr/EDIT> cat rundoilog-150302-162023

For object rutgers-lib:16250...
<?xml version="1.0" encoding="UTF-8"?>
<response>
<status_code>200</status_code>
<success>Metadata was successfully updated.</success>
</response><?xml version="1.0" encoding="UTF-8"?>
<response>
<status_code>200</status_code>
<success>The state was successfully changed.</success>
</response>

#4

In order to help Dave with the running of the two-stage project, I've combined and simplified the readmes into a single, canonical readme at:
/mellon/cvsroot/HHP-twostage-readme.txt
Dave should use this readme for both stages of the project.

#5

This was run in dryrun mode on rep-prod.

Running dryrun test...1110 ./putupdatehhpmodslist-prod objects
1110 retrieved and successfully parsed...
0 retrieved but unsuccessfully parsed...
0 retrieved for objects that do not exist...
Log file is "testfixdslog-170303-084627"

Log is attached.

#6

The dry run looks good. The realrun should be ready to go for stage 1.

#7

This was run on production in real mode.

Running fixds with ingest...1110 ./putupdatehhpmodslist-prod objects
1110 successfully ingested...
0 not successfully ingested...
Log file is "fixdslog-170303-151400"

report is attached

#8

Hi Dave,

The MODS modifications look good. I spot checked some and tested changing the DOI on one (rutgers-lib:14256) successfully. I think we could do the dryrun of the second part of the project (the update doi) whenever you are ready.

Thanks,

Jeffery

#9

stage 2 dryrun on rep-prod

nohup ./rundoidopublic.sh -dryrun &

Running dryrun test...1110 ./listofpids objects
1110 retrieved and DOIs successfully parsed...
0 have bad DOIs that cannot be managed...
0 objects do not yet have DOIs...
Log file is "testrundoilog-170321-133948"

report attached

#10

Hi Dave,

It looks good and ready for the realrun.

Thanks,

Jeffery

#11

Real run on rep-prod

nohup ./rundoidopublic.sh &

Running rundoi for real...1110 ./listofpids objects
1110 metadata successfully updated...
1110 retrieved and DOIs successfully changed to public...
0 have bad DOIs that cannot be managed...
0 objects do not yet have DOIs...
Log file is "rundoilog-170321-233057"

Full report is attached.

#12

Assigned to:dhoover» ananthan
Status:active» test

Dave,

According to my quick tests, it looks like everything worked. I think this is ready for testing. I'll reassign it to Kalaivani (though Rhonda may want to check it also).

Thanks!

Jeffery

#13

On all of the several records that I checked, the DOIs resolved successfully. Thank you to Jeffery and Dave for fixing the metadata on the records in this collection!

#14

Status:test» closed

Kalaivani says I can close this now.

Back to top