Skip to content
Snippets Groups Projects
Commit ca76d30d authored by Paul Millar's avatar Paul Millar
Browse files

Update facility data OAI-PMH metadata to record information as items

Motivation:

The current OAI-PMH information is recorded as `datasets`.  However,
this assumes that the items underlying the harvested OAI-PMH records
correspond to datasets.  This is not guaranteed, and there are
counter-examples.

OAI-PMH describes three concepts: resource, item and record.  The
OAI-PMH responses provide records (descriptive metadata of some item) or
identifiers thereof.  However, since OAI-PMH requires repositories to
support Dublin Core records, it seems a reasonable assumption for there
to be a 1:1 relationship between each Dublin Core record and some
corresponding item.

Therefore, we can use the metadata-agnostic `item` concept when
describing the information about the endpoint.

Modification:

Update script to record information under `items` node in the facilities
YAML file.

Update the Jekyll to consume this information when rendering the
corresponding HTML.

Result:

No observable change, but the facilities YAML file now uses the more
neutral 'items' instead of 'datasets'.
parent 2a6ce96e
Branches joss
No related tags found
1 merge request!64update_oai-pmh Add a back-off strategy, to allow a service to recover
...@@ -31,7 +31,7 @@ ...@@ -31,7 +31,7 @@
status: Active status: Active
adminAddress: adminAddress:
- mis@cells.es - mis@cells.es
datasets: items:
count: 20 count: 20
sets: sets:
count: 2 count: 2
...@@ -107,7 +107,7 @@ ...@@ -107,7 +107,7 @@
link: https://data.ceric-eric.eu/oaipmh/request link: https://data.ceric-eric.eu/oaipmh/request
last-check: 2024-12-22 last-check: 2024-12-22
status: Active status: Active
datasets: items:
count: 7 count: 7
sets: sets:
count: 1 count: 1
...@@ -152,7 +152,7 @@ ...@@ -152,7 +152,7 @@
status: Active status: Active
adminAddress: adminAddress:
- datapolicy@esrf.fr - datapolicy@esrf.fr
datasets: items:
count: 7202 count: 7202
pan-search-api: pan-search-api:
link: https://icatplus.esrf.fr/api link: https://icatplus.esrf.fr/api
...@@ -179,7 +179,7 @@ ...@@ -179,7 +179,7 @@
link: https://oai.panosc.ess.eu/openaire/oai link: https://oai.panosc.ess.eu/openaire/oai
last-check: 2024-12-22 last-check: 2024-12-22
status: Active status: Active
datasets: items:
count: 0 count: 0
adminAddress: adminAddress:
- max.novelli@ess.eu - max.novelli@ess.eu
...@@ -216,7 +216,7 @@ ...@@ -216,7 +216,7 @@
adminAddress: adminAddress:
- luis.maia@xfel.eu - luis.maia@xfel.eu
- krzysztof.wrona@xfel.eu - krzysztof.wrona@xfel.eu
datasets: items:
count: 5 count: 5
sets: sets:
count: 1 count: 1
...@@ -257,7 +257,7 @@ ...@@ -257,7 +257,7 @@
status: Active status: Active
adminAddress: adminAddress:
- icatmaster@helmholtz-berlin.de - icatmaster@helmholtz-berlin.de
datasets: items:
count: 28952 count: 28952
sets: sets:
count: 3 count: 3
...@@ -299,7 +299,7 @@ ...@@ -299,7 +299,7 @@
last-check: 2024-12-22 last-check: 2024-12-22
adminAddress: adminAddress:
- rodare-admin@hzdr.de - rodare-admin@hzdr.de
datasets: items:
count: 1020 count: 1020
sets: sets:
count: 39 count: 39
...@@ -453,7 +453,7 @@ ...@@ -453,7 +453,7 @@
last-check: 2024-12-22 last-check: 2024-12-22
adminAddress: adminAddress:
- data@ill.fr - data@ill.fr
datasets: items:
count: 0 count: 0
pan-search-api: pan-search-api:
link: https://fairdata.ill.fr/fairdata/api link: https://fairdata.ill.fr/fairdata/api
...@@ -541,7 +541,7 @@ ...@@ -541,7 +541,7 @@
last-check: 2024-12-22 last-check: 2024-12-22
adminAddress: adminAddress:
- carlo.minotti@psi.ch - carlo.minotti@psi.ch
datasets: items:
count: 0 count: 0
pan-search-api: pan-search-api:
link: https://dacat.psi.ch/panosc-api/ link: https://dacat.psi.ch/panosc-api/
......
...@@ -58,18 +58,18 @@ title: Open data resources ...@@ -58,18 +58,18 @@ title: Open data resources
<div class="tooltip-cont"> <div class="tooltip-cont">
<div class="trigger-tooltip">Endpoint:&nbsp;<a href="{{facility.odr.oai-pmh-endpoint.link}}">link</a> <div class="trigger-tooltip">Endpoint:&nbsp;<a href="{{facility.odr.oai-pmh-endpoint.link}}">link</a>
[<a href="{{facility.odr.oai-pmh-endpoint.link}}?verb=Identify">Identify</a>]<br/>{% [<a href="{{facility.odr.oai-pmh-endpoint.link}}?verb=Identify">Identify</a>]<br/>{%
if facility.odr.oai-pmh-endpoint.datasets if facility.odr.oai-pmh-endpoint.items
%}Datasets:&nbsp;<b>{{facility.odr.oai-pmh-endpoint.datasets.count | thousands_separated}}</b>{% %}Datasets:&nbsp;<b>{{facility.odr.oai-pmh-endpoint.items.count | thousands_separated}}</b>{%
assign total_oai_pmh_dataset_count = total_oai_pmh_dataset_count | plus: facility.odr.pan-search-api.dataset_count %}{% assign total_oai_pmh_dataset_count = total_oai_pmh_dataset_count | plus: facility.odr.pan-search-api.dataset_count %}{%
if facility.odr.oai-pmh-endpoint.datasets.sets%} if facility.odr.oai-pmh-endpoint.items.sets%}
<br/> <br/>
<span class="caret">Categories:&nbsp;{{facility.odr.oai-pmh-endpoint.datasets.sets.count | thousands_separated}}</span> <span class="caret">Categories:&nbsp;{{facility.odr.oai-pmh-endpoint.items.sets.count | thousands_separated}}</span>
<div class="toggled"> <div class="toggled">
<table> <table>
<tr> <tr>
<th>Category</th><th>Datasets</th> <th>Category</th><th>Datasets</th>
</tr> </tr>
{% for set in facility.odr.oai-pmh-endpoint.datasets.sets.details %} {% for set in facility.odr.oai-pmh-endpoint.items.sets.details %}
<tr> <tr>
<td>{{set[1].name}}</td><td>{{set[1].count | thousands_separated}}</td> <td>{{set[1].name}}</td><td>{{set[1].count | thousands_separated}}</td>
</tr> </tr>
......
...@@ -393,16 +393,16 @@ facilities.each do |facility| ...@@ -393,16 +393,16 @@ facilities.each do |facility|
oai_pmh['adminAddress'] = adminAddress oai_pmh['adminAddress'] = adminAddress
end end
oai_pmh.delete('datasets') oai_pmh.delete('items')
if status == "Active" if status == "Active"
datasets = {} items = {}
oai_pmh['datasets'] = datasets oai_pmh['items'] = items
datasets['count'] = total_count items['count'] = total_count
if !set_count.empty? if !set_count.empty?
sets = {} sets = {}
datasets['sets'] = sets items['sets'] = sets
sets['count'] = set_count.length() sets['count'] = set_count.length()
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment