Skip to content

Datacite harvester failes if not a json is returned, probably empty or denied request

Currently the datacite harvester can fail, if the graph query fails.

  File "/tmp/tmpe3iqk2x0prefect/data_harvesting/harvester/datacite.py", line 153, in query_graphql_api
    json_data = json.loads(req.text)
  File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Wanted behavior: Not clear, either fail exactly like it did, or catch, log and not recreate a new last_run file

Since the datacite harvester deals with several sources, it should not fail in such a case, but log the failure and not update the last_run file of the source.