Clean and unifiy interface of harvester run method.
Problem:
Currently all harvesters need to implement a run
method, which has no enforced interface.
This does not allow to write code to run a generic harvester type, which would require a common interface between all harvester types:
Solution: Unify the interfaces of all harvesters and enforce the interface in the base harvester.
proposed interface:
@abstractmethod
def run(self, source: str, since: Optional[Union[datetime, str]]=None, base_savepath: Optional[Path] = None, **kwargs) -> None:
"""Run the harvester
This method is required to be implemented for every harvester
"""
raise NotImplementedError
where source is the name of the source from the config that should be harvested or 'all'. since is a timestamp, that can overwrite what the harvester read in as last run. and the bsae_savepath allows also to overwrite the default outputpath of the harvester. I think none of the harvesters so far needs the kwargs, but we could still allow these in case there are optional harvester specific items. @gabriel.preuss84 the interface is open to discussion of course.