Skip to content

Clean and unifiy interface of harvester run method.

Problem: Currently all harvesters need to implement a run method, which has no enforced interface. This does not allow to write code to run a generic harvester type, which would require a common interface between all harvester types:

Solution: Unify the interfaces of all harvesters and enforce the interface in the base harvester.

proposed interface:

    @abstractmethod
    def run(self, source: str, since: Optional[Union[datetime, str]]=None, base_savepath: Optional[Path] = None, **kwargs) -> None:
        """Run the harvester

        This method is required to be implemented for every harvester

        """
        raise NotImplementedError

where source is the name of the source from the config that should be harvested or 'all'. since is a timestamp, that can overwrite what the harvester read in as last run. and the bsae_savepath allows also to overwrite the default outputpath of the harvester. I think none of the harvesters so far needs the kwargs, but we could still allow these in case there are optional harvester specific items. @gabriel.preuss84 the interface is open to discussion of course.