Test and code refactoring
The refactoring is still far from perfect, but these are some first steps.
- removed lots of code duplication
- created
helpers
package intests
- fixed some small bugs wrt. harvesters
- removed some useless dead code (and corresponding tests)
- removed lots of dumping and print commands polluting the test output
- improved some structures in the code
Notable changes:
- pydantic models for technical/provenance metadata (instead of plain dicts)
- instead of the
tests/run_all_*.sh
files that launch tests and must be run fromtests
, now tests can be run as is normal from the root directory, the old scripts are replaced withpoe
, i.e.poetry run poe test
Remaining problems (need discussion / support from @j.broeder):
-
fix sitemap FZJ test (some URLs are apparently not mapped) -
discuss the technical metadata modelling
Edited by Anton Pirogov