Do we really need `Parser.AbstractParser.check_max_elements` ?
While working on !10 (merged) , I stumbled across the need to implement the abstract parser method check_max_elements
.
It sort of feels redundant, to check data, retrieved from a file, which size was already confirmed to be smaller then a given threshold (in AbstractRawDataSource.check_max_file_size
), again.
As there are, at least to my knowledge, no such things as 'data bombs', where a given line of text expands to an enormous amount of data, I think we are save enough, if we only check for the size of a given raw data file and skip the element check.
So my proposal is to remove check_max_elements
from the AbstractParser
class.