Deleted Configuration stays in search list

When I open the search list, I get a list of configurations:

When I delete the abc configuration here, I got the message that the deletion was successful, but the configuration is still visible.

I would expect that this entry would be removed from the list.