I think, I'm going to choose a very pragmatic way and use -sample for all gif manipulations having a width < 17 px and leave the other transformations untouched

Maybe I can remove this workaround one day if this behaviour is fixed in newer IM versions.