Post any defects you find in the released or beta versions of the ImageMagick software here. Include the ImageMagick version, OS, and any command-line required to reproduce the problem. Got a patch for a bug? Post it here.
I am wondering if correct behavior would be as follows:
1) If no size is specified in -geometry, only offsets, then keep each image to its proper size as in my last example above. But allow spacing via -geometry offsets to be permitted without padding to have all images be the same size.
2) Use size 0x0 in -geometry to mean to pad the images to the same size as the max image, but not resize. Then permit -gravity to align them in the frame as desired.
3) When non-zero size is specified, then resize to that size as is currently done.
I do not know if this is practical or not. But it could be a possible enhancement for the future.