Last time I used convert for such a task I explicitly specified the size of the destination via resizing:
$ convert a.png b.png -compress jpeg -resize 1240x1753 \
-units PixelsPerInch -density 150x150 multipage.pdf
Where 1240x1753 are exactly DIN A4 when 150 DPI is chosen. I computed the values using bc and looked up the dimensions in inches in the Wikipedia article. The resize argument specifies the maximal page size.
This assumes that convert by default does not change the aspect ratio with the resize operation - which is the case:
Resize will fit the image into the requested size.
It does NOT fill, the requested box size.
(ImageMagick manual)
Thus the -page a4 should be added, such that DIN A4 is specified in the PDF header:
$ convert a.png b.png -compress jpeg -resize 1240x1753 \
-units PixelsPerInch -density 150x150 \
-page a4 multipage.pdf
Update: Tested it again with another viewer, and it seems that one has to use -repage a4 instead of -page a4 to get the right page information into the resulting PDF.