All Pantone colors or Pantone and CMYK colors?
I think as long as your colors are designated Pantone colors they should appear fine in the PDF/X document.
I applied Pantone colors to three objects including text and exported as PDF/X-4
In Acrobat > Tools > Ink Manager the three spot colors appear. The presence of CMYK colors are indicated too but there is nothing in the document that is CMYK. And a printer printing only Pantone colors would just not process the CMYK colors.
Bookmarks