1) Drag a colour to the page background - the Page Background layer appears in the Object gallery.
2) Ensure that Page Background is the currently selected layer.
3) Draw a Quickshape (you will have to click OK in the dialog box)...
4) ...and now try to delete it!

You can't since the Page Background layer is unselectable.
Nor can you drag it to another layer in the Object Gallery.
The only ways to get shot of it are to delete the entrie Page Background layer, or Undo to a point before the shape's creation.

Small bug, but annoying none the less!