It is because you have an image selected, and when you go to the bitmap gallery, your selecting the masking image too. The images need to match in size.

You could make a selection in PS, and save the selection. The saved selection is an alpha channel. You can use channels to look at the alpha channel, select the channel and paste back into a layer. That could give something with varaible transparency.

Rich