Vis Mask in Painter: no you're not going crazy.

[Thelonious]

Many people don't use the Visibility Mask in Painter. To them it is just an extra complex step that gets in the way of painting.

The same can be said for a lot of Painter's extras. Like the mosaic for example or the curves palette. Some folks just want to paint, and Painter is mighty good at it too.

Eventually everyone gets around to exploring the other stuff. And if you ever get round to the Visibility Mask, you can be in for a nasty surprise. You see the principal seems simple enough, there is a Vis Mask (not to be confused with the User Mask) which controls what parts of a layer is visible. That's what it says in the book.

However if you try to erase pixels on an unfilled layer using the Vis Mask. It seems to do what it is supposed to. However if you try and make the pixels visible again you will get nasty black artifacts in your stroke and also around the edges of your stroke.

You might wonder what you are doing wrong. But fear not you are not doing anything wrong. As crazy as it may seem (especially if you are used to a layer mask such as the one in Photoshop) this is how it works in Painter. Some people have come up with "work arounds" like selecting before erasing but this is not very effective for a complex stroke although it does help a bit.

The sad fact is you are not going crazy, it's just another badly engineered part of Painter. If you don't use Vis Masks then it wont bother you. However if you are a Photoshop user and think that the Vis Mask is a layer mask you will have a tough time. Actually I don't know what the purpose of the Vis Mask actually is. It is just a half baked Layer Mask.

Below are some samples: the green stroke has been made invisisble with the Vis Mask then it has been restored but as you can see the problems are obvious.

In case you are wondering Corel have decided not to do anything about this at all it P7, in fact they have indicated to me that it is only a possibility that they will even think about it for P8.

This post is just to reassure the unwary that it is not you who is doing something wrong. In the meantime you'll have to do your layer masking in Photoshop.

thelonious

[Thelonious]

Many people don't use the Visibility Mask in Painter. To them it is just an extra complex step that gets in the way of painting.

The same can be said for a lot of Painter's extras. Like the mosaic for example or the curves palette. Some folks just want to paint, and Painter is mighty good at it too.

Eventually everyone gets around to exploring the other stuff. And if you ever get round to the Visibility Mask, you can be in for a nasty surprise. You see the principal seems simple enough, there is a Vis Mask (not to be confused with the User Mask) which controls what parts of a layer is visible. That's what it says in the book.

However if you try to erase pixels on an unfilled layer using the Vis Mask. It seems to do what it is supposed to. However if you try and make the pixels visible again you will get nasty black artifacts in your stroke and also around the edges of your stroke.

You might wonder what you are doing wrong. But fear not you are not doing anything wrong. As crazy as it may seem (especially if you are used to a layer mask such as the one in Photoshop) this is how it works in Painter. Some people have come up with "work arounds" like selecting before erasing but this is not very effective for a complex stroke although it does help a bit.

The sad fact is you are not going crazy, it's just another badly engineered part of Painter. If you don't use Vis Masks then it wont bother you. However if you are a Photoshop user and think that the Vis Mask is a layer mask you will have a tough time. Actually I don't know what the purpose of the Vis Mask actually is. It is just a half baked Layer Mask.

Below are some samples: the green stroke has been made invisisble with the Vis Mask then it has been restored but as you can see the problems are obvious.

In case you are wondering Corel have decided not to do anything about this at all it P7, in fact they have indicated to me that it is only a possibility that they will even think about it for P8.

This post is just to reassure the unwary that it is not you who is doing something wrong. In the meantime you'll have to do your layer masking in Photoshop.

thelonious

[Wayne D]

Hi Thelonius!

I was reading along and suddenly realized thatI didn't know what in the worl you were taking about. But that doesn't surprise me, I'm still struggling with floater/layers/whatever. I did learn how to use the stroke selection thinga ma bob.

I guess Corel is taking the rap for something they got from Meta/Fractal people. I wonder what Zimmer and Derry were doing while they were consulting with Corel? Maybe they will leave soon and design a new package. Who knows.

I still get jazzed about what Painter5.5 can do let alone P7.

Still I hear lots a bad news overshadowing the good news. Most new product get real beta testing upon release to the consumer.
I remember the Pong video game when it came out in the late 60's. I watched three guys lose their jobs in SIlican Valley one Friday afternoon after lunch. They got to playing Pong and forgot to go back to work. And these weren't young guys either. They blew off end of the month shipments cuz' they were captured by technology.

Ah well -- enough of this rambling on -- off subject that is!

Wayne D
Arizona

[eky]

Though I've not yet tried using this feature it's good to be informed about problems others have experienced with a program. When I do find a problem I also like to be able to go to messageboards such as on talkgraphics and do a search for the problem and possible solutions.

I don't agree that Corel is taking a "rap" for a problem that existed in a program they purchased from metacreations. It's a bug that Corel was informed of and choose not to fix in their latest version. That was Corel's decision. It's strange when you think about it that we have come to accept (or live with) the fact that a lot of software will not always work the way we want or that when there is a major bug that it probably won't be fixed until a new version comes out that we have to pay for when we would not accept that kind of quality (or lack thereof) in any other product or service. Imagine if your car or television won't operate 10% of the time.

I appreciate reading about problems as well as successes with software as it keeps me well informed.

[freebird]

I once tried masking in Painter and after a long time of trying to figure out what the heck was going on...and a few expletives also [img]/infopop/emoticons/icon_smile.gif[/img] ...I saved it as a psd and just took it into Photoshop.

I am having a hard time acclamating to layers and masks in Painter as compared to Photoshop and so much appreciate your post and explanations.

[freebird]

Don't know if I am doing something wrong or if this is a painter bug but when I run filters on certain layers it also messes up pixels near the egde on other layers.
I was using some of the blur filters on lower layers and they worked fine on that layer but also kept messing up edge pixels on the upper layers.
I remade the file again from scratch and tried selecting the area on the layer before blurring it and it still messed up edge pixels on other layers. I even tried again by hiding the uppermost layers from view while I blurred the lower layer but it still continued to effect pixels on other layers. Not only did it blur edge pixels on other layers but it changed their color and I ended up taking it into Photoshop to repair the areas along the left side and bottom with the rubber stamp tool there.

[Thelonious]

Wayne, Eky, Freebird, thanks I've taken a lot of flak over this issue including being thrown off the Painter List which prides itself on being a serious forum but is in fact the worst forum I've ever seen anywhere.

Wayne, lol, you got lost. Don't worry I only gave a very rough description of the problem as I wanted to get it up on the forum. It would only be intelligible to users who have some knowledge of the Vis Mask. But fear not I will make a little explanation/tutorial soon.

The full explanation is a little mindbending and actually a rather fascinanting journey. For me this problem started way back when I did the "Flashbak" logo, on the wood one I tried to make one of the rings go behind and I wanted to use the Vis Mask to do this...when I erased too much and restored a bit I found that black pixels got into my layer.

I was new to Painter and I spent many days trying to figure out what I was doing wrong. I couldn't understand I tried different states on the Vis Mask, eg invert etc. I thought it might have something to do with preserve transparency... it went on and on. It was a chance discovery that made me realise the sad truth. This will astound you. The logic is unrefutable. Unfortunately Corel seems to have a bunch of cronies on forums all over the place who defend Painter to the death, these people cannot follow a reasoned argument and will brook no complaint against Corel.

It can be a confusing issue to talk about because people get their terms mixed up. Photoshop has a Layer Mask, also Alpha Channels which are used to store selections.

Painter has a Visibility Mask which at first glance appears to be like a Photoshop layer mask, in fact most people refer to the Painter vis mask as a layer mask, We will not do this as it leads to the most confusion. But as we shall soon see the Painter Vis mask is a strange beast that has gotten loose. Painter also has a User Mask, this is exactly like a Photoshop alpha channel and is used for the same purpose ie storing selections.

When discussing this issue people who can't follow the logic and want to defend Painter by
confusing the issue purposely mix up all these terms, so instead of discussing the issue I have to end up correcting mistakes, then I get a bunch of insults, then someone like Jinny Brown will write in saying that there is notthing wrong at all. This has been the usual pattern and indeed it has been repeated yet again recently on the Corel news group. Thank's Jinny do Corel pay you? They should.

Anyway back to the fun bit. The confusion comes when you think that the Vis Mask is analogous to a Layer Mask. it isn't. In fact Photoshop has a vis mask as well and it is this that the Painter Vis Mask is emulating.

Pause

What, do I hear you say that Photoshop doesn't have a vis mask? Well to be sure it does have one but it doesn't let the user see it or to have access to it. How can I say this. I know sounds mad doesn't it. Well I discovered it by accident and when I did suddenly it all fell into place. Because Painter will let you see Photoshops vis mask.

Do this, open a new file in Photoshop. Make new layer then paint a red stroke. Now get the eraser and rub out the middle of the stroke. Now save it in PSD. Close and reopen. Now there no way to restore the middle of your erased stroke is there? Not in Photoshop. You see the stroke has not been erased it has been covered with a vis mask. But you wouldn't know this because Photoshop won't let you see it or use it. It doesn't need to because if that's what you wanted to do you would have used the layer mask that it provides for you wouldn't you. But for whatever reason Photoshop keeps all your information about the original stroke somewhere in the depths of its viscera.

Close the Photoshop file with the stroke, now open it in Painter, highlight the layer with the stroke. you will note that it has a corresponding Vis Mask (BTW a Vis Mask is distinguished from a User Mask by the fact that it is italicised) Now highlight the Vis Mask in the Masks subpalette. Now you have TWO layers highlighted, "Layer One" in the Layers section and "Layer One Mask"in the asks section. Now leaving the mask highlighted turn of the EYE only on the Layer One Mask.

Now if you select a brush or pen and paint with black on you canvas where the missing stroke is you will restore your stroke. Because what you are in fact doing is you are painting on the Vis Mask, you can see the Vis Mask by turning it's eye on.

So there is the proof that Photoshop never actually got rid of the bit of the stoke that you erased.

What does this all mean? It means this, Metacreations knew they were going to sell Painter and they were in a hurry to develop P6 so instead of building proper layer mask they merely gave the user access to the Vis Mask that should in fact be hidden. Because it CANNOT work that the software and the user both paint on the same mask.

Corel didn't notice, they were done like kippers. However now that they have released P7 and Tanya Staples from Corel tells me that they haven't done anything about it in P7 it is fully Corel's baby now. This is not a little annoying but it is a cock up of major proportions and it treats users like mugs.

I'm late for work now gotta go, hope this gives you something to think about.

thelonious

PS here's a little message to JB. after all the crap I've had to endure from you on this topic over the months I wonder if you would be so sad as to still read this post to the end, my bet is you would. wouldn't you. Can't answer this can you without giving it away. Ha Ha so you just have to read it and fume. The truth will out.

[This message was edited by Thelonious Hink on August 21, 2001 at 00:54.]

[Thelonious]

Try this open the guitar file in the sample section of photoshop. goodies. you can bring this into Painter and restore the background.

Also you can use the extract command in Photoshop and also bring it into Painter and use the Vis Mask to see how Photoshop does the processing around the edge. It is quite interesting.

Thelonious

[eky]

Thelonious,

I actually had to draw a diagram to follow your explanation and it then I could follow all the terms. Now that I think I understand the problem, I will have to open PS6 and Painter6 to try it out. But I still don't know how Painter's vis mask is supposed to work. Is it supposed to function just like Photoshop's layer mask?

[Thelonious]

Hi Eky,

You can imagine how frustrating it is then to have a discussion on this when someone with an interest to disrupt the thread start using all the terms incorrectly. The terms I use are the "official" ones. ie User Mask in Painter, that's what they call it in the manual. This is the same as the Alpha Channel in PS.

It is very important not to call the Painter Vis Mask a layer mask. As Painter call it a Vis Mask. The reason for this is because if you call it a layer mask then the tendency is to get it mixed up with the PS layer mask.

If you have a layer in Painter that is completely filled with image like a photograph for example or even a painting if the layer has been filled first. Then in that circumstance the Painter Vis Mask will work in the same way as a Photoshop layer mask, no problem. Apart from the fact that in PS black makes invisible; white makes visible; and in Painter this is reversed.

This goes to prove that the Vis Mask is meant to act like a layer mask. It is when a layer is not filled with image that the Vis Mask doesn't work properly, it erases OK but it won't restore without damaging the layer.

The reason that it is so cockeyed is that in order to be able to highlight the Vis Mask you first have to highlight its corresponding layer first. Otherwise the Vis Mask will not show up in the mask palette. You can deselect the Vis Mask by selecting RGB image in the Mask Palette. But you can't deselect the layer that the Vis Mask belongs to and leave the Vis Mask highlighted because when you deselect the layer the corresponding Vis Mask disappears off the palette. This is why black appears on your layer when you paint on the Vis Mask and you go past the edge. The poor program has Both the Layer highlighted and the Vis Mask highlighted at the same time. So where does the paint go?

This is the sort of problem that Metacreations created when they make the Vis Mask user accessible. Unlike PS who kept it hidden and provided the user with their very own vis mask which they call a Layer Mask.

There is one more very frustrating thing about Painter that can be a real pain and that is Painter will allow you to paint on a layer even if it is invisible ie the eye is turned off. So it is possible to accidentally paint on a layer without realising until it is too late. Be careful on this one.

There is only one problem with doing layer masking in PS from a Painter file and that is if your image is in rif format for example if you have dynamic layers or wet layers and you want to bring it into PS you have to save into PSD which will set all dynamic layers.

Tanya Staples from Corel is trying to say that Corel never meant the Vis Mask to be like a proper layer mask. However this is rubbish. for two reasons one is that it was a metacreations idea in the first place so Corel can't speak for Metacreations. But more importantly they cannot possibly say that it was their intention that painting on the Vis Mask can completely destroy the image. I mean come off it.

If I complain that the scripts don't work properly are they going to say, "yeah we meant that too"? Corel has beta testers who do not even understand the difference between a User Mask and a Vis Mask. And Corel will not discuss this issue at all. In my opinion Corel have proven themselves to be fools. I beleive that Metacreations only provided the Vis Mask because they wanted to rush out P6 and didn't have time to engineer a proper layer mask. They probably thought that anyone serious would just do it in PS. Which I do. But it is an unnecessary pain. I'm sure that if Metacreations kept Painter that we would have working layer mask by now.

So remember boys and girls if the layer is filled with wall to wall image then the Vis Mask is OK. I hope this has made a very cocked up situation a bit more clear.

I remember way back in January someone on the Corel news group was asking for help about this black on the layer business. He even sent in a diagram in ascii code. I really felt sorry for him as he just met with total denial from the same morons who still deny that a problem exists.

The main problem now is Corel incredible arrogance. I see no reason at all to upgrade to P7. I don't see how Corel can move onto P7 before they release patches to fix P6. Like the scripts don't work very well in P6 everyone knows this. But what I don't know is are they fixed in P7. And even if they are why should I have to pay for a bug fix. I should only have to pay for new functionality. So it seems Corel have abandoned P6 and to add insult to injury we don't even know if all the P6 bugs have been fixed.

I'm happy to discuss this issue in however much detail is necessary. Usually by this stage I would have some complete idiot throw a spanner in the works. But she only enters the thread when she can detect that a moderator or someone in power is having trouble following because they only pretend to understand, then they insult me together. However it is more difficult to throw a hand grenade into a thread such as this as opposed to a newsgroup thread.

Regards

Thelonious

[This message was edited by Thelonious Hink on August 21, 2001 at 14:47.]