I have two texture2d , one of these is a mask texture and have 2kind color and i use that for mask (filter) second texture2D something like

     float4 tex = tex2D(sprite, texCoord);
float4 bitMask = tex2D(mask, texCoord);
if (bitMask.a >0)
{
 return float4(0,0,0,0);
}
else
{
      return float4(tex.b,tex.g,tex.r,1);
}

but because mask texture is just two color the result is too jagged i want know how i can do some antialiasing for edges that smooth these ty for reading and sry for my bad english