Page 1 of 1

Anti Aliasing in Post

Posted: 13 Jul 2018, 22:07
by oldmandarin
Hey all!

Does anyone know why applying AA in post (or using the effects panel for AA) after drawing is the standard for Japanese productions? Does it have something to do with using colored lines?

Thanks!

Re: Anti Aliasing in Post

Posted: 14 Jul 2018, 15:00
by schwarzgrau
I'm not really sure, but I guess it's easier to colorize if you don't have semi-transparent edges.

Re: Anti Aliasing in Post

Posted: 16 Jul 2018, 14:17
by Jet
I think it was mainly so the frames could be stored at a high resolution, if I remember right Studio Ghibli stored them as 5K, 150DPI, TIFF using CCIT Group 4 compression (aliased). Coloring techniques were then made around that workflow, HDD's weren't as cheap back in the day like they are today.

Re: Anti Aliasing in Post

Posted: 17 Jul 2018, 02:11
by Hironori Takagi
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg

Re: Anti Aliasing in Post

Posted: 23 Jul 2018, 21:08
by oldmandarin
Hironori Takagi wrote:
17 Jul 2018, 02:11
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg
This makes so much sense! I've also just discovered the OLM smoothing plugins for AE, so I'm going to try that with this method as well. Thank you so much (and to everyone else who replied) !

Also, follow up question, how do you colorize your blue and red lines? Or do you just color to the line, and turn the layer off?

Re: Anti Aliasing in Post

Posted: 24 Jul 2018, 06:45
by melanie
You can use the new FX Line Colorize Enhanced to color your lines:

https://youtu.be/M-6mnirDBbc



The full tutorial will be out very soon!

Re: Anti Aliasing in Post

Posted: 24 Jul 2018, 17:30
by oldmandarin
I spent a decent amount of time messing with the Line Colorize Enhanced, and unfortunately can't get it to work. At best, the blue only partially disappears. I have the FX applied to the colored line layer as well.
LCE ex.jpg

Re: Anti Aliasing in Post

Posted: 25 Jul 2018, 07:06
by melanie
Is your colorization layer under your line layer?
Is your blue line completely opaque? If not, increase the value of the range.

Re: Anti Aliasing in Post

Posted: 25 Jul 2018, 22:16
by oldmandarin
Yes, the line layer is above, and the opacity is 100%. I tried boosting the range all the way up to 255, and still no change :/

I can send you the working file if possible!

Re: Anti Aliasing in Post

Posted: 26 Jul 2018, 06:44
by melanie
Of course, you can send me your file in a Private Message :)

Re: Anti Aliasing in Post

Posted: 30 Jul 2018, 07:18
by Hironori Takagi
oldmandarin wrote:
23 Jul 2018, 21:08
Hironori Takagi wrote:
17 Jul 2018, 02:11
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg
This makes so much sense! I've also just discovered the OLM smoothing plugins for AE, so I'm going to try that with this method as well. Thank you so much (and to everyone else who replied) !

Also, follow up question, how do you colorize your blue and red lines? Or do you just color to the line, and turn the layer off?
Hello, oldmandarin.
Thank you for your feedback.

In Japan's way of making, use "paint bucket" and "filled stroke" for images with anti-aliasing turned off.
I will introduce the video of Youtube although it is explanation of Japanese.
Although FX Line Colorize Enhanced is effective, there are aspects that have not been able to respond to the special needs of Japan yet.

Re: Anti Aliasing in Post

Posted: 30 Jul 2018, 12:19
by Jet
I miss two tools from Retas PaintMan.

Color Drag:
Image

This tool is similar to a color smudge tool, it picks up whatever color you click on and allows you to drag into only unfilled areas. So if there is many colors with unfilled gaps it allows quick color dragging without eyedropping. It's adjustable, you can choose gap size etc.

Fill Unfilled Area Brush:
Image

It's a brush that paints a transparent green mark, when you let go it disappears and fills any small gaps you painted over, it's adjustable, you can pick the size of unfilled area to be filled, size of the brush head, etc and it also allows you to include the red, green, blue color trace lines as if they were unfilled areas, so if you draw highlights using red lines they can be changed easily to a highlight color.

This works similar to shape fill in TVP but a brush design is faster to use, like a marquee shape fill brush I guess.

Re: Anti Aliasing in Post

Posted: 31 Jul 2018, 21:43
by oldmandarin
Hironori Takagi wrote:
30 Jul 2018, 07:18
oldmandarin wrote:
23 Jul 2018, 21:08
Hironori Takagi wrote:
17 Jul 2018, 02:11
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg
This makes so much sense! I've also just discovered the OLM smoothing plugins for AE, so I'm going to try that with this method as well. Thank you so much (and to everyone else who replied) !

Also, follow up question, how do you colorize your blue and red lines? Or do you just color to the line, and turn the layer off?
Hello, oldmandarin.
Thank you for your feedback.

In Japan's way of making, use "paint bucket" and "filled stroke" for images with anti-aliasing turned off.
I will introduce the video of Youtube although it is explanation of Japanese.
Although FX Line Colorize Enhanced is effective, there are aspects that have not been able to respond to the special needs of Japan yet.
Thank you so much, this is supremely helpful!

Re: Anti Aliasing in Post

Posted: 31 Jul 2018, 21:44
by oldmandarin
Jet wrote:
30 Jul 2018, 12:19
I miss two tools from Retas PaintMan.

Color Drag:
Image

This tool is similar to a color smudge tool, it picks up whatever color you click on and allows you to drag into only unfilled areas. So if there is many colors with unfilled gaps it allows quick color dragging without eyedropping. It's adjustable, you can choose gap size etc.

Fill Unfilled Area Brush:
Image

It's a brush that paints a transparent green mark, when you let go it disappears and fills any small gaps you painted over, it's adjustable, you can pick the size of unfilled area to be filled, size of the brush head, etc and it also allows you to include the red, green, blue color trace lines as if they were unfilled areas, so if you draw highlights using red lines they can be changed easily to a highlight color.

This works similar to shape fill in TVP but a brush design is faster to use, like a marquee shape fill brush I guess.
Woah, this is really cool. I would second this for TVP!

Re: Anti Aliasing in Post

Posted: 06 Jan 2019, 06:36
by Nexxorcist
I just wanted to say this is AMAZING. I came on to look for an OLM Smoother type of thing in TV Paint, cause I was so used to coloring characters with anti-alias off in Photoshop, and boom, Anti-alias in the FX Panel with awesome directions from Hironori Takagi.

This program really is the gift that keeps on giving. Arigato! :D