Anti Aliasing in Post Topic is solved

Please use this part to report bugs & errors, ask questions & "How to..."
Post Reply
oldmandarin
Posts: 6
Joined: 12 Jan 2017, 04:41

Anti Aliasing in Post

Post by oldmandarin » 13 Jul 2018, 22:07

Hey all!

Does anyone know why applying AA in post (or using the effects panel for AA) after drawing is the standard for Japanese productions? Does it have something to do with using colored lines?

Thanks!

User avatar
schwarzgrau
Posts: 1037
Joined: 23 Jan 2012, 22:08
Location: Offenbach / Germany
Contact:

Re: Anti Aliasing in Post

Post by schwarzgrau » 14 Jul 2018, 15:00

I'm not really sure, but I guess it's easier to colorize if you don't have semi-transparent edges.
Mac OS X 10.11.6 / TVP 11.0.8 PRO / Cintiq 22HD
Mac OS X 10.11.6 / TVP 11.0.8 PRO / Cintiq Companion 2
SCHWARZGRAU.COM

Jet
Posts: 40
Joined: 31 Jul 2015, 10:48

Re: Anti Aliasing in Post

Post by Jet » 16 Jul 2018, 14:17

I think it was mainly so the frames could be stored at a high resolution, if I remember right Studio Ghibli stored them as 5K, 150DPI, TIFF using CCIT Group 4 compression (aliased). Coloring techniques were then made around that workflow, HDD's weren't as cheap back in the day like they are today.
OS: Win 10 [x64] / CPU: 1950X Threadripper [4.0GHz] / RAM: 128GB / GPU: 2x GTX 1080Ti / Display: Cintiq 27QHD
TVP: 10.5.7 Pro [x64] / 11.0.8 Pro [x64]

Hironori Takagi
Posts: 6
Joined: 14 May 2018, 10:15
Location: Tokyo, Japan
Contact:

Re: Anti Aliasing in Post

Post by Hironori Takagi » 17 Jul 2018, 02:11

Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg

oldmandarin
Posts: 6
Joined: 12 Jan 2017, 04:41

Re: Anti Aliasing in Post

Post by oldmandarin » 23 Jul 2018, 21:08

Hironori Takagi wrote:
17 Jul 2018, 02:11
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg
This makes so much sense! I've also just discovered the OLM smoothing plugins for AE, so I'm going to try that with this method as well. Thank you so much (and to everyone else who replied) !

Also, follow up question, how do you colorize your blue and red lines? Or do you just color to the line, and turn the layer off?

User avatar
melanie
Site Admin
Posts: 313
Joined: 12 Sep 2017, 07:13

Re: Anti Aliasing in Post

Post by melanie » 24 Jul 2018, 06:45

You can use the new FX Line Colorize Enhanced to color your lines:

https://youtu.be/M-6mnirDBbc



The full tutorial will be out very soon!
Mélanie Do Monte • Sales & Support

oldmandarin
Posts: 6
Joined: 12 Jan 2017, 04:41

Re: Anti Aliasing in Post

Post by oldmandarin » 24 Jul 2018, 17:30

I spent a decent amount of time messing with the Line Colorize Enhanced, and unfortunately can't get it to work. At best, the blue only partially disappears. I have the FX applied to the colored line layer as well.
LCE ex.jpg

User avatar
melanie
Site Admin
Posts: 313
Joined: 12 Sep 2017, 07:13

Re: Anti Aliasing in Post

Post by melanie » 25 Jul 2018, 07:06

Is your colorization layer under your line layer?
Is your blue line completely opaque? If not, increase the value of the range.
Mélanie Do Monte • Sales & Support

oldmandarin
Posts: 6
Joined: 12 Jan 2017, 04:41

Re: Anti Aliasing in Post

Post by oldmandarin » 25 Jul 2018, 22:16

Yes, the line layer is above, and the opacity is 100%. I tried boosting the range all the way up to 255, and still no change :/

I can send you the working file if possible!

User avatar
melanie
Site Admin
Posts: 313
Joined: 12 Sep 2017, 07:13

Re: Anti Aliasing in Post

Post by melanie » 26 Jul 2018, 06:44

Of course, you can send me your file in a Private Message :)
Mélanie Do Monte • Sales & Support

Hironori Takagi
Posts: 6
Joined: 14 May 2018, 10:15
Location: Tokyo, Japan
Contact:

Re: Anti Aliasing in Post

Post by Hironori Takagi » 30 Jul 2018, 07:18

oldmandarin wrote:
23 Jul 2018, 21:08
Hironori Takagi wrote:
17 Jul 2018, 02:11
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg
This makes so much sense! I've also just discovered the OLM smoothing plugins for AE, so I'm going to try that with this method as well. Thank you so much (and to everyone else who replied) !

Also, follow up question, how do you colorize your blue and red lines? Or do you just color to the line, and turn the layer off?
Hello, oldmandarin.
Thank you for your feedback.

In Japan's way of making, use "paint bucket" and "filled stroke" for images with anti-aliasing turned off.
I will introduce the video of Youtube although it is explanation of Japanese.
Although FX Line Colorize Enhanced is effective, there are aspects that have not been able to respond to the special needs of Japan yet.

Jet
Posts: 40
Joined: 31 Jul 2015, 10:48

Re: Anti Aliasing in Post

Post by Jet » 30 Jul 2018, 12:19

I miss two tools from Retas PaintMan.

Color Drag:
Image

This tool is similar to a color smudge tool, it picks up whatever color you click on and allows you to drag into only unfilled areas. So if there is many colors with unfilled gaps it allows quick color dragging without eyedropping. It's adjustable, you can choose gap size etc.

Fill Unfilled Area Brush:
Image

It's a brush that paints a transparent green mark, when you let go it disappears and fills any small gaps you painted over, it's adjustable, you can pick the size of unfilled area to be filled, size of the brush head, etc and it also allows you to include the red, green, blue color trace lines as if they were unfilled areas, so if you draw highlights using red lines they can be changed easily to a highlight color.

This works similar to shape fill in TVP but a brush design is faster to use, like a marquee shape fill brush I guess.
OS: Win 10 [x64] / CPU: 1950X Threadripper [4.0GHz] / RAM: 128GB / GPU: 2x GTX 1080Ti / Display: Cintiq 27QHD
TVP: 10.5.7 Pro [x64] / 11.0.8 Pro [x64]

oldmandarin
Posts: 6
Joined: 12 Jan 2017, 04:41

Re: Anti Aliasing in Post

Post by oldmandarin » 31 Jul 2018, 21:43

Hironori Takagi wrote:
30 Jul 2018, 07:18
oldmandarin wrote:
23 Jul 2018, 21:08
Hironori Takagi wrote:
17 Jul 2018, 02:11
Hello, oldmandarin
I will explain the flow using not anti-aliased images in Japan below.
The big reason is that it is easy to paint without gaps.

I can not answer a detailed question because I am not good at English.
Thank you.
20180717.jpg
This makes so much sense! I've also just discovered the OLM smoothing plugins for AE, so I'm going to try that with this method as well. Thank you so much (and to everyone else who replied) !

Also, follow up question, how do you colorize your blue and red lines? Or do you just color to the line, and turn the layer off?
Hello, oldmandarin.
Thank you for your feedback.

In Japan's way of making, use "paint bucket" and "filled stroke" for images with anti-aliasing turned off.
I will introduce the video of Youtube although it is explanation of Japanese.
Although FX Line Colorize Enhanced is effective, there are aspects that have not been able to respond to the special needs of Japan yet.
Thank you so much, this is supremely helpful!

oldmandarin
Posts: 6
Joined: 12 Jan 2017, 04:41

Re: Anti Aliasing in Post

Post by oldmandarin » 31 Jul 2018, 21:44

Jet wrote:
30 Jul 2018, 12:19
I miss two tools from Retas PaintMan.

Color Drag:
Image

This tool is similar to a color smudge tool, it picks up whatever color you click on and allows you to drag into only unfilled areas. So if there is many colors with unfilled gaps it allows quick color dragging without eyedropping. It's adjustable, you can choose gap size etc.

Fill Unfilled Area Brush:
Image

It's a brush that paints a transparent green mark, when you let go it disappears and fills any small gaps you painted over, it's adjustable, you can pick the size of unfilled area to be filled, size of the brush head, etc and it also allows you to include the red, green, blue color trace lines as if they were unfilled areas, so if you draw highlights using red lines they can be changed easily to a highlight color.

This works similar to shape fill in TVP but a brush design is faster to use, like a marquee shape fill brush I guess.
Woah, this is really cool. I would second this for TVP!

Post Reply