AA coloring issues

This section is dedicated to the feature & improvement requests (be sure what you are asking does not exist yet in TVPaint Animation ;) )
Post Reply
User avatar
Peter Wassink
Posts: 3781
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

AA coloring issues

Post by Peter Wassink » 24 Nov 2020, 18:57

Precise coloring with Anti-Alias can be problematic
with the paintbucket set to anti alias we get a 'balooning'effect on the fill

this also happens when exporting the C&T layer set to anti-alias


This Anti-Aliasing effect seems to me to be a bug, or a symptom of some non optimized algorythm.
It was reported earlier i believe, cant relocate that but ran into this issue again.

The fill tool with AA or the Col&Texture layer set to extract with AA, both give the same distortion of the filled shape,
making it look as if its blown up.

i would think that a perfect square should look the same with or without AA,
only when a line has an angle other then 0° 90° 270° or 360° should AA kick in?
to me it seems weird that the AA would cause a square to blow up like a balloon even growing outside of its fill boundaries.





Another thing i noticed with is that the fill border goes up to the border on top and Left of the fill.
But it goes over 1 pixel to right and bottom boundary?
Why is the fill asymmetrical?
You can see it in this image, where i put the layer with black boundaries at 30% opacity

colo&texture-AA-issue.png
colo&texture-AA-issue.png (16.04 KiB) Viewed 2532 times
Peter Wassink - 2D animator
• PC: Win10/64 Home - Xeon Quadcore E5410@2.33Ghz - 12Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM

User avatar
Xavier
Site Admin
Posts: 808
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier » 25 Nov 2020, 09:14

Yes, i think you reported it in the beta thread. Thanks anyway, it's better here :)
Short but strong

User avatar
Peter Wassink
Posts: 3781
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: AA coloring issues

Post by Peter Wassink » 25 Nov 2020, 09:36

yes we are currently struggling with this issue.
And when i revisited my post, i saw you suggested i put it here.

i mean this a-symmetrical filling.
we use the C&T and these edges cause us problems in compositing.
colo&texture-AA-issue2 .png
colo&texture-AA-issue2 .png (12.49 KiB) Viewed 2501 times
Last edited by Peter Wassink on 25 Nov 2020, 09:47, edited 4 times in total.
Peter Wassink - 2D animator
• PC: Win10/64 Home - Xeon Quadcore E5410@2.33Ghz - 12Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM

User avatar
Xavier
Site Admin
Posts: 808
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier » 25 Nov 2020, 09:39

The multiply blending mode is a pain to do. I also have an aliasing issue...
Short but strong

User avatar
Peter Wassink
Posts: 3781
Joined: 17 Feb 2006, 15:38
Location: Amsterdam
Contact:

Re: AA coloring issues

Post by Peter Wassink » 25 Nov 2020, 09:49

Xavier wrote:
25 Nov 2020, 09:39
The multiply blending mode is a pain to do. I also have an aliasing issue...
Yes i imagine Aliasing is hard to get right
i'm not sure what multiply has to do with this issue specificly, could you elaborate?
Peter Wassink - 2D animator
• PC: Win10/64 Home - Xeon Quadcore E5410@2.33Ghz - 12Gb RAM
• laptop: Win10/64 Pro - i7-4600@2.1 GHz - 16Gb RAM

User avatar
Xavier
Site Admin
Posts: 808
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier » 25 Nov 2020, 10:10

Peter Wassink wrote:
25 Nov 2020, 09:49
i'm not sure what multiply has to do with this issue specificly, could you elaborate?
Completely unrelated, just to let you know on which topic I am currently :)
Short but strong

User avatar
Xavier
Site Admin
Posts: 808
Joined: 01 Oct 2019, 11:08

Re: AA coloring issues

Post by Xavier » 25 Nov 2020, 10:11

Peter Wassink wrote:
25 Nov 2020, 09:36
yes we are currently struggling with this issue.
And when i revisited my post, i saw you suggested i put it here.

i mean this a-symmetrical filling.
we use the C&T and these edges cause us problems in compositing.
colo&texture-AA-issue2 .png
Thanks for the detail level, no worries I'll have a look.
Short but strong

User avatar
artfx
Posts: 656
Joined: 20 Sep 2006, 18:52
Location: Shanghai, China
Contact:

Re: AA coloring issues

Post by artfx » 21 Dec 2020, 22:19

I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
Terrence Walker
MacBook Pro (Retina, 13-inch, Early 2015)
OS X Catalina Version 10.15.7
2.7 GHz Intel Core i5, 8 GB 1867 MHz DDR3, Intel Iris Graphics 6100

User avatar
Hironori Takagi
Posts: 148
Joined: 14 May 2018, 10:15
Location: Tokyo, Japan
Contact:

Re: AA coloring issues

Post by Hironori Takagi » 24 Dec 2020, 09:37

artfx wrote:
21 Dec 2020, 22:19
I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.
(2) When compositing, use the color area as mask.
After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
TVPaint 11.5.0, Windows10 Pro, DELL Precision M3800 / TVPaint 11.5.0 Mac OS 11.0.1, Apple Mac Pro(Late 2013) & Mac OS 10.12.6, Apple MacBook Air(Mid 2011)

User avatar
artfx
Posts: 656
Joined: 20 Sep 2006, 18:52
Location: Shanghai, China
Contact:

Re: AA coloring issues

Post by artfx » 28 Dec 2020, 11:26

Hironori Takagi wrote:
24 Dec 2020, 09:37
artfx wrote:
21 Dec 2020, 22:19
I wonder if this is why I notice in videos about Japanese studios (that use TVPaint) that they use no antialiasing, on their lines or colors... none... nowhere.

Back in the day when we had a 2D team at the studio in Shanghai, I don't know what software they were using for ink and paint, but I was surprised to see the same thing. No antialiasing on anything.

I wonder what resolution they must work at to make that look good.
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.
(2) When compositing, use the color area as mask.
After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
Wow! Thanks for answering and explaining that. I didn't know that antialiasing could be added later. This is very good information.
Terrence Walker
MacBook Pro (Retina, 13-inch, Early 2015)
OS X Catalina Version 10.15.7
2.7 GHz Intel Core i5, 8 GB 1867 MHz DDR3, Intel Iris Graphics 6100

User avatar
D.T. Nethery
Posts: 2950
Joined: 27 Sep 2006, 19:19

Re: AA coloring issues

Post by D.T. Nethery » 28 Dec 2020, 20:31

Hironori Takagi wrote:
24 Dec 2020, 09:37
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.

(2) When compositing, use the color area as mask.

After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
.

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?

Animator, TVPaint Beta-Tester, Animation Educator and Consultant.
MAC OS 10.14.6 , Macbook Pro Retina 15-inch, Late 2013,
2.3 GHz Intel Core i7, 16 GB Memory , TVPaint PRO 11.5.1 - 64bit ,
Wacom Cintiq 21 UX ,Wacom Intuos Pro 5 , Wacom driver version 6.3.39-1

User avatar
Hironori Takagi
Posts: 148
Joined: 14 May 2018, 10:15
Location: Tokyo, Japan
Contact:

Re: AA coloring issues

Post by Hironori Takagi » 29 Dec 2020, 10:37

D.T. Nethery wrote:
28 Dec 2020, 20:31
Hironori Takagi wrote:
24 Dec 2020, 09:37
There are two reasons why we don't color with antialiasing.
(1) It takes time to color.

(2) When compositing, use the color area as mask.

After adding textures and effects with each color area, antialiasing is handled with plug-in effects.
.

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?
For example, we use After Effects plug-in "OLM Smoother".
https://olm.co.jp/rd/technology/tools/?lang=en
TVPaint 11.5.0, Windows10 Pro, DELL Precision M3800 / TVPaint 11.5.0 Mac OS 11.0.1, Apple Mac Pro(Late 2013) & Mac OS 10.12.6, Apple MacBook Air(Mid 2011)

User avatar
D.T. Nethery
Posts: 2950
Joined: 27 Sep 2006, 19:19

Re: AA coloring issues

Post by D.T. Nethery » 29 Dec 2020, 15:19

Hironori Takagi wrote:
29 Dec 2020, 10:37
D.T. Nethery wrote:
28 Dec 2020, 20:31

Are the plug-ins to add antialiasing to the image plug-ins for TVPaint or in another application ?
For example, we use After Effects plug-in "OLM Smoother".
https://olm.co.jp/rd/technology/tools/?lang=en

Thank you. I've downloaded it and will try it.

Do you apply the OLM Smoother to a single composite layer (characters and backgrounds) in After Effects or is OLM Smoother applied to
separate layers of characters and backgrounds in AE before doing the final export to .MOV or .MP4 from Adobe Media Encoder ?

Animator, TVPaint Beta-Tester, Animation Educator and Consultant.
MAC OS 10.14.6 , Macbook Pro Retina 15-inch, Late 2013,
2.3 GHz Intel Core i7, 16 GB Memory , TVPaint PRO 11.5.1 - 64bit ,
Wacom Cintiq 21 UX ,Wacom Intuos Pro 5 , Wacom driver version 6.3.39-1

Post Reply