Automated inbetweening

This section is dedicated to the feature & improvement requests (be sure what you are asking does not exist yet in TVPaint Animation ;) )
Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 19 Mar 2013, 23:17

I don't get it. If you want smooth moving robots, do CGI. And even from the examples I don't believe the software could create perspective correct inbetweens of a character turning 120° from just 2 keyframes.
Eh, I think you are just stating an opinion. hand drawn rotation by key frames will look more traditional organic than just using CGI.
CGI will have everything correct from every angle hence providing a stiff feeling.

There is a software that can do a rotation of 120 degrees from 2 Keys? I can tell you we can not do that sort of results.

From two keys comfortably a character of 30-40 degrees angle change is ok.
Last edited by Jotokutora on 20 Mar 2013, 01:18, edited 1 time in total.

User avatar
slowtiger
Posts: 2097
Joined: 08 May 2008, 21:10
Location: berlin, germany
Contact:

Re: Automated inbetweening

Post by slowtiger » 19 Mar 2013, 23:23

There is a software that can do a rotation of 120 degrees from 2 Keys? I can tell you we can not do that that sort of results.
From two keys comfortably a character of 30-40 degrees angle change is ok.
Now this is honest advertising! I've heard improbable claims before quite often, it's refreshing to meet somebody who doesn't do that.
TVP 10.0.18, Mac Pro Quadcore 3 GHz, 16 GB RAM, OS 10.11, QT 7.7.3

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 19 Mar 2013, 23:27

idragosani wrote:It sounds like it ultimately is vectorizing everything in the end, it's just an automated way of going from bitmap to vector tweening/morphing.
No, the user needs to clean up the keys in the software. No post conversion.

Draw the rough keys loosely if the user wants to on 3rd party software or inside this technology. The tools for clean up are vector. The key aspect of this tech is the efficiency of its tools to make the process effective.
Last edited by Jotokutora on 20 Mar 2013, 01:14, edited 2 times in total.

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 19 Mar 2013, 23:32

idragosani wrote:
slowtiger wrote:I don't get it. If you want smooth moving robots, do CGI. And even from the examples I don't believe the software could create perspectively correct inbetweens of a character turning 120° from just 2 keyframes.
I think for more complex motion, you'd need to do some kind of vector transformation or use skeletal armature/kinematics to get the proper movement, if you want it automated. Just tweening the lines might be ok for small point A to point B motion, but there are going to be severe limitations without lots of keys and extremes... at which point you might as well just hand-draw the inbetweens.

I will disagree, for more complicated movement the user simply draws more Keys or additional breakdowns. If a user wants to do Inverse Kinematics and such then maybe they should stick to using Toon Boom/ Anime studio/ Celaction or the likes.

I dont believe any true 2D animator not wanting to draw its keys. It is the inbetweens that most do not want to do.
Last edited by Jotokutora on 19 Mar 2013, 23:40, edited 1 time in total.

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 19 Mar 2013, 23:35

Paul Fierlinger wrote:
at which point you might as well just hand-draw the inbetweens.
I think the developers of such programs have people in mind who can't draw anything. I had students like that sign up for my class called, "Hand Drawn Computer Animation" asking me if it will matter that they can't draw. Others asked if it will matter that they know nothing about how to use a computer.

This is incorrect! This software was created for industry professionals and has over 10 years of R&D. Will some flavor be offered to the regular joe, most likely.

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 19 Mar 2013, 23:37

slowtiger wrote:My animation teacher Tahsin Özgür said: "If you can't draw - forget it."

If you plan to make a living of 2D animation, then this is a true statement.

User avatar
Fabrice
Posts: 10018
Joined: 17 Jul 2007, 15:00
Contact:

Re: Automated inbetweening

Post by Fabrice » 20 Mar 2013, 01:54

Ok, I'm back during a very short moment of internet connection.
Not that easy to participate during a trip, between planes and jetlag management. :)

First, I need to say a BIG or even a HUGE thank you to Jotokutora.
In contrary to someone else in this topic, and in spite of many questions, bad or good remarks, etc ... you keep answering, debating and explaining the method you are introducing !
And this is interesting, because you point the limits of the method, and the advantages.

I will let the animators discuss about the interest of the method in their workflow and will more focus on the process itself.
I still have some questions but I need to read again everything before posting anything. hopefully soon.

To everyone : we have seen this is a topic generating a lot of passion (touching the heart of the animator job), so please stay all open minded to new and old ideas.
Fabrice Debarge, TVPaint Team

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 20 Mar 2013, 09:00

slowtiger wrote:OK, here's a way to keep your precious confidentiality while showing a bit more. I did two fast drawings and challenge you to inbetween them. Nothing fancy, just my usual style, nothing difficult for a human inbetweener.
inbetweentest1.png
inbetweentest2.png
If you can do that, I might get interested. At least you could demonstrate how you break up a drawing like this, if necessary (and I'll get an idea whether it will fit my style and workflow or not.).
Hello SlowTiger, I am not sure if ever got those two Key Frames you posted earlier in-betweens, so I took the liberty to do a quick interpolation during my lunch break. Here are the results.
It is on my youtube page. If you like it I can keep it there or I can take it out once you see it.

The process was simply trace the images and interpolate them. Color one frame the system auto colored the rest. End :)

Let me know your thoughts.


User avatar
slowtiger
Posts: 2097
Joined: 08 May 2008, 21:10
Location: berlin, germany
Contact:

Re: Automated inbetweening

Post by slowtiger » 20 Mar 2013, 10:15

Woah! You just made my day. I'm sitting here with my mouth open. (And this is the first time some other than me does this character.)

This is pretty convincing, given that the keys are not really perfect. I'd say this quality would be good enough for a whole film of mine.

Now what about the line quality? I like a sketchy, rough quality in my lines, the thin smooth lines of most japanese animation don't fit my style. Does your software have a way to apply brushes to the lines? (Side note: Anime Studio can create muliple outlines over one vector line, each of it randomly offset to give a wiggling effect. One user had combined this with brushes and got a really convincing result of an animated sketch: http://www.lostmarble.com/forum/viewtop ... =9&t=20054)

After you traced the two keys, how does the software identify which lines from both frames belong together? Do you have to tag the lines?

(Of course the whole approach reminds me of the RETAS software, of which I've only seen some video so far, never used it myself.)

As you told me in PM, you took about 45 min to do this. So I had to beat this (man vs. machine!), but I needed the same amount of time for inbetweening and coloring by hand (30 and 15 min). This is the result:
inbetweentest1.gif
inbetweentest1.gif (91.24 KiB) Viewed 7552 times
(And yes, you can't do good inbetweens from bad keys. Now I see what's all wrong with my drawings ...)

Or as a friend commented: "Draw faster, or I'll replace you with a very short script!"
TVP 10.0.18, Mac Pro Quadcore 3 GHz, 16 GB RAM, OS 10.11, QT 7.7.3

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 20 Mar 2013, 12:30

slowtiger wrote:Woah! You just made my day. I'm sitting here with my mouth open. (And this is the first time some other than me does this character.)

This is pretty convincing, given that the keys are not really perfect. I'd say this quality would be good enough for a whole film of mine.

Now what about the line quality? I like a sketchy, rough quality in my lines, the thin smooth lines of most japanese animation don't fit my style. Does your software have a way to apply brushes to the lines? (Side note: Anime Studio can create muliple outlines over one vector line, each of it randomly offset to give a wiggling effect. One user had combined this with brushes and got a really convincing result of an animated sketch: http://www.lostmarble.com/forum/viewtop ... =9&t=20054)

After you traced the two keys, how does the software identify which lines from both frames belong together? Do you have to tag the lines?

(Of course the whole approach reminds me of the RETAS software, of which I've only seen some video so far, never used it myself.)
I am glad you like it. I will explain the process to the extend that I am allowed. First, I will comment on the pencil sketch effect. At this point and time we do not offer such options. We are planning on creating in-betweens in which the lines arent perfect (CGI) so it helps with the effect of slight imperfections (manually) to provide a more man made result. How is it going to work or specifics I can not comment.

The other aspect is texture brushes. I don't think anything competes with TVpaint when it comes to brushes and quality for media creation. We are planning to add textures and with other abilities we have the results will broaden the type of art styles a user can create. However, at the moment the tech is targeted at Studios more main stream workflow.

On the subject of the tracing.
The first technology was invented in 1968, yes that far back and its core tech still the same. "Tic-Tac Toons" during the 90's had a derivative of that technology. Even the new Disney "Meander" abides by the laws of stroke correspondence as well, just cleverly done. The key aspect is not the core interpolating process of vectors, but the tools provided to the user in order to manipulated them. (That is the core key of it all)

As I just mentioned the key is how the tools are design making process not tedious or making the user re-learn what as an animator have gotten use to for so long.
when you trace the keys you need to set a stroke correspondence. The way we tackle it is in a way that is comfortable and not disruptive. Actually almost no difference to the conventional method.

Once that is done, interpolation is done and fix minor tweaks to your personal liking as final results.

Worflow Scenario
So a work scenario for you would be. Rough out the keys in TVPaint due to its superior bitmap drawing tools then, import the rough cleans to the our tech for real clean up and interpolation and color. Maybe back to TVpaint to ad gradients and composting.

I hope this answer some of your questions
Last edited by Jotokutora on 20 Mar 2013, 14:57, edited 1 time in total.

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 20 Mar 2013, 12:36

BTW, I forgot to mention that the test you just did was great and the unevenness jitter of your result is not possible to us...yet. Later this year possible. Again We are currently not selling the technology as is been tested at the moment and see it performance testing.

Good work!


As you told me in PM, you took about 45 min to do this. So I had to beat this (man vs. machine!), but I needed the same amount of time for inbetweening and coloring by hand (30 and 15 min). This is the result:
inbetweentest1.gif
(And yes, you can't do good inbetweens from bad keys. Now I see what's all wrong with my drawings ...)

Or as a friend commented: "Draw faster, or I'll replace you with a very short script!"[/quote]

User avatar
Fabrice
Posts: 10018
Joined: 17 Jul 2007, 15:00
Contact:

Re: Automated inbetweening

Post by Fabrice » 20 Mar 2013, 14:48

It's interesting in terms of results, because it's better than Sykosan ones.
There is no fading lines.

That's said, I'm not sure that you could make move a whole image (line + color + texture) created in TVPaint. Or am I wrong ?
This point was possible with Sykosan method.

Also, what happen if the line is not done with the penbrush but with a watercolor brush or something else with a complicated shape ?

I wish to spend more time in this topic, hopefully this week-end.

@ Slowtiger : From what I have seen in Japan, Retas is not working in this way.
Fabrice Debarge, TVPaint Team

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 21 Mar 2013, 13:04

Fabrice wrote:It's interesting in terms of results, because it's better than Sykosan ones.
There is no fading lines.

That's said, I'm not sure that you could make move a whole image (line + color + texture) created in TVPaint. Or am I wrong ?
This point was possible with Sykosan method.

Also, what happen if the line is not done with the penbrush but with a watercolor brush or something else with a complicated shape ?

I wish to spend more time in this topic, hopefully this week-end.

@ Slowtiger : From what I have seen in Japan, Retas is not working in this way.
I'm glad you find them interesting. I didn't see Sykosan results although, I have seen some of his sample before I found this thread webpage.

At this time, textures like Sykosan's samples I have seen, I can not say we can. But I can't say that we can't eventually.

Its seems he separated the image assets on layers and then use some form of interpolating warp effect of some kind. Clever, I strongly feel and say that good results will depend on the complexity of the provided keys ( not for all scenarios). But the problem is that they are not true in- between in the full sense (my observation). There will be a point in which the warping effect will reach a warp distance limit, therefore there will be few frames where the lines will look blurry. On those blurry section the only solution will be to skip those frames when filming OR blur them as a fast effect. But I doubt they actually progressively interpolated the strokes. It seems his effect is not limited only on the outlined strokes but also on the texturing of the characters render. I have a felling that the images without texture will reveal more sliding effect but could be well hidden on a good rig. Again this is just an opinion.


Retas Pro, has both bitmap and vector abilities. It can into a certain degree do conversion of each format toward the other format. Retas to my knowledge cant handle drawing with textures. In Japan's Anime shows this isn't necessary since Anime don't tend to use much mixed media on their productions.
That said, on one module called Traceman they can do gray-scale vectorization of images (TGA preffered), in which it will retain the pencil texture on top of the underlined mono/vector trace.
However their type of vectors to my knowledge do not have interpolation ability.

sykosan
Posts: 10
Joined: 09 Jan 2013, 19:54

Re: Automated inbetweening

Post by sykosan » 05 Apr 2013, 21:17

Hello Jotokotura.

Just popped in and saw your posts.
Great technique you have developed here !
I am very impressed with the results. I wish I were a programmer myself so that I could develop my own animation programme :)
This said my own technique is very different from yours and would require different tools.
You managed to create a very accurate tool which indeed is perfect for thin clean lines and flat shaded animations. Respect !
I can see the limitations, but quite honestly you manage to work around them with great efficiency.
I wish your programme turns into a commercial success. It deserves it ;)
It looks like you're close to having a complete software. When do you plan on releasing the first commercial version ?
Good thing you don't overlap too much with TV Paint ^_^

Keep it up Jotokotura and thanks for sharing.
Oh and by the way, that Calypso Angency film totaly rocks !! :shock:
Awesome stuff ! Seriously, this is a masterpiece and it needs a lot more viewing. I'll advertise a bit for you ;)
It's a real pleasure to discover your work.

Jotokutora
Posts: 15
Joined: 18 Mar 2013, 07:31

Re: Automated inbetweening

Post by Jotokutora » 09 Apr 2013, 03:54

Thanks for the kind words! Yes it seems the techniques are very different. I will say that TV Paint works really well with this technology. The only thing to make it better is sharable file formats! :)

We have been very close a Anime Studio in Japan for many years and they are staring to test TV Paint to hopefully migrate from pencil and paper. They had several units with large Cintiqs, nice setups. So the popularity of TV paint is growing exponentially from I can see.

I have tried almost every digital software to draw with for my Calypso Key frames back in 2010. But simply nothing compares to the natural feeling and responsiveness that TVpaint provided me. Another thing that I was highly impressed while drawing my work was the stability of the program. The main limitation I had at the time was not been 64 bit not now of course .

So will see if TVpaint and our technology crosses path in the future.

Thanks for the kind words I hop you can thrive to find ways to implement your technology on different kinds of animations. Maybe a new kind of animation workflow and style just as Flash did back in the late 1990's :D

Post Reply