Is there a guide on how to export correctly? I tried out a very big Spine file, but some images are completely black and shaped like their mesh. It would also be nice if this supports multiple skins as well.
Edit 1: I found out that this does not accept multiple pngs.
@skarasuko Sorry you are having issues.
SilverStraw wrote@[deleted] Sorry you are having issues.
No worries at all. This is already feels very usable among most aspiring V-Tubers! The Spine Team should consider fast-tracking official support for this, because it's still the perfect time to be part of the market!
Spine Vtuber Prototype
Update 1.1.3
https://silverstraw.itch.io/spine-vtuber-prototype
Updated Spine Vtube Test Model for synchronizing both eyes on "left eye open" track and keyframe only the right eye on "right eye open" track.
https://silverstraw.itch.io/spine-vtube-test-model
Some ninja patching with the help of LustFire's bug catching.
A user notified me an issue with the experimental motion capture not exporting the JSON file. I patched the bug.
I have tried 1.1.3, and it certainly seems to make it easier to detect eye wink! Here is the result of my test (the model was also modified slightly):
It had been a while since I tested this tool, but I found it enjoyable once again 
Thank you for keeping this cool tool updated!
One thing I thought was it might be a good idea to add a threshold value for the eye pupils, such that subtle changes in position could be ignored. My model's pupils wobbled while winking, so I adjusted the left/right pupil pitch strength and left/right pupil yaw strength parameters, but if these values are set too low, the model’s eyes will not follow the eye movement at all and it is not ideal. FYI, in the model in the video above, left/right pupil pitch strength is set to 4 and left/right pupil yaw strength to 5. I would be happy if you could consider it!
Here is my latest model data:
chara-for-Spine-Vtuber-Prototype_20221223.zip
1.1.4
https://silverstraw.itch.io/spine-vtuber-prototype
Misaki wroteOne thing I thought was it might be a good idea to add a threshold value for the eye pupils, such that subtle changes in position could be ignored. My model's pupils wobbled while winking, so I adjusted the left/right pupil pitch strength and left/right pupil yaw strength parameters, but if these values are set too low, the model’s eyes will not follow the eye movement at all and it is not ideal. FYI, in the model in the video above, left/right pupil pitch strength is set to 4 and left/right pupil yaw strength to 5. I would be happy if you could consider it!
I had to reinstate the threshold mechanism after I forgo it for moving average. You cannot export those settings to .svp yet. I want you to test it out before I make more commitments to the pupil threshold.
Thank you so much for adding left/right pupil pitch/yaw threshold settings!! I have tried them and changing the threshold values actually helped to reduce the eye wobbling. Here is the result:
The settings for my model related the eyes are as follows:
While testing these settings, I found I should modify my animations, so the model in the video above is updated. Here is the updated project files:
chara-for-Spine-Vtuber-Prototype_20230117.zip
As shown in the following image, I made the pose will not be changed immediately after detecting changes in eyelids and mouth movements captured in some animations:
This adjustment was made so that minute movement changes would not cause the eyelids or mouth subtle opening. Also, since I set the thresholds for pupil pitch higher values than for pupil yaw, I adjusted pupils in the pitch down/up animations do not move from frame 0 to frame 10, so that these do not appear as if the position suddenly jumps when the threshold is exceeded.
By the way, you said:
You cannot export those settings to .svp yet.
but somehow I can export the threshold settings to .svp. (Maybe you have updated this tool after replying to this thread?)
Anyway, I am happy with the results this time! There are some things I would like to fix in my rig (e.g., The half-eye pose is not very good, although I have adjusted it many times), but I think the current specification of this tool is already great for vtubing. I am looking forward to the day when facial expression animations can be added. Great work!! :yes:
but somehow I can export the threshold settings to .svp. (Maybe you have updated this tool after replying to this thread?)
It has been a while since I worked on the source code. I forgot that I have a function that exports settings from a list of default setting values. I updated that list for the left/right pupil pitch/yaw threshold default values, therefore, they got exported. :lol:
Update 1.1.5
Add "Backface culling" checkbox to "Canvas Settings" menu. The backface of attachments are invisible if checked. The back side is kept consistent with Spine editor.
Add "Flip Horizontal" checkbox to "Canvas Settings" menu . When checked, the world X-axis is flipped.
Add empty animation tracks: custom tracks 1 to 4.
Add customizable expression buttons 0 to 20 underneath the rendering canvas. Button 0 resets all custom tracks thus removing all customizable expressions active. The buttons are usable once the expression slot has been setup.
Add customizable expression setup interface under "Model Settings" menu.
Add customizable expression slot drop-down menu ( 1 to 20 ) in the setup. Each slot allow you to setup multiple custom track index ( 1 to 4 ), transitional animation, and animation loop that follow after the transitional animation.
Include a button to add a setup interface row for adding more custom tracks ( 1 to 4 ), transitional animations, and animation loops.
Include a button to add a setup interface row for adding more custom tracks ( 1 to 4 ), transitional animations, and animation loops.
Add a button to remove the last setup interface row. You do not want any rows with incomplete information as you will not be able to finish setting up the customizable expression slot.
Add a button to assign all the custom tracks ( 1 to 4 ), transitional animations, and animation loops to the customizable expression numbered slot ( 1 to 20 ).
Each setup interface row has three parts: custom track index ( 1 to 4 ), transitional animation, animation loop. Custom track index ( 1 to 4 ) and transition animation are required for setup while animation loop is optional. The transition animation and animation loop input fields are drop-down menus that lists all the animation found in file ( .json | .skel ). Transition animation does not loop and the animation loop plays after the transitional animation. There is not any mix duration between transition animation and animation loop.
The customizable expression slots are saved into and can be loaded from SVP file.
Remove animation selection from Single Value Properties drop-down list with "Model Settings"
https://silverstraw.itch.io/spine-vtuber-prototype
Updated model for testing custom expressions.
The expressions feature is really fun and wonderful!! I haven't been able to test it much because preparing the animations still takes more time, but here are the results of a quick test I did:
I know that the transition animations back to the default pause also need to be registered in the expression buttons, but I have not yet been able to do that at this time.
My current model is in a very half-assed state, but I'll leave the data here for anyone who wants to test it:
chara-for-Spine-Vtuber-Prototype_20230130.zip
When I make more improvements, I will share the data here again.
I should add another setting for 'returning to default pose' animation to the customizable expression feature. That way you save another expression slot.
After this feature, facial expression recognition AI should be as easy as setting which expression slot you want to use for the AI result.
Spine Vtuber Prototype body track test 1. Preview only.
SilverStraw Wow, the tool can finally capture body movements!! That is definitely an inovative update I'm really looking forward to the day when it will be available!!
This is amazing! looking forward to creating a cute prototype for this when I have time too >.< thank you for your work!
There is an updated tracking framework but I would have break down the application. It is almost like starting over. Wish me luck 
.
Some possible gestures if it works 
.
Face gestures
1 - browDownLeft
2 - browDownRight
3 - browInnerUp
4 - browOuterUpLeft
5 - browOuterUpRight
6 - cheekPuff
7 - cheekSquintLeft
8 - cheekSquintRight
9 - eyeBlinkLeft
10 - eyeBlinkRight
11 - eyeLookDownLeft
12 - eyeLookDownRight
13 - eyeLookInLeft
14 - eyeLookInRight
15 - eyeLookOutLeft
16 - eyeLookOutRight
17 - eyeLookUpLeft
18 - eyeLookUpRight
19 - eyeSquintLeft
20 - eyeSquintRight
21 - eyeWideLeft
22 - eyeWideRight
23 - jawForward
24 - jawLeft
25 - jawOpen
26 - jawRight
27 - mouthClose
28 - mouthDimpleLeft
29 - mouthDimpleRight
30 - mouthFrownLeft
31 - mouthFrownRight
32 - mouthFunnel
33 - mouthLeft
34 - mouthLowerDownLeft
35 - mouthLowerDownRight
36 - mouthPressLeft
37 - mouthPressRight
38 - mouthPucker
39 - mouthRight
40 - mouthRollLower
41 - mouthRollUpper
42 - mouthShrugLower
43 - mouthShrugUpper
44 - mouthSmileLeft
45 - mouthSmileRight
46 - mouthStretchLeft
47 - mouthStretchRight
48 - mouthUpperUpLeft
49 - mouthUpperUpRight
50 - noseSneerLeft
51 - noseSneerRight
52 - tongueOut
Hand gestures
["None", "Closed_Fist", "Open_Palm", "Pointing_Up", "Thumb_Down", "Thumb_Up", "Victory", "ILoveYou"]
SilverStraw That sounds tough 
I hope it works out! 
@SilverStraw oh, a new MediaPipe model? That's super cool! Likely worth the time investment.
Misaki https://silverstraw.itch.io/spine-vtuber-prototype-2
I need your help breaking it the application again 
. This is using the new facing tracking. I haven't added any tracking smoothing and I am not sure if it is needed. You can tell me your thoughts.
Hopefully I can transfer the body animation code to the new body tracking without much problem like the face animation. The draw order won't be ready with the body tracking because I haven't figure a good solution to handle depth for Spine skeletons.
SilverStraw The new face tracking seems very fine! You mentioned you did not add any tracking smoothing, but it already seems very smooth. The following video is the result of testing on my end:
It may be hard to tell from the video, but I feel that the movement has become less wobbly, and when I want to stop the movement, it can be stopped properly.
I'm looking forward to body tracking becoming available! 
2.0.1
https://silverstraw.itch.io/spine-vtuber-prototype-2
Test model updated.
https://silverstraw.itch.io/spine-vtube-test-model
SilverStraw Awesome!! I tried it with your test model and it certainly did body tracking on my PC. It's absolutely fun to have the model on the screen react to my movements 
When I get a chance, I would like to modify my model for body tracking. Great job!! 
