-
rhubarb-lip-sync
Rhubarb Lip Sync is a command-line tool that automatically creates 2D mouth animation from voice recordings. You can use it for characters in computer games, in animated cartoons, or in any other project that requires animating mouths based on existing recordings.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
Anyway, I used Rhubarb Lip Sync for this. Made tool plugin for Godot to pass the parameters to Rhubarb and grabbed the stdout (json) and saved that and the audio file to a custom resource file. I'm simply playing the audio, getting the playback position and check if it's more than the Rhubarb's mouthCue "end" value, I change to the next mouth shape.