ai_sadtalker

An authenticated client can post a request to sadtalker

This AI has the ability to produce an animated video from a single image upload and a sound file.

GitHub - OpenTalker/SadTalker: [CVPR 2023] SadTalker：Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face AnimationGitHub

sadTalker github oage

This AI can run AI alongside a Daemon on GPU only

Get the JSON config file for this AI

GET https://opensourceais.com/api/v1/public/config/sadtalker

200: OK The full JSON config file

{
    "engine": "sadtalker",
    "name": "SAD Talker",
    "location": "http://0.0.0.0",
    "port": 5309,
    "version": "1.0.7",
    "description": "Image to Video",
    "github": "https://github.com/OpenTalker/SadTalker/",
    "requiresGPU": true,
    "isFree": false,
    "default_cost": 1000,
    "params": [{
        "in": "url_upload",
        "out": "-filename",
        "isMandatory": true,
        "type": "string",
        "default": null,
        "ui": {
            "title": "Image input",
            "widget": "uiSelectPicture",
            "column": 1
        }
    }, {
        "in": "url_audio",
        "out": "-audio",
        "isMandatory": true,
        "type": "string",
        "default": null,
        "ui": {
            "title": "Audio file input",
            "widget": "uiSelectFile",
            "column": 1
        }
    }, {
        "in": "still_image",
        "out": "--still_mode",
        "isMandatory": false,
        "type": "boolean",
        "default": false,
        "ui": {
            "title": "Still image mode",
            "widget": "uiSwitch",
            "column": 1
        }
    }, {
        "in": "blink",
        "out": "--use_blink",
        "isMandatory": false,
        "type": "boolean",
        "default": true,
        "ui": {
            "title": "Eye blink effect",
            "widget": "uiSwitch",
            "column": 1
        }
    }, {
        "in": "scale_image",
        "out": "--use_enhancer",
        "isMandatory": false,
        "type": "boolean",
        "default": false,
        "ui": {
            "title": "Scale image",
            "widget": "uiSwitch",
            "column": 1
        }
    }, {
        "in": "preprocess",
        "out": "--preprocess_type",
        "isMandatory": true,
        "type": "string",
        "accept": {
            "$in":["crop", "resize", "full", "extcrop", "extfull"],
            "display":["crop", "resize", "full", "extcrop", "extfull"]
        },
        "default": "crop",
        "ui": {
            "title": "Input image pre-processing",
            "widget": "uiMultiToggle",
            "column": 2
        }
    }, {
        "in": "res",
        "out": "--res",
        "isMandatory": false,
        "type": "int",
        "default": 256,
        "accept": {
            "$in":[256, 512],
            "display":["256x256", "512x512"]
        },
        "freeAccess": {"$max": 256},
        "ui": {
            "title": "padding in px to the eye center",
            "unit": "px",
            "widget": "uiMultiToggle",
            "column": 2
        }
    }, {
        "in": "pose",
        "out": "--pose_style",
        "isMandatory": true,
        "type": "int",
        "default": 0,
        "ui": {
            "title": "Pose style",
            "tooltip": "",
            "displayAsPercent": false,
            "unit": "",            
            "min": 0,            
            "max": 46,
            "widget": "uiRangeSlider",
            "column": 2
        }
    }, {
        "in": "cimg",
        "out": "--batch_size",
        "isMandatory": true,
        "type": "int",
        "default": 1,
        "accept": {
            "$in":[1, 2, 3, 4],
            "display":["1", "2", "3", "4"]
        },
        "ui": {
            "display": false,
            "title": "Number of videos to produce",
            "widget": "uiMultiToggle",
            "column": 2
        }
    }, {
        "in": "output",
        "out": "--output",
        "isMandatory": true,
        "type": "function",
        "default": "$output_uid.mp4"
    }]
}

Post a request to AI sadtalker via OpenSourceAIs

POST https://opensourceais.com/api/v1/private/client/ai/sadtalker

Request Body

Name	Type	Description
url_audio	String	A well formed URI starting with https://... and accessing a .WAV file. Note that this file has a limit of 2Mb. You should only upload files of at most 15sec of sample voice.
url_upload*	String	A well formed URI starting with https://... and accessing a .PNG or .JPG file. Note that this file has a limit of 2Mb.
still_image	Boolean	Set to true if you want to avoid too much movement in the video ; this is interesting if you have a full body video. Default = false
blink	Boolean	Set to false if you do not want eye blinking animation. Default = true
scale_image	Boolean	Set to true if you want to use gfpgan style AI to improve the quality of images generated to make the video. Default = false
preprocess	String	Takes one of those values: "crop", "resize", "full", "extcrop", or "extfull". Default is "crop"
res	Number	The video output resolution, either 256 (for 256x256px), or 512 (for 512x512px). Default = 256
pose	Number	A "pose" indicator, between 0 and 45

201: Created Confirmation that the request was processed

{
    "data": {
        "uid": 1678361974000,
        "created_at": "2023-03-09T11:39:44.000Z"
    }
}

400: Bad Request This error should not happen unless authentication is tampered with

{
     data: null,
     status: 400,
     statusText: "A client token is required"
}