​
Integrate with Vercel AI SDK

This demo shows how to integrate with Vercel AI SDK https://github.com/vercel-labs/ai-chatbot to create a chatbot with avatar display.

Using the useChat hook creator, gives you helper config to setup your vercel ai person easily with emotion mapping.

  const {
    messages,
    input,
    handleInputChange,
    handleSubmit,
    isLoading,
    setMessages,
  } = useChat()

Integrate the avatrar display in your react tree.

chat.tsx
  const {
    messages,
    input,
    handleInputChange,
    handleSubmit,
    isLoading,
    setMessages,
  } = useChat()
  
  const [text, currentEmotion] = getAIReplyWithEmotion(messages, isLoading)

  const {
    avatarDisplay,
    handleFirstInteractionAudio,
    availableEmotions,
    context,
  } = useAvatar({
    // Avatar State
    text: text,
    currentEmotion: currentEmotion,
    avatarId: 'af3f42c9-d1d7-4e14-bd81-bf2e05fd11a3',

    // Loader + Plugins
    avatarLoaders: defaultAvatarLoaders,
    blendshapesService: defaultBlendshapesService_2,

    audioService: elevenLabs,

    // Style Props
    scale: 4,
    className: 'w-[400px] h-[400px]',
  })

  // Set initial prompt
  useEffect(() => {
    if (!availableEmotions) return

    setMessages([
      {
        content: buildCharacterPersonaPrompt({
          name: 'Ava',
          context: 'Im ava, a virtual idol from avatechs.',
          exampleReplies: ['I am ava!', 'I love next js!', 'What are you working on recently?', 'npm i @avatechai/avatars'],
          emotionList: availableEmotions,
        }),
        role: 'system',
        id: '1',
      },
    ])
  }, [availableEmotions])

Using the avatar hook you created!

chat.tsx
// Including the avatar display in your react tree
return <>{avatarDisplay}</>

Make sure to trigger the audio playback setup on first interaction

chat.tsx
<ChatPanel
  id={id}
  isLoading={isLoading}
  stop={stop}
  append={(input) => {
    handleFirstInteractionAudio()
    return append(input)
  }}
  reload={reload}
  messages={messages}
  input={input}
  setInput={setInput}
/>

In order to remove the emotion tag from the prompt, you can use the removeEmotionFromPrompt helper function.

chat-message.tsx
import { removeEmotionFromPrompt } from '@avatechai/avatars'
...
const text = removeEmotionFromPrompt(message.content)