🖼️Modalities integration

From start to finish, set up a chat-based application with custom modalities in minutes with NLX

Prerequisites

Before we begin, make sure you completed the Chat assistant guide or have the following:

Integrated an NLP
Created an intent with an API flow
Set up a bot with an API channel

Checklist

You'll complete the following to successfully use a custom modality:

Create a Modality
Link the Modality to a flow node
Build + deploy
Set up a web page with the Chat SDK

Step 1: Set up a Modality

Est. time to complete: ~1 minute

Modalities allow for information to be transmitted to a user that goes beyond a standard bot message. Images, audio, structured text, haptic signals, videos, and more can be modalities you may want to provide users during conversation.

Select Settings from workspace menu > Choose Modalities tab
Select + Add modality
Enter "Video" as the name name
Select Schema and change the Type to Object
Select Properties and then + Add property
Enter "src" as the Field name
Click Create modality

Step 2: Link the Modality to a flow node

Est. time to complete: ~1 minute

Navigate to one of the intents you created (see Prerequisites), then select the API conversation flow. You'll enable the video modality you created, on an end node, and provide it with a video URL.

Select Intents from workspace menu > Choose one of the intents linked to your bot
Select the API flow card > Click on one of the end nodes
Select Add functionality > Choose Modalities
Toggle the Enable to on
Enter dQw4w9WgXcQ as the value for "src"
Click Save

🧠 Looking for more? See Construct flows

Step 3: Deploy changes

Est. time to complete: ~5 minutes

Now that you added the modality to the flow, we need to deploy those changes to your application.

Click Deployment tab of bot > Select Create or Review & build
Wait for validation to complete > Select Create build*
When satisfied with a successful build, click Deploy

*After a build status appears as 🟢 Built, you may use the Test feature to test the conversation with your bot using the latest build.

🔧 Need to troubleshoot your build? See how to troubleshoot build errors

Step 4: Set up a webpage

Est. time to complete: ~10 minutes

Using the Chat SDK, you can quickly build an HTML page to test out the Video modality. You'll need to copy the bot URL and API key from the bot deployment tab.

Picking up from Step 3, click on the Details link next to Deployed
Under Setup instructions, expand the API row
Copy the Bot URL and API key
Copy the HTML file below and replace REPLACE_WITH_BOT_URL and REPLACE_WITH_API_KEY with the values you just copied
Save the HTML file you just edited and serve the HTML file with your preferred command line tool such as serve

<html lang="en">
  <head>
    <title>NLX Widget Sample HTML</title>
    <meta name="viewport" content="width=device-width, initial-scale=1">
  </head>
  <body>
    <script defer src="https://unpkg.com/@nlxai/chat-widget/lib/index.umd.js"></script>
    <script defer src="https://cdnjs.cloudflare.com/ajax/libs/htm/3.1.1/htm.js" integrity="sha512-RilD4H0wcNNxG2GvB+L1LRXCntT0zgRvRLnmGu+e9wWaLKGkPifz3Ozb6+WPsyEkTBLw6zWCwwEjs9JLL1KIHg==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
    <script>
      window.addEventListener("DOMContentLoaded", () => {
        // EMBEDDABLE COMPONENT SETUP
        // Destructure dependencies
        const React = nlxai.chatWidget.React;
        const useConversationHandler = nlxai.chatWidget.useConversationHandler;

        // Use the https://github.com/developit/htm package as a JSX alternative
        const html = htm.bind(React.createElement);
 
        Video = ({ data }) => {
          const videoSrc = `https://www.youtube.com/embed/${data.src}`;
          return html`
            <iframe
              width="264"
              height="160"
              src=${videoSrc}
              allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
              allowFullScreen
            ></iframe>`;
        };

        const widget = nlxai.chatWidget.create({
          config: {
            botUrl: "REPLACE_WITH_BOT_URL",
            headers: {
              "nlx-api-key": "REPLACE_WITH_API_KEY"
            },
            languageCode: "en-US"
          },
          titleBar: {
            "title": "Modalities"
          },
          // CUSTOM BEHAVIOR SNIPPET
          onExpand: (conversationHandler) => {
            const checkMessages = (messages) => {
              if (messages.length === 0) {
                conversationHandler.sendWelcomeIntent();
              }
              conversationHandler.unsubscribe(checkMessages);
            };
            conversationHandler.subscribe(checkMessages);
          },
          customModalities: { Video },
          // CUSTOM BEHAVIOR SNIPPET END
          theme: {
            "primaryColor": "#2663da",
            "darkMessageColor": "#2663da",
            "lightMessageColor": "#EFEFEF",
            "white": "#FFFFFF",
            "fontFamily": "-apple-system, BlinkMacSystemFont, \"Segoe UI\", Roboto, Oxygen-Sans, Ubuntu, Cantarell, \"Helvetica Neue\", sans-serif",
            "spacing": 12,
            "borderRadius": 8,
            "chatWindowMaxHeight": 640
          }
        });
      });
    </script>
  </body>
</html>

Step 5: Test the modality

Finally, to test the modality you will have to either host your HTML file or serve it locally using a command line tool like serve. Once you can access the page, click on the chat widget at the bottom right of the page and converse with your bot until you reach the node in the conversation where you set the modality.

Last updated 17 days ago