Handling Conversational Flows

Handling conversations with Slang

Before we begin

In this section, we will dive deep into Slang's Android SDK and understand how to handle conversational flows. This section assumes that you've already reviewed the sections on Terminology and Android Quick Start. If you have not, please review those sections before exploring the following sections.

To make the following sections more intuitive, we will use the following example:

• The app with which Slang is being integrated is a banking app

• The app's buddy is configured with the following intents and entities:

  • Intent: show_balance

    • Entities: None

  • Intent: transfer_funds

    • Entities: recipient (required), amount (required), mode (optional)

Conversational flows and multi-step actions

The intent action example that you saw in the Android Quick Start section uses a single-step intent action (SlangIntentAction). Single-step actions are perfect for handling actions that do not require a conversational flow. For example, the banking app could use a single-step action for displaying account balance, when a user Alice says something like "What's my account balance?".

However, not all types of actions can be completed using single-step intent actions. For example, with the banking app, if Alice wants to transfer money to Bob, she could begin by saying "I'd like to transfer money to Bob". However, the app will need more information to complete the transaction, such as the amount to transfer. With Slang, this additional data can be collected automatically through voice with multi-step intent actions (SlangMultiStepIntentAction). With correctly configured multi-step intent actions, Slang would automatically ask Alice "How much money would you like to send Bob?", collect this information and pass it down to the app for completion. Let's now look at how to correctly configure and implement multi-step intent actions.

Configuring multi-step actions

The first step towards using multi-step intent actions is to configure the use-cases correctly in the Slang Console. Specifically, you will need to mark entities that require automatic prompting by Slang as "required" entities. Marking these entities as required informs Slang that the action can be completed only when the values for these entities is available, and it will therefore automatically collect the values for these entities when they're missing. You can mark the entities as "required" by navigating to Your Buddy -> Intent Page -> Entities Section and checking the box under the "Required" column for the appropriate entity. Please see the image below for reference. You should also configure prompts for these entities to ensure that the correct question is asked by Slang when the entities are missing.

Implementing multi-step actions

Once you've configured the required entities for all of your buddy's intents, the next step in handling conversational flows is to implement a multi-step intent action using the SlangMultiStepIntentAction interface and registering it with Slang at initialization time.

// First, implement the interface
public class BankingAppAction implements SlangMultiStepIntentAction {
    // implement API methods here
}

...

// Then, register with Slang
SlangBuddyOptions options = new SlangBuddyOptions.Builder()
                                .setBuddyId(BUDDY_ID)
                                .setAPIKey(API_KEY)
                                .setIntentAction(new BankingAppAction())
                                // initialize other required options
                                .build();
SlangBuddy.initialize(options);

The SlangMultiStepIntentAction interface

The SlangMultiStepIntentAction interface is the primary means through which apps handle conversational flows, so let's look at it in more detail. The interface definition consists of the following methods:

void onIntentResolutionBegin(SlangIntent intent, SlangSession session);

Status onEntityUnresolved(SlangEntity entity, SlangSession session);

Status onEntityResolved(SlangEntity entity, SlangSession session);

Status action(SlangIntent intent, SlangSession session);

void onIntentResolutionEnd(SlangIntent intent, SlangSession session);

Once an implementation of SlangMultiStepIntentAction is registered with Slang during initialization, the Slang SDK invokes the interface methods in a specific order whenever it detects an intent from an utterance spoken by the user. The order is described in the flowchart below:

Using the banking example from before, when Alice says "I'd like to transfer money to Bob", which maps to transfer_funds intent in the app's configuration, Slang performs the following steps:

1. Begin intent resolution: onIntentResolutionBegin()

First, the app is notified that intent resolution is beginning, via the onIntentResolutionBegin() method. As part of this method, the app can examine the name of the intent (funds_transfer) and begin work necessary to fulfill the action, such as kicking off background network or database fetch operations.

void onIntentResolutionBegin(SlangIntent intent, SlangSession session) {
    String intentName = intent.getName();
    if (intentName.equals("funds_transfer") {
        // perform any pre-processing necessary for funds transfer
    }
}

2. Process entities: onEntityUnresolved() and onEntityResolved()

Next, Slang begins processing entities that are part of this intent. Slang iterates through all the entities that are part of this intent and processes them in the following way:

• If the entity is resolved as part of the user-utterance, Slang calls the onEntityResolved() method and passes the entity value as parameter to the method. The app is expected to return Status.SUCCESS to inform the Slang to proceed further (and speak the affirmative prompt, if configured) and Status.FAILURE to inform Slang to stop further processing (and speak the negative prompt, if configured).

• If the entity is not resolved, and is configured as required, Slang calls the onEntityUnresolved() method and passes the entity as parameter to the method. As part of this method's implementation, the app can choose to resolve the entity's value via the SlangEntity.resolve() method, or do nothing - in which case Slang will prompt the end-user for the entity value, using the prompts configured for the entity in the Slang Console. Once the entity is resolved, either by the app, or by the end-user, Slang calls onEntityResolved() for the entity before processing the next entity.

Status onEntityUnresolved(SlangEntity entity, SlangSession session) {
    if (entity.getName().equals("recipient") {
        // navigate to activity showing list of available recipients
    }
    return SUCCESS;
}

Status onEntityResolved(SlangEntity entity, SlangSession session) {
    if (entity.getName().equals("recipient") {
        String recipient = entity.getValue();
        return validRecipients.contains(recipient) ? SUCCESS : FAILURE;
    }
    
    if (entity.getName().equals("mode") {
         String mode = entity.getValue();
         return validModes.contains(mode) ? SUCCESS : FAILURE;
    }
    
    return SUCCESS;
}

3. Perform the actual action: action()

• Once all the entities are processed and all the required entities are resolved, Slang calls the action() method and pass the values of the intent and the corresponding entities. As part of this method, the app is expected to take the appropriate action for the given intent, such as opening an activity, completing a transaction or displaying relevant information. Upon completion, the app should return Status.SUCCESS to inform Slang that the action was successful or return Status.FAILURE to indicate a failed action.

Status action(SlangIntent intent, SlangSession session) {
    if (intent.getName().equals("funds_transfer") {
        int amount = Integer.parseInt(intent.getEntity("amount").getValue());
        String recipient = intent.getEntity("recipient");
        SlangEntity modeEntity = intent.getEntity("mode");
        String mode = modeEntity.isResolved() ? modeEntity.getValue() : defaultMode;
        confirmTransfer(amount, recipient, mode);
        return SUCCESS;
    }
}

4. Finish intent resolution: onIntentResolutionEnd()

• Finally, after the action method has been completed, the onIntentResolutionEnd() method is called to give the application a chance to clean up after the action.

void onIntentResolutionEnd(SlangIntent intent, SlangSession session) {
    if (intent.getName().equals("funds_transfer") {
        // perform cleanup
    }
}