CONVA Search
Adding an accurate, multilingual Voice Search capability into your app
By now you must have configured and published your Assistant via the Slang Console and also maybe customized it as required. Congratulations! :) If you have not already done that, you can do so by following the instructions here.
Let's start coding!
For testing, we recommend using a physical device instead of an emulator because most emulators don't work well with microphones.
The first step is to update the app's build system to include Slang's Retail Assistant SDK.
Android
iOS
React Native
Flutter
Web
Add the path to the Slang maven repository to your top-level Gradle file
# Add this to your top-level Gradle file
allprojects {
repositories {
…
maven { url "https://gitlab.com/api/v4/projects/25706948/packages/maven" }
}
}
Add the Slang Retails Assistant dependency to your app's Gradle file
# Add this to your app's Gradle file
dependencies {
…
implementation 'in.slanglabs.assistants:slang-retail-assistant:8.+'
}
Add the path to the Slang Cocoapod repository to your Podfile
# Add this to your podfile
source 'https://github.com/SlangLabs/cocoapod-specs'
source 'https://github.com/CocoaPods/Specs.git'
Add the dependency to SlangRetailAssistant in your Podfile
pod 'SlangRetailAssistant'
Add support for granting microphone permission
In iOS, the user must explicitly grant permission for an app to access the user’s data and resources. An app with the SlangRetailAssistant requires access to User’s device microphone for voice interactions.
To comply with this requirement, you must add
NSMicrophoneUsageDescription key to the Info.plist file of your app and provide a message about why your app requires access to the microphone. The message will be displayed only when the SlangRetailAssistant needs to activate the microphone.To add the key:
- 1.In the Xcode project, go to the Info tab.
- 2.In the Custom iOS Target Properties section, hover over any key in the list and click the plus icon to the right.
- 3.From the list, select Privacy - Microphone Usage Description.
- 4.In the Value field to the right, provide a description for the added key. This description will be displayed to the user when the app is launched. For example: "We require microphone permission to enable the voice assistant platform"
The next step is to install the required packages inside your code repo
yarn setup
If you use yarn for install packages, run the below command
yarn add @slanglabs/slang-conva-react-native-retail-assistant
npm setup
If you use npm for managing your packages, run the below command
npm install @slanglabs/slang-conva-react-native-retail-assistant --save
Because Slang uses native libraries, you need to link the package to your codebase to run the automatic linking steps
react-native link @slanglabs/slang-conva-react-native-retail-assistant
Run the below command to install the required packages inside your code repo.
$ flutter pub add slang_retail_assistant
Once done, run the command '
dart pub get' and ensure Slang assistant is added to the dependencies.
dependencies:
slang_retail_assistant: ^8.0.0
import 'package:slang_retail_assistant/slang_retail_assistant.dart';
You can add our NPM package to your project using any of your preferred package manager -
yarn
$ yarn add @slanglabs/[email protected]
npm
$ npm install @slanglabs/[email protected] --save
If you are building a simple HTML project, you can still use our SDK using the script below -
<script type="text/javascript" src="https://unpkg.com/@slanglabs/[email protected]/index-iife.js" defer></script>
The next step is to initialize the SDK with the keys you obtained after creating the Assistant in the Slang console.
Android
iOS
React Native
Flutter
Web
The recommendation is to perform the initialization in the
onCreate method of the main Activity where the search bar will be visible. // Your main activity class
protected void onCreate(Bundle savedInstance) {
...
AssistantConfiguration configuration = new AssistantConfiguration.Builder()
.setAPIKey(<API Key>)
.setAssistantId(<AssistantId>)
.build();
SlangRetailAssistant.initialize(this, configuration);
}
The recommendation is to perform the initialization in the
viewDidLoad method of the main ViewController where the search bar will be visible. import SlangRetailAssistant
override func viewDidLoad() {
super.viewDidLoad()
...
let config = AssistantConfiguration.Builder()
.setAPIKey("<APIKey>")
.setAssistantId("<AssistantId")
.build()
SlangRetailAssistant.initialize(with: config)
}
This should ideally be done in the componentDidMount of your main app component
import SlangConvaTrigger, {SlangRetailAssistant} from '@slanglabs/slang-conva-react-native-retail-assistant';
SlangRetailAssistant.initialize({
assistantId: '<assistant id>', // The Assistant ID from the console
apiKey: '<API Key>', // The API key from the console
})
This should ideally be done inside the main method.
import 'package:slang_retail_assistant/slang_retail_assistant.dart';
var assistantConfig = new AssistantConfiguration()
..assistantId = "<AssistantId>"
..apiKey = "<APIKey>"
SlangRetailAssistant.initialize(assistantConfig);
If you're using async/await, then -
import SlangRetailAssistant from '@slanglabs/slang-retail-assistant';
await SlangRetailAssistant.init({
assistantID: '<assistant id>', // The Assistant ID from the console
apiKey: '<API Key>', // The API key from the console
})
If you're using Promise callbacks, then -
import SlangRetailAssistant from '@slanglabs/slang-retail-assistant';
SlangRetailAssistant.init({
assistantID: '<assistant id>', // The Assistant ID from the console
apiKey: '<API Key>', // The API key from the console
}).then(() => {
// other integration code...
})
If you are using a framework like React, we recommend putting this code inside a
useEffect block.
If you are building a simple HTML project, then put this code inside an document.addEventListener('DOMContentLoaded', () => {}) event listener.Trigger refers to the UI element that will appear on the screen, which the end user will click to bring up the Assistant.
The most common way to add Voice Search into apps, i.e. adding the trigger (which by default is a microphone icon) is either inside or next to the search bar.
Inline Trigger
Android
iOS
React Native
Flutter
Web
Add the below XML element to your UI definition in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).
<in.slanglabs.assistants.retail.SlangConvaTrigger
android:id="@+id/your_id"
android:layout_width="45dp"
android:layout_height="45dp"
/>
///1. Just set the custom class as `SlangConvaTrigger` for an UIImageView on Identify Inspector panel
///2. Drag the element into the ViewController and you should see the line below.
@IBOutlet weak var trigger: SlangConvaTrigger!
weak var trigger: SlangConvaTrigger!
override func viewDidLoad() {
super.viewDidLoad()
/// Init the SlangConvaTrigger
self.trigger = SlangConvaTrigger()
/// Add the button to the view
self.view.addSubview(self.trigger)
self.trigger.translatesAutoresizingMaskIntoConstraints = false
/// Align the trigger on the view
let views = ["trigger" : self.trigger!]
let verticalButton = NSLayoutConstraint.constraints(withVisualFormat: "V:|-(>=0@299)-[trigger(64)]-40-|", options: NSLayoutConstraint.FormatOptions(), metrics: nil, views: views)
let horizontalButton = NSLayoutConstraint.constraints(withVisualFormat: "H:|-(>=0@299)-[trigger(64)]-20-|", options: NSLayoutConstraint.FormatOptions(), metrics: nil, views: views)
self.view.addConstraints(verticalButton + horizontalButton)
}
Add the below element to your UI definition in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).
const styles = StyleSheet.create({
triggerStyle: {
width: 45,
height: 45,
},
});
<SlangConvaTrigger
style={styles.triggerStyle}
/>
To specify a custom image for the microphone icon, please specify the
src field in the trigger element.src: A string representing the remote URL of the image.
Example :
<SlangConvaTrigger
src={{uri: Image.resolveAssetSource(require('../Images/trigger.png')).uri}}
style={styles.triggerStyle}
/>
Incorporate the following component into your UI definition at the desired location to display the trigger (typically represented by a microphone icon), often positioned alongside the search bar.
Container(
height: 45,
width: 45,
child:SlangConvaTriggerView()
)
To specify a custom image for the microphone icon, please specify the imageUrl or imageAsset field in the trigger element.
imageUrl:- A string representing the remote URL of the image.
imageAsset:- A string representing the asset path of the image.
Example:-
Container(
height: 45,
width: 45,
child:SlangConvaTriggerView(imageUrl: "<url link>")
)
Or
Container(
height: 45,
width: 45,
child:SlangConvaTriggerView(imageAsset: "<assets/your_image>")
)
Add the below HTML element to your HTML markup in the place where you want the trigger (the default image is a microphone icon) to appear (usually next to the search bar).
<slang-conva-trigger></slang-conva-trigger>
To specify a custom image for the microphone icon, please specify the
src attribute in the trigger element.src: A string representing the remote URL of the image.
Example:
<slang-conva-trigger
src="/images/custom-img.png"
></slang-conva-trigger>
If you are using typescript, then you might get the following error -
Property 'slang-conva-trigger' does not exist on type 'JSX.IntrinsicElements'.
To resolve this, create a
declaration.d.ts file and add the following to this new file - 1
declare namespace JSX {
2
interface IntrinsicElements {
3
"slang-conva-trigger": any;
4
}
5
}
Note that by default CONVA will show a coachmark on the trigger when it shows up for the first time on the screen. Refer to the API below for details to control it
Once the app has integrated the trigger, the next step is to register a callback to handle the voice search commands from the end user. CONVA will process the utterance spoken by the end user and if it detects that the user is trying to do a valid search operation, it will invoke the registered callback with the search string. Note that the search string will always be in English irrespective of which language the end user spoke it in. CONVA will automatically translate other language inputs into English.
Android
iOS
React Native
Flutter
Web
SlangRetailAssistant.setOnSearchListener(
new OnSearchListener() {
@Override
public void onSearch(SearchInfo searchInfo, final SearchUserJourney searchUserJourney) {
String searchString = searchInfo.getSearchString();
// Fire the actual search using the searchString
}
}
);
Or, utilize the onSearch callback provided by the SlangConvaTriggerView component.
SlangConvaTrigger convaTrigger = findViewById(R.id.your_id);
convaTrigger.setOnSearchListener(
new OnSearchListener() {
@Override
public void onSearch(SearchInfo searchInfo, final SearchUserJourney searchUserJourney) {
String searchString = searchInfo.getSearchString();
// Fire the actual search using the searchString
}
}
);
SlangRetailAssistant.onSearch = { (searchInfo, searchUserJourney) in
let searchString = searchInfo.searchString
// Fire the actual search using the searchString
...
}
SlangRetailAssistant.setAction({
onSearch: (searchInfo, searchUserJourney) => {
String searchString = searchInfo["description"]
// Fire the actual search using the search string
}
});
SlangRetailAssistant.setOnSearchListener((searchInfo, searchUserJourney) {
String searchString = searchInfo.toString();
// Fire the actual search using the searchString
});
Or, utilize the onSearch callback provided by the SlangConvaTriggerView component.
Container(
height: 45,
width: 45,
child: SlangConvaTriggerView(
onSearch: (searchInfo, searchUserJourney) {
String searchString = searchInfo.searchString;
// Fire the actual search using the searchString
}))
SlangRetailAssistant.setOnSearchListener({
onSearch(searchInfo, searchUserJourney) {
const searchString = searchInfo.item.description;
}
})
Last modified 27d ago