Speech To Text Awesomeness in Siebel Open UI - Part 1

If you have been following the various articles in the Siebel Open UI realm over the past two years you have probably seen a demonstration of speech recognition to provide field values via the device microphone.

The example code with this demo boils down to one line like this:

$('input[name="' + controlInputName + '"]').attr("x-webkit-speech","x-webkit-speech");

The above code is adding the x-webkit-speech attribute to a text input control so that we can simply click or tap a microphone icon and start speaking. Note that only Google Chrome supported this feature. Supported? Read on...

As you also probably know, this humble writer and his fellow authors are working on the Siebel Open UI Developer's Handbook (to be released early 2015). Of course, we aim to provide as many working examples as possible and the speech-to-text demo is something we have on our list.

However, while creating the demo code for the book we found to our dismay that the x-webkit-speech attribute is deprecated. Bummer...

HTML5 Rocks

This is where HTML5 (and any browser that supports it) comes to the rescue with the Speech Recognition (Web Speech) and Speech Synthesis API. Support for these APIs is currently limited to Google Chrome and Apple Safari browsers.

So we set out to give speech recognition in Siebel Open UI another chance and are happy to provide a working solution using the HTML5 APIs. In this and the following post we will describe how to use the following ingredients to create a custom physical renderer for a form applet:In 

• Display a microphone button in a text field using a scalable icon by Font Awesome
• Allow speech input using the HTML5 Web Speech API
• Animate the microphone icon during recording with CSS animation
• Write the recognized text to a field according using the Siebel Open UI API (of course)
• Play back the recorded text using the HTML5 Speech Synthesis API

Let's get started with a screenshot:

The highlighted microphone icon (in the Job Title control) is all the user can see. Tapping/clicking the icon will cause the browser (again: only Chrome or Safari support currently support this) to prompt the user that the web page wants access to the device microphone. After allowing that, the user can talk. Once she or he finishes talking, the field will be updated with the recognized text.

In the remainder of this first part we will lay out the foundation in terms of creating and registering a boilerplate PR and downloading the necessary files for Font Awesome.

1. Create and Register a Boilerplate PR

We assume a certain familiarity with Open UI at this point, so all we do here is create a new custom physical renderer file in the siebel/custom folder from a template.

• We suggest that you name the file ContactSpeech2TextPR.js.
• If you use the Siebel Essentials PR Template, continue by replacing all occurrences of 'PRTemplate' with 'ContactSpeech2TextPR'.
• Next, ensure that the call to SiebelJS.Extend looks like the following to have a proper extension for a form applet:

SiebelJS.Extend(ContactSpeech2TextPR, SiebelAppFacade.PhysicalRenderer);

At this point, the PR does not do much apart from printing messages to the console (at least that's what our template does), but that's ok for now.

Next, we register the new file in the Manifest Files view:

Off we go to the Manifest Administration view and associate the file with the Contact Form Applet for the sake of demo.

Finally, log off and on again and verify that the PR is loaded (look at the JavaScript console for the log messages if you're using the Siebel Essentials template).

If you are already a Siebel Open UI veteran, you might well be annoyed by this level of detail here. Here at Siebel Essentials we aim to please juniors and seniors alike. So if you can't wait and want to jump ahead, here is the complete example PR (with full speech recognition and synthesis). Note that you also need Font Awesome files and some custom CSS, so reading on is recommendable.

Prepare for Font Awesome

Font Awesome is an open source icon and CSS toolkit which provides hundreds of vector icons. The benefit of using scalable font icons instead of image files becomes clear when you consider different zoom levels, the amount and size of files to be downloaded and whether you want to be old-school or just cool ;-)

For "ninja-style" testing you can include Font Awesome in any web page (including Open UI) with one line of JavaScript like the following:

$("head").append('<link href="//maxcdn.bootstrapcdn.com/font-awesome/4.2.0/css/font-awesome.min.css" rel="stylesheet">');

Of course the above will require that your browser can reach the public internet. For a real-life Siebel production environment, we will be better off if we download the source files and put it on our own Siebel web server. Here is how to accomplish this:

1. Go to Fort Awesome and download the latest source archive.
2. Open the archive and extract the font-awesome.css file to the PUBLIC/%Language%/FILES/custom folder. Do not use the minified version (some editing is necessary so we better use the original file).
3. Create a FONTS folder in PUBLIC/%Language% (Note: This folder will be present in future Siebel versions, so you probably only have to do this only if you're on IP 2013).
4. Create a custom sub-folder within the FONTS folder.
5. From the Font Awesome archive, extract all font files to the PUBLIC/%Language%/FONTS/custom folder.
6. Edit the font-awesome.css file and replace all occurrences of '../fonts/' with '../../fonts/custom/' to comply with the placement of files in the Siebel PUBLIC folder.
7. Edit the custom theme.js file (that you hopefully have at this point, if not look here for a start on custom themes) and add a reference to the font-awesome.css file similar to the following example:

SiebelApp.ThemeManager.addResource(
    "TANGERINE_TAB",
    { css : {  fa: "files/custom/font-awesome.css"
        }}); 

Note that this applies to IP 2013. There is a high probability for future posts on custom themes in IP 2014...
Repeat the above step for any standard and custom theme you might be using.

Quite a stretch for a small icon but now we have the full power of Font Awesome at our fingertips.

Please also visit the next part where we will actually bring all the pieces together to capture your beautiful speaking voice in the Siebel database ;-)

have a nice day

@lex