|
Speech Quality Testing Because of its importance to the success of converged communication systems, CT Labs maintains a strong focus in the testing and verification of speech quality performance with a recognized ability to verify speech quality under real-world conditions. CT Labs utilizes the latest automated quality measurement techniques including PESQ. Through its qualified test lab partners, CT Labs can also conduct true live-listener Mean Opinion Score testing. |
 Automated Voice User Interface (VUI) Testing
CT Labs has extensive experience developing customized automated tests that exercise the voice-controlled user interface (VUI) of dial-in applications. CT Labs can create tests using automated call generators that interact with these interfaces by simulating live caller utterances. Such utterances are previously recorded and categorized based on speaker demographics and characteristics, noise environments, and telephone device types. Such tests can be designed to:
CT Labs can support VUI tests with telephone line densities from a single line to hundreds of lines. For further information on CT Labs' capabilities to test core speech recognition performance, click here. |
Automated Telephone User Interface (TUI) Testing CT Labs has extensive experience developing customized automated tests that exercise telephone user interfaces (TUI) that allow callers to control call information using their touch tone telephones. CT Labs can create suites of tests using both commercial and in-house automated call generator products that can interact with these interfaces by simulating live caller actions. Tests can be designed to:
Test runs can be designed for very short (minutes) or days long durations as your needs dictate. CT Labs can support telephone line densities from a single line to hundreds or even thousands of lines on special request. Comprehensive post-test analysis is usually provided via the concluding test report. |
Real-World Stress and Load Testing CT Labs specializes in creating high density VoIP and TDM-based inbound and outbound automated call test scenarios. The goal of this class of tests is to provide a worse-case call environment that evaluates the product-under-test for its ability to handle high call loads without excessive delays, dropped calls, degraded voice quality, or call-handling errors. Testing methodology can include the creation of database-driven call scenarios that allow a specific set of initial test conditions and a list of originating calls-and the actions and parameters within each call-- to be exactly specified. This technique allows for exactly repeatable test runs, thus providing a better chance that any detected problems can be reproduced. Call arrival profiling makes it possible for CT Labs to develop incoming call scenarios that closely mirror actual call volumes over a typical work day or work week. Such real-world call profiling results in a higher probability that all critical bugs will be exposed before the product is officially released. CT Labs engineers can construct positive path and negative path tests that exercise a product's VUI and/or TUI for all possible correct (positive) touch tone or spoken command sequences as well as a wide variety of erroneous (negative) caller command inputs. An example of negative TUI tests is where the simulated caller enters their password incorrectly three times in a row, or perhaps waits too long at various menus to enter or speak their command, or even hangs up prematurely. In each "negative" test, the product-under-test should have a well-defined and predictable response that gets the caller back on track. Combining both positive with negative test strategies can create a testing environment that has the best overall chance of success in preparing a product for operation in the unforgiving real world. |
|
Speech Recognition Performance Testing CT Labs has considerable experience in testing speech recognition-based products, including the latest voice portals, interactive voice response (IVR) systems, and auto attendants. These systems are characterized by their ability to respond to spoken commands from callers without the need to individually train each user on the system. CT Labs also has expertise in evaluating core recognition engines from many leading recognition technology vendors. Speaker-independent systems can be tested by CT Labs using a variety of test scenarios that play specially-recorded spoken commands from a variety of callers selected to match the demographics of the target caller population. The results of these test calls are then accumulated and used to quantify the degree of success in recognizing and appropriately responding to these commands. If desired, CT Labs can generate special spoken-command files with degraded audio conditions added, such as traffic or wind noise that simulates a cellular phone caller in less-than-perfect call environments. Speech recognition tests are typically performed on your product in our lab. Results of these automated test exercises are typically compiled and used to produce a confidential product performance report that is delivered to our client. CT Labs evaluations can include an assessment of a wide variety of speech recognition performance metrics, including:
|
Text-to-Speech Performance Testing CT Labs has experience in evaluating the speech quality and performance of text-to-speech (TTS) based systems. This includes validation of the TTS component as part of any turnkey application as well as performance testing of core TTS engine products. CT Labs uses its own proprietary live listener evaluation methodology that divides the assessment into two phases:
The quality evaluation listening phase involves having trained CT Labs listeners produce listening effort and listening quality scores (using MOS standard scoring definitions) for selected TTS speech segments. The anomaly count listening phase utilizes trained CT Labs listeners counting the number of occurrences of audible anomalies in a text segment including mispronounced words, incorrect punctuation, incorrect acronym expansions, and more. Tests can be performed on single TTS products as well as multiple competitive products for comparative purposes. Results of these evaluations are typically compiled in a confidential performance report that is delivered to our client. |