When setting up an upRTC WebRTC monitoring script, what you are looking for is to reduce the amount of false positives to a minimum. You don’t want to get alerted when what is broken is the test script itself and not your service. Here are a few suggestions that you should implement in your monitoring script to reduce the number of potential false positives.
Set a short total run time
- Keep the total test length at 3-4 minutes. Less if you can handle it
- Have the media flowing in the session itself at 60-120 seconds, so that there’s enough data to look at and analyze
Reduce the number of fields and forms that need to be filled
If your service has long forms or many fields that need to be filled to get to the media itself, then this might be a bit of a challenge to automate. The reason for that is that automation tools may fail with form filling the UI elements every once in a while which will again lead to false positives.
Our suggestion here is to reduce the number of fields and forms that the script will need to fill to get to the actual media. If possible, have these filled out using an API call, making the flow more deterministic and simple in nature.
Take care of clean up via an API
Oftentimes, getting a monitor to run on a scheduled service is a bit of a challenge. Your application might have some logic that keeps session state or user state open or at a condition that won’t let the next session start properly. This usually boils down to having call queues clogged or emptied, sessions marked as open when they shouldn’t be, etc.
Getting to such a condition can happen when a single sporadic failure happens which doesn’t enable the whole script scenario to be used for the monitor to go from start to finish nicely, leaving things hanging around and causing failures in the following runs.
If your service is like that, you should have an API or a similar mechanism that can remotely clean up the context that the monitoring script uses. The script should call this API at the beginning using rtcActivateWebhook() to make sure it is running properly and that the session that is about to start isn’t affected from old data of a previous run.
Avoid user related pop ups
Some services like to have pop-ups and messages appear to users. These are used to collect metrics and statistics like NPS (Net Promoter Score), customer feedback, or to notify a user about new features or promotions.
These tend to kill automation, since it changes the behavior of the UI. While humans can easily avoid such distractions and close redundant windows, automation will have a hard time anticipating this and scripting for it in advance.
Our suggestion? Ensure that the user account utilized in the monitoring test script has all marketing automation tools disabled.
Use waitForElementVisible() instead of pause()
This is just best practices in writing automation scripts, but it is doubly important in WebRTC stress testing, where processing time can stretch a bit longer than usual.
client
.waitForElementVisible('#login', 30000)
.click('#login);Use pause() after waitForElementVisible()
Now that you are using .waitForElementVisible(), here’s a non-obvious suggestion – add a pause() after it. Specifically when you are running a load test. The reason for it is that sometimes, with slow servers and high CPU load, there can be delays between the changes in the DOM to the screen rendering.
In such cases, .waitForElementVisible() will return but it might be too early for the .click() command.
client
.waitForElementVisible('#login', 30000)
.pause(1000)
.click('#login);