Teaching Patience to Automated Tests

Have your automated test steps ever outrun the web application it is testing? Imagine a test that enters an invalid password on a login form, clicks the “Sign In” button, and immediately tests for the presence of an “Invalid username or password” message. If the test does not find that message when it looks for it, it will fail – even if the application displays the message a few milliseconds later.

Oddly enough, the slower execution speed of manual test steps is an advantage here, as is the human capacity for patience. Humans barely notice the milliseconds it takes the server to reject those faulty credentials. If they notice anything, it will be visual cues on the browser – hourglasses, spinning circles – counseling patience.

The trick, then, is to teach automated tests to be patient.

One naive but common solution is to scatter fixed pauses at strategic points in your tests. Pause your test execution for four or five seconds at the hot spots, to let the web application catch up. Inevitably:

  • For the vast majority of runs, the pause is far longer than needed, meaning your runs take much longer than they need to;
  • For an (annoyingly large) minority of runs, the pause is not long enough, meaning your test has a not insignificant failure rate, even when the application is functioning as expected.

Fixed pauses produce tests that are simultaneously unbearably slow and unacceptably flaky – a most unhelpful combination.

A better solution is to pause only until the expected condition obtains, or until a maximum acceptable timeout expires – whichever comes first. Unlike fixed pauses, these “smart waits” go as fast as the application under test does.

Smart Waits in Toffee

Toffee implements these smart waits in several commands:

wait 30 seconds for pages to load – sets the maximum amount of time to wait for pages to finish loading. Feel free to use large timeouts here: if it only takes 2 seconds to load the page, this command will only take 2 seconds. This command is “sticky” – that is, it applies to all subsequent page loads in your workspace (until you clear it) or your script (until it completes).

wait 15 seconds for commands to execute – sets the maximum amount of time to wait for an element that is not present. This command is also sticky. Again, feel free to use large timeouts, but be aware that negative existence tests will use the full timeout. For example, if you issue the following command sequence:

wait 60 seconds for commands to execute
test that element with id "errorMessage" does not exist

The second command will take 60 seconds (!!) to complete.

wait 15 seconds until button "Archive" exists – an example of the wait…until command, which wait up to a maximum duration for some condition to obtain. You can test whether an element exists (or not), is visible (or not), or is enabled (or not). This command is not sticky: the timeout applies only to this command. It returns as soon as the condition obtains – even negative existence – affording finer-grained control over timeouts than the wait <duration> for commands to execute command.

Get Started Today!

Does your web test automation solution support smart waits? If not, join Toffee today. For more information about Toffee’s “smart waits” see the Toffee documentation.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s