In this blog, we’ll cover a relatively simple and maintainable way to create tests for your website using the buzzwords, “AI (Artificial Intelligence)” and “ML (Machine Learning).”
I would love to live in a world where everything just works. Can you imagine that?
Imagine if parents and grandparents knew to “turn it off and on again” before they call because this is the time they remember that their problem has always been fixed by first turning the device off, and then back on again. Incredible.
Imagine you are responsible for creating a new website for your latest company-wide initiative in this perfect world.
Finding information on your target demographics would just be available and defined. The perfect color scheme to help convey the right look and feel of your brand and idea to the users has already been made into a style guide. You work with a team of user experience and visual design experts to help bring the concepts to life. It’s all so easy.
The engineering team builds the website with no bugs. Every pixel is where it should be, and every form is submitted with ease. The mobile experience is stunning.
Every critical data point is coming through to your analytics dashboard, in real-time. The website took two days to build from start to finish and conversions are through the roof. Bravo, you did it. Incredible. You created a perfect website from start to finish in a perfect imaginary world.
Even if you are somehow able to create a perfect website, the site will soon start to unravel and decay as various teams update it and the site weathers the ravages of time. Your beautiful and informative modal was perfect until you found out that Steve somehow changed the text copy a week after launch to “like, be way more chill”. Steve was the intern who misread an acceptance criterion.
The best way to defend the critical path of your site from you, your team, and time is to create an instance of the fabled maintainable test suite.
In general, there are at least two main problems with test suites. First, test suites are difficult to create due to the technical expertise required to write the tests. Second, they are difficult to maintain, since every tiny site update requires manual adjustments to the test suite. These two reasons are why well-maintained test suites are as rare as five-leaved clovers. Testim.io solves both these problems with the use of clever UX and AI.
UI Tests
UI tests help you verify that your general user experience stays as it was initially set up. You can create a test that confirms a certain widget exists on a page or confirms that a form submits when filled out correctly. You can also use UI tests to confirm error states when submitting bad information. The tests can be run on a schedule or even connected to your CI pipeline to be run when a developer makes a commit.
Running a set of tests that ensure the critical parts of your website are working, and things are where they should be, can catch errors before they happen. UI tests also help prevent collisions when multiple people or departments are working on one website with shared pages.
Testim.io
There are many ways to create UI tests. In this post, we’ll cover testim.io specifically. Testim.io is one of several of the new breed of test creation tools that allows you to both create, and more importantly, maintain a suite of complex multistep tests.
Testim.io is a service that uses AI to help make the process very easy for general test cases. While they offer more in-depth tooling geared for developers, you can get started in just a few steps with their codeless testing feature.
Getting Started
Take a look at this video to understand how to start using Testim with their Chrome browser extension. You’ll have to use Chrome, Brave, or any other chromium-based browser with the testim.io extension. The extension is what we will be using to integrate with the test service.
So, go create an account and install the Testim chrome extension. I’ll wait. Make sure to go to chrome://extensions and toggle “Allow in private” for the extension. Take a look at the Testim documentation too, if you have a chance.
With the account created and the extension added you are ready to create your first test. When you run the test the testim.io service will open a new window that will record what you do. Try to be accurate with your keystrokes, clicks, and scrolling because it will detect those events and replicate them in the test. When you are done you hit the stop button and the steps will be displayed in the test suite.
Premise
Let’s say we have a contact form and some users are reporting that it is not on the page sometimes. It’s just not there. A very good way to make sure your contact form is there and that it also submits is to write a test that confirms that those things are true. While you are at it you should make sure it doesn’t submit when it is not filled out correctly as well, right? Better safe than sorry.
Test Case: Contact Form Submits When Filled Out
We want to verify that if you fill out all required fields of the contact form it will submit and in this case take you to a thank you page.
We want to capture the following for our test:
- Filling out the form
- Submitting the form
- Being shown a thank you page on successful submit
Let’s Start
- Navigate to the contact page in the browser
- Click the Testim extension in your toolbar and select “Create automated test”. You can create new tests from inside the editor as well.
3. A new window will open which will record your actions
- Fill out the form completely (be accurate with your keystrokes, clicks, and scrolling)
- Submit the form
- Click on a part of the confirmation message
- Hit the stop button and return to the other browser window
At this point you should see the steps you performed in the testim editor. It recorded the start (setup), clicking select boxes and filling out the text fields, and clicking the submit button.
The last step was clicking on “Thanks for reaching out” text on the confirmation page. We don’t actually care about recording the click event on that text, but we do want to verify the text does actually say “Thanks for reaching out”, so we need to modify that step.
CRITICAL NOTE: Before going any further, make sure you use the dropdown by the play icon in the editor and select “run in incognito mode” – this will really help mitigate any issues caused by cookies and ensures your tests start from a point with no prior information saved.
We can use this last recorded click event we performed and set a Testim predefined step. The predefined steps offer a lot of the functionality you will be looking for, like:
- Validating email addresses
- Validating CSS properties
- Validating visibility
- Validating text
- Setting Cookies
- Waiting for other elements to become visible
Let’s update that click event, and instead make sure the text we expect is there.
- Go to the plus icon next to the event and select the “M” button (predefined steps)
- In the window that opens select the Validations drop-down and scroll to “Validate element text”
- In the test window click on the “Thanks for reaching out!” text
- The new text validation event is now the final item in our test. I used the trashcan icon to delete the previous click event and remove it from our test.
Note: If you record any events that are not valuable to the test, you can find them in the editor and simply remove them with the trashcan icon. Here is a window resize event that I do not need or want:
Now you have a basic test that runs in a browser, submits the contact form, and also makes sure the confirmation message is what it is supposed to be. There may be times when you need to modify certain steps to use validation when it’s useful. If you experience tests not running or getting stuck on a specific step, check that you are running incognito mode and if needed scrap your test and try again.
Managing UI Updates
A big pain point with test suites is that if you update your markup, you have to update your test. This is not the case with testim.io. Let’s change the name and ID of one of our form elements and see if testim.io picks up the change automatically.
We changed the textarea name and id from “field[10]” to “test”. This should normally break a UI test because the identifiers of those particular elements have changed.
….But, we run the test, and voila! Testim.io picked up the change with no problem and the test passed.
This ability significantly reduces the amount of upkeep required when maintaining a test suite. If we removed the textarea completely, then the test fails because it’s expecting to fill out an additional form element that is not there, but name and style changes should be picked up by the system.
Test Case: Contact Form Does Not Submit With Error
This is very similar to our first test but we are going to leave a required field blank and try to submit, then we will verify that we saw an error message saying “This field is required”. This will ensure the form does not submit when not all required fields are filled out and that our form validation is working.
- Run a new test (either from the editor or the extension) and go to the contact form URL
- Fill out the form but leave a required field empty
- Submit form
- Click on error message
Now the only thing we need to do is change the click on “this field is required” event to perform validation that the right text is displayed. Use the “M” button to use the same “Validate element text” validation and click on the error message in the test window. You can now delete the unnecessary click event.
Running Your Test Together
You can set your tests to run on a normal schedule but first, let’s do some organizing. I now have 2 tests and we’re going to move over to our test list.
We want to create a test suite so we can run as many or few tests as we want. Let’s create a new test suite called “General Test” and add our tests to it.
Now we have a test suite, you can run it by selecting it and hitting the play button
Check “Run as incognito”
This will take you to your runs and display the result. We passed!
Running On A Schedule
It would be nice to have these tests run every night and email you in case they break, let’s head to the scheduled runs section in the runs area.
I want to run my test suite every day of the week at midnight
And I want to be notified of every test failure at my email address
Bonus points if you set up slack integration. It would be very useful to have these tests running throughout the day alerting a “UI Test Suite” Channel.
Final Notes
We covered some extremely basic functionality and thetestim.io suite can go much, much deeper. With tools like this you can rapidly deploy tests and bring stability to your website especially when there are too many cooks in the kitchen.
