Getting Started with AI for Testing
Preparing the Human Brain for the Rise of the Machines


Last week I had the honor of giving the opening keynote at StarWest on the topic of Artificial Intelligence for Software Testing (AIST). The talk, entitled “Rise of the Machines: Can Artificial Intelligence Terminate Manual Testing?, created quite a bit of buzz at the conference. You can check out the video below if you missed it. Aside from the “Do you think AI will take over the world?” type questions, one of the most popular things I was asked about after the talk was: “How do I get started with AIST?”

Video: Rise of the Machines StarWest 2017 Keynote

The short answer to that question was to point delegates to general resources on AI and machine learning. However, it is important to recognize that such a question is loaded and therefore deserves a more detailed answer. First let’s start by revisiting the definition of AIST.

Artificial Intelligence for Software Testing (AIST) is an emerging field aimed at the development of AI systems to test software, methods to test AI systems, and ultimately designing software that is capable of self-testing and self-healing.
 — AISTA (

Examining the definition we can see that how to start depends on two things: what you are trying to achieve and where it lies in the intersection of using AI to test software, testing AI-based systems, and building software with intrinsic self-testing and self-healing capabilities. A safe place to start is pursuing knowledge and understanding in the areas of artificial intelligence, software testing, and self-managing systems.

Artificial Intelligence

AI is not new. It has been around since the 1950’s and knowing its historical progression and major advances over the years will help you to position your work in the AI space. Creating the vision for your tool, method, or approach, acquiring the right talent, aligning team members, developing the solution, and communicating the value of your work all start with positioning.

The field of AI is broad and so I recommend that you gain an understanding of that breadth before you do a deep dive into specific areas. Artificial Intelligence: A Modern Approach by Stuart Russell and Peter Norvig is a great reference book for all things AI. When you are ready to go deeper, you’ll want to get your hands on more specialized books. For example, Deep Learning by Ian Goodfellow presents a broad range of topics on hierarchical learning architectures. In terms of freely available self-paced multimedia learning, check out the video courses on YouTube from Andrew Ng (Stanford University) and Lex Fridman (MIT).

Software Testing

For as long as there has been software, there has been software testing. Despite the rise of other approaches such as formal verification and model checking, testing remains the primary means of validation and verification (V&V) used in the software industry today. Like AI, the field of software testing is extremely broad, but chances are that if you are looking into AIST you may already have a background in testing. However, if you don’t and are looking for online resources on software testing, check out this post on 45 Websites Every Software Tester Should Bookmark in 2017. For books, some old favorites of mine are James Whittaker’s series on How to Break Software, Lee Copeland’s Practitioner’s Guide to Software Test Design, and Cem Kaner’s Testing Computer Software.

Self-Managing Systems

In 2001, software giants IBM, Microsoft, and HP all set out on their own initiatives to define the next generation of computing systems. Such systems were envisioned to be self-managing, and able to adapt to changes in order to meet the needs of its users. IBM coined the term autonomic computing and modeled their approach against the human autonomic nervous system (ANS). The ANS regulates critical bodily functions such as homeostasis without conscious involvement. Applying this idea to computing, systems should adapt to their environment through self-configuration, self-optimization, self-protection and self-healing. What? No testing? Although the giants weren’t talking about it, in order for autonomic computing to work in practice, self-testing needed to be an inherent part of the system.

It was for this reason that I decided to pursue this research area during my graduate studies from 2005–2009. Unfortunately to date there is still a general lack of research in the area of self-testable autonomic software. However, the recent advances in AI/ML, Cloud Computing, DevOps, and Microservices, have improved the potential for self-testing to provide a formidable solution to the grand challenges of software testing. We’ll be posting some self-testing and autonomic computing foundation papers on AISTA’s website for those interested in learning to how self-testing systems can be designed and developed.

Other AIST Presentations

In addition to the Rise of the Machines keynote, StarWest this year featured two other talks on AIST. Jason Arbon gave an updated version of his talk “AI and Machine Learning for Testers”, originally presented earlier this year at StarEast. Paul Merrill from Beaufort Fairmont presented a concurrent session on “Machine Learning and How it Affects Testers.

Video: Machine Learning and How it Affects Testers

What’s Next?

As much as you can read books and watch videos about AIST, there is nothing more powerful than getting hands to keyboard. In our next post, we’ll cover some tools that you can play with to get a feel for AI and ML.

 —Tariq King  ( Co-Chair, AISTA)


Survey: When does all this impact the world?


When will AI begin to impact Manual or Automated Software testing?  In the footsteps of Nick Bostrom, we created a survey to poll the crowd to see what they thing.  Please take the survey and we'll share the results soon:

--Jason Arbon(Co-Chair, AISTA)


Hello, AI for Software Testing


Welcome to the Artificial Intelligence for Software Testing Association (AISTA). Software testing has yet to fully benefit from the advances in Artificial Intelligence (AI). Classic approaches to software testing are being disrupted by new AI-based products that require new approaches to data driven testing, validation, and measures of quality. AI also presents the opportunity to consider novel approaches to enable software to, well, test itself. AISTA exists to explore these intersections of AI and software testing.

AISTA was born in the hotel bar of a recent software testing conference. We realized there were other software, AI, and testing nerds just like us, but we didn’t have a focused forum to share our findings and collaborate. 

To support AISTA’s mission, we will:

  • Organize online and offline events.
  • Distribute a blog and newsletter highlighting work in the intersection of AI and software testing.
  • Host repositories of open data and code.

Join AISTA at to keep in the loop on AI in Testing, and submit papers on recent work to share with the community.

 — Jason Arbon (Co-Chair, AISTA)