Is Domain Knowledge Necessary for Testing?

Domain knowledge is necessary. However, how much domain knowledge is needed is far lower than technology professionals believe today, primarily as it depends on what your goal is.  At Webomates, we have successfully ramped up in completely different domains with a few hours of involvement from our clients. to clarify complex and/or customer specific domain questions.

As a matter of fact, for end User Interface testing we have consistently managed to ramp-up in new domains and create test cases with a target that 85% of the test cases do not require domain clarification from the customer. We achieve this typically within 2 weeks with any new customers and currently operate in numerous domains.

What is Domain Knowledge?

Here is the definition from Wikipedia

Domain knowledge is knowledge of a specific, specialized discipline or field, in contrast to general knowledge, or domain-independent knowledge. The term is often used in reference to a more general discipline, as, for example, in describing a software engineer who has general knowledge of programming, as well as domain knowledge about the pharmaceutical industry. People who have domain knowledge, are often considered specialists or experts in the field.

The implication in the very definition of Domain Knowledge is based on becoming a business or technology expert in a particular domain. This is confusing as it implies that either you are an expert or you DON’T have Domain Knowledge! However, in reality, the knowledge that you have about a domain lies on a continuum. We can define the level of Domain Knowledge at a couple of different levels:

1. End User level – the person understands the domain well enough to operate as a user in the domain. For example, if you are in the retail banking domain the person should be able to log in to their bank account and check their balance, look at the balance etc. This is called blackbox testing knowledge. Here the user has no understanding of how the application works – it’s a “blackbox” to them. Experience can be as low as a few weeks to achieve this level.
2. Skilled level – Here the person needs to understand not just how to use the system but also what happens within the application and why. For example, in online travel, the person would need to understand not only how to book a flight but that most online travel systems connect to a GDS (like Sabre) to get the available inventory.
3. Knowledgeable level – At this level of knowledge, a person can give input and feedback on a new feature that needs to be developed for a system.

1. Expert level – this person is steeped in the domain and can define a new product or new features in a product that will address underserved business or market needs.

Often times when people say that domain expertise is incredibly time consuming to acquire they are talking about System or Business understanding. And they are right. It takes time, sometimes years of effort and experience to pickup that kind of knowledge and Expert understanding. But like many continuums, there is a disproportionate increase in effort in order to achieve master or expertise in a particular subject area.

How Much Domain Knowledge do You Need?

The above graph outlines the level of Domain Knowledge in the Technical and Business domain that are needed in a typical technology company across several job functions. In each job function, the minimum level of domain knowledge is identified in order to carry out a job function. The table is not a comprehensive list but is used primarily to illustrate the level of domain knowledge that is necessary.

User Interface testing, as the name should and does imply, needs only End User level understanding! This is the most basic level of Domain Knowledge that is required.   A product manager that is defining the roadmap and the features for a particular product or service is at the other end of this spectrum as (s)he needs to not only understand the current product or service but also understand the market intimately and define what new features the company can create and sell. Thus, product managers have many years of industry business experience. In a similar manner, a software architect has, for example, many years of experience in the technology domain. In addition, often there are people that have varying levels of expertise in both the Technology and Business domains. However, the key point is that user level of understanding requires low levels of business and technology domain knowledge.

How Quickly can You Acquire User Level Domain Knowledge?

Very quickly – if you spend the time, focus the effort and keep an open mind. We use a variety of techniques outlined below to rapidly get to 100% of the test cases that we set out to create. Typically, this effort is completed in 2 weeks and requires only 8-10 person hours of time on the part of the application creators.

You May Like to Read

• Software Testing and the 5W and 1H technique of Metrics Reporting
• Importance of Artificial Intelligence Software Testing Tool

Explore: User Interfaces are for Humans – Humans can Explore and Discover!

The first technique for acquiring domain knowledge rapidly is to explore the User Interface. Get access to the system and navigate through all the capabilities. If there are any help manual user docs etc read them.

1. Taking a systematic approach to the exploration and understanding the user of the application is necessary for this step. Putting yourself in the end user’s shoes, like being the call agent for a call center application, or someone who wants to book a flight for an online travel website is essential to really understand the application and creating relevant test areas and test cases.
2. Thorough review – Obviously go through every screen and every option in the application. Relating input to output presented and asking “why” is another important step to acquiring a better understanding.
3. Identify key flows – Understand the purpose of the application and identifying the key flows that users will take through the User interface. It may be that there is an incredibly complicated reporting package, but if the main purpose of the User Interface is to provide the balance in your bank account then the focus needs to remain in that area. Start building a thesis of where the end users spend most of their time and get most of their value out of the application.
4. Identify domain areas and questions that don’t make immediate sense.

On average we find that over 50% of the test cases can directly be identified by pure exploration!

Research – Google it! Or YouTube it! (Still Google)

There is an amazing amount of information on the Internet. Everyone knows this. And Google puts it at your fingertips. Almost any acronym or concept that is exposed in an application is explained by someone, somewhere on the Internet. Go and find it. This is just part of doing your homework to understand how an application works. Often companies and end users have videos on Youtube for the application. Review them and understand what is important to users. Go check forums to see if there are any comments on the product or service. It’s all out there!

We can raise the number of test cases that we find from 50% to 70% to 75% by carrying out research!

Pair Researchers and Document it

Have multiple persons follow the above techniques and start creating an overall knowledge document for the application that is under review. Different people use different techniques for approaching how to test an application. This results in two things happening:

1. Non overlapped test case areas increase the coverage to between 80% and 85%
2. The focus of the tests is thoroughly reviewed for applicability

Customer

The people with the greatest domain are the people that created the application that we are focused on. By having short duration calls we get domain clarification for the areas that are non-public information as well as guidance on what area to focus the testing. As the calls are short (typically 30 to 45 minutes) we do not take a lot of the customers’ time yet at the same time are able to refine and focus the test cases to the area that the application creators feel will yield the most breakage in the system.

In this manner, 100% test case creation is achieved in an incredibly short period of time by Webomates!

Conclusion

Domain knowledge is very important and requires almost a logarithmic increase in time spent in a domain for expertise to be achieved. But feature/regression testing particularly focused on testing as an end user or what is called blackbox testing, can be achieved incredibly quickly by focusing a large amount of energy and using a structured process as defined above. Even more importantly the application creators do not have to spend a large amount of time (8-15 person hours) in order to achieve 100% test case creation and approval of the test cases.

At Webomates, we have worked hard to create the structured process defined above. And we have applied this technique in multiple different domains. We specialize in taking many different software creators’ builds across domains, on 18 different Browser/OS/Mobile platforms and providing them with defects in 24 hours or less. If you are interested in a demo click here Webomates CQ

Tags: Domain Knowledge, Software Testing