Turning Tester Experience and Tacit Knowledge into Explicit Knowledge

The Importance of Experience in Testing

Normally, software testing is conducted using systematic testing techniques based on specifications and design documents. However, in actual practice, there are cases where specifications are not sufficiently reviewed, or where there are operational assumptions that cannot be read from the documents alone.

The “intuition” and “instinct” of experienced test engineers compensate for these site-specific assumptions and user-specific ways of using software. Through the accumulation of past similar projects and defect cases, sensitivity to risk naturally increases. Testers with good “intuition” are sometimes called “god hands,” and they can find defects at an early stage, having a major impact on quality improvement. They know from past experience and knowledge where defects are likely to occur. As a result, they can anticipate and find defects that are difficult to catch with specification-based testing alone.

On the other hand, if the criteria for judgment exist only in an individual’s head, the quality of testing will decrease if the person in charge changes. It is important to be aware that while experience is an asset that improves the quality of testing, it can also be a risk factor in terms of becoming dependent on specific individuals.

The True Nature of “Intuition” Born from Experience

A tester’s “intuition” is thought to be the result of past experiences being unconsciously patterned and memorized. KKD (Keiken/Experience, Kan/Intuition, Dokyo/Guts), which has supported Japanese manufacturing, can also be said to be an accumulation of such empirical rules at its root.

However, intuition in testing is different from haphazard ideas. Based on the points that should be covered in specification-based and structure-based testing, it integrates and organizes empirical information such as “past bugs,” “modification history,” and “user behavior” in the mind, and functions as a “navigation” tool to preferentially search high-risk areas.

The more experience you gain, the more accurate the “navigation” becomes, allowing you to find defects effectively even in a short time. On the other hand, unless the process is verbalized and shared, you cannot escape from relying on the senses of veterans.

• View PDF for details

How to Efficiently Gain Experience

Experience increases over time, but simply accumulating years is not enough. By consciously choosing your environment and actions, your experience as a tester can be built up faster and deeper.

Participate in Diverse Projects

If you are only involved in projects with the same development style, the patterns you learn will be limited. By participating in as diverse a range of projects as possible, such as business systems, BtoC apps, and embedded software, you can broadly learn about points where defects are likely to occur and patterns of user behavior.

Expand the breadth and depth of your experience by participating in support for other teams.

Analyze Defect Patterns

When you find a defect, reflect carefully on why it occurred and where it could have been prevented.

Check bug tracking tools and tickets, and organize them by labeling them according to the cause or location of occurrence, such as “input check,” “concurrent execution,” “boundary value,” and “error handling.” You will then notice that similar patterns appear repeatedly.

By consciously collecting and analyzing defect patterns, it becomes easier to predict risks, and the accuracy of intuition also increases. Some companies consciously accumulate information such as past troubles and utilize it in development.

Learning from Failure

Rework caused by bugs missed during testing or errors in judgment is something we would normally want to avoid, but depending on how we reflect on it, it can be turned into a valuable learning experience.

Analyze questions such as “Why were we unable to establish this testing perspective?” and “At what point did we have a chance to notice it?” calmly as a team, and reflect the findings in the next test plan or checklist. Having testers and developers gather to conduct a retrospective for each testing phase or agile development iteration is also effective for effective test execution.

Learning from Experienced Testers

Observe experienced testers closely as they work, and ask about their thought processes, such as “Why did you perform that operation just now?” or “Why are you focusing so much on that screen?”

By drawing out the background, such as “There was a similar bug before” or “This screen is business-critical,” you will gradually begin to see the tacit rules that support the intuition of experienced testers.

Attempts to Convert Tacit Knowledge into Explicit Knowledge

Converting the “rules of thumb” and “intuition” of individual testers into a format that can be shared across the entire team is essential for preventing dependency on specific individuals and stabilizing quality. The key here is to organize this knowledge into explicit knowledge that is refined to a level where it can actually be used in the field.

Creating Checklists and Guidelines

An easy first step is to write down the tacit rules in the minds of experienced testers, such as “Always check here” or “Always test boundary values under these conditions,” as checklists or guidelines. It is also important to draw out the methods actually used in the past to uncover bugs.

Rather than aiming to create a perfect list from the start, it is easier to adopt in practice if you start by narrowing the scope to areas like “login function” or “payment flow” and listing only the key perspectives in short sentences.

Creating Opportunities for Knowledge Sharing

In converting tacit knowledge to explicit knowledge, the process of dialogue that generates it is more important than the document itself.

By establishing regular study sessions or retrospective meetings to bring together and discuss topics such as “What kind of bugs occurred this time?” and “Why did we notice them, or why did we miss them?”, individual tacit knowledge will naturally be verbalized.

To share know-how across the entire team, it is essential to create an environment where failure cases can be reported without fear.

Systematizing Testing Perspectives

Reorganizing scattered rules of thumb and case studies along axes such as functions, quality characteristics (performance, security, usability, etc.), and processes makes it easier to handle testing perspectives as a system.

For example, this involves grouping common defect patterns and effective testing perspectives as a set for each category, such as “screen input systems,” “batch processing systems,” and “external integration systems.”

• View PDF for details

The Balance Between Experience and Methods

Experience-based testing and exploratory testing are powerful in situations where specifications are insufficient or when you want to focus on checking high-risk areas in a short time. However, relying solely on them reduces test reproducibility and leads to variations in quality depending on the person in charge.

On the other hand, systematic testing techniques such as equivalence partitioning, boundary value analysis, and state transition testing can ensure a certain level of coverage even for inexperienced testers. However, they cannot fully cover unexpected usage or assumptions not written in the specifications.

The important thing is not to choose one over the other. It is to take a two-pronged approach: first, ensure foundational quality with test cases based on methods, and then complement any gaps with experience-based exploratory testing.

Summary

In testing, experience and intuition are necessary when discovering defects that cannot be fully captured by specifications alone. On the other hand, relying too heavily on them poses risks such as increased dependency on specific individuals, inconsistent test quality, and a failure to accumulate know-how.

By converting tacit knowledge into explicit knowledge through the development of checklists, creation of knowledge-sharing opportunities, and systematization of testing perspectives, individual experience can be accumulated as an organizational asset, allowing you to build a sustainable testing system that balances experience and methods.

• View PDF for details