By Wasi Wahedi
Published in “The Guide to Translations & Localization” by Lingo Systems

Whether you are releasing a computer game, income tax software, a complicated business enterprise solution, a children’s toy that interfaces to a computer, or any other technology product, it has to be thoroughly tested before being released to market. You don’t want your customer to spend hours in frustration pouring over their new product, or on endless calls with your customer support department. Not only will you lose an opportunity to establish product loyalty with the customer, but dealing with such problems is both expensive and unproductive.

A third party testing and integration provider can play a key role in the product development lifecycle. Prerelease product testing and quality assurance is critical part of any technology product release. Whether this step is performed by in-house resources or is outsourced to a testing company, there are many types of tests that can and should be performed to ensure the product functions as intended.

If you choose to outsource, the testing and integration provider will most likely have a basic testing model that can be easily modified depending on whether they are testing a software program or a digital device, and depending on the degree and scope of changes being made. Multilanguage testing may require further customization to allow for cultural and linguistic differences, as well as internationalization issues. For instance a tax program for the United Kingdom would be substantially different from a tax program in the US, with entirely different functionality-and different outputs. Even products as simple as toys may differ from one cultural base to another, so great care must be taken to ensure that these differences are appropriately tested and localized.

Each testing vendor may have slightly, or even radically different approaches to test planning and execution. At nResult, our testing model has four basic phases. Not every phase will be needed on every project. Moreover, products that contain major changes to only one process, or minimal changes to a few processes/applications, may be candidates for combining test phases. In all cases, however, a comprehensive test plan must be prepared and approved prior to any work being performed. Once the project does begin, it does not proceed to the next phase of testing until specific release criteria have been met for the current phase.

Unit Testing

Unit testing is the first-phase of product testing, and is intended to be the lowest level and most detailed test phase. nResult has defined two levels of unit testing.

Level I Unit Test

Level I unit testing is conducted by the client’s development stuff during the coding phase and is intended to ensure that changes are working at a load module level as outlined in the business requirements. The developer focuses on functional, navigational, and logic testing:

• Every major program branch executed at least once,
• Boundary conditions checked on all major branches; such as greater than, equal to, and less than cases,
• Minimum and maximum limits just reached, and then exceeded,
• Initialization logic exercised,
• First and last record on file processed, and
• Empty file encountered.

For Online Programs

• All edit conditions violated,
• All screen flow braches taken, and
• All function keys used.

For reports all the following are carefully checked:

• Headings,
• Page breaks,
• Totals and subtotals, and
• Format of all fields.

Level II Unit Test-Informal Integration

Level II unit testing consists of an informal integration test of all load modules for the product, and is normally conducted by a testing and integration specialist such as nResult. Our analyst verifies that abbreviated logic changes work with other pieces of the product, verifies functionality, and executes a limited regression test. The tracking of incidents begins with this phase of testing.

User Requirements Verification Test

Client representatives (typically end users) conduct user requirements tests to ensure that the developer has captured all business requirements in the coding changes. Prior to moving to the next phase of testing, the project sponsor must review the changes and agree that the developer met business requirements as submitted. There is no testing for bugs in this phase. Additionally, this is not an opportunity to add new enhancements or make major changes to requirements outside the scope of the original system documentation.

System Integration/Regression Tests

A designated tester or test team conducts systems integration tests. This phase of testing incorporates logic, functional, and regression tests of all requirements from a technical perspective. The objective is to verify that changes or enhancements do not adversely affect preexisting functionality and interfaces to other products or systems. Testing companies such as nResult are ideally equipped to perform integration and regression tests.

Final User Acceptance/Regression Test

The final phase of testing is often referred to as Beta testing, production review, or, in some cases, a pilot phase. Beta testing ensures that the various integrated components match, operate with, and/or communicate with each other. The objective of final user acceptance testing is to complete logic, functional, and regression tests to confirm that changes are in accordance with the business requirements and that they have not negatively impacted the business process and/or production environment in any way. The testing process utilized for this phase of testing should emulate the business process in the business unit (i.e., emulate a real world scenario).

The business sponsors requesting/supporting changes are responsible for the verification and acceptance of all changes. They are also required to formally approve the acceptance.

Localization Testing

The above discussion covers the basic testing model that would be appropriate for most hardware and software applications. For global releases, the application may be translated into other languages in order to reach consumers in those countries. In this instance, the testing plan will also need to incorporate localization testing to ensure that the translated versions function properly in each native language, that there are no unique problems to that environment or language, and that it is user-friendly in every language. In many cases, this testing will also need to be performed on native operating systems (e.g., a German version of Windows).

There are many other tests that can be performed by a testing company to ensure your product is ready for the global marketplace. Applications or products that are released in more than one language should be tested in each language on native systems to ensure full functionality.

Accessibility Testing

Accessibility testing is used to identify and resolve
functionality and usability limitations within websites and products that would otherwise prevent disabled people from using the technologies to their maximum
advantage. Resolving these problems fulfills legal
obligations and provides assurance that your product will be accessible by the widest possible audience.

Benchmark Testing

Benchmark testing is used to measure the performance of your hardware, software, Internet application, or other product against some standard of excellence or achievement by identifying the measurable successes of others and applying them to your own organization.

Competitive Analysis

Competitive analysis is a process by which detailed information on functionality is gathered on a competitor’s company, device, or product for the sake of understanding and for future product development.

Interoperability Testing

Interoperability testing ensures that two or more systems (computers, communication devices, networks, software, and other information technology components) are able to interact with one another and exchange data according to a prescribed method in order to achieve predictable results.

Load Testing

Load testing (sometimes called volume testing or performance engineering) determines how an application deals with large tasks. Stress testing examines application behavior under peak bursts of activity.

OEM Product Validation

Frequently, an OEM company differentiates itself from its suppliers by adding unique features to generic products. OEM validation testing verifies product claims and validates added features.

Performance Testing

Performance testing focuses on how web applications function under real world loads and connectivity, and includes the interaction of customers, web applications, and the Internet.

Sustaining Testing

Sustaining testing is similar to acceptance testing in that it is done on a percentage of randomly-selected units off the production line. Unlike acceptance testing, which is
based on a customer’s requirements and is generally non-destructive, sustaining testing is aimed at catching problems that may creep in due to design changes and often includes the same tests (including destructive tests) used during qualification testing.

Usability Testing

Usability testing is similar to verification testing except that it is performed throughout the production cycle and usually includes only three evaluators.

Verification Testing

Verification testing is executed at the end of the production cycle when functionality of the product is at or close to one hundred percent completed. Typically, the testing is extensive, and consists of areas such as usefulness, effectiveness, learnability, and likeability. Each test requires as many as one hundred evaluators of varying skill levels and includes interviews before, during, and after evaluation.

About nResult

nResult provides enterprise-wide testing services for hardware, software, and web/e-commerce applications. The company’s state of the art testing lab is unrivalled in the Pacific Northwest. With a rolling inventory of more than 1000 computers, nResult can provide large scale network testing for any size client or system configuration. Core testing solutions include:

• Compatibility
• Functionality
• Load and Stress
• Performance
• Benchmarking
• Accessibility
• Competitive Analysis

nResult also offers a comprehensive outsourcing solution to supplement or replace in-house resources, providing greater capacity at a lower cost. Since its founding in 1990, the Company has tested more than 10000 products. Clients include educational companies, toy companies, shrink-wrapped software companies, medical device companies, and service companies.