1、 2001 i2 Technologies, PROPRIETARY and CONFIDENTIALHuawei TechnologiesHUAWEI APS PROJECTINTEGRATED TESTING STRATEGYREVISION 1.0 / 4-SEPTEMBER-2002Overall Solution Architecture Huawei APS Project 2001 i2 Technologies, PROPRIETARY and CONFIDENTIALTABLE OF CONTENTSPURPOSE .3INTRODUCTION .3SCOPE 4FUNCTI
2、ONS/SCENARIOS 5ARCHITECTURE.10TEST ENVIRONMENT REQUIREMENTS .10DEMAND PLANNING TO FORECAST NETTING10FORECAST NETTING TO S verify the readiness of application for system integrationVerify that data has been migrated as per the defined mappingsSystem Integration To test the coexistence of products and
3、 applications that are required to perform together in the production-like operational environment (hardware, software, network) To ensure that the system functions together with all the components of its environment as a total system To ensure that the system releases can be deployed in the current
4、 environmentVerify the ability of the application to be integrated with other systems and modulesUser Acceptance To ensure the system meets the business user requirements and business processesVerify the integrated system meets business requirements including service levelsOperations Acceptance To e
5、nsure the product can operate in the production environment To ensure the product meets the acceptable Verify that the application can operate in the production environment. Operability tests will be concurrent with, User Acceptance Tests.Overall Solution Architecture Huawei APS ProjectPage 16 of 28
6、Level Focus Benefitlevel of service as per Service Level Agreement To ensure the product operates as stated in the Operations Standards To ensure the system can be recovered / restarted as per standards To ensure that UNIX Scripts are as per standardFor the APS BR1 implementation project the followi
7、ng levels of test will be required:Levels of Test Where neededUnit test All data migration componentsAll components where code has been changedIntegration test All separate and unique functions and transactions within the Data Migration suiteAll customized transactions and functions within the Oracl
8、e modulesAll Type and 2 systems that required change to functions and transactionsSystems Integration Test (SIT)The integrated suite of implemented Oracle 11i modulesOracle 11i and all of the interfacing Type 1, 2 and 3 applicationsUser Acceptance Test (UAT)The total solution plus desktop procedures
9、, business processesSCOPE OF TEST LEVELS AND TEST TYPESOne of the key principle behind all tests is to ensure that tests are performed as early as possible so that errors can be corrected early which in turn reduces the cost and time required for fixes.Given that principle, the following Types of Te
10、st should be included in the scope of each Level of Test.LevelsTypesUnit Integration SIT UATAudit & Control Conversion/Migration Documentation and ProceduresError-handling Function Installation Interface/Inter-System Overall Solution Architecture Huawei APS ProjectPage 17 of 28LevelsTypesUnit Integr
11、ation SIT UATRegression Transaction Flow Usability Backup and Recovery Contingency Disaster Recovery N/AJob stream Operational Performance Security Stress/Volume TEST PHASESThe current approach to SIT and UAT is based on the criticality of the modules for go live. The following diagram indicates the
12、 different phases of test and the occurrences. 8/269/29/99/169/239/3010/710/1410/2110/2811/411/1111/1811/2512/212/912/1612/2312/30 1/61/131/201/272/3DP DP DP DPFN FN FN FN DP DP DP DPFP FP FP FP FN FN FN FNDF DF DF FP FP FP FPSCCSCCSCCDF DF DF DFMP MP MP SCCSCCSCCSCCMP MP MP MPDP DP DP DP DP DP DP D
13、P DPFN FN FN FN FN FN FN FN FNFP FP FP FP FP FP FP FP FPDF DF DF DF DF DF DF DFSCCSCCSCCSCCSCCSCCSCCSCCMP MP MP MP MP MP MP MPERPAPS2003Develop Integration Programs2002Unit TestingEU Material Prep EU TrainingUAT C3UAT C2UAT C1Data Prep Extended SITLimited UAT Full UATCore SITEnvironmentAPS UAT 2APS
14、UAT 1ERP UAT C3ERP SITERP UAT C1APS SIT 1ERP UAT C2APS SIT 2Overall Solution Architecture Huawei APS ProjectPage 18 of 28TEST RESPONSIBILITIESThe following chart identifies the groups responsible for performing the testing activities at each level of test:Level of test Test activity responsibilities
15、Planning Preparation Execution ReportingUnit Module TeamIntegration Team Module Team Integration Team Module Team Integration Team Module Team Integration TeamIntegration Module TeamIntegration Team Module Team Integration Team Module Team Integration Team Module Team Integration TeamSystems Integra
16、tion TestAcceptance TeamIntegration TeamERP & Satellite Application TeamsModule TeamKey UsersData Team Acceptance Team Integration Team ERP & Satellite Application Teams Module Team Key Users Data Team Acceptance Team Integration Team ERP & Satellite Application Teams Module Team Key Users Data Team
17、 Acceptance Team, Integration Team ERP & Satellite Application Teams Module Team Key Users Data TeamUser Acceptance TestAcceptance Team Integration TeamERP & Satellite Application TeamsModule TeamEnd Users RepresentativesData Team Acceptance Team Integration Team ERP & Satellite Application Teams Mo
18、dule Team End Users Representatives Data Team Acceptance Team Integration Team ERP & Satellite Application Teams Module Team End Users Representatives Data Team Acceptance Team Integration Team ERP & Satellite Application Teams Module Team End Users Representatives Data TeamTESTING PROCESSThe basic
19、testing process consists of four basic steps. They are: Plan for the tests Prepare for tests Execute the tests Report the resultsTesting is an iterative process. These steps are followed at the overall project level and repeated for each level of testing required in the development life cycle.Overal
20、l Solution Architecture Huawei APS ProjectPage 19 of 28TEST PLANNINGPlanning in itself is a process. Performing each step of the planning process will insure that the plan is built systematically and completely. Documenting these steps will ensure that the plan is itself testable by others who have
21、to approve it.During test planning activities, the test focus, test levels and test types specific to the application module being tested will be identified and a testing strategy developed that is aligned with this test strategy and the needs/risks of the specific application module. The testing wi
22、ll be based on the business and technical requirements for the application.TESTING TECHNIQUES AND PROCESSESThere are several testing techniques and processes that will be used. These include: Review Procedures, Code Walkthrough procedures, Test Case Design, Test Results Evaluations, Problem/Variance
23、s Tracking, Problem/Change Management, Configuration Management of test objects, Migration to User Acceptance Test Environment, Acceptance of the tested system.Some of these processes are defined through the Project Control Office, while others are specific to testing and will be defined and documen
24、ted by the Integration/Acceptance team.TEST DOCUMENTATION AND APPROVALSThere are several documents that together constitute the test documentation. The Detailed Test Plan for each module for SIT or UAT will identify the documents required for the project and will identify the acceptor for each docum
25、ent. The project deliverables will require formal acceptance, while the intermediate work products will require a less formal acceptance.The Acceptance Test Cases is the most critical of the work products, as it forms the basis for the User Acceptance Testing. This will require input from the Key Us
26、ers in their development and formal signoff of the results once the test cases have been executed and results reviewed. Examples of other documentation that may be required are test log, fault (problem/ variance) log, backup/restore log, test change log, and deliverable acceptance forms.TEST PREPARA
27、TIONThe test design must consider whether the tests involve on-line or batch systems, and whether they are input or output driven. Test cases would be built to accommodate these system characteristics.Another consideration to take into account will be the grouping of valid and invalid test condition
28、s. The integration approach (top down, bottom up, or a combination of both) can sometimes influence the test case design. It is recommended that the development of regression test packages be considered part of the test design. Test cases should always be built for reuse. Test case design strategy e
29、ntails the following steps:Overall Solution Architecture Huawei APS ProjectPage 20 of 28 Examine the development / testing approach. Consider the type of processing - whether on-line, batch, conversion program etc. Determine the techniques to be used - white box / black box, and error guessing. Deve
30、lop the test conditions. Develop the test cases. Create the test script. Define the expected results. Develop the procedures and data (prerequisites, steps, expectations, and post-test steps).In developing a list of conditions that should be tested, it is useful to have a list of standard conditions
31、 to test for some common application types.TEST DATA PREPARATION AND MANAGEMENTTest data set up is a very tedious and time-consuming process. After the test case design has been selected, tester will have to consider all the available sources of data and build new or modify existing test data. Frequ
32、ently, the testers tend to look at the functional specifications and set up data to test the specifications. This tends to overlook how the software will be used. It is therefore important that the production data are analyzed to understand the types, frequency and characteristics of data so that th
33、ey can be simulated during testing.During the test preparation process consideration may need to be given to the following sources while setting up test data: Production data Data from previous testing Data from centralized test beds (if available) Data used in other systems (for interface testing)I
34、n many situations, data from the selected source(s) have to be supplemented to cover additional conditions and cases. When setting up effective test data, the goal is to provide adequate requirements and conditions coverage. Consideration should be given to some of each of the following types: Frequ
35、ently occurring types and characteristics of data that have high risk. For example, deposit or withdrawal type of transactions at a banking terminal. Frequently occurring types of data with very little exposure. For example, maintenance-type of transactions such as name and address changes. Low freq
36、uency errors that have very little consequence.Overall Solution Architecture Huawei APS ProjectPage 21 of 28 Low frequency errors resulting in heavy losses. For example, the printing of a few additional zeros in a financial report.Once the data source has been identified, testers have to determine t
37、he files and sizes of files to use. Data is then extracted using the appropriate methods.The key to successful testing is to state the expected results when defining the test conditions and documenting the test scripts. Documentation should include: What is being tested? How is testing done (procedu
38、res)? Where to find the test data? The expected results.Test Data Management will be critical during the Integration, SIT and UAT phases of the project. The module teams will define prerequisite data for test cases as part of the test preparation process. It is critical that this data is created and
39、 stored in a database that is quarantined from any other data and accessible only by systems test. A procedure to backup and restore this data to multiple base points will require to be implemented. The module team will require adding to this data, deleting from the base data, and modifying any of t
40、he data elements as required.CONFIGURATION MANAGEMENTIt is essential that there is an adequate configuration management process adopted for this entire project. Items under test must be isolated from the development environment and fully documented as to their content and configuration. Code should
41、be released from development with fully documented release notes that identify not only the content of the release, but also any known variances to behavior, defect fixes or change control items and any known issues that might affect testing.TEST TRAININGApplication TrainingTraining is required to f
42、amiliarize the testers with the various application system components prior to test case and data preparation. These requirements will be identified and communicated in the Detailed Test Plans.Test Process EducationAppropriate Test Process education will be provided to the Test Teams for the purpose
43、s of test design preparation and execution. The ISC Acceptance/Integration Manager is responsible to ensure that all test team members have undertaken the necessary test process education.Overall Solution Architecture Huawei APS ProjectPage 22 of 28RISK ASSESSMENTRisk Assessment a part of the test p
44、lanning process. The reason for risk analysis is to gain an understanding of the potential sources and causes of failure and their associated costs.Measuring the risk prior to testing can help the process in two ways: High-risk applications can be identified and more extensive testing can be perform
45、ed. Risk analysis can help draw attention to the critical components and/or focus areas for testing that are most important from the system or user standpoint.In general, the greater the potential cost of an error in a system, the greater the effort and the resources assigned to prevent these errors
46、 from occurring.The Table below lists the risks that have been identified at this stage of the testing process. Further risks, as identified, will be documented in the Detailed Test Plans and managed in the same manner as the Risk Management Process defined for the project.RISK PROBABILITY IMPACT MI
47、TIGATIONCompletion of all Specifications in time to allow test scope to be defined3 3 Attempts must be made to ensure specifications are signed off in a timely mannerAcceptance of all tests levels prior to UAT.1 5 Acceptance to be completed after each test stage.Key User Staff not available during k
48、ey testing activities.5 5 Huawei must ensure staff available. If not agree that the testing and acceptance will be conducted by current SME who is within the project. Staff not available for Integration and System Integration testing5 5 Huawei must allocate more staff to assist the SMEs should their
49、 tasks conflict with testing preparation and execution. Not all tests cases defined in the Inventory will be completed.3 3 Each test case will be prioritized and risk assessment completedPerformance Testing will be the first time that transactions are tested with near live frequency and volumes of data.5 5 A mechanism will need to be defined to allow for these functional errors to be passed back to the teams responsible
