You have been asked to test the performance of a software and system update to a North American mobile trivia quiz game application. The quiz app is available on major app stores serving English-speaking users. The number of concurrent users planned for is 5 million Winning quiz participants are promised a share of an allocated cash prize Once participants start playing, their quiz sessions must not belost especially if all previous questions were answered correctly. The application owner wants to ensure that 99% of quiz sessions are maintained (up to 4.95 million sessions) until the last answer has been successfully recorded.
You performed 100 test runs that provided 5 million virtual concurrent users. You have measurements of participant concurrency at the last question for each test run In your analysis, you order all the concurrency test results by measured concurrency value from best to worst. You determine 85%. 90%. 95% and 100% values by selecting the 85th. 90th. 95th and 100th (last and worst) measured values by counting list positions starting from the best value going towards the worst until you reach the desired list position (ex. 85). Keep in mind, the results in lower list positions are as good or better than the results chosen for reporting.
After analyzing the results of the performance tests, you present the following results
85% 4.92 million valid concurrent users
90% 4.90 million valid concurrent users
95%: 4.0 million valid concurrent users
100%. 3.8 million valid concurrent users
Based on these results, what should you report to the stakeholders?
A . The performance tests clearly have failed to produce a realistic and compatible load. The results should confirm the test oracle in that the 99% of five million participants concurrency objective has been achieved. If the performance test plan had worked correctly, the reported values should have been 85% 4.95, 90% 4.95, 95% 4 95 and100% 4.95 million valid concurrent users.
B . These results clearly show that the system update successfully supports 99% of five million participants
C . Four samples from the first fifteen ordered test results are presented. The system update performed much worse than what is presented here. The system update has failed to meet the 99% of five million participants concurrency objective.
D . If we pick 85 out of 100 test runs as the minimum level of reliability, the system update does not reliability meet the 99% of five million participants concurrency target, However not all results are as bad as these
Answer: A