|
Chapter 13
1: What motives were behind the break-in of the NORAD computer system in the movie WarGames?
A1: Use/Leverage. Maybe a bit of Challenge/Prestige.
2: True or False: Threat modeling is an informal process done by the software testers to decide where best to apply their tests for security vulnerabilities.
A2: False. It is a formal process performed by the entire team.
3: The JPEG Virus was caused by a buffer overrun bug. Look back to the Generic Code Review Checklist in Chapter 6. What two categories of checks best describe why this overrun occurred?
A3: Computation Errorsthe value was only expected to be positive. When it went negative, it became a huge positive number. Data Reference Errorsbecause when the value became a huge positive number, the destination buffer was not limited to the size of the comment (65533 bytes).
4: The Most Recently Used (MRU) file list that appears when you attempt to open a file in a standard Windows application is an example of what type of data that could be a security vulnerability?
A4: Latent Data.
5: What are the two types of extra, potentially unsecure, data that can be unintentionally written when a file is saved to a disk?
A5: RAM Slack and Disk Slack.
Chapter 14
1: What basic elements of a web page can easily be tested with a black-box approach?
A1: The elements that are similar to what's in multimedia CD-ROM softwaretext, graphics, and hyperlinks.
2: What is gray-box testing?
A2: Gray-box testing is when you can take a peek at the underlying code and use that information to help you test. You're not examining it to the same level of detail as you would with white-box testing. It's helping you test, but you're not basing all of your tests on it.
3: Why is gray-box testing possible with website testing?
A3: Because many websites are principally created with easily viewable HTML, a mark-up language, not an executable program. You can quickly and easily take a peek to see how the pages are built and design your tests accordingly.
4: Why can't you rely on a spell checker to check the spelling on a web page?
A4: Because a spell checker can only check ordinary text. It can't check graphical letters or dynamically generated text that changes on each viewing or over time.
5: Name a few areas that you need to consider when performing configuration and compatibility testing of a website.
A5: The hardware platform, the operating system, the web browser, browser plug-ins, browser options and settings, video resolution and color depth, text size, and modem speeds.
6: Which of Jakob Neilsen's 10 common website mistakes would cause configuration and compatibility bugs?
A6: Gratuitous use of bleeding-edge technology. Existing hardware and software is always susceptible to new technology being run on it for the first time. This was a bit of a trick questionit wasn't mentioned in the chapter, but hopefully you could arrive at the answer by applying what you've learned in Part III, "Applying Your Testing Skills," of the book.
Chapter 15
1: Name a few benefits of using software test tools and automation.
A1: They can speed up the amount of time it takes to run your test cases. They can make you more efficient by giving you more time for test planning and test case development. They're precise and relentless.
2: What are a few drawbacks or cautions to consider when deciding to use software test tools and automation?
A2: Because software can change during the product's development, your test tools will need to change, too. You can fall into a trap of spending too much time designing tools and automation, neglecting actual testing. It's easy to rely on automation too much. There's no substitute for testing the software yourself.
3: What's the difference between a tool and automation?
A3: A test tool will help you test, making it easier for you to perform a manual testing task. Automation is also a tool, but it will run without your intervention. Think power saw and hammer building a house while the carpenter sleeps.
4: How are viewer tools and injector tools similar and different?
A4: Both types of tools hook into the software at points not normally accessible to the average user. Viewer tools are non-invasive as they allow you to see only what's happening. Injector tools are invasivethey allow you not only to see what's happening, but also to manipulate it. You can try test cases that might otherwise be difficult or impossible to perform at the normal user level.
5: True or False: An invasive tool is the best type because it operates closest to the software being tested.
A5: False. An invasive tool may provide you with better information and control in some situations, but it has the possible downside of affecting the software and the test results. It's best to carefully evaluate each situation and select the tool that works best, with the fewest side effects.
6: What's one of the simplest, but effective, types of test automation?
A6: Recording and replaying your test cases so that you only have to manually perform them once is very effective. It frees you from the monotonous repetition and can give you more time to hunt for those hard to find bugs.
7: Name a few features that could be added to the test automation you described in question 6 to make it even more effective.
A7: Simple programming of steps rather than captured steps. The ability to pause or wait for the software to react to the actions. Some type of simple verification so that the macros know whether a bug has occurred.
8: What advantages do smart monkeys have over macros and dumb monkeys?
A8: They're almost self-aware. They know the software's state table so they know where they are and what they can do.
Chapter 16
1: Describe the pesticide paradox and how bringing in new people to look at the software helps solve it.
A1: The pesticide paradox (described in Chapter 3, "The Realities of Software Testing") is the situation that occurs if you continue to test software with the same tests, or the same people. Eventually, the software seems to build up an immunity to the tests because no new bugs are found. If you change the tests or bring in new testers, you'll find new bugs. The bugs were always there, it's just that the new approach made them visible.
2: What are a few positives to having a beta test program for your software?
A2: It gets lots of additional people looking at the software. It's a good way to find configuration and compatibility problems.
3: What are a few cautions to consider with a beta test program?
A3: A beta test is no substitute for an organized, planned, methodical test approachit's not good at general bug finding. You should know who the beta testers are in regards to their experience level, equipment, and needs to ensure that you get what you expect out of the test.
4: If you're testing for a small software company, why would it be a good idea to outsource your configuration testing?
A4: The expense and overhead to stock and manage a configuration testing lab is very high and would likely be prohibitive for a small company or project.
Chapter 17
1: What's the purpose of a test plan?
A1: To paraphrase the IEEE 829 definition, the purpose of a test plan is to define the scope, approach, resources, and schedule of the testing activities and to identify the items being tested, the features to be tested, the testing tasks to be performed, the personnel responsible for each task, and the risks associated with the plan. In short, to tell and get agreement from the rest of the project team exactly how the heck the test team intends to test the software.
2: Why is it the process of creating the plan that matters, not the plan itself?
A2: Because all the issues and questions defined in a test plan either impact or are influenced by other project functional groups or team members. Getting everyone to understand and agree to the contents of the plan is what matters. Privately creating a paper document and putting it on a shelf is not just a waste of time, but also jeopardizes the project.
3: Why is defining the software's quality and reliability goals an important part of test planning?
A3: Because left to their own, everyone will have different ideas of what quality and reliability mean to them. Since they're all different, they all can't be achieved.
4: What are entrance and exit criteria?
A4: These requirements must be met to move from one testing phase to another. A phase can't be left until its exit criteria are met. A new phase can't be entered until its entrance criteria are met.
5: Name a few typical testing resources that should be considered when test planning.
A5: People, equipment, offices and labs, software, outsourcing companies, and miscellaneous supplies.
6: True or False: A schedule should be made to meet absolute dates so that there's no question when a testing task or phase is to start and when it is to end.
A6: False. Because testing depends so much on other aspects of the project (for example, you can't test something until it's coded), a test schedule is best made relative to the delivery dates.
Chapter 18
1: What are the four reasons for test case planning?
A1: Organization, repeatability, tracking, and proof of testing.
2: What is ad hoc testing?
A2: Ad hoc testing is testing without a plan. It's easy and fun but it's not organized, it's not repeatable, it can't be tracked, and when it's over, there's no proof that it was ever done.
3: What's the purpose of a test design specification?
A3: The purpose of the test design spec is to organize and describe the testing that needs to be performed on a specific feature. It outlines the features to be tested and the approach to be used. It identifies the test cases, but doesn't specify them, and what the pass/fail criteria is.
4: What is a test case specification?
A4: This document defines the actual input values used for testing and the expected outputs. It also lists any special environmental needs or procedure requirements and any inter-dependencies among test cases.
5: Other than a traditional document, what means can you use to present your test cases?
A5: Tables, matrices, lists, graphical diagramswhatever means most efficiently presents the test cases to you, other testers, and other members of your product team.
6: What's the purpose of a test procedure specification?
A6: The purpose of the test procedure spec is to identify all the steps required to perform the test cases, including how to set up, start, run, and shut down the test. It also explains what to do in case the test doesn't go as planned.
7: At what level of detail should test procedures be written?
A7: That's a question without a specific answer. It greatly depends on who will be using the procedures. Too little detail makes the test procedures ambiguous and variable. Too much detail can bog down the test process. The level of detail should be set by the industry, the company, the project, and the test team.
|
