7 Reasons Why QA is DifficultPosted on 08/13/2012 in Testing Trends by Jamie Saine
Paul Smyth recently wrote a post for finextra detailing seven reasons why software development is so hard and often runs into deadline and bug issues. He makes some good points and does a nice job looking at the software development process from both the developer and tester point of view.
Here’s an overview of Paul’s seven points, along with my thoughts on how they pertain to testing in particular:
1. The software industry is young
Humans have been building house, roads and bridges for thousands of years. We’ve no idea how many house or bridges collapsed in the early days as humans learned the correct techniques for building these structures. …
In comparison the software industry is only about 50 years old. We still have a long way to go before we have the body of experience behind us that the construction and manufacturing industries have.
Yes, the industry is still relatively new so we’re still trying to figure everything out. But, unlike building physical structures, software is constantly changing right under our noses. Once a bug is finally common enough to be routinely addressed, another one pops up – sometimes as a result of fixing the initial bug. Plus the continual development of new technologies – hardware, operating systems, add-ons – adds new challenges while testers are still trying to overcome previously-known challenges. The earth stays mostly the same no matter when or where you’re building a bridge, software doesn’t.
2. Every line of code is a potential point of failure
Each line of code will have dozens, even thousands, of possible inputs, outputs, states or dependencies to deal with. It can impact, or be impacted by, other lines of code or by external factors. Even if it was possible to document every test case for a line of code you still couldn’t be sure that there wasn’t some unknown factor that could cause an error.
And testing a single line of code is only part of the challenge. No line of code exists on its own. It is part of the whole system and the whole needs to be tested to ensure that all parts of the application function correctly. …
This one pretty much hits the nail on the head. Without unlimited time and resources, there is no way to test 100% of the new software. Even with unlimited time and resources, the fact that many projects are essentially brand new means there may be hidden bugs that testers don’t think to look for. The fact that development and testing teams traditionally don’t work together only exacerbates the code challenge – though we’re starting to see a shift away from this divided practice.
3. Lack of user input
Without the involvement and input of a user representative the project is doomed to failure. This person should be a subject domain expert with the authority to make decisions and a commitment to the project timescales.
As a usability tester, how many times have you looked at a new application, been confused and wondered how in the world the developers thought it was a good idea? Then, when you log the bug, the only response you get is “works as designed.” Sometimes developers can be so heads-down in their project that they don’t realize the software isn’t end-user friendly. This can also be a problem for testers. If a tester has been working on a particular project since the beginning, they can be blind to the fact that it’s not intuitive. Since they already know how the software is supposed to work it seems easy to use to them.
4. Users don’t know what they want until they see it
Even with good input from the users no amount of analysis of user requirements can take away an immutable fact that users only think that they know what they want. In truth, it’s not until they start seeing something, and using it, that they begin to really understand what they need. This is especially true when the software is being developed for a new idea or process that they haven’t used before.
Paul looks at this point more from a development standpoint, noting that requirement changes are common but drastically effect development and project deadline. From a tester standpoint it speaks to the fact that in-lab testers don’t necessarily match end-user demographics, so they can only guess at how users will respond to the software. In-the-wild testing, using professional testers who mimic your target demographic is a way to address this issue.
5. There are no barriers to entry to become a programmer
There is one argument that states that software development is so hard because programming is so easy. In other words it is relatively easy to learn how to write code but there is a huge gap between that and delivering great software. …
There is no barrier to entry into the programming world and thus it is awash with many poor programmers who adversely affect projects. In addition, even potentially good young developers will still make mistakes that a more experienced developer will have learned to avoid.
Switch the word “programmer” with “tester” and the sentence is just as true. Though there are tester certification programs out there, they are by no-means necessary like some training programs are (think doctor or lawyer).
6. All software is affected by external factors
Software is “mindware” and therefore doesn’t obey physical laws but it usually must conform to external constraints such as hardware, integration with other software, government regulations, legacy data formats, performance criteria, scalability etc.
Understanding and catering for all of these external factors is a near impossible task. Even a seemingly simple requirement, such as supporting multiple browsers, exponentially increases the difficulty of both building and testing software. If you then add in a requirement to support multiple versions of each browser then you are again exponentially increasing the complexity and the difficulty.
This is the challenge of the testing matrix. Testers must take a developer’s work and test it against a variety of hardware/software combinations, plus real world factors such as carriers, network connections and unexpected load spikes.
7. Estimating is an art not a science
Of course experience guides you in your estimating and the more experience you have the more likely you will be to anticipate the unknowns. Too many projects run over because overly optimistic estimates are set by inexperienced people who expect everything to flow smoothly and who make no allowance for the unknowns.
Testers get particularly frustrated with this because it it usually not the QA department setting the deadline. Plus, if development runs over the deadline typically isn’t extended. Instead, testing is expected to be completed in a shorter time frame. These practices put added pressure on testers. In fact, companies under-appreciating QA and underestimating the difficulty of the job are two recurring complaints among testers.
This was just a sneak peak at Paul’s full list. He makes some great points that I didn’t even touch on here, so be sure to read the original post in its entirety at finextra >>> =========================================================== comments: said: It may sound vague, but I’m going to say all of the above. I like this list so much because I feel like it more or less pertains to any and all parts of software development – including both testing and QA. Posted on 08/14/2012 at 3:11 pm said: software development is so hard main reason is Lack of user input. Posted on 08/21/2012 at 1:11 am said: Referring to the addage: “There are no bad children, only bad parents.”, I cringe every time I hear people talk about bad programmers or testers. I’ve been in the industry for over 15 years. If you’re going to take a department of professional Programmers and QA people, then you must put a proper mentoring programs in place to mitigate the bad habits that can sprout up. Carpenters, electricians, and plumbers all use apprenticship programs, yet the software development world has failed to realize this need for some strange reason. Passing the buck to the end programmer or QA person is an irresponsible way to deflect the problem. Use pair-programming/pair-QA techniques, internal graduate programs, performance measures, to fill these gaps in knowledge. Guide new people for petes’s sake. Don’t blame the person who doesn’t know what they don’t know. Take responsibility for your department and industry and put something in place to support those people. Posted on 08/22/2012 at 2:14 pm said: There will always be bugs to eliminate no matter how good the QA or tester tries to be. User feedback and the desire and willingness to eliminate bugs for the next release is always the key mini-cycle in the SDLC of any software application. Posted on 08/22/2012 at 2:18 pm said: QA and testing are not hard, bad requirements and lack of communication make both tasks harder than they need to be. I have yet to meet a business analyst or product development person who is not as useless as tits on a bull. Posted on 08/22/2012 at 2:23 pm said: I LOVE this list.. Especially number 7, where testing timeframe is either squished, or where other than the QA Manager sets the testing timeframe. I also love when someone outside of QA says “Oh, this should only take 5 minutes to verify”, then QA has to explain, justify the “proper” time it takes to test. It’s funny.. QA can’t state how long DEV should really take for coding, but they and others think they can tell us how much time we’re given for testing… humm…. What’s wrong with this picture? I’ve been a QA Manager for 10+ years.. I’ve protected, and supported my current and past QA employees in reference to all seven reasons why QA is difficult. Posted on 08/22/2012 at 2:26 pm said: As areas mature, narrow poorly educated technicians (software and otherwise) are in the end problematic by their limited training in other key areas of the arts & sciences more broadly, plus there is of course the little problem of hubris within the dot.com folks. Posted on 08/22/2012 at 3:28 pm said: Point 7: “Instead, testing is expected to be completed in a shorter time frame”. The test team can test in whatever time frame they are given. The skill comes from articulating the associated risk that the project manager (not the test team!!) must accept if the timescales are reduced. Posted on 08/22/2012 at 3:48 pm said: I don’t think QA is that hard. I’ve been in the industry 20 years and leading QA departments for 15. In the passed 5 years, 96% of my teams’ projects are zero defect. Computer engineering is called “computer engineering” because it is based on engineering principles the same as engineering as building. People just want to make the business of building software easier than it is – but it just isn’t. Just like building a house isn’t easy – it takes many teams of people coordinating and designing to make it work. On another note – this article doesn’t address the most important factor. People. You need the RIGHT people. Right means that they have the proper skills to be a good tester. You can’t buy or learn that. Posted on 08/23/2012 at 12:00 am said: “Instead, testing is expected to be completed in a shorter time frame”. The test team can test in whatever time frame they are given. The skill comes from articulating the associated risk that the project manager (not the test team!!) must accept if the timescales are reduced Posted on 08/23/2012 at 1:09 am said: Software, QA is not a cup of tea for all. I may be able to suppose to be for any consisting software testing to make career. Posted on 08/23/2012 at 1:31 am said: 96% of my teams’ projects are zero defect—-Great As per my knowladge and Standerd Software Quality Defination this achivment is awsome…….. Posted on 08/23/2012 at 1:41 am said: Software Engineering is hard for those who are lacking of technical knowledge, Especially Testing & QA is difficult only when they are lacking of Domain knowledge & creativity. Posted on 08/23/2012 at 1:48 am said: @ mailman Agree with what you say but dont ever see it happen due to 'Practicalities’ As the autor pointed out , this is 'Mindware’ and there is no defined write or wrong (specially in test) . And companies dont want to invest time/money on things like internal graduate programs, performance measures, and other programmes/courses to fill these gaps in knowledge. All the mgmt. care about is pushing the product out of the door ASAP as soon as it arrives at QAs desk ! within their planned Budget (which is never enough) . Untill that mindset changes Software industry will be plagued but the 'bad programmer/bad tester’ issues . like you rightly say ' “There are no bad children, only bad parents’ … children = programmers/testers/tech support . parents = Mgmt. (sponsors, CTOs, Head of XXXXXXX ) Posted on 08/23/2012 at 5:31 am said: Raising defect is not a crime it’s a job description Posted on 08/23/2012 at 5:43 am said: Development and Deployment methods are always changing based on issues, requests and deadlines. Just like they need to come up with second option, we need to provide testing options as well. You may not always have time for full testing. In that case, you provide options to the client of what can be accomplish in the limited time frame. Then let the client determine what they are willing to take as “Risks”. Posted on 08/23/2012 at 10:24 am said: ->After bug fixing by developer, new problem may be arising in Site, so a tester must test his sites again and again… So a qa must be Patient said: I just loved this article because I am a tester. Yes, QA is difficult for testers who want to deliver perfectly and not for cases where testing is done for namesake. QA becomes real fun when Dev and QA teams understand each other’s concerns and responsibilities weather its same organisation or 3rd party. As a team the challenges are met easily and in a fast way. We are proud to be part of this young software family because we will be setting the rules for our successors Posted on 09/04/2012 at 3:43 pm said: I think work of tester is very hard to justify.I executed hunderds of testcases those are not documented in testcase doument. Posted on 09/27/2012 at 1:34 pm said: Doing a known thing is easy compared to finding unknown things. Testers job is very tedious because because he has to identify unidentified loopholes of the code. As paul said this industry is new and evolving rapidly. So tester has major role in this evolving environment and his role is getting more importance day by day. Posted on 02/06/2013 at 9:22 am said: QA might become even harder for the novice tester. Moreover too much reliance on manual testing can make it even tougher. Posted on 02/08/2013 at 6:01 am |
|
|
来自： hildaway > 《goodpoints》
