Established firms perform QA activities to maintain their goodwill and increase their market share. However, startups have to strive to survive the competition as well as attract and retain customer base from scratch. Ignoring QA & Testing activities and considering them a cost burden could prove to be very riskier and costlier for startups.

Implementing SQA & Testing activities can benefit the startups in many ways; some of which are as follows:

v  To verify, validate and to determine whether the software/application meets all the requirements and specifications that the client wants. , Testing comes in very handy as test results help you identify the glitches in your build and can be fixed before you launch a product full of bugs and earn a bad name for your start up. A defect free product then keeps your client happy (which is very important, especially if you are in the early stages of your business) and also helps in establishing long-term relationships between you and your client.

v  As P. D. Coward has quoted: “the principle objective of software testing is to give confidence in the software.” Being a startup firm; client’s confidence in your software means a lot as it attracts more customers & prospects

v  If you are looking for increased system integrity then making it quality assured is a must. Testing and quality assurance measures ensure that the system integrity will not be compromised as it has been made error free. Firms that employ security measures for confidential information are deemed to become number one choice for the clients.

v  Being an IT startup firm, even though you manage to hire good and certified developers you can’t just trust them blindly and not test the software they create because it doesn’t matter how good a developer is at his job they won’t know (at least for the time being) how the software would work in the real world. As the famous saying goes “there is always one more bug to find”, testing the software/application might reveal some bugs/errors.

v  Testing software simply adds value to it as it confirms that the software meets all the predefined goals and does exactly what it is supposed to do. It then also establishes even the software is doing things that it is not supposed to. Startup firms which provide tested and quality assured software are much preferred by both individuals and organizations. This preference will help such firms to make progress by leaps and bounds.

Anyhow, SQA & Testing measures are beneficial for the established and startup firms alike. Startups put a lot of efforts to acquire customers and if they fail to provide them what they want then all of their efforts are a waste of resources. Tested and Quality Assured software/applications keep the clients satisfied and for the startups; clients’ good word of mouth about their firm is a valuable asset, like none other. Many IT startups fail as soon as they ignore testing services because considering quality assurance and testing services to be just an additional cost is not a wise move. In fact such measures will help the firms to reduce development & maintenance costs.

Furthermore, testers and developers should work side by side in order to have a better understanding about the product because although developers might have great skills but they are still not immune to human error. Hence good development, coupled with great testing and Quality Assurance is the recipe for an Amazing Product Launch and many steps closer to becoming a very successful startup.

In this discussion, based on my experience I will share few points which will be helpful during the (functional) testing execution and a testing team can achieve their desired results by following these simple points.

First of all, as a tester you must have the crystal clear idea of the application scope/functionality which you are going to test. It’s the most important part of the test execution because every subsequent step will base on it. So take your time and thoroughly explore the application, discuss it with your team lead and send your concerns to the client if something still isnt clear.

If possible, informally document your findings so that a clear picture of the application can be drawn.  I used this technique while testing K-Tango application and it brought great results for me. K-Tango was a big enterprise application having multiple user roles interacting with each other over the web and mobile. At the start of execution, I wasn’t able to figure out most critical scenarios covering multiple user roles. But after proper application exploration and drawing application flow diagram, I not only figured out all the critical scenarios but also found out lots of critical bugs as well.

Another plus point of the application exploration is that you can get application domain knowledge and in case of lacking the required extent of it (which is compulsory for testing an application), you can get it prior to the execution.

This exploration also helps you to identify necessary dataset (software and hardware) required for testing. You must list down all the software and hardware requirements for testing and pass this list to your superiors to make sure all the dataset elements are available before execution.

During exploration you can also figure out scenarios which can’t be tested due to certain limitations (Time constraints or data sets unavailability etc…). During Ford Focus Rally application testing, there was a scenarios of pagination in which there were options of 25-50 and 100 users on one page.  We only tested it till 50 users because creating a new user and registering it with Ford Focus Rally was taking time and we left the last scenario with client approval due to  time constraints.  So by careful exploration of the application you will also identify these scenarios as well and notify the stakeholders about these scenarios in advance to avoid any hassle.

Another point is, that by prioritizing application scenarios you can mitigate lots of risk factors and can define the sequence of your activities.  You must have got the application idea after exploring it, so now you can identify its critical areas as well. You can define the sequence of your testing activities based on your requirements and with the mutual understanding of the client. But it’s recommended to start from the most critical areas first and so on. But you can also follow the application flow if there are not many risk factors involved.

Another important factor to complete the test execution on time is to divide the whole activity and set daily targets. By achieving the daily targets, you can easily meet the schedule. This strategy helped me a lot during Motista application testing, where we have to complete its testing within 10 working days and we divided the complete functionality in 10 parts. During execution we found, it was a very tight estimate but only due to daily targets we were able to deliver reports on time. Similarly on another occasion, where application testing reports were required in 5 days and although the scope wasn’t that big but due to absence of daily targets, I wasn’t able to deliver its reports on time.

What if you received an application and have to deliver it on urgent basis?

In that case you just try to setup a quick meeting with the client (even with team lead if client is not available) to get as much understanding of the application as possible, its critical scenarios and discuss how you can minimize the execution time (Most often by reformatting reporting mechanism) to meet the tight schedule. But make sure you have acquired necessary knowledge of all application aspects to complete its testing successfully.

In conclusion of this discussion, I would say planning is essential for the success of any task and without proper planning planned results can’t be achieved. Same is the case with software testing where you can get the desired outcomes by considering all the above mentioned factors while you are testing an application.

Many organizations get deceived by the sales executives of tool vendors, giving demo on how easy it is  to create test scripts using the record/playback functionality. In many cases, organizations do not investigate how the tool is capturing the application and automating the functionalities and they buy the licenses after being inspired by the flowery words of the marketing personnel.

Eventually, they find out that the tool captures by identifying the application and its controls using coordinate positions, which is not a reliable method, and the tool gets shelved.  Hence a systematic approach must be taken to evaluate tools. I recommend that the selection of automation tool be divided in to three phases:

• Requirements Gathering
• Tool selection
• Proof of Concept (POC)

Requirements Gathering

We have to gather all the requirements or the expectations from the tool under evaluation respective to your environment of application. Some of the items that need to check during requirements phase will be:

• The testing tool must be compatible with operating systems that your product supports.
• The tool must be compatible with third party software that your product is integrated with.
• The tool must be version control respective to build so as to update scripts with lesser man effort.
• the organization must  have the skill set for operating the respective tool
• management must be ready to invest on licenses or training the resources for tool
• the extent of test coverage that can be done using the tool
• Degree to which regression tests are possible with this tool
• test data setup should besupported in tool in different formats
• the testing tool should increase the efficiency of the testing process?

There could be more points that will help in order to map your requirements during the selection process as per company processes and requirements. These all points will set the evaluation criteria for the tool.

Tool Selection

The second main step requires the selection of the tool as per the revelations from our evaluation criteria made in the requirements phase. One thing that must be must be known is that no tool will ever support your environment completely. So, you have to prioritize your requirements in a way that clearly identifies the most valuable features to you and the tool that best supports them.

Categorize the features for tools under consideration upon following basis:

Compulsory Features: These are the features that are mandatory in order to accomplish the automation goals as per your requirements within all constraints.

Unique Features: These will be all those features that will show value/difference/uniqueness of one tool  compared to the other one

There will be some features within the tool that you do not require or that are of no use so just ignore all of them. Hence, rate all features for the tools and prioritize them as per your requirements. Shortlist the most feasible tools and perform a POC on it.

Proof of Concept

The POC is always performed on the tool/tools that you have short listed that map on your all requirements for automation testing. In order to perform the POC you have to identify some core features of the application under test with technical constraints that need to be considered.  Select any one scenario from a sample of common features that will be a part of POC too. Once the POC has been performed document all the analysis, failures and success that will lead to choose the tool for automation testing. The same process will be applied on all the tools in the list.

Selecting the automation tool requires intensive focused research following the process Described above. This effort will cut down the time and cost risks that have the probability to appear without proper analysis of the tool.

Even though the adverse effects of the global financial crisis (which started in early fall of 2008) have been felt by almost all the industries; however the IT industry (particularly the software testing industry) hasn’t been much affected by this crisis as compared to other industries. In fact it wouldn’t be wrong to say that this crisis was a blessing in disguise for the software testing industry.

Huge corporations and software houses that have been affected by this crisis have been looking for innovative and efficient ways for decreasing costs while improving customer satisfaction. This need makes them outsource their businesses to places where they can get the same quality standards at cheaper rates.

Forrester Research (an independent research company that provides pragmatic and forward-thinking advice to global leaders in business and technology) has predicted in one of its research reports that around 1.2 million (or even more) European IT industry jobs will be outsourced to other territories by 2015.

One of the studies commissioned by SQS (Software Quality Systems: one of the world’s leading specialists in software quality) entailed that most of the firms have already been working to optimize their software testing activities and the rest of the firms are looking forward to do this via measures like comprehensive standardization of software testing.

Majority of the firms are increasingly engaging with offshore software testing service providers for the sake of achieving higher quality standards while reducing costs. The study also shows that due to this financial crisis; majority of the firms are now seeking blended services delivery (a combination of onsite and external test services.)

Software Testing activities are definitely the integral part of software development processes as they guarantee that the software meets all quality standards. Although testing software sometimes costs you a lot of time, money and effort but (if done the right way) in the long run this cost can turn into a fruitful investment.

Anyhow; as ironic as it may sound, Software Testing Industry Owes Big time to this start of the century, global financial crisis!!

Testers from all over the world have benefited from this industry as the demand for software testing and quality assurance is increasing day by day

Earlier, it has been perceived by many, that testing alone consumes nearly half of the software development costs so testing processes are merely additional cost.  However, history has proved several times that avoiding these costs turns out to be several times costlier for the company. Software Testing is therefore looked upon as an investment and is not just a “Plus” for software companies but a “Need”The software testing industry provides an abundance of business prospects and opportunities.

Ovum (a world renowned global advisory and industry consulting firm) claimed in one of its reports that despite the worldwide economic crisis; the global software testing industry is projected to reach around $56 billion to $59 billion at a compound annual growth rate of 9.5% by the year 2013 and around $13 billion of which will be outsourced to the Asian countries. Alexander Simkin (a senior Ovum analyst and author of the report) further added that both; outsourced and in-house testing service markets will grow during 2009-2013 and testers of Asian region (especially India) will give a tough time to the European and North American testers in terms of quality while offering cheaper rates.

The analysts at TechNavio (an internationally recognized research platform) have forecasted that the Software Testing Industry will grow at a compound annual growth rate of 11.1% during 2010-2014 globally as the need for identifying and removing costly errors is increasing. The report further entails that this software testing market will witness more outsourcing than before, however shortage of skilled testers will be a great challenge for the growth of this industry.

According to Gartner (the world’s leading information technology research and advisory company) the trend of independent testing as a business is growing at 40% to 50% and also the offshore testing services are growing at 35% to 40%. The report provides a clear indication that the software testing industry has a lot of growth potential. A recent study by PAC (Pierre Audoin Consultants: a privately held research & consulting firm for software and Information & Communication Technology (ICT) services market) revealed that around 91% of IT decision-makers consider software testing and IT quality management to be one of the most important IT tasks. The study also entails that again 91% of managers prefer that the testing team should be different from the software development team in order to improve transparency of services, test quality and objectivity of testing performance.

CONCLUSION

Software Testing Industry has gone through lots of changes and has evolved a lot mainly because of the changing technologies and business dynamics since the last decade. Today, all major IT corporations across the globe have invested in this industry which is evident by the fact that all of them have now established their own centers of excellence and are providing independent testing services as well. Just a few years ago, the average deal of outsourced testing projects ranged from $50,000 to $60,000 with only certain types of testing but today the independent software vendors outsource end-to-end testing projects with the project deals ranging between $2 million to $4 million.

Due to the kind of remarkable recognition that this industry has witnessed over the past couple of years it won’t be too far fetched to hope that the Software Testing Industry has an even brighter future ahead and the software testers will definitely be an integral part of this industry because of the ever increasing demand for software testing, however the industry might be facing some difficulties to meet these demands due to lack of skilled testers. In future we still hope to witness the growth of both the software testers and the software testing industry.

]]> http://www.kualitatem.com/software-testing-industry-%e2%80%93-unlimited-employment-opportunities/feed 0 http://www.kualitatem.com/testing-tools-more-thoughts http://www.kualitatem.com/testing-tools-more-thoughts#comments Thu, 22 Mar 2012 10:38:50 +0000 admin http://www.kualitatem.com/?p=795 ]]> Software organizations, clients, and test engineers, all turn to testing tools for some of their own good reasons. Over here we don’t exactly discuss the individual reasons of tools usage, or pros & cons, but just some aspects that will help us remember the factors that should be considered in the selection of test tools.

Tools exist to make life easy. Software testing, too, is made fun and interesting with the use of testing tools. These tools serve the purpose of improving efficiency and productivity; all the while making testing a more interactive activity for the test engineers.

Let me ask! Is it really as wonderful as it appears? Are our expectations from these tools justified? And do we consider and take into account all the important factors before setting these expectations?

True, tools might facilitate the technical feasibilities; however, one might come across scenarios where the usage of testing tools is not preferred.

Generally speaking, following are the criteria elements that should be considered before using any specific test tool in order to maximize the benefits you gain through its usage:

• Project needs
• Nature and frequency of application code update
• Test tool license considerations
• Precision of the tool’s performance
• Acknowledgement of client
• Availability of supported software & hardware infrastructure for test tool
• The ability & skill of the team using the tool

Knowing, learning and applying a test tool can be an interesting as well as a challenging task. Using testing tools becomes Challenging when it comes to the integration & extension of test tools with other tools, framework and software mechanisms. Linking the tools up with other interfaces might at times require out of a test engineer to learn the skills required to work on these connected interfaces in order to achieve the optimum level of results from the testing tools.

Things I’ve learned:

Problems might occur when test engineers begin to blindly rely on the tool’s execution output.  In my opinion, it is advisable that we cross check the results through manual tests.

Sometimes the validity of a testing tool can be put to test by using it across different applications, browsers, OS and other such changeable factors.

It might be a good idea to compare the results of multiple tools over the same platform or OS along with manual test results. You might get somewhat different results in this comparison that may reveal the accuracy of a tool’s performance.

Another factor to be considered in using a test tool, e.g. automation test or load test can be the behavior of application under test. An automation tool, for example, may not always run exactly the same way because the website in question is built in a different way than a standard way of W3 consortium. Test tool will thus fail to recognize some application page elements or their values. This may happen, either due to incompatibility between application under test and test tool, or because the user of test tool may need to apply some explicit workarounds to make things work.

Therefore, its not always the case of following a tutorial or checklist to use the tool perfectly, rather it’s a dynamic process that requires real time decision making; which is subject to variations  and changes in real life. It is, then, a game and art of technical test engineer and not a non-technical team.

The success or failure of a test tool results is thus, codependent on many more factors that we tend to ignore. These, however, are critical enough to affect your commitment to project specifications and deadlines. It is, in the field of software testing, that every connecting dot matters!

Check again! “Tool based tests Vs. manual tests” is an on-going battle, and is perhaps one of those “Egg or The chick” scenarios, where you can never decide who came first! The Only difference lies in the way you may interpret the word “First”!

About the Author: Zeeshan Dilawar is a senior QA engineer, majors in Automation Testing, with a little over 6 years of Testing and QA experience.

________________________________________________________

There’s a lot that goes into developing a valuable product. The startup team first identifies a problem or opportunity that they believe they can build the right solution for. Then the team starts to focus on building the minimum viable product. This is where the product people, UI designers and backend ninjas come together and start using broad brush strokes to design something that will test the main assumptions. Hopefully at the end of this development process you have something you can share with the world

Continue Reading Here

With the passage of time, ‘Software Development Industry’ has also become aware of the importance of ‘Quality Assurance’ in order to successfully comply with the concept of ‘Total Quality Management’ (TQM). People/Customers now have more confidence in such software/programs which have undergone critical tests and QA measurements. For this purpose in this article it has been tried to provide a better understanding about the importance of ‘Software Quality Assurance’ (SQA) to head towards ‘Total Quality Management’ (TQM).

In today’s technology driven world, software perform a key role in daily business and even in private life i.e. software are used in cars, trains, planes, banking systems, enterprises and so on. Therefore, software’s quality plays a crucial role and is important for the acceptance of software by end users and hence proves to be a key factor in success of software product. Developing software is an expensive process as it takes a lot of skilled people to develop it yet efficient companies spend excessive amounts to make it a “quality software” and also to fulfill the customers’ quality needs. On the other hand there are some companies as well which don’t take quality assurance measures seriously thinking of it as mere additional costs. But as a result the possible software failure causes millions of breakdown costs, loss of market share and reputation or can even cause injuries to people. Thus, it is important that SQA measures should be strongly considered to prevent a lot of financial losses as well as life threats and such measures will eventually result in the firm’s proper implementation of TQM principles. The International Organization for Standardization (ISO) has provided all the characteristics regarding SQA in its ‘ISO 9126’ standard which was first provided in 1991 and was revised and divided in 4 parts in 2001 as “ISO/IEC 9126-1 to ISO/IEC 9126-4”.

Implementing SQA measures can result in numerous benefits for the organization. Some of which are as follows:

• Although implementing SQA measures can be costly for the firm but it will eventually decrease the amount of time and money spent on software revisiting requirements, design code and the documentation procedures etc. which might occur in future.
• Quality Assured Software mostly no longer requires modifications and repairs which will save a lot of time and money.
• Software Quality Assurance decreases the time between ‘Development’ & ‘Deployment’
• SQA measures can help in catching errors and bugs etc. before they are too costly to fix.
• Customers have more confidence and interest in quality assured software. Therefore SQA can help the firms in increasing their customer base, market share and sales volume.

On the other hand, neglecting SQA can result in a number of adverse effects. The history is full of incidents where lack of quality assurance has caused the firms to suffer huge losses and sometimes even proved to be fatal to the people.

• In 1985, Canada’s “Therac-25” radiation therapy machine malfunctioned and delivered lethal radiation doses to patients. As a result 5 people died and a few were critically injured. The situation was caused by a bug called “race-condition” i.e. making the electron beam to fire in high-power mode without proper patient shielding
• A 2002 study commissioned by National Institute of Standards and Technology found software bugs cost the U.S. economy about $59.5 billion annually and also that about $22.2 billion could be eliminated by improving testing.

‘Software Quality Assurance’ is just as important as ‘Software Development’ and IT based organizations cannot hope to successfully implement the principles of ‘Total Quality Management’ without seriously considering the Software QA measures in that

(a) SQA saves a lot of repetitive work, modifications and repairs of software

(b) SQA helps in meeting the timelines

(c) Due to increased efficiency increases the profit per product

“Software bugs caused the bank accounts of 823 customers of a major U.S. bank to be credited with $924,844,208.32 each in May of 1996, according to newspaper reports. The American Bankers Association claimed it was the largest such error in banking history”

We know that software bugs can be annoying and costly but they can never be costlier than faulty software. Software errors cost a little over $60 Billion annually to US economy alone. Today, Delivering quality product to marketplace in a timely and efficient fashion does not promise remarkable success in software industry. It takes a lot more. Today’s software products are increasingly complex, often operating within multi-layer, multi-platform environments. Being created in rapid development environments, these softwares are pushed into the market at breakneck speed.

Fresh graduates often work as Testers at their first jobs even though their titles may reflect upon them as developers and might know nothing about testing. It is done because management fears bearing too much damage if the resource fails. In time, they are then promoted to being developers, their area of specialization.

Using testing as a transitional job for new programmers is not uncommon and is one classic hiring mistake. It does have a couple benefits; why else would someone do it otherwise. For instance, it really does keep bad programmers from causing trouble. A bad tester can be much less damaging than a bad programmer. Moreover, it might serve as a good orientation and introduction to the business and its products for the programmer.

S/he might learn some tricks as a tester that could help them later as a programmer.
All these benefits are outweighed by one huge disadvantage: the new hire HATES his job as a tester. That’s hardly conducive to good work. One could argue that the testers “have” to perform well to get their so called promotions. However, because people tend to be as motivated by effort as by results; they might not care much for their jobs.

Another classic mistake in organizing testing teams is to hire testers from amongst failed programmers. There could be several examples where bad programmers turn out to be brilliant testers. However, what happens when someone’s a bad programmer not because of the skill set but because of inappropriate work habits? For Instance, programmers who make lots of bugs because they are inattentive will also miss lots as a tester, for the very same reason.

So how should we then staff our software testing teams? What are the DOs for staffing QA experts?
1. When interviewing, we should concentrate less on formal qualifications than on intelligence and the Character of the candidate’s thought. Know that a good tester has these qualities
o Methodical and systematic.
o Tactful and diplomatic (but firm when necessary).
o Skeptical, especially about assumptions, and wants to see concrete evidence.
o Able to notice and pursue odd details.
o Good written and verbal skills (for explaining bugs clearly and concisely).
o A knack for anticipating what others are likely to misunderstand. (This is useful both in finding bugs and writing bug reports.)
o A willingness to get one’s hands dirty, to experiment, to try something to see what happens.
2. Try to dodge the trap of testers who are not domain experts. Domain experts know more about the area of study and will help find the most relevant and useful bugs.
3. Good technical writers develop a sense of what’s important, what’s confusing, and so on. Those areas that are hard to explain are often fruitful sources of bugs. Therefore; it is advisable to have some technical writers in the team. Like testers, technical writers often also lack detailed domain knowledge. However, they’re in the business of translating a product’s behavior into terms that make sense to a user.

If you keep in mind these guidelines while selection of your testing resources, you will successfully dodge a testing team that lacks, skills, quality and diversity. All of the members might lack some skills, but the team as a whole will have them all. Over time, in a team with mutual respect, the non-programmers will pick up essential tidbits of programming knowledge, the programmers will pick up domain knowledge, and the people with a writing background will teach the others how to deconstruct documents.