Best practices in Test Data Generation

With emerging trends, the technology is also shifting from code generation paradigm to data model. The main idea behind test data generation is testing the competence of a software or an app. Testing an app with real data is important to bridge with real time scenarios and make the necessary changes accordingly.

Classification of Test Data Generators

Test Data Generators can be broadly classified into:

Arbitrary Test Data Generator: As the name suggests, it is a random test data generator. It is the most uncomplicated data generation technique and is based on prospects. Thus, it can’t achieve high quality coverage of test data.

Download Whitepaper: Test Data Management – Key Challenges and Test Approach

Aim-Oriented Test Data Generator: Here, input set is generated for any path, instead of entry to exit block of code. Control flow graph plays a very vital role in these types of test data generation technique, thus reducing a probability-prone and infeasible path based test data generation and providing an opportunity of direct search.

Path-Oriented Test Data Generator: This is the best test data generation technique among the lot. In this, an unsurpassed specific path is offered, instead of multiple paths for a control flow. This technique is centralized on fault based testing. Another name for this type of testing is Mutation Testing. The changes done in the code after this type of test are called ‘mutants’.

Intelligent Test Data Generator: This technique draws upon the complicated analysis of code to pave way for the search of test data. Here, test data generation method is utilized along with the comprehensive analysis of the code. This technique involves thorough analysis to anticipate different upcoming situations.

Test Data Generator Life Cycle

Steps involved in Test Data Generation are as follows:

  • Control Flow Graph Creation: It consists of the representation of possible transfer of control.
  • Path Selection: In this step, the path of program – especially the control flow, is identified.
  • Input Data Derivation: After the selection of path, set of realistic input data are generated for the selected path, determining the control flow. This is the test data generation step.

A test data generator takes help of Program Analyzer for the same. Program Analyzer has many tasks to complete during the process. The Program Analyzer firstly retro inspects the control flow graph and approaches the path selector to gather the set of selected paths. Again, it’s the Program Analyzer which mulls over the control flow graph and data dependence and approaches Test Data Generator to generate test data set for each flow. Test Data Generator also consults the Path selector before test data set generation to ensure the authenticity of available path information.

Best Practices for Test Data Generation

  •  Naming Canon: The name of the test data should be in accordance of module name or functional area to make the reference very clear to the future onlookers.
  •  Test Data Requisites: The expected performance should be clearly mentioned. Dependencies should also be declared with clarity. The functions and modules which will use the test data should be clearly queued as well.
  •  Range of test data: The range should be specified well in advance for each data.
  • Re usability should be taken care of: The Test data should be written in a way that they can be used in the future too.
  • Maximum Coverage: Test data should be optimum in number and should have the maximum coverage.
  • Anon clause should be clear: Changes to be made to test data should be clearly mentioned to be used for later test case in case of identical functionality.
  • Scope field: The scope field like test boundaries, OS, database types etc. should be clearly declared.
  • Description: A brief description of test data should be given, which specifies the objective of the test data.

Challenges faced in Test Data Generation

Test data generation is quite complex as there is no standard skeleton for finding out the test data. The following are the various areas which require further study for test data generation:

Arrays and Pointers: The main problem exists during the symbolic execution, especially dynamic allocation of array and pointers and index or array or structure of the input of the pointer.

Objects: The OO features intensify the complexity as objects aredynamic by natureand it’s difficult to find out the exact code that would be called at run time. Use of mutation has been attempted to combat this problem.

Loops: Which path will be followed at the run time always remains a question mark, thus making the entire process of test data generation complex.

Despite these and a few other prevailing problems and challenges, Test Data Generation is making tasks easier with various available possibilities of creating large quantities and/or random data for testing purposes, thus reducing code conversion efforts.

Lets Connect Meet, Connect with TechArcis Quality Engineering Experts at QAI QUEST 2017

How can TechArcis assist?

TechArcis has expertise in enabling independent testing services, and is far ahead of the curve in following the process of Continuous testing Integration.

We’re focused on delivering high value added engagements with measurable returns on your investment. Simplify your test data management to reduce your software development and testing costs. Talk to us today

Other Resources

Agile demands a holistic view of testing and Automation

With a growing move towards the cloud, mobility and eCommerce, there’s an increasing complexity in both developer and customer sides with respect to IT businesses. Despite competition heating up and the need to market quality products fast,

With a growing move towards the cloud, mobility and eCommerce, there’s an increasing complexity in both developer and customer sides with respect to IT businesses. Despite competition heating up and the need to market quality products fast, the testing budgets aren’t expanding as they should have. With a limited budget that demands a lot to be done, agile practices and test automation have emerged as the driving force. Though agile is often perceived as applicable just to development teams, it needs the entire organization to adjust to become truly effective. Let’s see how a holistic view of testing and automation will power agile practices.

Download Whitepaper: Automation Testing in an Agile Environment

The entire team at work

Testing isn’t just the responsibility of testers. To bring stability and quality to applications early in the development process and ensure the business processes are flexible enough to keep pace with agile methodology’s iterative nature, the entire agile team needs to think holistically about testing and automation plans, approaches and tasks. Thus, from deciding what tests should be automated in agile environment, the most beneficial areas to automate within each sprint and for overall release, implementing TDD and BDD, maintaining and reviewing automated tests in agile environment and deciding when not to use automated tests, broad and collective thinking needs to be done by the agile team. It’s important to address the crucial pain points, highest risks and greatest value at first and then assign the work within the team to someone who has the expertise to handle it. since the decision are content-driven and need holistic thinking, you shouldn’t be surprised to find priority given to setting up a unit test framework and creating a unit test as compared to automating functional testing.

Tying tool technology with knowledge        

A holistic view of testing needs will help set a good framework for your automation. From reusing elements (such as a button press function or edit filed) and codes across applications of a similar technology type, maintaining and updating changes to location from a centralized point, to maintaining flow control in error (where scripts stop, report error and return to a clean slate to get ready for the succeeding test) and detailed reporting of error, you can create an automated framework that’s not limited to a specific automation technique or tool. Holistic view will also let you expand the benefits of automation to areas like data mining, data cleanup, application sanity tests (conducted by environment support teams and in case of failure, offering a preliminary diagnosis). Thus, by blending automation with tools, techniques and business understanding, delivering high quality software consistently and at a pace will become a reality.

Bringing testers and users into the fold

How many times have you faced problems in your end product due to late involvement of testers in the development cycle? Does late user involvement bother you as you often find no direct link between a user’s requirement and the system features your developers and testers have implemented and tested? Solution to such problems lie in a holistic view where you use an automation approach with a scriptless, behavior-driven, tool agnostic approach to utilize end users effectively while involving testers early on, thus blending technical and domain skills of the team to deliver quality products that meet the end-users’ demands and needs effectively.

Download Whitepaper: The Role of Testing in DevOps and Agile

Increasing the automation footprint

Since each release in agile testing has multiple iterations and sprints, the number of changes and volume of work is quite high. Thus, increasing the automation footprints in all aspects of the delivery procedure will help organizations save cost and time as they will no longer depend on manual processes. For example, if you consider the operations layer of test environment, a few key areas you can automate involve knowledge management, reports and dashboards, and service desk. Apart from establishing a single point of contact, such automation will pave way for easy knowledge base access and better control and visibility through fast and detailed reporting. Again for the test environment layer of data and application, you can automate app installment and code deployment, test data generation, version management, service virtualization. These will help in quicker installation and code deployment, higher test data availability, testing at early stage and better test coverage. Thus, for continuous integration and continuous delivery, knowing what and how to automate is crucial.

The collective agile team also needs to consider all validation, verification and automation work to arrive at the most beneficial solution to the business at a given point of time.  Want to know how you can use a holistic view of testing and automation in the agile setup for better decisions and quality outcomes? Feel free to drop us a line at contact us or reach us via Ph: 678-361-4357.

Things You Need to Know about Software Testing in 2017

The past half-decade, and most of 2016 markedly saw a surge in new technological trends in various walks of the software industry. While IoT and machine learning driving a shift towards artificial intelligence, the year left a gaping requirement in many fields of software development to catch-up. Software Testing, a phase which has always been a part of the SDLC did see a lot of transformations and new tool set being implemented. 2017 has already seen some old habits fade away and new ones being adopted that are perfect from the technology perspective.

Swifter Releases

Agile Software development isn’t a new concept. It has been around for a long time but over the past year, the concept has undergone a lot of development. Organizations are increasingly favoring adoption of Open source software especially in case of Agile because it allows collaboration among the teams. The result is faster development cycle and reduction in the costs. Developers prefer to use the code that is readily available as solution and thereby increase the efficiency.

The IOT Challenge

IoT is a complex network of connected devices embedded with electronics, sensor and software to enable collection and analysis of data being exchanged.  This unique architecture of systems call for various kinds of tests to ensure that the IoT applications function perform as per expectations and are secure.   The development of interfaces and connection and communication algorithms, were in full swing during the most of 2016, there wasn’t much interest shown into the area of testing. If IoT as a concept has to manifest itself into the real world, a proper end to end testing of the various communication and interfacing components needs to be developed.

Digital Transformation through Agile and DevOps

If one is to zero in on a trend which has markedly made its mark within the enterprise-wide technology space, it must be DevOps and the Agile methodologies which come bundled with it. A research commenced in 2016 asserted that as many as 54 percent of all the big enterprises across the world, were running digital transformation projects and revolution arising procedures entailing one agile technology or the other. As corporate move towards faster deliveries and pushing releases through the agile pipeline, it is needless to stress upon how important the testing wrapper around it will be. The real challenge in this is not just around having the testing procedures developed, but in deciding as to which ones are the most effective ones to have, that can be turned around in the least of time frames. This will be crucial in terms of minimizing technology spends for companies, while maintaining a competitive advantage over their peers.

Download Whitepaper: The Role of Testing in DevOps and Agile

Security Stand point of Software Testing

Do you remember what the buzz words were for the past year, in terms of the technological innovation space? There undoubtedly were ‘IoT’, ’Mobile’ and ‘Cloud’. Look at each o these and try and answer as to what is one common theme which comes out as a concern for all of these? Security! With the enterprise systems and resources moving largely out of premise on cloud networks, and being increasingly access by mobile users, there are grave threats to the working of such technologies from a security standpoint. Trust the world Security Report to realize the true potential of this threat, which clearly states more than 86 percent breaches occurred at the application level in enterprises. Needless to say, the focus in 2017 has to move from just developing applications on Cloud or in the mobile or IoT space, to having they being water tight in terms of security.

Download this Free checklist: Importance of Security Testing in IoT

Enhanced automation Testing

As technological innovation came on to transform all the fields of working as we know, automation has come on to manifest itself within the testing arena as well. Automation needs to become an integral part of modern application delivery methodologies like continuous delivery. This is the reason why a whole lot of tools such as Selenium were adopted to bring in automation of test cases in the various testing paradigms followed in enterprises. Most software testing professionals focused on implementing the testing automation for the basic functionality within the application, and this was the direction of the flow in 2016. For an automated software delivery, it makes sense  to have a code that can automatically run through a build process to include automated tests. As testing automation is manifested and companies start to realize the benefits in terms of cost savings, customers  become all the more demanding. This is why a smarter approach to software testing automation needs to be developed in 2017, stressing customer experience within the entire software testing framework. The automated processes will enable organizations to overcome the seemingly impossible range of variables they have to cater to.

Any new initiative needs to be properly funded, if it is to have its desired effect in the long term. Software Testing and quality assurance is no different and the corporate have surely realized this. Therefore the progression from 2014 to 2016 saw a surge of about 38 percent in the spends on testing budgets. Although 2016 was all about swelling quality assurance budgets and spends, there is a dire need to make sure that this investment bears maximum fruit. This is the reason why optimizing these spends, and putting the money on only the most optimized testing approaches should be the driver in 2017.

7 Ways to Improve Your Mobile App Testing

As mobile users are growing steadily, mobile devices too are undergoing changes with devices offering various network connection options for users along with using various OS versions and hardware platforms. No wonder mobile app testing has to combat such matrix of combinations fast as heightened competition needs mobile apps to have quicker development/release cycles. With confined testing budgets and such challenges, people often wonder how they can improve their mobile app testing. If you too belong to this league, here are some ways that can help you improve your mobile app testing effectiveness and raise your mobile app quality.

1. Testing on both real and emulated devices:

With device fragmentation being an important aspect in today’s mobile world, chances are high that your app won’t run on a single device or a screen size that’s uniform across different devices. That’s why you need to test your app with as many real-life devices as possible to check how it looks and works on different devices, platforms, screens, network connectivity options and OS versions with emulators and simulators. But you won’t get adequate confidence in your apps with just emulators and simulators. Cloud testing gives you an ideal solution in such cases by offering you real-time access to a wide variety of new, and sometimes even unreleased devices and network operators, along with inbuilt collaboration and test frameworks. You should also use scaling testing to test your apps on remote devices and involve other people in diverse geographical regions in testing along with using device farms for remote testing.

5 Mobile Application Testing Goof upsDownload this checklist

2. Stick to agile development:

A smart approach to mobile app testing involves finding the defects and defect trends early on and support rapid development cycles. So, you should adopt an agile culture to get your testing closer to being fast and more effective. Agile testing methodologies will not only improve your team’s ability to adapt to change, but will also improve stakeholder and customer engagement together with paving way for more useful documentation and creation of deliverables that are of higher quality.

3. Test automation and continuous automated testing:

By creating automated tests, you can run them more frequently as compared to human testers and even run them when no testers are available. With continuous automated testing, you can run tests automatically when an app’s source code has been updated and compiled successfully. With every change in the app, automated tests offer more traceability and visibility in the source code, thus letting developers know of early warning of failures faster. This in turn lets them investigate and fix problems quickly. To use test automation more effectively, you should choose test automation interface carefully (which could be ad hoc, publicly and officially supported, or a custom interface embedded in the mobile app). You can even use test monkeys, which are automated programs that let you test your apps.

4. Assess if your automation is delivering results:

Though automation is crucial for mobile app testing, it’s equally important to assess if you are getting the desired results from test automation. Unless you evaluate tests within a framework continuously, you may end up automating tests (e.g exploratory testing) that could have been done better manually. Measuring the value you get from test automation is also important to take the right decisions. For example, you should easily automate build smoke tests that deliver high value, while compatibility tests that are necessary could be given somewhat low priority due to the value you derive from them.

5. Using virtualized platforms and services:

Virtualizing your testing resources (by leveraging virtualized platforms and services) can cut costs in terms of setup and infrastructure, and even bring down the ongoing expenses involved. You can use virtualized test resources on both server and client sides of your mobile apps. Thus, with less costly infrastructure and hardware, you will be able to scale up or down, as and when required, which would help significantly in stress, load and performance testing.

Download Whitepaper:Security Testing Guidelines for Mobile Apps

6. Consider potential storage issues:

Despite having limited storage, mobile users these days like to install and play HD mobile games, run hi-resolution videos and photos, or use music streaming services with gigantic caches. So, while testing your mobile apps, you have to consider the extent of data your app will download to your users’ devices, and how the users’ monthly mobile data plans may get affected by the amount of such data downloads. Since unavailability of disk space often make mobile apps behave unpredictably, you should try to minimize your app’s storage requirements on mobile devices.

7. Focus on end-user experience to know about problems:

You won’t know the real performance of your app by just reviewing ratings and comments in app stores. Rather, try using modern crash/analytics SDKs that have a small run-time overhead, as they can give you real-time insights into the interaction of your users with your app apart from providing you with detailed crash reports that would help find problems immediately and fix them.

Despite mobile app testing becoming a tough proposition with the introduction of new devices with different features, you can improve your mobile app testing with test automation and use of virtualized and cloud resources together with leveraging new tools and techniques. Join hands with TechArcis and see all this happening in reality to give your app users an unmatched, seamless experience.