Version control of test data also plays a great role in the same manner as you can’t live without source code version control. You should definitely think of this dimension while choosing the test automation tool. Otherwise, you will face various issues once you have adopted the tool in the later run. The following issues are centered around version control:
- Test data Auditing (Who changed what data at what time)
- Review process
- Minimizing merge conflicts
- Team collaboration or sharing issues
- Backup and restore test data
I am writing this post in context of API Automation data. We lack version control in the vREST cloud version as of now. After the hard-earned experience of five or more years and learning from several customers, we re-built vREST NG version, solely to solve this particular problem. In vREST NG, we tried to resolve this problem at its root.
First, we will see, how some of the tools in the market handles this particular issue, although I will not name any of the tools:
Issues when tool exports all the test data in a single file
Some tools export all the data in a single large JSON file. If you are working on a large project then this problem become worsen. Think of a situation where multiple testers are working on a testing project if they all work on a single large file, then how you are going to handle the conflicts. The review process will also become cumbersome for you with this strategy.
There are also some variations among tools. Some tools don’t even export all the test data. So you have to manually export all the data via some hacks or via some cumbersome manual process repeated every time. These hacks consume a lot of effort at a later stage.
Further, if the tool doesn’t import the exported data back then you will face issues like backup and restore test data. So, it should be possible to import the data back into the tool to support the backup and restore capability. Some tools partially export and partially import the data. This makes the backup and restores procedure cumbersome.
Issues when tool provides inbuilt version control capabilities
Git is a famous version control system and it took several years of development to become GIT a leader in this domain. If an automation tool provides inbuilt capabilities of version control then it is obvious that they will pick the basic and bare minimum features of GIT and implement it in their tooling. However, there are several issues with this approach:
- Each company has its own workflow of the review process, but they have to stick with the workflow provided by the tool.
- Some companies might be using other version control systems e.g. CVS, Mercurial, etc. but they have to stick with the integration provided by the tool.
- Mostly these inbuilt capabilities lack important features like branching, tagging, merging, custom review process, and conflict resolution, etc.
- Some tools lack integration with famous tools like Github, Gitlab, Bitbucket. Then it becomes harder for the companies which are using Gitlab or any other similar tool and the automation tool doesn’t provide such integration.
Some companies even have a policy that they will not expose their data to any third-party applications. In such situations, cloud-based solutions become out of scope.
The solution which vREST NG provides
vREST NG is designed to solve the above-mentioned issues. vREST NG stores all the test data on the file system. It not only stores the data on the file system but also in a well-structured manner. It creates separate files for each of the following categories of data:
- Test Cases Data
- Test Suites Data
- Authorization related data
- Environment-related data
- Global schemas (Either defined or imported from Swagger spec)
- Custom Response Validation Scripts
- CSV data for data-driven testing
If you are still not sure on how vREST NG stores data then I will recommend you to see these example repositories:
We create separate files for each test case to minimize the merge conflicts. We have done several iterations and discussion within the team on how we should store the data. Now we are in the market to have your feedback. Also, we provide several other features to enhance the version control process. e.g. if the expected response body is of type JSON then we store the expected body in JSON format instead of String format. So that any changes in the test case expected response body can be easily reviewed by QA Leads.
As vREST NG stores all the data on the file system. So, you may use any version control system of your choice. You may store your test cases on Github, Bitbucket, Gitlab or any other online version control systems without any integration issues. Sharing or Team collaboration becomes easier without any further learning process. You may apply the same workflow process or review processes as you apply for your application source code.
Finally, We would like to have your feedback regarding the version control feature supported by vREST NG. Just visit vREST NG website and try out vREST NG and let us know what do you think about this tool. I will definitely assure you that you will love this product. This tool is designed to be enterprise-ready and after several years of experience and hard-learned lessons in the domain of API Testing.