As I see it, it is important to have a test plan to help the people outside the team understand the details of testing. The next tool we also mentioned is cucumber, this is more of a way of testing than a tool, it supports Behaviour Driven Development , it helps explain the application flow aspect in Gherkin syntax. It is one of the most wildly used automation testing tools due to many reasons, first of all its open source and free and it supports multiple languages like Python, Java and JavaScript. It is usually hard to create an exact copy of the production environment, in this chapter I will explain the difference between test and staging environment and I will explain the role of these test environments and how we used them. For any successful UI automation locators or element ids are needed, it’s how the automation tool knows where to find a specific element on the view.

  • Automating ios, we would be sometimes faced with the issue of the Appium automation tool unable to load certain views, this is because of the huge amount of information a certain view has, sometimes it would take up to five minutes to load a view which is not sustainable, the solution here is to limit the tool to load up to a certain depth in the view where the element we need is located, making it easier and faster for the tool.
  • The best practice here would be using a function that waits for the element to show on the view instead of using sleeps, which is definitely not a good practice, when the waitForVisible function runs, it checks for the element we want to find every 500ms, this ensures we go on with the test case once we find the element opposed to putting a five second sleep where even if the element is present, the test will still be on hold for five whole seconds.
  • We’re starting off by our requirement analysis, in this chapter we’re going to talk about the chosen tools, every tool has to have a purpose and lets face it, some tools also have cons so when it comes down to it have to makes sure the pros outweigh the cons.
  • As for web automation we chose the most used framework, Selenium, which is used for webapps and does not support system or mobile application hence the necessity of using both Appium and Selenium as automation frameworks.
  • A staging environment replicates the prod environment, it is important that this environment is an exact copy of production, a very detailed documentation is needed to achieve this.
  • We managed these environments through automation by build a deployment automation CI, as mentioned before the tool we used is Gitlab CI, this tool helped us automate and run test suits through scripts in which we mentioned the targeted repository the tests we want to run, the chosen environment and obviously a platform of choice ios/android/web, which was launched on browserstack, our cloud provider.
  • That was the goal of our framework, we aimed to test the webapp and ios/android app with one single test on the User Interface side.

As the name suggests,@beforehook gets executed before any other test scenario, gets executed after executing the scenario, this helped us control some of the precondition. I think many will be questioning this choice versus the Maven tool, so let me just say both are able to cache dependencies locally and download them in parallel. As a library consumer, Maven allows one to override a dependency, but only by version meanwhile Gradle provides customisable dependency selection and substitution rules that can be declared once and handle unwanted dependencies project-wide and that’s why we chose it over Maven.

Some Of The Issues We Faced And How We Were Able To Solve Them

We also have to think about compatibility between tools, that means that every tool we chose has to be compatible with all others and vice versa, so a lot of research has goes into it.

We as QA engineers always try to optimise execution and maintenance times and even testing strategies making it easy to write tests in an efficient way we even wanted to involve our POs or anyone who had interest in writing test for that matter in the creation of tests using the cucumber tool. For mobile automation we chose Appium, an open-source framework that allows the QAs to conduct automated app testing on different platforms like Android, iOS, and Windows. As for web automation we chose the most used framework, Selenium, which is used for webapps and does not support system or mobile application hence the necessity of using both Appium and Selenium as automation frameworks. That was the goal of our framework, we aimed to test the webapp and ios/android app with one single test on the User Interface side. Another feature of the framework is profiling, we can basically create a Json profile that has chosen capabilities like screen size and browser for web tests, exact mobiles and operating system version to execute our tests. Our Hooks had more uses one of which was having our tests named in BrowserStack, making easy to distinguish between tests, specially if multiple are running, this was done by calling a function in the before hook to send the name to browserstack as a capability or preset to the test.

Reporting

We chosen this particular cloud based platform because not only is easy to integrate but it also only offers ‘real devices’ and that is important when it comes to testing, we want to make sure we get as close to the real thing as possible, in my opinion a simulated or an emulated device will never be the same as a real device, of course it can come close but there will always be differences. A staging environment replicates the prod environment, it is important qa framework that this environment is an exact copy of production, a very detailed documentation is needed to achieve this. Same goes for gitlab CI, sometimes we had to wait for ports to clear so that the automation process could pick one and launch the automation, we shared the repository with other teams so that part was difficult to adapt. The next tool on the list is GitLab CI/CD which is the part of GitLab that you use for all of the continuous methods .

qa framework

While a test environment is focused on testing individual components, the staging environment is focused on testing the whole application. Basically, the staging environment is a safe playground in which you can test the whole application and this is what makes staging ideal for end-to-end testing or performance testing. This description should detail what unit, feature, or function is being tested or what is being verified,m this is done through the cucumber language as seen before, it is a step by step process explaining how the test will run and with the help of a feature name and a scenario name a simple look at the test case should be enough to understand the goal of said test. UI testing, graphical user interface testing is the process of testing a product’s graphical user interface to ensure it meets its specifications, we also had envisioned API testing but we didn’t get that far, it is safe to say that our framework was prepared to integrate API testing. The scope of testing is very important, we should know what part of the app is assigned to our team, what should, what shouldn’t and what can’t be tested via automation testing, there are a lot of advantages to automated testing but like everything is has some limitation as well. As mentioned before, Browserstack is a provider of real remote devices and we had to pay a monthly fee for the five threads we had available.

A Framework For Qa Testing

We’re starting off by our requirement analysis, in this chapter we’re going to talk about the chosen tools, every tool has to have a purpose and lets face it, some tools also have cons so when it comes down to it have to makes sure the pros outweigh the cons. This tool can be set up using Gradle to generate any configured extended reports, you can even customise it by overriding property values by passing in other properties to override the ones by default, all this is done in the serenity.properties file. The best practice here would be using a function that waits for the element to show on the view instead of using sleeps, which is definitely not a good practice, when the waitForVisible function runs, it checks for the element we want to find every 500ms, this ensures we go on with the test case once we find the element opposed to putting a five second sleep where even if the element is present, the test will still be on hold for five whole seconds. In most cases, following a convention for this naming ID helps with organisation, clarity, and understanding of the test case, in the second part of the chapter I explain more about IDs and why they should be selected over xpaths. We came together with the developers to define what tests should be prioritised, each member of the team had assigned a number of test cases, we each made sure the tests were completed before the end of the sprint, making sure the continuity of the cycle.

qa framework

Neglecting QA activities may create the illusion of faster feature delivery or reduced costs/resources for the project but in the long run not paying enough attention to the quality assurance process will lead to serious risks both technical and commercial and would even jeopardise the security of the product, in my humble opinion QA is the key to building a stable secure product that users will find comfortable and pleasant to use. We managed these environments through automation by build a deployment automation CI, as mentioned before the tool we used is Gitlab CI, this tool helped us automate and run test suits through scripts in which we mentioned the targeted repository the tests we want to run, the chosen environment and obviously a platform of choice ios/android/web, which was launched on browserstack, our cloud provider. After each execution we get a report generated by serenity where details of the run are shown, serenity is a tool used for reporting as well as other thing like writing actual tests using its framework, we had it implemented because of the rich reports it offers, serenity reports can be customised and adapted to the needs of the framework, in addition to standard HTML reports it provides emailable reports and even HTML reports as a single page react application. A test environment is an environment that allows you to run already defined test cases, it enables you to create identical environments every time you need to test a product, it’s very important for a testing engineer to ensure reliable results, a testing engineer uses test environments to test a very specific part of an application, and what makes this environment so different than production is the setup needed for testing different parts of the app. We created a readme file with all the documentation and all the steps necessary to prepare a testing environment, automation tools, environment access, testing mobiles setup, accounts used for testing and even local machine setup for running test in local.

Resource Planning

Quality Assurance is a very complex process that includes many steps to ensure the security and quality of the software under development, when the QA process is properly organised and executed it enables you to track and eliminate bugs, performance issues and even security issues before the release. The framework we created was meant to do all this for three platforms while keeping an optimal performance and needing as little maintenance as possible regarding the core architecture of the framework, we adapted it to both our needs and the clients needs. An issue all QA automation engineers face is execution times specially when it’s UI automation and have a regression test to run, regression testing is a software practice that ensures an application still runs as expected after any code changes in the app, the regression is responsible for the application stability overall.

qa framework

With GitLab CI/CD, you can test, build, and publish your software with no third-party application or integration needed. One more important thing that one should keep in mind is that not all tools are free, some need a monthly fee, like a subscription an obvious example would be Browserstack our cloud platform. Apiumhub brings together a community of software developers & architects to help you transform your idea into a powerful and https://globalcloudteam.com/ scalable product. A Test Plan is a detailed document that describes the test strategy, objectives, schedule, estimation, deliverables, and resources required to perform testing for a software product. To finish this chapter, the is only thing left to mention is the development language, we chose Java because it has significant advantages over other languages and environments that make it suitable for just about any programming task.

Test Planning

The number of threads our cloud platform had was another resource to consider, we only had five threads, so between us members of the team and the automation running in the background we had sometimes to calculate, adjust and adapt our time to avoid wasting time waiting for threads to clear. Test planning is where all elements connect together, all the teams, ios, android, web including our POs, after they evaluate the customer needs and define our plan/strategy, each team develops new functionalities that needed testing, thats where we come in as QAs, our objective is to test each of theses functionalities before they get to production. The test plan helps us determine the effort needed to assure the quality of the application under testing, it also serves as a blueprint to control software testing as a whole which is monitored by the QA team.

The documentation is essential for the continuity of any project, reliable documentation is a must it helps keep track of all aspects of an application and it improves the quality of a software product, the main goal of documentation is development, maintenance and knowledge transfer to other developers in case it is needed. This Article is about the steps we took to make that possible, what challenges we encountered and what possible solutions were found and the ones we decided to adapt, we also explain the use of tools like browserstack cloud platform and how it makes the automation run smoother and faster. Automating ios, we would be sometimes faced with the issue of the Appium automation tool unable to load certain views, this is because of the huge amount of information a certain view has, sometimes it would take up to five minutes to load a view which is not sustainable, the solution here is to limit the tool to load up to a certain depth in the view where the element we need is located, making it easier and faster for the tool. A test case is a scenario measuring the functionality of many actions or conditions to verify an expected result, they can be for manual testing or automated testing, a very important thing to keep in mind is that test cases are intended to test basic behaviour such as a login action, many make the mistake of creating a long test case which becomes a nightmare to maintain specially in automation defying the purpose of automated testing, having a small controlled test case allows more flexibility in how to test features.