We have performed an empirical study to identify the main issues faced on the GREat Test Factory. The objects of the study are five tools we have developed in a partnership with industry. The following table presents the target platform, tools and their number of releases. The tools' names and versions have been changed for reasons of confidentiality agreement. Two web tools are of global use (T3 and T4), their complexity demands constant and accurate testing since they present critical information about the company's internal processes such as schedules, activity monitoring and resource allocation. Mobile applications (T1 and T2) are based on Android technology and are continuously updated with new features and changes in the interaction flow to cover usability issues. The major challenge faced on such mobile applications is that they currently have together over 3 millions active users, so they must be tested in several Android versions running on different target devices to ensure they work as users expected.
| Platform | Tool | Releases |
|---|---|---|
| Mobile | T1 | 5 |
| Mobile | T2 | 4 |
| Web | T3 | 3 |
| Web | T4 | 4 |
| Web | T5 | 5 |
The results shows that 18 releases are launched without any tests in 2019 as illustrated in the graphic below. In addition, T1 and T4 during one release had some tests, which were initially planned, postponed.
