GCAT: Detecting and Summarizing GUI Changes in Evolving Mobile Apps 

Team Members: Kevin Moran, Cody Watson, John Hoskins, George Purnell, & Denys Poshyvanyk

College of William & Mary --- SEMERU

 
1447968923388.png
 

Purpose

This project was created by the Software Engineering Maintenance and Evolution Research Unit (SEMERU) at the College of William & Mary, under the supervision of Dr. Denys Poshyvanyk.  The major goal of the Gcat project is to help improve and automate the process summarizing the changes to a mobile application's graphical users interface as it evolves over time in order to improve program comprehension tasks.

Video Demonstration

 
Coming-Soon.png
 

Publications

  • Detecting and Summarzing GUI Changes in Evolving Mobile Apps
    ‣ Kevin Moran, Cody Watson, John Hoskins, George Purnell, and Denys Poshyvanyk
    ‣ Proceedings of the The 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE'18)Montpellier, France, September 3rd-7th 2018, to appear, 12 pages (21% Acceptance Rate)
    ‣ [pdf]

Gcat is going to be open sourced on GitLab!

We will be open sourcing the Gcat tool on GitLab. The project will be released in October 2018, but if you would like early access please email kpmoran@cs.wm.edu. 

 
logo.png
 

Gcat

Gcat Workflow Overview

Figure 1: Gcat Workflow Overview (Click for more detail)

There are three main components of the Gcat approach depicted in Figure 1, (i) Version Control Integration, (ii) Automated GUI-Based Dynamic Analysis, and (iii) GUI Interpretation and Change Detection.  Gcat is able to analyze subsequent commits from a software repository that utilizes a version control system such as Git, and automatically compiles and executes target commits. It then filters and matches screens discovered during automated exploration of a target app's GUI, and finally detects and reports changes related to GUI-components.  Gcat was implemented for the Android platform and is written in Java. It was designed to allow for integration into Continuous Integration and version control systems to facilitate fully automated generation of documentation. In this section we describe each component of the Gcat approach.

Tools used for Gcat Implementation

We provide the tools we used in our implement GVT below:

Accessing the Gcat Open Source Project & Documentation

We will be open sourcing the Gcat tool on GitLab. The project will be released in October 2018, but if you would like early access please email kpmoran@cs.wm.edu. 


GVT Study

Research Questions

RQ1: How well does Gcat's screen matching and filtering procedure function?
RQ2: How well does Gcat perform in terms of detecting and classifying GUI changes that occur during the evolution of mobile apps?
RQ3: Is Gcat able to more accurately detect and classify GUI changes in evolving mobile apps compared to manual efforts from developers?

RQ4: Do developers find Gcat reports useful for documenting and summarizing GUI changes in evolving mobile apps?

Study Subjects

The table below contains the Name, Versions, LoC, and the number of downloads (if listed on Google Play) of each app used in our evaluation of Gcat. Please click on an app in the table to go to its page in the FDroid Repository. 

 
App name App Versions LoC Old LoC New # Downloads
FastHub Libre 3.0.0 | 4.6.2 88444 152107 100,000+
Authorizer 0.2.4 | 0.2.5 46984 47667 100+
AnyMemo 10.9 | 10.9.993 44847 44093 100,000+
Gear Shift 0.9.96-Beta | 1.0.5 39637 42246 N/A
Book Catalougue 3.7 | 3.8.1 31607 31946 100,000+
Easer 0.4.9 | 0.5.5.3 23928 30149 N/A
Calendar 3.1.0 | 3.3.2 22326 23412 500,000+
ServeStream 0.4.6 | 0.4.8 19878 20086 100,000+
SkyTube 2.7.0 | 2.8.0 17670 20425 N/A
Flym 1.9.0 | 1.9.1 17384 17359 50,000+
LnkShortener 1.0.2 | 1.1.2 16464 30196 50+
fwknop2 1.0 | 1.2.3 15540 20810 1,000+
SysLog 2.0.2 | 2.1.2 13340 13141 100,000+
Equate 1.4 | 1.6 12706 13924 10,000+
SGit 1.3.0 | 1.3.2 11097 11863 50,000+
Simple Weather 4.1 | 5.3.2 10210 10662 10,000+
Earmouse 1.2.2 | 1.2.3 6022 6114 500+
Simple Dilbert 4.3 | 4.5 3659 3841 N/A
Bullseye 0.1 | 0.3 3272 3185 N/A
Fon 1.5.1 | 1.5.2 3151 3152 100,000+
Sound Recorder 1.2.5 | 1.3.0 3025 3211 5,000+
Sensors20SC 0.1.0 | 0.3.0 2670 2637 N/A
Currency 1.0 | 1.5 2546 5559 N/A
Simple-Calculator 2.0.0 | 3.1.2 2484 2801 10,000+
StageFever 1.0.8 | 1.0.9 2461 1971 N/A
Counter 13 | 18 2012 2880 100,000+
Draw 1.14 | 2.3.0 1993 2288 100,000+
App Launcher 2.0.0 | 3.1.1 1807 1781 10,000+
Dumbphone Assistant 0.4 | 0.5 1566 1600 N/A
Snotepad 1.0 | 1.2.1 1406 1479 N/A

 

Gcat Reports for Empirical Evaluation

Reports for Empirical Study

 
Screen Pair Identifier % PID Pixel Difference GUI-Change Severity Category
com.a5corp.weather-4.1-4-vs-5.3.2-4 9.228334 HIGH
com.a5corp.weather-4.1-5-vs-5.3.2-5 91.81071 HIGH
com.simplemobiletools.calculator-2.0.0-3-vs-3.1.2-3 94.89196 HIGH
com.simplemobiletools.calculator-2.0.0-vs-3.1.2 93.25475 HIGH
me.tsukanov.counter-13-2-vs-18-2 18.14931 HIGH
net.fred.feedex-1.9.0-3-vs-1.9.1-3 91.423065 HIGH
org.billthefarmer.currency-1.0-2-vs-1.15-2 59.858547 HIGH
com.eleybourn.bookcatalogue-3.7-2-vs-3.8.1-2 0.92021275 MEDIUM
com.simplemobiletools.calendar-3.1.0-3-vs-3.3.2-3 1.1721597 MEDIUM
com.simplemobiletools.draw-1.14-2-vs-2.3.0-2 3.908329 MEDIUM
com.tortel.syslog-2.0.2-2-vs-2.1.2-2 11.835075 MEDIUM
com.tortel.syslog-2.0.2-4-vs-2.1.2-4 1.9393876 MEDIUM
me.sheimi.sgit-1.3.0-vs-1.3.2 1.7022176 MEDIUM
me.sheimi.sgit-1.3.0-4-vs-1.3.2-4 2.963039 MEDIUM
net.fred.feedex-1.9.0-7-vs-1.9.1-7 3.767209 MEDIUM
net.tjado.passwdsafe-0.2.4-3-vs-0.2.5-3 7.3318005 MEDIUM
org.sensors2.osc-0.1.0-5-vs-0.3.0-5 4.325908 MEDIUM
org.sensors2.osc-0.1.0-5-vs-0.3.0-5 3.4219935 MEDIUM
org.sugr.gearshift-0.9.96-4-vs-1.0.5-4 5.5033703 MEDIUM
x653.bullseye-0.1-vs-0.3 3.3350704 MEDIUM
com.danielkim.soundrecorder-1.2.5-2-vs-1.3.0-2 0.26885355 LOW
com.fsck.k9-5.208-vs-5.304 0.24173349 LOW
me.sheimi.sgit-1.3.0-6-vs-1.3.2-6 0.34786674 LOW
net.sourceforge.servestream-0.4.6-2-vs-0.4.8-2 0.24603619 LOW

REports for User Study

 
Screen Pair Identifier % PID Pixel Difference GUI-Change Severity Category
de.hirtenstrasse.michael.lnkshortener-1.1.2-2-vs-1.0.2-2 94.97614 HIGH
de.hirtenstrasse.michael.lnkshortener-1.1.2-vs-1.0.2 94.6314 HIGH
net.fred.feedex-1.9.1-2-vs-1.9.0-2 93.88904 HIGH
com.a5corp.weather-656-vs-578 88.306564 HIGH
com.oakley.fon-1.5.2-vs-1.5.1 84.10892 HIGH
com.oakley.fon-1.5.2-2-vs-1.5.1-2 80.72212 HIGH
com.github.yeriomin.dumbphoneassistant-0.5-vs-0.4 66.68771 HIGH
com.llamacorp.equate-1.6-vs-1.4 63.78696 HIGH
com.a5corp.weather-656-2-vs-578-2 56.958313 HIGH
net.fred.feedex-1.9.1-vs-1.9.0 49.698788 HIGH
com.tortel.syslog-26-2-vs-23-2 20.65635 HIGH
me.tsukanov.counter-18-vs-13 15.899084 HIGH
com.tortel.syslog-26-3-vs-23-3 14.144452 HIGH
net.tjado.passwdsafe-0.2.5-vs-0.2.4 14.109943 HIGH
org.liberty.android.fantastischmemo-10.9.993-2-vs-10.9-2 13.890984 HIGH
org.liberty.android.fantastischmemo-10.9.993-3-vs-10.9-3 13.649076 HIGH
org.liberty.android.fantastischmemo-10.9.993-vs-10.9 13.052783 HIGH
com.simplemobiletools.applauncher-3.1.1-vs-2.0.0 11.826948 MEDIUM
com.fastaccess.github.libre-4.6.2-vs-3.0.0 11.759409 MEDIUM
org.cipherdyne.fwknop2-1.2.3-vs-1.0 7.5255957 MEDIUM
com.simplemobiletools.calculator-3.1.2-vs-2.0.0 7.3374505 MEDIUM
com.a5corp.weather-656-4-vs-578-4 6.509115 MEDIUM
org.billthefarmer.currency-1.15-2-vs-1.0-2 6.1474714 MEDIUM
net.sourceforge.servestream-58-2-vs-56-2 6.0621996 MEDIUM
me.sheimi.sgit-110-3-vs-108-3 5.9370737 MEDIUM
ryey.easer-0.5.5.3-vs-0.4.9 5.5706487 MEDIUM
org.sugr.gearshift-1.0.5-vs-0.9.961 4.44169 MEDIUM
com.mareksebera.simpledilbert-38-2-vs-36-2 4.393839 MEDIUM
org.sensors2.osc-3-vs-1.1 4.3309064 MEDIUM
me.sheimi.sgit-110-2-vs-108-2 2.9849436 MEDIUM
com.simplemobiletools.calendar-3.3.2-vs-3.1.0 2.7350829 MEDIUM
x653.bullseye-0.3-vs-0.1 2.430721 MEDIUM
com.a5corp.weather-656-3-vs-578-3 2.336105 MEDIUM
com.mareksebera.simpledilbert-38-3-vs-36-3 2.0599935 MEDIUM
org.billthefarmer.currency-1.15-vs-1.0 1.8113931 LOW
com.tortel.syslog-26-vs-23 1.7986155 LOW
info.aario.snotepad-2-vs-1 1.2906796 LOW
free.rm.skytube.oss-12-vs-10 1.1355215 LOW
org.sugr.gearshift-1.0.5-2-vs-0.9.961-2 1.0561607 LOW
pk.contender.earmouse-1.2.3-vs-1.2.2 1.0135248 LOW
com.eleybourn.bookcatalogue-3.8.1-vs-3.7 0.918822 LOW
com.mareksebera.simpledilbert-38-vs-36 0.65900844 LOW
com.simplemobiletools.draw-2.3.0-2-vs-1.14-2 0.51384664 LOW
com.fsck.k9-5.304-2-vs-5.208-2 0.42544574 LOW
net.sourceforge.servestream-58-vs-56 0.2473835 LOW
com.fsck.k9-5.304-vs-5.208 0.23803926 LOW
net.alaindonesia.silectric-1220161217-vs-1120168131 0.18101762 LOW
com.simplemobiletools.draw-2.3.0-vs-1.14 0.15980835 LOW
com.fr3ts0n.stagefever-10009-vs-10008 0.1304283 LOW
me.sheimi.sgit-110-vs-108 0.11026211 LOW
com.danielkim.soundrecorder-130-vs-8 0.0926167 LOW

Study Results

Results for RQ1

Our first RQ investigates the performance of Gcat’s screen filtering and matching procedure. Running CrashScope through all 61 of our subject apks resulted in 3,854 total extracted screens, or ≈ 63 screens per apk. Gcat’s filtering procedure was able to reduce this set to a much more manageable 316 screens for the matching proce- dure. This results in an FS measurement of (3, 854 − 316/3, 2854) × 100 = 91.8%, meaning that over 90% of the collected screens were filtered out as redundant, drastically reducing the information bur- den on developers for reading GUI-change reports. These filtered screens resulted in 158 matched screen pairs, which exhibited aMatching Precision (MP) of 84.8%. This illustrates that Gcat is able to both effectively filter and match corresponding screen pairs that were automatically extracted from automated dynamic analysis of subsequent app versions.

Results for RQ2

 
 Detection Precision (DP), Classification Precision (CP), and Recall (R) of Gcat during typical use.

Detection Precision (DP), Classification Precision (CP), and Recall (R) of Gcat during typical use.

 

Results for RQ3

 
 Detection Precision (DP), Classification Precision (CP), and Recall (R) of Humans.

Detection Precision (DP), Classification Precision (CP), and Recall (R) of Humans.

 

Results for RQ4

 
 Participant Responses to the Likert-Based UX questions illustrating favorability for Gcat

Participant Responses to the Likert-Based UX questions illustrating favorability for Gcat

 

Data and Replication Package

We are committed to open science practices and will release a replication package for the Gcat study in October 2018. If you are interested in the data before this time, please contact kpmoran@cs.wm.edu for early access.