Are Thermal Attacks Ubiquitous? When Non-Expert Attackers Use Off the shelf Thermal Cameras

Recent work showed that using image processing techniques on thermal images taken by high-end equipment reveals passwords entered on touchscreens and keyboards. In this paper, we investigate the susceptibility of common touch inputs to thermal attacks when non-expert attackers visually inspect thermal images. Using an off-the-shelf thermal camera, we collected thermal images of a smartphone’s touchscreen and a laptop’s touchpad after 25 participants had entered passwords using touch gestures and touch taps. We show that visual inspection of thermal images by 18 participants reveals the majority of passwords. Touch gestures are more vulnerable to thermal attacks (60.65% successful attacks) than touch taps (23.61%), and attacks against touchscreens are more accurate than on touchpads (87.04% vs 56.02%). We discuss how the affordability of thermal attacks and the nature of touch interactions make the threat ubiquitous, and the implications this has on security.


Introduction
Recent work showed that thermal attacks are effective in retrieving passwords by using expensive thermal cameras [1] and/or employing automated image processing approaches [1,18,23].To date, it is not well understood to what extent a non-expert with no technical skills can conduct thermal attacks using an off the shelf camera.If non-experts can perform thermal attacks without any technical background, then this means the majority of the population can perform the attack, making the threat ubiquitous.Compared to previously considered threats, studying threats by untrained attackers sheds light on a) how realistic the risk of thermal attacks is, and b) how ubiquitous thermal attacks can be.
We close this gap by investigating how well untrained attackers infer touch taps (Fig. 1.1 and 1.3) and touch gestures (Fig. 1.2 and 1.4) by visually inspecting thermal images.In a first study (N=25), we collected thermal images of gestures and taps resulting from authenticating using two graphical authentication schemes on a smartphone's touchscreen and a laptop's touchpad.In a second study, 18 new participants inspected the thermal images visually to infer the passwords.Inputs are correctly guessed 60.65% and 23.61% of the time in case of gestures and taps respectively.Guesses based on thermal attacks are fully correct at almost equal rates across touchscreens (43.06%) and touchpads (41.2%).More guesses against touchscreens (87.04%) are partially correct compared to touchpads (56.02%).Tapping is more secure on touchpads than on touchscreens, and touch gestures are more secure on touchscreens than on touchpads.We discuss how the nature of interactions and physical properties of interfaces contribute to the success of thermal attacks.Our results highlight that thermal attacks are becoming ubiquitous and have significant implications on touch-based authentication.

Background and Related Work
Users unlock their mobile devices around 40 times a day, thereby creating many occasions in which users are subject to side channel attacks, such as observation attacks [13], video attacks [33], smudge attacks [6], and thermal attacks [1].While observation, video and smudge attacks were extensively researched in previous work [12,26,28,29], thermal attacks are relatively under investigated.
In thermal attacks, thermal cameras capture heat traces left on interfaces after authentication [1,23,31].When the user touches a surface, heat is transferred from the users' hand to the touched surfaces.This generates a temperature difference at the point of contact referred to as heat traces.Heat traces can be detected using thermal cameras.Mowery et al. [23] were among the first to explore thermal attacks by using an $18,000 thermal camera and an automated approach to find PINs on plastic ATM keypads.They found thermal attacks ineffective against metal keypads as they reflect heat signatures.Further research underlined the threat's significance on mobile authentication [1,4].Abdelrahman at al. [1] studied thermal attacks on PINs and Android Patterns on smartphone touchscreens.They found that password properties, such as duplicate digits in PINs and overlaps in Patterns, impact the attack's success.They also used a high end thermal camera and an image processing algorithm to detect passwords up to 30 seconds after authentication.Kaczmarek et al. [18] studied thermal attacks on external keyboards.Using an automated approach, their attacks could recover key presses up to 30 seconds after entry.
While prior work used expensive thermal cameras (e.g., $5,900-$18,000 [1,23]), or automated approaches [1,18,21,23,31], we study thermal attacks where non-expert attackers visually inspect thermal images taken by an affordable (<$450) off-the-shelf thermal camera.This means that we consider a threat model that is more realistic, more likely to happen, and potentially more ubiquitous.In addition, we compare the thermal attack resistance of touch gestures and touch taps when entered on touchcsreens and touchpads.

Threat Model
In our threat model, the attacker waits until the victim authenticates on a laptop or a mobile device and then leaves it unattended.To ensure optimal but realistic attack conditions, the user does not interact after authenticating (e.g., quickly checking messages, emails, etc.) before temporarily leaving the device to attend to something else (e.g., get coffee).The attacker takes a thermal image of the interface and visually inspects it to guess the password.

Evaluation
Our evaluation entailed two phases: 1) dataset collection, and 2) thermal attack execution.Both complied with ethics and privacy regulations of the university in which they took place.Both study phases were designed as within-subjects experiments where all participants went through two independent variables.IV1) Input Type: Gestures (Drawmetric) and Taps (Locimetric): We studied two input types: touch gestures and touch taps.Gestures are commonly used for drawmetric graphical passwords (aka recall authentication schemes) [8], such as Draw-A-Secret [17], Pass Shapes [30], and free form gestures [22,32].Gestures are sometimes also used for cued-recall schemes [26].Recent research prototypes, such as SwiPIN [27], XSide [12] and CueAuth [20], also make use of touch gestures.Lock Patterns on Android is an example of a commercial adoption of authentication using touch gestures.On the other hand, taps are used for PINs, and for locimetric graphical passwords [5] where the user selects multiple points on one or more images.Examples include PassPoints [24], and CGP [14].Windows 10's image password is a sample Locimetric scheme.
IV2) Input Interface: Touchscreen and Touchpad We compared a smartphone's gorilla glass touchscreen (touchscreen for short), and a laptop's capacitive touchpad (touchpad for short).Materials exhibit different thermal conductance [2], which means that touchscreens' resistance to thermal attacks (e.g., in [1,4]) is not necessarily similar to that of touchpads.
Implementation To collect input from participants, we implemented Android and Windows versions of drawmetric and locimetric schemes.The drawmetric scheme is similar to Draw-A-Secret [17] and allows participants to freely draw on the touchscreen and touchpad using touch gestures (Figures 1.2 and 1.4).While the locimetric one follows prior implementations of cued-recall passwords [3].The scheme overlays a picture over a 3×3 grid on which the user has to tap some positions on the touchscreen, or navigate the pointer then tap using the touchpad (Figures 1.1 and 1.3).

Phase 1: Data Collection
To collect a dataset of thermal images to be used in the subsequent security study, we invited 25 participants (9 females) aged between 18 and 28 (Mean=22; SD=2.7) through university mailing lists.

Procedure
The experiment was conducted in a temperature controlled room in our lab (24 • C).Upon arrival, participants were explained the study and asked to sign a consent form and a demographics questionnaire.We recorded the participants' hand temperature, as well as that of the touchscreen and the touchpad.After introducing the two authentication schemes, we gave the participants 4 minutes to familiarize themselves with them.We then told the participants the password they had to enter one at a time according to a predefined list of passwords.We explain how we generated the list in the following section.The passwords in our list were illustrated on paper and handed to participants.To prevent heat traces of different entries from mixing up, participants waited for one minute before entering the following password to allow the older heat traces to fade.Each participant entered 24 different passwords (2 input interfaces × (6 drawmetric passwords + 6 locimetric passwords)).The order of conditions was counter balanced using Latin-Square.A thermal image of the interface was taken 4 seconds after completing the entry (see Figure 1) .We chose 4 seconds as our pilot tests using the Flir C2 Compact thermal camera showed that the heat traces fade away significantly after 4 seconds.We discarded the data of P19 because her hand temperature was too low (≈ 25 • C) that few heat traces were visible due to cold fingertips.

Choice of Passwords
To ensure ecological validity, our choice of passwords to be entered by participants is inspired by common passwords as identified in prior work.For drawmetric passwords, half of the passwords in the list were letter-shaped gestures (e.g., gestures that look like T, S, and Z), while the other half were shapes such as squares, circles and stars.This was motivated by Yulong et al.'s [32] finding that users tend to use letters and shapes as

Phase 2: Analyzing Thermal Attacks
To simulate thermal attacks against the collected images, we invited participants to visually inspect and infer the passwords.We recruited 18 new participants (8 females) aged between 18 and 54 (Mean=26; SD=11) through mailing lists to take the role of attackers.We considered two dependent variables to evaluate attacks: DV1 Correct Guess: an attack is considered successful if the whole guess is entry correctly.For gestures, a correct guess means successfully uncovering the shape and the direction of the input.For taps, it means successfully uncovering the positions and the order of input.
DV2 Partially Correct Guess: this refers to uncovering the shape but not the direction in case of touch gestures, or the positions of input and not their order in case of touch taps.

Procedure
We first explained how the authentication schemes work and how to provide input.We then showed the participants a sample thermal image for each condition to explain how thermal attacks take place.We also explained that heat traces fade over time and this could be used to determine the order and direction of entry.After filling a consent form and a demographics questionnaire, participants were then provided with the thermal images one after another, and a pen and paper to write down their guesses.In total, each participant performed 24 attacks (6 attacks × 2 input types × 2 input interfaces).Participants made up to three guesses per attack; only the best of the three was considered for analysis.For each guess, we logged the guessed password, and the guessed direction/order of input.Participants were not told whether their guesses are correct until the end of the study to avoid any potential biases.The order of conditions in which we presented the thermal images was counter balanced using Latin square.To encourage participants, we created a scoring mechanism and a scoreboard.
Participants received two points for each Correct Guess (DV1), and one point for each Partially Correct Guess (DV2).Scores were also based on the best of the three guesses.

Limitations
Participants with high hand temperatures left more visible heat traces (e.g., P4's hand temperature was 45 • C, while P19's was 25 • C).This is mitigated by following a within-subjects experiment design, which controls individual differences [16].Nevertheless, we expect relative results to remain generalizable.For example, we expect gestures to remain more vulnerable compared to taps, and a higher accuracy of attacks against touchscreens compared to touchpads.

Effect on Correct Guesses
A two-way repeated measures ANOVA revealed significant main effects for input type  1,17 = 42.43, < 0.001, but not for input interface ( > 0.05) on Correct Guesses.We found a significant two-way interaction effect between input type and input interface  1,17 = 11.642, < 0.005.This means that Correct Guesses depend on a) input type and b) combination of input type and input interface.Thus, we carried out additional one-way ANOVA tests.

Effect on Partially Correct Guesses
A two-way repeated measures ANOVA revealed significant main effects for input type  1,17 = 44.677, < 0.001, and input interface  1,17 = 154.082, < 0.001 on part.corr.guesses.We found a significant two-way interaction effect between input type and input interface  1,17 = 219.68, < 0.001.This means that part.corr.guesses depend on a) input type, b) input interface, and c) the combination of both.To distinguish the impact of input type from that of input interface, we carried out additional one-way ANOVAs.

Summary of the Results
The results (summarized in Figure 2) indicate that in terms of security against thermal attacks, a) tapping is significantly more secure than touch gestures, b) touch gestures are significantly more secure when entered on touchscreens than on touchpads, and c) tapping is significantly more secure on touchpads than on touchscreens.

Discussion and Future Work
Results show the possibility to use a low-cost thermal camera to conduct thermal attacks by visually inspecting the thermal images.
Lesson 1: Touch input is Highly Vulnerable to Thermal Attacks, but Taps are Relatively More Secure than Gestures.
The results indicate that both tapping and touch gestures are highly vulnerable to thermal attacks.We recommend using taps rather than gestures as the latter are more vulnerable.This is inline with previous work in which Android Patterns, which require touch gestures, are inferred using automatic approaches and high end cameras.Although our evaluation of taps was performed on a graphical locimetric password scheme, these outcomes are also relevant for passwords that require tapping, such as PINs.
Our results compare types of touch input and not types of graphical authentication schemes.While we followed the most common implementations of drawmetric and locimetric schemes [3,17,26], this does not generalize to all graphical passwords.The security of graphical passwords can be improved by, for example, using contactless input for, such as eye gaze or mid-air gestures.Indeed, one direction to address thermal attacks is to employ schemes that use modalities that do not leave heat traces [9,10,14,19].An alternative could be to use cue-based authentication where the user's input depends on system cues [7,20,25,27].While cue-based authentication leaves heat traces, the dependency on cues requires adversaries to learn which cues the user responded to when providing input, thereby complicating attacks.Another direction is to employ biometric schemes that are usually more usable, such as keystroke dynamics [11,15], and facial or fingerprint recognition.Future work should study how resilient biometrics are against thermal attacks, and improve their usability to maximize adoption.
Lesson 2: Touchpads are more Secure against Thermal Attacks.While successful attacks against touchscreens (43.06%) are as high as touchpads (41.2%), guesses are significantly more accurate on touchscreens (87.04% partially correct guesses) than touchpads (56.02% partially correct guesses).This means attacks are less effective when using touchpads of laptops.This is due to the nature of interaction on touchpads compared to touchscreens; to authenticate using a touchscreen, the user's finger touches the screen at the first input position, while on touchpads the user needs to navigate their mouse pointer to reach the first input position.The interactions that move the mouse pointer create additional heat traces that blend into those resulting from authentication.Therefore, the threat is relatively lower on touchpads.
Thermal Attacks are Becoming Ubiquitous and can be Performed by Anyone Overall, the results indicate that both taps and gestures are highly vulnerable to thermal attacks.Previous work employed image processing to analyze thermal images and infer entered passwords using high end thermal cameras that cost more than $5,900 [1,23].Our work shows that attackers can achieve an alarming success rate by visually inspecting thermal images taken by an off the shelf thermal camera that costs less than $450.These results underline how critical and timely it is to address thermal attacks.Thermal cameras will continue to become cheaper and accessible to a many potential adversaries who could use them maliciously without any technical expertise.

Conclusion
We evaluated the effectiveness of thermal attacks by non-expert attackers using an off the shelf thermal camera.We collected a dataset of thermal images of a smartphone's touchscreen and a laptop's touchpad after participants entered graphical passwords using touch taps and gestures.In a second study, 18 participants visually inspected the thermal images to infer the passwords.They recovered 60.65% of touch gestures and 23.61% of touch taps.Attacks against touchscreens and touchpads are almost equally successful, but are more accurate on touchscreens.These results highlight that thermal attacks are likely to become ubiquitous, especially with the affordability of thermal cameras and the feasibility of attacks without any image processing as done in previous work [1,23].We discussed how the user's behavior and the physical properties of the two studied interfaces impact the success of thermal attacks, and future work directions to counter the ubiquity of thermal attacks.

Figure 2 :
Figure 2: Attack success rates for the different input schemes and input interfaces.Taps are significantly more secure against thermal attacks compared to touch gestures.Tapping on a laptop's touchpad is significantly more secure than tapping on a smartphone's touchscreen, and touch gestures are significantly more secure on touchscreens than on touchpads.free-formrecall graphical passwords.Our choice of locimetric passwords was motivated by common distributions of password points according to prior field studies on cued-recall passwords[3].4.1.3ApparatusWe used a Flir C2 Compact [? ] Camera (80 px × 60 px), which is a low-cost off the shelf thermal camera (<$450).The camera was mounted on a 25 cm high tripod placed 30 cm away from the interface.Passwords were entered on a Lenovo Tango Phab 2 Pro smartphone with a gorilla glass screen (1440 px × 2560 px) pixels, and a Lenovo z50 Laptop (1920 px × 1080 px).