``I Added `!' at the End to Make It Secure'': Observing ...

"I Added `!' at the End to Make It Secure": Observing Password Creation in the Lab

Blase Ur, Fumiko Noma, Jonathan Bees, Sean M. Segreti, Richard Shay, Lujo Bauer, Nicolas Christin, Lorrie Faith Cranor

Carnegie Mellon University

{bur, fnoma, jbees, ssegreti, rshay, lbauer, nicolasc, lorrie}@cmu.edu

ABSTRACT

Users often make passwords that are easy for attackers to guess. Prior studies have documented features that lead to easily guessed passwords, but have not probed why users craft weak passwords. To understand the genesis of common password patterns and uncover average users' misconceptions about password strength, we conducted a qualitative interview study. In our lab, 49 participants each created passwords for fictitious banking, email, and news website accounts while thinking aloud. We then interviewed them about their general strategies and inspirations. Most participants had a well-defined process for creating passwords. In some cases, participants consciously made weak passwords. In other cases, however, weak passwords resulted from misconceptions, such as the belief that adding "!" to the end of a password instantly makes it secure or that words that are difficult to spell are more secure than easy-tospell words. Participants commonly anticipated only very targeted attacks, believing that using a birthday or name is secure if those data are not on Facebook. In contrast, some participants made secure passwords using unpredictable phrases or non-standard capitalization. Based on our data, we identify aspects of password creation ripe for improved guidance or automated intervention.

1. INTRODUCTION

Despite decades of research investigating passwords, many users still make passwords that are easy for attackers to guess [9, 22, 35, 62]. Predictable passwords continue to cause problems, as evidenced by the recent release of celebrities' private photos obtained in part through a password-guessing attack on Apple's iCloud [11, 37]. While most everyone would prefer a world without the burden of remembering a portfolio of passwords [18, 53], passwords are familiar, easy to implement, and do not require that users carry anything. As a result, passwords are unlikely to disappear entirely in the near future [7]. Although expecting users to remember complex and distinct passwords for dozens of accounts is absurd, singlesign-on systems, software password managers, and biometrics [4] promise to reduce this burden. Passwords also remain useful for frequently accessed accounts, as master passwords for password managers, and as an integral part of two-factor authentication.

Copyright is held by the author/owner. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee. Symposium on Usable Privacy and Security (SOUPS) 2015, July 22?24, 2015, Ottawa, Canada.

Researchers have identified common, predictable choices that result in easy-to-guess passwords [9, 22, 35, 62]. While some users may be making informed cost-benefit analyses and creating weak passwords for low-value accounts, other users may have misconceptions about what makes a good password. Existing security advice [10, 28, 36, 49, 73] and real-time feedback [1, 12, 15, 32, 59, 63] may be insufficient in disabusing users of these misconceptions.

To understand where users fall short in their attempts to create passwords, we conducted the first qualitative laboratory study of the process of password creation. Whereas analyses of large sets of passwords can reveal common patterns, a qualitative study is better suited to discern precisely why these patterns appear because researchers can probe the rationale behind behaviors through context-based follow-up questions. Prior lab studies of passwords have focused on password management [2, 20, 27, 29, 52, 55], how users cope with password-composition requirements [45,66], novel password systems [19], and the external validity of password studies [16]. In this paper, we report on the first lab study focusing exclusively on how users craft and compose passwords step-by-step.

We conducted in-person lab sessions with 49 participants, each of whom created passwords for a banking website, news website, and email account in a think-aloud, role-playing scenario. We also explored participants' general strategies and inspirations. This enabled us to pinpoint participants' misconceptions and identify strategies that seem both usable and secure against large-scale guessing attacks, such as an offline attack [6, 31, 70].

We found that most participants had a well-defined process for creating passwords. Commonly, participants either had a base word or a systematic human "algorithm" for generating passwords based on the site. While many strategies led to predictable passwords, some participants successfully mixed unrelated words or crafted unique phrases to create more secure passwords. Some participants desired passwords of different security levels across the three websites, yet nearly half did not, indicating that some people may routinely waste effort creating and remembering strong passwords for low-value accounts. Participants struggled to create passwords that matched their desired security levels, sometimes creating strong passwords that they inteded to be weak, and vice versa.

Participants were concerned primarily with targeted attacks on their passwords, rather than large-scale, automated attacks. As a result, some participants believed the (common) name of their pets or birthdays would be strong passwords because they had not posted that information on their Facebook page, not accounting for the types of automated guessing attacks often seen in the wild when sites like LinkedIn [9], eHarmony [57], Gawker [5], or Adobe [43] had their password databases compromised.

We identified numerous other security misconceptions. Most participants knew that dictionary words make bad passwords, yet

1

others incorrectly expected common keyboard patterns (e.g., "qwerty" or "1qaz2wsx") to be a secure replacement. Some participants had learned that phrases make secure passwords, yet chose obvious phrases (e.g., "iloveSiteName"). Commonly, participants believed that adding a digit or symbol to the end of a password would make it secure, whereas such an action is very predictable. Other participants conflated difficulty for users with difficulty for attackers, such as thinking that words that are hard to spell are secure.

In contrast, some participants employed strategies that resulted in strong passwords. These strategies included combining unrelated words or developing unique phrases. Whereas many participants insecurely capitalized the first letter of their password in deference to the rules of grammar, others employed non-standard capitalization to make far stronger passwords. Whereas some participants ill-advisedly used the website name as a core part of their password, others used songs and concepts they associate with the site. These related concepts would be far less obvious to attackers.

Many misconceptions we identified might derive from misinterpretations of well-meaning security advice. For example, some participants seem to have misconstrued the idea that "a strong password should contain letters, digits, and symbols" as the false statement "any password that contains letters, digits, and symbols is secure." Similarly, the admonition to avoid dictionary words in passwords does not mention birthdays or keyboard patterns, which some participants incorrectly believed to be secure. Building on our results, we discuss aspects of abstract password guidance and data-driven tools that could help users create better passwords by avoiding the misconceptions we observed in this study.

We next discuss related work in Section 2. Then, we present our methodology in Section 3. We present our findings in Section 4, discuss their implications in Section 5, and conclude in Section 6.

2. RELATED WORK

Password-based authentication remains ubiquitous for online accounts [7]. Even if passwords are replaced with devices that do not rely on human memory [41, 53], the deployment of such systems and subsequent decline of passwords would be gradual. Even recent multi-step authentication systems, such as two-factor authentication systems from Google [26] and Microsoft [40], tend to retain passwords as one part of the approach.

The literature on passwords is vast; below, we briefly discuss the most relevant prior work. However, prior studies of password characteristics focus post-facto on passwords that have already been created, in contrast to our qualitative focus on passwords in the process of being created. Prior studies with a similar qualitative approach have generally examined complementary topics, such as password management and novel password systems.

2.1 Analyses of Password Characteristics

Many password databases have been leaked in recent years [5, 9, 43, 57]. Both the popular press and academics have mined these password corpora to identify common passwords characteristics. For example, popular media reported on the leaked set of RockYou passwords, noting the most common password was "123456" [62]. Researchers found that RockYou passwords commonly included digit sequences, names, and phrases about love [69].

Researchers have also focused on the semantic content of passwords [60, 64]. Historically, researchers have found that some of the most prevalent semantic themes in passwords include names and locations [39], as well as dates and years [65]. Researchers have also noted love, animals, and money as common semantic themes [64]. While two-word Amazon payphrases are not as predictable as general English text, common themes include music,

television, and sports [8]. Combining multiple words and substituting characters are also common strategies [30].

Other studies have entailed collecting passwords created under controlled conditions in online studies. For example, our group has used this technique to study password-composition policies [31,38, 51] and password-strength meters [59]. While controlled experiments can be used to collect some behavioral metrics, our qualitative methods allow us to collect far more explanatory data.

We also aim to understand password characteristics. However, qualitatively observing password creation as it happens, rather than after the fact, lets us not just learn what users do, but also why.

2.2 Laboratory Studies

Other laboratory studies have focused on complementary aspects of the password ecosystem. These aspects have included passwordmanagement practices [2, 20, 27, 29, 52, 55] and how users respond to password-creation requirements [45, 66].

Researchers have studied how users recall multiple passwords. Their participants learned six passwords each, including text passwords and graphical passwords. Participants were asked to authenticate two weeks later [13]. Researchers have also explored automatically increasing password strength. Participants created passwords in the lab, and the system added random characters, which participants could shuffle until arriving at a configuration they liked. The authors found that inserting two random characters increased security, yet adding more characters hurt usability [19].

More recently, researchers interviewed 27 participants about their strategies for password management and usage. Participants had an average of 27 accounts and five passwords. They often made tradeoffs between following password advice and expending too much effort [55]. While our methods resemble those of prior lab studies, we are the first to focus on how users create passwords.

3. METHODOLOGY

To uncover precisely how average users construct passwords, we conducted face-to-face interviews in our lab. Participants created passwords for three different types of accounts we hypothesized would elicit different security levels. Each participant created all three passwords under a single password-composition policy that we randomly assigned from three possibilities. Participants engaged in a think-aloud process while creating each password and answered follow-up questions about their processes, decisions, and general habits related to password creation. The study was approved by the Carnegie Mellon University IRB.

3.1 Recruitment and Logistics

We recruited participants for a study on passwords through ads on our local Craigslist and flyers at public places in and around Carnegie Mellon University's Pittsburgh campus. Each session was designed to last between 45 minutes and one hour. We compensated participants $25 for the session. The study took place in a room in our laboratory with either one or two moderators. Participants used a laptop from our lab for the study. We audio-recorded the interviews and subsequently transcribed them.

3.2 Study Protocol

We began the study with demographics questions. We then asked participants to create passwords for three websites while thinking aloud. Next, we asked participants about their general passwordcreation approach and strategies. Finally, we had participants recall each of their three passwords. The text below provides more detail about each step, and the appendix contains the full interview script.

2

Figure 1: The design of the news (top), banking (middle), and email (bottom) sites for which participants made passwords.

Our demographic questions included age, gender, and occupation. We also asked about familiarity with different computer devices and Internet usage in order to understand the context in which participants created and recalled passwords. In order to introduce participants to the technique of thinking aloud, we next had them perform a warm-up activity in which they thought aloud while crafting a slogan for a bumper sticker.

We then asked participants to create passwords on three different websites, which we assumed would be of different value to participants. These were mock-up websites that we created for the purpose of this study. The three sites, presented in randomized order in the study, were a news website ("National Daily Times"), a banking website ("First Trust National Bank"), and an online email website ("SwagMail"). Figure 1 shows each site's visual design.

We hypothesized that participants would view the password for the news website as having minimal value, whereas the banking and email account passwords would be of higher value. That is, participants would find those accounts more important to protect. Because participants each created three passwords, we could examine the passwords' similarity. Previous research documented that users often reuse passwords verbatim or with minor, predictable modifications [14, 17, 20, 72].

We asked participants to role-play and "pretend that [they] are actually creating new passwords to sign up for new services" and act as if they will "need to use those passwords again to log in to the account [they] sign up for." Furthermore, so that we could understand precisely where in the process of password creation participants came up with different ideas, as well as in what order, we had participants think aloud when creating their password.

Each participant created passwords for all three accounts under a single password-composition policy assigned round-robin from the following three possibilities:

? 1class6: passwords must include at least 6 characters;

? 2class8: passwords must include at least 8 characters, among which are at least 2 of the following: a lowercase letter, an uppercase letter, a digit, a symbol;

? 3class12: passwords must include at least 12 characters, among which are at least 3 of the following: a lowercase letter, an uppercase letter, a digit, a symbol.

As participants met each requirement, a checkmark appeared next to the requirement, as shown in Figure 2. Participants needed

Figure 2: As participants created a password, checkmarks indicated which requirements they had completed. The password appeared as asterisks.

to re-enter their password correctly before proceeding. We chose the 1class6 and 2class8 policies to represent minimal and typical password-composition policies, respectively. We chose 3class12 as a policy that has relatively complex requirements, yet prior research studies have found to be reasonably usable [51]. We expect policies that require longer passwords to see increasing adoption in the real world given the vulnerability of passwords containing eight or fewer characters [24, 54]. We chose to have participants create all three passwords under a single composition policy because we were more interested in how a single participant's behavior differed across sites of potentially different value, as opposed to how a participant's behavior changed across password-composition policies.

We then asked participants about their general strategies for creating passwords and whether the strategies they employed in the study resembled their usual behavior. We excluded from further analysis behaviors they said were atypical. We also asked whether and how they make modifications if they reuse a password, and whether an account of theirs had ever been compromised.

The final part of our study tested password recall. First, to distract participants so that they would think about something other than their passwords for a few minutes, we asked participants to count backward from 100 in increments of seven. Then, we asked participants to log on using each of their three study passwords. We gave each participant up to five attempts to do so, simulating the rate-limiting that many websites use to prevent online attacks.

3.3 Analysis of Password Security

To inform our qualitative analyses of password-creation behaviors, we needed an objective metric of password security. We therefore measured each password's guessability, or how quickly an attacker would guess that password in a large-scale guessing attack [6, 31, 70], using the software tool Hashcat [54]. This tool is widely used by attackers [22, 23, 24, 34, 44] and, relative to other guessing approaches, is generally successful at guessing a large fraction of target password sets in the configuration we used [61]. We made 100 trillion (1014) guesses against participants' passwords, which represents about 6 hours of guessing on a single modern GPU (AMD R9 290x) for passwords stored unsalted using the NTLM hash function, 3 weeks for passwords stored using SHA256, and 904 years for passwords stored using SHA512crypt.

Hashcat takes as input a word list and a set of mangling rules, or transformations (e.g., "add a 1 at the end" or "change every A to @") to apply to word list entries. It is impossible to model every attacker or to study all word lists. We thus chose settings and training data that prior work found to represent a reasonable step

3

beyond Hashcat's default configuration [61]. Our word list comprised large sets of leaked passwords and natural-language dictionaries. The passwords were taken from breaches of MySpace [48], RockYou [62], and Yahoo! [21]. We used dictionaries found effective in past studies [31, 70]: all single words in the Google Web corpus [25]; the UNIX dictionary; and a 250,000 word inflection dictionary [50]. The combined set of passwords and dictionaries contained 19.4 million unique entries, ordered by descending frequency. The mangling rules comprised the "generated2" set included with oclHashcat and a Hashcat translation of rules originally released by Trustwave SpiderLabs [58] for the tool John the Ripper.

Although this approach simulates a large-scale guessing attack, it does not simulate an attacker who knows personal details of the user. Therefore, members of our research team manually examined the study passwords alongside participants' think-aloud transcriptions. If the password was derived primarily from a date significant to the participant or the name of a participant's family member or pet, we marked the password as vulnerable to a targeted attack. Similarly, if the password was mostly derived from the name of the website on which the participant was making a password, we marked it as vulnerable to an attack targeted to that site. Automated cracking methods do not natively support these sorts of highly targeted attacks, necessitating this limited manual analysis.

3.4 Qualitative Analysis

Because our objective was to gain a nuanced perspective on how users craft passwords, we relied heavily on qualitative methods. Rather than approach the study with well-defined hypotheses or very targeted research questions, we instead chose to let participants' strategies and misconceptions emerge from the data.

To that end, one member of the research team first tagged each self-contained thought, representing a distinct password-creation strategy or behavior, mentioned by any participant either during the think-aloud portion of password creation or in response to an interview question. For example, one of the tagged thoughts was, "swap the g for a $ because gold is something related to money." We identified 546 thoughts across our 49 participants.

The members of the research team then collaboratively analyzed these thoughts in a process derived from affinity diagramming [3]. The members of the research team began with each of the 546 thoughts, as well as the corresponding password, printed out on an individual piece of paper. We then iteratively grouped these thoughts into distinct clusters, continuously refining, collapsing, and separating clusters. These clusters represented thoughts the team felt related closely to each other. At the end of our full-group session, we had grouped these 546 thoughts into 18 initial clusters, with themes such as using the website itself as inspiration for a password or adding random characters to a password.

While these clusters represented closely related behaviors, they conflated secure and insecure actions. To separate successful strategies from security misconceptions, two members of the research team went back through all quotes in each cluster and discussed whether that particular behavior would be beneficial for security, negatively impact security, or whether the security impact was uncertain. As a result, we split some clusters to distinguish between secure variants of a strategy and those that were likely predictable by attackers, transforming the initial 18 clusters into 25 clusters.

Finally, within each of the 25 clusters, we performed an additional round of affinity diagramming to further disambiguate distinct behaviors from each other. For instance, within the broad category of "use words inspired by the website," we created distinct sub-clusters of "passwords derived from the website name," "words a participant associates with the site," "phrases a participant asso-

ciates with the site," "songs the participant associates with the site," "people the participant associates with the site," "emotions the participant associates with the site," and "descriptions of the website's visual design/logos." This process resulted in 122 distinct behaviors that we report within the context of the 25 broad themes.

In addition to our formative analysis of strategies for creating passwords, we had more targeted research questions related to how participants approach creating and managing passwords. We based these questions on a combination of prior work and our own expectations and experiences. These targeted questions covered the security levels participants desired for different websites, the reuse of whole passwords or elements thereof [14,72], the order in which participants would think of different chunks of their password [60], and how participants manage passwords [18, 55]. We tagged each instance of a participant discussing or exhibiting behaviors related to these areas. Using the same group process we used to analyze creation strategies, we again clustered these behaviors.

Throughout the paper, we focus on reporting the theme captured by each cluster of behaviors and providing relevant quotes where illustrative. In a few cases, we report the frequency of different behaviors to provide a better sense of our data; these frequencies are not intended to suggest that any quantitative analyses of our data are appropriate. To protect participants, some of whom might have used their real passwords in the study, we adopt Fahl et al.'s suggestion and report in this paper sanitized passwords that replace potentially personalized information with analogous content [16].

3.5 Limitations

Our study suffers from limitations typical of small-scale, qualitative studies. We used a small sample that is not representative of any larger population. For instance, more participants than average have technical backgrounds. Despite these limitations, qualitative studies offer rich insight into not just what users do, but why. Password characteristics have been very widely studied post-facto, yet the moment-to-moment decisions of password creation had not previously been studied in such depth.

A lab study can only capture a sliver of the many ways in which people use passwords, limiting ecological validity. For example, we had participants create three passwords in succession, whereas password creation for different sites is often spread out over time. Furthermore, we test password recall during the same lab session it was created, albeit following a distraction task. In contrast, users need to recall passwords very frequently for some accounts, yet infrequently for others. Similarly, some users log into accounts using different devices or using password managers, which we do not test. However, only two of the 49 participants reported that they normally use password managers.

Our participants made passwords for a study, not a real account. As a result, they had little incentive to make the passwords hard to guess or easy to remember. To gauge the generalizability of different types of password studies, Fahl et al. compared students' actual university single-sign-on passwords with passwords the same students created for an online or lab study [16]. They found passwords from lab studies to be acceptable proxies for real passwords.

4. RESULTS

Our participants generally wished to create strong passwords, at least for some accounts; they just did not always know how to do so. Even worse, they sometimes wrongly believed their choices were contributing to a strong password even when these choices were actually making the password more predictable. In this section, we discuss the passwords participants actually made, alongside their considerations and micro-decisions along the way.

4

Table 1: The average length and number of character classes in unique passwords participants created.

Policy

# Length (characters)

# Classes

Median Mean 1 2 3 4

1class6 37 2class8 47 3class12 47

10 10.1 3.5 6 12 8 11 9 9.9 2.2 ? 7 9 31 13 14.4 3.5 ? ? 17 30

We begin by describing our 49 participants in Section 4.1. We then briefly summarize the characteristics and guessability of the passwords they created in Section 4.2. Even though many participants made passwords that exceeded the minimum requirements of their assigned password-composition policy, roughly half of the passwords were vulnerable to an automated guesing attack or to a targeted attack. Next, in Section 4.3, we describe participants' desired security level for each of the three sites for which they were creating passwords. Unfortunately, the value participants assigned to accounts diverges from what a security researcher might expect.

The main contributions of this paper rest in the qualitative analyses we detail in the subsequent sections. In Section 4.4, we explore participants' security considerations, as well as their abstract, broad approaches for generating a password. We found that participants try to create passwords to match their perceived value of different accounts. We also found that some participants reused passwords or base elements verbatim across sites. We highlight general approaches and human algorithms participants used to craft a password. Some approaches, such as generating a unique phrase, appear secure and also memorable to participants. Sadly, other participants unwittingly employed very predictable approaches.

Despite their desire to create secure passwords, many participants struggled to distinguish approaches that increase password security from those that make a password easier to guess. In Section 4.5, we delve into participants' low-level strategies and microdecisions. Subtle differences often separated choices that increased security from those that made passwords predictable. For example, basing a password on a song or visual image the participant associates with the website for which he or she is creating a password is far better than using a password like "iloveSiteName!" Many of participants' misconceptions can be viewed as twisted interpretations of advice about how to create a strong password.

4.1 Participants

We interviewed 49 participants, 21 male and 28 female. Their ages ranged from 19 to 63. Young participants were overrepresented relative to the general population as the mean age was 31 and the median 24. Of the 49 participants, 24 were students, 13 of whom studied a technical discipline like engineering. Of the nonstudent participants, 16 were employed in a variety of occupations, while the other 9 were currently unemployed or retired. All participants used text passwords regularly and were frequent Internet users. To preserve anonymity, we refer to each participant as PN .

4.2 Password Characteristics and Security

The 49 participants each created 3 passwords, resulting in a data set of 147 passwords, of which 131 were unique. No participant created the same password as any other participant, but 13 participants reused a password verbatim across two or three of the three accounts. When we report password characteristics and guessability in this subsection, we report on unique passwords, counting a password that a participant reused multiple times only once.

Table 2: The number of passwords created under each policy that were vulnerable to a general attack of 1014 guesses using Hashcat, as well as the number manually identified as vulnerable to a site-specific attack using the website name, or a userspecific attack. We also present the number that appear secure against all three attacks.

Policy

Vulnerable to attacks Site- User-

# General specific specific Secure

1class6 37 21

0

0

16

2class8 47 19

2

3

23

3class12 47 10

8

3

26

The quantitative metrics we report in this subsection are not intended to suggest generalizability, which would be inappropriate for a small-scale, qualitative study. Instead, we present these numbers to give a broad sense of the passwords our participants created.

Participants often significantly exceeded the requirements specified by their assigned password-composition policy, as shown in Table 1. For example, the median length of a 1class6 password was 10 characters, rather than 6, and 84% of 1class6 passwords included multiple character classes despite the lack of any characterclass requirement. Although 2class8 passwords were only required to contain characters from two distinct character classes, 66% of these passwords contained all four character classes.

Across password-composition policies, 38% of the passwords participants created were guessed within 1014 guesses in the automated guessing attack using Hashcat. Table 2 gives an overview of how many passwords created under each composition policy were vulnerable to attack. Sanitized examples of passwords vulnerable to this automated guessing are Tyrone1975 (1class6), Gandalf*8 (2class8), and Triptrip1963 (3class12). In contrast, sanitized examples of passwords that were not guessed include 5cupsoftoys (1class6), AfNaHiLoco (2class8), and 7301Poplarblvd$ (3class12). Using lists of common passwords, six passwords were trivially cracked, including three 1class6 passwords (gabriel, password, and qwerty), two 2class8 passwords (1Qazxsw2 and Password1!), and one 3class12 password (Newspaper123). None of the other passwords were among the most commonly used passwords [9, 35].

Our automated, large-scale Hashcat attack did not specifically focus on site-specific information, such as the name of the site on which an account was being created. We manually evaluated vulnerability to site-specific attacks, considering a password to be vulnerable if the name (e.g., "First Trust Bank" or "1sttrust") or function of the site (e.g., "email" or "breakingnews") was the majority of the password. We marked ten additional passwords (e.g., 1234SwagMail@ and nationaldailytimesP@ss2) as vulnerable.

In addition to general attacks, passwords can also be guessed in attacks targeted to a user's personal information. We manually examined passwords not guessed by Hashcat alongside participants' explanations to determine whether a password would be vulnerable to a user-specific attack. We marked passwords vulnerable if the name of the participant, immediate family member, or pet, or a date or geographic location of well-known significance to the participant, formed the majority of the password. We marked six additional passwords (e.g., structured Firstname.Lastname715 and hOMETOWN!123) as vulnerable.

4.3 Security Level of Each Site

On the assumption that some or all of the participants would create fundamentally different types of passwords based on their de-

5

 ................
................

In order to avoid copyright disputes, this page is only a partial summary.

Google Online Preview   Download