How did LastPass master passwords get compromised?

A number of LastPass users recently received an email like the following, indicating that someone else attempted to log into their account:

Email with the LastPass header: Login attempt blocked. Hello, Someone just used your master password to try to log in to your account from a device or location we didn't recognize. LastPass blocked this attempt, but you should take a closer look. Was this you?

The mail is legitimate and has been sent out by the LastPass service. The location however was typically very far away from the user’s actual location, e.g. in a country like Brazil or India. Yet this isn’t merely an attempt to guess the password, as LastPass will only send a mail like this one if the correct master password is provided in the login attempt.

One affected user created a thread on Hacker News and at least a dozen others chimed in with similar experiences. This indicates that a large-scale attack is underway, with the total number of affected users being quite significant.

As online password managers go, a user’s master password is the most critical piece of information. So the important question is: how do the attackers know the master passwords? There are some explanation being discussed: credential stuffing, phishing, malware, LastPass compromise. As I know a thing or two about LastPass, I’ll write down how likely these are and why.

TL;DR: It appears that LastPass infrastructure has been compromised, all other explanations being rather unlikely. And, surprisingly, it isn’t given that the attackers actually know these master passwords.

Credential stuffing

The Hacker News thread prompted AppleInsider to write about this issue. The article quotes a LastPass spokesperson with the following statement:

LastPass investigated recent reports of blocked login attempts and we believe the activity is related to attempted ‘credential stuffing’ activity, in which a malicious or bad actor attempts to access user accounts (in this case, LastPass) using email addresses and passwords obtained from third-party breaches related to other unaffiliated services

Credential stuffing attacks are quite common and rely on password reuse: a password leaked by one service happens to be used for another service as well. But is password reuse likely when talking about a password manager whose entire goal is eliminating password reuse? With LastPass having a history of downplaying security issues, we should take their statement with a grain of salt.

Several affected Hacker News users claim (and I believe them) that their master password has not been reused anywhere. Worse yet, several users reported changing their master password in response to the notification, yet getting notified about another login attempt with the new master password shortly afterwards.

A tweet by Valcrist @Valcristerra: Someone tried my @LastPass master password earlier yesterday and then someone just tried it again a few hours ago after I changed it. What the hell is going on?

If true (and I have little reason to doubt this statement), this completely rules out credential stuffing as the attack vector here.

Update (2021-12-30): LastPass published an expanded statement that I was unaware of at the time of writing. It also claims that credential stuffing is the source of the issue but has an important addition (emphasis mine):

Our investigation has since found that some of these security alerts, which were sent to a limited subset of LastPass users, were likely triggered in error.

This seems to indicate that the email messages were mistakenly triggered by login attempts with incorrect password. This would be good news and make the rest of this article obsolete. But note how they say “likely” here. If they found and fixed a bug in the email notification mechanism, why don’t they just state so? Unfortunately, this kind of statement sounds like they still don’t have a clue about what’s going on but want to calm the crowd nevertheless.

Phishing

Phishing, mentioned as another possible explanation, is quite unlikely for the same reason. The suspicion here is that the users have been lured to a fake LastPass login page and entered their master password there. While this is also a common attack, it cannot explain repeated login attempts after a master password change.

Also, several affected LastPass accounts haven’t been in use for at least a year. This means that the phishing attack must have happened at least a year ago in this case. Yet anybody collecting passwords via phishing would have attempted to use them as soon as possible. The longer they wait, the more likely the user will recognize their mistake and change the password.

Malware

As it seems that the attackers react to changes, some people concluded that they are indeed being watched. The options are a malicious (or severely vulnerable) browser extension or a standalone application compromising the entire system. This explains at least how the attackers would get the new master password after a change.

This theory is also unlikely however, for multiple reasons. First of all, malware provides a level of access that makes hacking LastPass accounts unnecessary. If it can intercept or extract the LastPass master password, it can do the same for all other passwords as well. Logging into LastPass accounts is simply an additional step, one that unnecessarily exposes the entire campaign.

It’s also notable that the attacks here are thwarted by rudimentary protection based on geographical location. Why would attackers who control user’s browser or their entire system run into this trap? They could log into LastPass from the user’s own browser instead. Better yet: they could hijack the existing LastPass session. Last time I checked these took two whole weeks to expire.

Finally, there is the question of timing. Malware can only steal a password when that password is used. Yet some affected accounts were abandoned, last used more than a year ago. Did the attackers collect credentials and thoroughly prepare over a lengthy period of time? It doesn’t look like that. These attacks aren’t sophisticated, or they wouldn’t attempt to log into accounts of US users from Brazil. With real preparation they wouldn’t repeatedly run against this geographical protection.

LastPass compromise

As far as I can tell, this only leaves the option that LastPass itself has been compromised. But this option also raises questions: not even LastPass servers are supposed to know users’ master passwords. Master password is only ever transmitted as a hash, one that is difficult to reverse.

My initial suspicion was that someone abused the vulnerability I described in this post from 2018. The issue allowing websiteBackgroundScript.php to be loaded by any website was obvious enough that someone could have discovered it independently from me. And cracking the master password when hashed with merely 5,000 PBKDF2 iterations (LastPass default before 2018) is totally possible. However, this option is ruled out by the fact that users also reported brand new accounts being affected, as well as recently changed master passwords.

The new default (100,000 iterations) makes recovering master passwords from hashes require considerably more resources, too much for unsophisticated attackers to pull off such a large-scale attack even in case of another similar data leak. But that’s assuming that LastPass servers are in fact unaware of your master password.

LastPass users will often enter their master password into web pages hosted on lastpass.com, even when using the browser extension. This is a design flaw, as there is no real way of verifying that the password is never sent out unhashed. It is for example possible that some logging functionality inadvertently sent the plaintext password to a server. And with log4j vulnerabilities being actively exploited right now, someone might have compromised the very server receiving this data. Update (2021-12-31): LastPass reached out to me to state (among other things) that they aren’t using log4j anywhere.

I tried logging into LastPass via the web interface and couldn’t see any unexpected data being transmitted. This doesn’t prove anything of course, as some other form might be the culprit here. Or this functionality might only be enabled for US users for example. But there is also another explanation, one that doesn’t require LastPass to know the unhashed master password.

Pass the hash

It is in fact not given that the attackers know the master password. That’s because LastPass authentication (unlike proper PAKE protocols) is vulnerable to pass the hash attacks. The same master password hash is used each time to log in, so the attackers might have used that hash rather than the master password itself. And lifting that hash from some LastPass server should be much easier than getting your hands on the unhashed master password.

So one can log in using only the master password hash, and what then? Once logged in, attackers can download https://lastpass.com/getaccts.php which holds all of account’s data. This gives them the addresses of websites but the corresponding user names and passwords cannot be decrypted without knowing the master password.

It doesn’t mean that this attack is worthless however. Once the attackers have the data, they can spend all eternity bruteforcing the master password locally. They can even prioritize the accounts holding passwords to more valuable websites. So this might very well be what happened here.

Comments

  • zakius

    so in systems where pass the hash is possible database basically contains a plaintext password being a jumbled mess of bytes seeded by the user intended password, did I get that right?

    Wladimir Palant

    Yes, that sounds about right.

  • mike epson

    If LastPass is setup with Google authenticator would this prevent any unauthorized login attempts?

    Wladimir Palant

    I’ve never had a proper look at their MFA implementation. If implemented correctly – sure, it would prevent such attacks and is very much recommendable for this very reason.

  • Michael

    Since the password should never reach the server, is it possible that the captured hash could be used to decrypt the database? Or is a separate hash and/or a different browser-side salt used for that purpose? Even if the latter is true, the page receiving the hash for unlocking the password file could theoretically have been compromised as well.

    I guess we'll know for sure if we ever get reports of affected users having Lastpass-generated account passwords compromised.

    Wladimir Palant

    Yes, the captured hash can be used to bruteforce the master password. This takes considerable time (that’s the purpose of PBKDF2 used here) but as I said: the attackers could decide which accounts are worth the effort.

  • I told you so

    Use LastPass instead of local KeePass (or XYZ). It's safe. Trust me. Give me your money. We know what we do.

    Wladimir Palant

    Well, local password managers have their own share of issues, particularly when browser integration is involved. And the attack surface of online password managers can be minimized. It isn’t black and white. It’s rather that LastPass has numerous design issues and little interest in addressing them.

  • Michael E

    I've read that LastPass doesn't hold the unhashed master password. But I've also read that they offer ways to recover your account if you forget the password, unlike bitwarden. To me that indicates that they do in fact have unhashed master passwords.. Or what am I missing?

    Wladimir Palant

    They have a recovery mechanism called One Time Passwords (OTP). These OTPs hold the encryption key necessary to decrypt your data without a master password. Cryptographically this approach is sane, there are some problematic implementation details however. You can read about it under https://palant.info/2018/07/09/is-your-lastpass-data-really-safe-in-the-encrypted-online-vault/, the section titled “A few words on backdoors.”

  • Michael

    Might be nice to update your article after LastPass confirmed the emails were sent erroneously by their system and their was no hack. Especially your tl;dr piece and title are now more clickbate than actually true.

    Wladimir Palant

    Yes, I want to update the article. They didn’t actually state that the mails were sent out erroneously however, they said “likely.” In this context that’s synonymous with “we don’t really know but we’ll claim it anyway.”

  • John

    Thanks for this insightful post.

    Just to confirm... assuming someone has MFA in place and has recently updated their master password, they should not be vulnerable to previously captured hashes, and due to MFA, it is unlikely some could have previously captured getaccts.php. Is that all correct?

    Wladimir Palant

    Yes, that’s correct.

  • Mahesh

    Thank you for a well written article that is intelligible even to a non technical person like me. I have used LastPass for a long time, and therefore, am very interested in keeping up with any security issues related to LastPass. I use Google authenticator for MFA. I sure hope that is enough to keep my account safe from hacking. I don't know what else to do, or whether any other password manager is any less vulnerable than LastPass

  • Niki

    There is, of course a very simple reason too. I actually did this to myself, and had a panicky few minutes before I figured out it was me. I went shopping on a global Amazon EU site to see what was available to Europeans and not to Northern Ireland even though still in the EU customs union (the answer - a LOT!). I then tried to use LastPass for my Amazon password, no go. It was several hours later that I got the email, by which time I'd initially forgotten I'd even tried to log on whilst VPNed into an EU country

    Wladimir Palant

    Someone already brought up VPN as a possibility on Hacker News. However: no, the IP addresses mentioned don’t belong to a VPN. And several accounts affected haven’t been in use at all: no legitimate logins whatsoever, over VPN or otherwise.

  • Mosh

    Were there actually any breached credentials?

    Wladimir Palant

    Who knows? I don’t know any more than what’s said in the article.

  • Donald

    What do you recommend for managing passwords then?

    Wladimir Palant

    I don’t currently have good recommendations. Most online password managers have a similarly bad security story while offline password managers typically have usability issues prompting users to install insecure third-party additions. I used to recommend 1Password but another security researchers with considerable experience in the area isn’t really fond of them.

  • zor

    I spoke with the proxy IP provider that I found in the email from lastpass after doing a whois from the IP. I explained them the situation before lastpass even made their first statement. They answered that they blocked the user who tried to access my account. I am not sure if this is true or they just responded like that to make me believe they took action. But if this is true then we might have a confirmed attack?

    Wladimir Palant

    I think there is little doubt whether an attack is going on. Then again, credential stuffing attacks aren’t unusual and they are happening all the time. The important question at this point is: do the attackers have the correct master passwords?

  • Oran Kelley

    When I look at all the "information" you have here, NONE of the scenarios you've outlined explains it all. Nothing explains everything that users have reported. So, I really think your tl;dr is unjustified.

    If the purpose here is to gather hashes and bruteforce the master password, why the repeated "successful" login attempts?

    Why are new accounts, which presumably have nothing valuable in them also "successfully" logged into? If this is a real hack, isn't the new account phenomenon a pretty strong indication that this isn't just a LastPass thing, but is also related to something these individuals have done?

    OR: How's this for a scenario that both seems plausible and would explain what's been reported: There is indeed something wrong with the email alert system at LastPass. The "likely" that LastPass added to their announcement should be interpreted as "we aren't yet sure this bug explains everything that's been observed," rather than "we are just making this up because we have no idea what is going on."

    And that you just insist on preferring the second interpretation for your own selfish reasons?

  • Russell

    Why are you assuming that the hash just contains the password? It probably is combined with the users Mac address or another identification from the device to make the hash, as an example. Then lastpass can prompt that you have never logged from that device and make you authenticate the log in. Then they store the location and Mac address etc. The hacker could not get ahold of passwords from lastpass and create a hash or use an existing hash. They will not have enough information. This article is just instilling fear into people. Don't simplify the process, it's more complicated and you know it.

    Wladimir Palant

    I’m not assuming anything. The hash is calculated on the client side and I’ve looked into the approach a while ago. I’ve reported a number of LastPass security vulnerabilities, you know?

    The hash sent is a PBKDF-HMAC-SHA1 hash of your master password, calculated with 100,000 iterations by default (old default was 5,000, changed after a vulnerability report by me), using your email address as the salt. That’s it. Nothing else goes into it. You are the one assuming here. The client-side code doesn’t even have access to your MAC address, so it couldn’t use it even if it wanted.

  • Jason J

    I got the same email headed "Login attempt activity blocked...". I don't remember having a login failure and I haven't been blocked, and I always use a VPN to access it on my phone (only one attempt from a PC of which they talked about switching and I declined). There is no info on where the attempt came from, but LastPass did recommend to change my master password if it wasn't me. My master password is 21 characters so surely brute force from my end is not possible, most likely something was got from LastPass IF there was a breach. Good piece I must add, but we still don't know unfortunately.

  • linus

    First of all, I am not a nativ english speaking person nor particular tech-savvy so I might have misunderstood this. When I use my browser to log into my lastpass account and then open lastpass.com/getaccts.php I get a set of data with (I assume) all my account data. Its encryptet. All values like username etc are just long strings of lower-case letters and numbers. But contrary as statet in the above blog post also the URL is encrypted like this. Just a few minutes ago I changed my masterpassword. So I don't know if the url was plain text when it was encrypted with my old masterpassword. Could it be possible that the assumption in the blog post is correct and in response lastpass also started to encrypt the URL when choosing a masterpassword?

    Wladimir Palant

    The URLs are not encrypted, that’s merely hex encoding.

  • HackedUser

    I wish I saw this earlier.

    I logged into my cryptocurrency wallet on Thursday and over $100,000 of coins had been transferred out on December 26th. I thought it was the wallet app, but after finding this I know it was Lastpass.

    I looked on the account activity and there was a login from United Kingdom that was not mine that was still logged in!!! I closed it all down and changed my master pass, but I can see on the Lastpass account history, they logged into every single exchange where I have an account. They had the time to go through all the secure notes and find the coded saved screenshot of the 12 words and then found the password and recreated the exodus wallet and accessed like it was theirs.

    I've been going over everything on my end and finding no security threats on my local machines. I always use a VPN and I even used coded titles for my secure notes....they've had access to my account long enough to pour over every single thing--all of my identity docs, social security, bank accounts, birth certificates.

    I've filed a report with the FBI but of course, I can't get anyone from lastpass to talk to me about anything.