Introduction
Have you ever been trying to solve a systemic problem, like users getting infected by malware, and the only advice you get is completely impractical, such as to instruct users to not click on links or open attachments? This seems to be one of the top security recommendations lately, as if the solution was so simple. The good news is that there are some practical solutions out there for nearly every organization. The key is to figure out your core goals, determine what solutions supposedly work in your environment, and then test them to make sure they actually do what they say on the tin. It’s going to take more than a single blog post to explain everything, but we wanted to start by going over a concrete example and then future posts can generalize.
The problem
Our example will be to cover the damage done by people getting phished and downloading malware (either from an attachment, or by clicking a link and downloading something). You may have noticed that we said “the damage done by” this attack, which is slightly different than attempting to thwart the attack itself. There’s a subtle difference between the two that we’ll discuss later.
To be clear, this falls under the “protect” phase of the NIST Cybersecurity Framework, which means we’re skipping the step of “Identify” and just taking it as a given that the person being phished has something (cryptographic keys, session tokens, etc.) which provides access to information we want to protect.
We mentioned that there was a difference between choosing a goal of protecting information versus having a goal of preventing users from executing or being infected by malware. It’s worth highlighting why we make this distinction. Unless you are running an antivirus company, your goal probably isn’t to prevent malware infections, it’s going to be to protect information. If you’re in the financial sector this would be financial data, for the medical sector, it’ll be health data, and so on. It is easy to get bogged down by the details of how to accomplish this and lose sight of what the real goals are.
There’s ample evidence that users are going to click on things which are e-mailed to them. They click on links, they open attachments, and while there is significant evidence that user awareness training does help, it doesn’t eliminate the problem. 1 Even when limiting the data to widespread phishing (as opposed to spear phishing) and only considering users who fell for one phishing e-mail, there was still a 15% failure rate in the second round of testing. 2 Furthermore, most research doesn’t quantify the cost of false negatives, that is, people not clicking on things which are benign.
Why do people click links and open attachments? Because they need to do their job. People who are in charge of hiring are going to need to receive e-mails from people they don’t know and open resumes which are probably going to be attached, potentially malicious, PDF or Word files. We have seen malicious resumes which exploited memory corruption bugs in popular PDF reading software in the wild, so this is not merely a theoretical threat.
The goal is to protect the company data while still allowing people to do their job. This means that simply stripping all incoming attachments and links is probably not a viable option (not to mention that this solution misses things in end-to-end encrypted e-mail). So what about a solution which acknowledges that no e-mail filtering technology is perfect, and there is a real business requirement to open attachments from strangers, yet still doesn’t allow an attacker to gain access to sensitive data? As you may have guessed, we’re talking about sandboxing dangerous activities using virtual machines (VMs).
One possible answer
Qubes OS is a possible solution, as it can be configured to have the e-mail client open all attachments in a separate, on-demand and disposable VM. Links can also be opened in a fresh browser in their own VM. The system can be configured so the disposable VM doesn’t have network access, or has access to different networks than other VMs. For example, perhaps e-mail is only accessible on the VPN, but when the user clicks a link in the VM that has VPN access it’ll open in a new disposable VM which is directly connected to the internet, rather than connected through the VPN. Perhaps you want those links to go through a special proxy which has exceptionally harsh filtering rules. That is not a problem. When the user is done with the website or document, they close the browser or attachment and the VM is destroyed. This gives a level of separation between the system with access to the VPN and the attachments which may be malicious.
The short explanation of how to set this up is to configure the default program to open attachments and links to be /usr/bin/qvm-open-in-dvm and just pass the URL or filename as an argument. The documentation on disposable VMs is small and worth reading in its entirety. In fact, all the documentation for Qubes is very well written and covers things like copying and pasting text between domains, moving files between domains, and includes several security guides.
Software with enhanced security is often harder to use than its less secure counterparts, so you may be wondering what this looks like to the user. First, they get an e-mail with some links:
You may have also noticed the green bar indicating the e-mail message was decrypted and the signature was valid. This is because I have Enigmail set up so outbound messages are automatically signed and encrypted and incoming messages are automatically decrypted and the signatures are validated. If you want to know how to set that up, you probably also want to know how to prevent a compromised e-mail VM from stealing your private PGP key. Check out this guide for the answers.
Whether e-mail encryption is set up or not, when the user clicks on the link, it is brought up in a new window and it looks like this:
If you look closely, you’ll see the title bar has “[work-email]” in the first screenshot, which indicates it is running in the work-email VM, and “[disp10]” in the second one, showing it is in a disposable VM. The user might not even notice this, and that’s the point, really. We don’t want users to have to be security experts. They should be able to do their job and have the technical systems work for them, not against them, causing problems.
Let us consider what happens if a user opens a malicious document. First, it’s probably targeted at Adobe Reader or Word, but for the sake of argument let us assume that it’s targeted at LibreOffice and that it is either a 0-day or the system isn’t patched. The attacker would then gain control of the LibreOffice process in this disposable VM. If disposable VMs are configured to have network access, it can call out to the internet, download more malicious code, give the attacker a shell, and so forth. However, this disposable VM is not going to have access to things like the VPN, file shares, or other critical infrastructure. This means the data we want to protect remains safe and the attacker loses their foothold when the window is closed and VM self destructs. What about the use case of opening a malicious link in a webpage? We’ll assume that the attacker has a browser-based exploit, or is able to serve up a Linux executable which the user runs. Normally links are opened in a user’s web browser, which means there’s the potential for leveraging the fact that requests opened in frames will include the cookies. This means an attacker can open up a small frame to request access to a site which the user is logged into and then read the content of the page with javascript (unless the server has X-Frame-Options set to block this). However, in the case of our Qubes environment, the link was opened in a fresh browser, meaning the browser in this disposable VM does not have access to these cookies which are stored in a different VM altogether (in my case, work-web, pictured below).
Taking this a step further and demonstrating my use, that web browsing VM doesn’t have access to the VPN, the VPN doesn’t have access to my fuzzing boxes, which are segregated from the VM used for reversing binaries and so on. A couple of my VMs don’t even have network connectivity. Obviously my setup is more compartmentalized than most; it all depends on what your threat model is and what the capabilities are of the attackers you intend to defend against.
The remaining gaps
Of course, this solution does not stop all attacks, nor does it even completely eliminate the damage which can be caused from every phishing attack. Users can still be tricked into opening a link providing their credentials to an attacker-controlled server. Clever attackers will also grab two factor authentication tokens and immediately trade them in for a session token, so two factor auth is not a panacea either. Limiting logins to LAN or VPN users might be an answer as long as there’s some way to make sure the attacker can’t obtain VPN credentials from a user (e.g. using certificates which the O/S can access but the user can not). Other options might be to require client-side TLS certificates to access critical servers. There are a lot of options, and we regularly help organizations figure these things out.
There’s also the risk that the attacker has an exploit to escape the VM and obtain access to the host or other VMs, in which case they’ll be able to do anything the user is able to do. There doesn’t seem to be much evidence that VM escapes are common, and many people assume that if the attacker has a VM escape they probably don’t need to phish users to get access to servers which hold sensitive data. However, some organizations’ threat models include attackers with these capabilities. Whether you need to worry about this or not, it’s important to make a conscious decision on what is an acceptable risk. Performing this kind of threat modeling and building your threat profile can be difficult, especially the first time; practice makes perfect, and the more experience the better. We’ve helped many of our clients build out these threat models, which has allowed them to determine what areas should be prioritized in terms of budgeting. If this is documented, not only does it help your cognizance, but it will also help make security assessments go more quickly.
It’s important to acknowledge the attacks which will be successful as it gives us an idea of the level of security we have. Requiring a VM escape is a much higher level of security than being one malware infection away from having data stolen or being locked out of your own files. This strategy works well with the defense in depth approach, which could be implemented by hardening the disposable VM with exploit mitigations such as Address Space Layout Randomization (ASLR), Data Execution Protection (DEP), PageHeap, and Control Flow Guard (CFG)
Finally, while Qubes does work with Windows VMs, in my experience it works much better if the user is running Linux. This is typically not a problem since Thunderbird runs just as well under Linux as it does on Windows and there is software for opening Word documents and PDF files. In cases where a Windows program needs to be run, if Windows 7 is acceptable Qubes is still very user friendly, as there are guest tools which make things run smoothly. For those who want or need to run later versions of Windows, you can do so, but there currently aren’t any working guest tools, which means usability probably will not be not be much better than any other VM-based solution. If you want the guest tool support for Windows 8/8.1/10, you can contribute to the bounty on this feature, or improve the guest tools (which are open source) to work with later versions of Windows.
Conclusion
The two main take-aways are not to conflate what the security objectives are with the details of how to accomplish them, and that there are practical technological solutions out there which address problems without putting so much of a burden on the users and interrupting their workflow. Qubes won’t be the right answer in all situations, but it’s a solid contender for many use cases.
Qubes OS happens to be open source and free, however if you want cool technology like this to continue, you can show your support by donating. Disclosure: We are not affiliated with Qubes other than being users, donors, and fans ourselves.
If you’d like help sorting out either the big picture or digging into the implementation details, feel free to contact us.