Personal Data breach
What has happened
On the afternoon of Sunday 24 May 2020, we became aware that a stolen copy of a database backup containing some customers’ personal data from 30 October 2018 had been posted online. If you only became a customer after this date, you will be unaffected by this.
This is of course deeply unfortunate and we are very sorry for the inconvenience and confusion it has caused. We believe openness is the best policy in responding to this incident. This statement aims to do that.
What data is involved
This database backup contains names and contact details for everyone who was or had been a customer, including their email addresses, postal addresses and phone numbers. It also holds cryptographically hashed copies of their control panel passwords, some details of every payment they had made, and the content of every support ticket they had filed. Details of your services, including hostnames and IP addresses, were also leaked.
While this is clearly a very serious data breach, the database does not have any credentials for accessing servers, unless they were disclosed in support tickets and not changed in the following 18 months. Nor do the payment details contain any credit card or bank account numbers, and as all payments are received via PayPal, we could not access that information if we wanted to. No filesystem snapshots are included in the compromised data, so we are completely confident that any data on your server remains secure.
Potentially compromised passwords
Cryptographically hashed passwords are the industry standard for storing login details to websites, and they provide a reasonable degree of security in the event that the database is compromised, as has happened here. The hash used is the standard crypt $6$ hash (i.e. 5000 round SHA512 with a salt). However, when insecure passwords have been used, such as dictionary words, common names or dates of birth, they can be cracked fairly easily offline. Mindful of this, we temporarily took our control panel offline on Sunday afternoon and only restored access once we had disabled the accounts of users whose password has not been changed since 30 October 2018. Everyone should now be able to log in to the control panel again, but if you have not reset your password since that date, you will have to do a password reset before you can log in. There is a link to do this on the login page.
If you use the same password on other systems, please reset those passwords too. It is best practice to use a separate, randomly generated password for each site, and store these in a password manager or other secure location, than to memorise a single password which you use on everywhere.
If you’ve sent us or were provided with a server password in a ticket before 30 October 2018, you should consider this compromised too, but this is not a common scenario and the emails containing the initial password for newly installed servers are not stored and have not been compromised. In general, it is best to avoid ever putting passwords in email, including in online ticketing systems. If it is necessary to do so, the password should be immediately reset, or better still, replaced with an SSH key. Customers who have been following this practice will not need to worry about compromised server passwords, but if in doubt, change your password.
We became aware of a threat to post this data online at 12:44 (BST) on Sunday, but we believe the link to the data was not distributed until 13:07. We took the control panel offline at 13:10. If these timings are accurate, we do not think it practically possible for a password to have been cracked and your account accessed during this three minute window.
How was the data stolen
We can be certain that this copy of the database backup was removed from our server no later than 4 November 2018, as that is when we deleted our last copy of this particular backup. Even if the file was somehow not deleted, one of the first things Mythic Beasts did on buying VMHaus on 26 November 2018 was to move VMHaus’s billing system and backups to newly installed hardware to ensure only authorised people had access to it. The old server, together with any old backups it might have, has been powered off in storage ever since in case we ever need to access it.
We have clear and compelling evidence that this data was posted online by a former director of VMHaus Ltd named Wai Hoe Au Yong, who also uses the online name Auriga. He resigned as a director on 17 October 2018, but still had access to the server in question when the copy must have been made. It is difficult to see what lawful business he could have had copying this backup after his resignation, and why he did not disclose its existence when we were negotiating to buy the company, so it is tempting to suppose it was made with malicious intent. But regardless of the original reason for the copy, posting this data online is unfortunately just the latest in a series of actions he has taken to disrupt our business and harm our reputation. We have already referred the matter to our lawyers in Malaysia, where Au Yong lives, and will be reporting the breach to the Information Commissioner's Office here in the UK when it opens on Tuesday. The advice we receive will determine how best to inform law enforcement agencies in the two countries.
It would be almost impossible for a small business to have technical measures in place that would prevent a technical director from taking a private copy of data, and it usually isn't possible to tell after the event whether this happened. Ultimately, when we acquire a business – and VMHaus is the fifth time we've done this, so we have some experience of it – we have to trust the previous owners to be honest about what data they may have and to delete anything that is subsequently found, or at the very least not to engage in criminal activity.
We became aware of this data breach when a tweet appeared on the @VMHaus Twitter account saying “Giving away Bonsai source code and customer database in mysql dbdump. DM to get it now.” This was one of a series of malicious posts made on Sunday. Another simply read “We are closing down.” These were not posted by us and unfortunately the @VMHaus account is controlled by Au Yong, not us. This is something we have known about for some time but have been unable to resolve.
When we bought VMHaus, we were given access to the Twitter account. Most unfortunately we failed to notice that Au Yong had a phone number set up for emergency backup access to the account, and long after the acquisition, he used this to change the password and lock us out. We've tried repeatedly to get Twitter to restore it to us, but so far without success.
Bonsai source code
Bonsai is the name given to the software which runs our control panel. The contract of sale included all intellectual property rights to Bonsai and specifically stated Au Yong and the other sellers had no licence to use, sell or sublicense Bonsai. We do not know whether Au Yong really has retained a copy of the Bonsai source, and if so whether he has distributed it. However anyone who has been given a copy should be aware that they are in receipt of stolen property and we reserve the right to take legal action should they make use of it.
Appendix: Background information
It is reasonable for our customers to want to know how this situation has occurred. Why are relations so poor between Wai Hoe Au Yong and the company he founded that he is willing to engage in criminal activity to hurt us? We cannot answer this as there seems to be no rational explanation why he should jeopardise his own reputation for no possible tangible gain. However in light of some of the misleading information that has been circulating, we feel it appropriate to provide further background to allow customers to reach their own informed view on the matter. Most of what follows is already in the public domain, and this section simply puts it together in one place.
Wai Hoe Au Yong (who also goes by the name Auriga) founded VMHaus in 2017 to provide cheap virtual servers. This was not his first attempt at doing this – his previous unsuccessful attempt was called Gigavest. Au Yong lent the business the capital it needed to get started and handled the administrative and financial side of the business, but relied on a succession of English teenagers to do all the technical work and customer support. VMHaus was originally set up as an unregistered Malaysian sole proprietorship, owned wholly by Au Yong. In November 2017, a UK limited company was set up, owned jointly by Au Yong and his English business partners, who were each directors of the company.
No-one involved in VMHaus received any salary, dividends or other payments – their shares in the company and the potential for future income was their sole incentive. The company was profitable only when staff time was excluded, and the small profit that was generated was reinvested in growth, such as when VMHaus opened a second hosting site in California. This is not unusual in small businesses, and Mythic Beasts went through a similar initial period when we started trading in 2000.
The small print on the VMHaus website said that it was the UK company that was trading, as did the occasional invoice the company issued, however most assets remained in the name of the Malaysian sole proprietorship. There has been some public speculation that this arrangement may have been allowed to persist to avoid paying UK corporation tax and/or to avoid giving the English business partners a meaningful stake in the company. However we think it likely it was simply an administrative oversight that both entities remained. The matter was finally resolved in November 2018 when Au Yong agreed to transfer all the assets from the Malaysian proprietorship to the UK company.
In the summer of 2018, Mythic Beasts offered a paid summer job to one of the English co-owners, who at that time was still in school. We were fully aware of his involvement with VMHaus at the time. As the summer progressed, he decided to leave school and asked whether he could work full time for us. We were extremely happy with the quality of his work and agreed on the condition that he cut his involvement with VMHaus. What was a manageable conflict of interest for a few months over the summer would not have been acceptable with a full time employee. This condition was accepted.
We suspected this might leave VMHaus without adequate technical manpower and did not want to be accused of pushing them out of business. We therefore did not require an immediate resolution and made several suggestions that we hoped might help VMHaus find a suitable replacement. One of these suggestions was that if they chose to sell the company, we would be willing to consider buying it at a fair market rate and pay off its debts so no-one was out of pocket. We gave VMHaus a provisional valuation which we thought to be a fair and believed VMHaus were minded to sell, but we would have been perfectly happy if they chose to continue trading. Acquiring a business generates a vast amount of work and from a purely financial consideration this would clearly not make a company as small as VMHaus a desirable acquisition. Our offer was made solely as a possible exit strategy if the other owners wanted one. With hindsight we can only suppose Au Yong was deeply unhappy with this, though he never expressed this to us.
The matter came to a head at the end of October. We received a panicked phone call from our new employee who needed urgent advice. He had received a message from a supplier saying that there was a large invoice very overdue, and if it wasn’t paid immediately, VMHaus’s servers would be turned off. Thinking this was an oversight, he went to the company PayPal account to pay the invoice and discovered that Au Yong had blocked his access to it. Au Yong refused to use the money in the account to pay the outstanding invoice, later transferring the balance to his personal bank account. We also discovered from a chance look at the Companies House website that Au Yong had resigned as a director about a week earlier. The third owner, who we believe to be blameless in this, had also resigned as a director. This left our employee as the sole director of a company which by any reasonable definition was now insolvent.
Even though the amount Au Yong removed from the PayPal account was less than the outstanding amount of his loan to the company to finance its initial development, it is simply not lawful for a shareholder to help himself to the company’s money without the consent of the directors, which he did not have. Nor is there any doubt that the money he took was the rightful property of VMHaus Ltd, as the only income to the account (aside from the initial loan) was the result of trading activity by the UK company. We would have preferred these facts to have remained private, but this was not to be and as a result VMHaus lost a significant number of customers.
In such a situation, the best thing to do is normally to allow the company to go into administration, and subsequent liquidation as the company’s debts exceeded its assets. For the sake of our new employee, we wished to avoid this. Not only would this have caused a lot of stress, it was also likely to have a long term effect on his credit record. There was also a risk that he could have been held personally liable for the company’s outstanding debts if a court found he had traded while insolvent. He had not knowingly done so, but with all the company’s financial records in Malaysia, he may have struggled to prove this. After taking professional legal advice we decided to see if we could continue with the acquisition.
Obviously the situation had now changed. We were no longer buying a viable business, but rather one that was only solvent because we were underwriting and paying their day to day expenses while negotiations continued. VMHaus was haemorrhaging customers due to the public uncertainty over its future. Moreover, we would have to deal with the fact that VMHaus had not been keeping adequate accounts and it would soon be necessary to produce and file the first set of annual accounts. We therefore judged that the company had no marketable value – it was a liability.
Our offer was to buy the company for £1 and repay Au Yong the outstanding part of his loan to the company. The previous owners would have no further responsibilities or liability, and we would be able to assure suppliers that the company was again solvent and ensure proper accounts were filed. In the circumstances, we felt this was an extremely generous offer, not least because the only other outcome we could see was insolvency proceedings which would not see any of Au Yong’s loan repaid, and may well have resulted in legal proceedings over the money removed from the PayPal account.
This offer was far lower than we were initially considering, but that was before Au Yong’s actions rendered the company functionally insolvent and materially harmed its reputation. All three owners agreed to these terms and VMHaus was sold to Mythic Beasts on these terms on 26 November 2018. The other two owners seemed immensely relieved to be out of this toxic partnership. We repaid the balance of Au Yong’s initial loan to VMHaus in full on the same day, a total of several thousand US dollars. This included several sums which Au Yong claimed to have lent the company – and indeed probably did – but was unable to document.
Even though Au Yong consented to the sale of VMHaus, we suspected he was never happy about it. From first hand experience with Mythic Beasts, we know it is extremely hard to build a successful business, and we do not think that a business model which involves getting teenagers you’ve never met to work for free is a viable way of achieving it. They were not in indentured servitude, and it was always likely that at some point one of them would need a source of income and leave. That is exactly what happened.
Mythic Beasts was under no obligation to help VMHaus by offering to buy it, either in the summer of 2018 or that November. We did so because it felt the right thing to do, and fairest for all concerned. Au Yong was not forced to sell against his will. His stake in the company was big enough to prevent that, and there were always alternatives he could have explored. Even at the very end, he could have chosen to decline our offer and either allowed the company to go into administration or found alternative investment. But he did not, and in truth, our offer was far more generous than any realistic alternative. Rather than be grateful, he evidently harboured a grudge which over time seems to have developed such that he is now willing to engage in criminal activity with the sole purpose of harming us.