Sometimes, the intruder is a polite guest who doesn’t want to leave.

In 2005, the rules of the internet were still being written. We didn’t have automated patch management or sophisticated EDR; we had manual deployments and a lot of trust. But as I found out through a single text file left on my private server, trust is a poor substitute for a security policy.

A short while after the dot com bubble burst, I worked as a software developer for a small company. Part of my job was deploying our software to the production system.

“Deploy” meant manually copying a handful of binaries to a Linux server.

That server was special in another way: for reasons nobody could explain, it was the only machine from which I could establish an SSH connection to my private Linux VPS on the internet. So I did that regularly.

At some point I received an email from my VPS provider. According to them, SSH brute-force password scans were originating from my server. They offered to have one of their technicians investigate - billed in 15-minute increments at a rate that made me briefly consider a career change. They asked what my intentions were.

I replied that my intention was to take a vacation on a remote island somewhere in the Caribbean, without any technology newer than 50 years - except maybe the fridge for the drinks. And that, in order to ever afford that, I would take care of the issue myself. Thanks very much. I immediately stopped the SSH scanner on my VPS, changed my password, and started digging. I collected logs and suspicious scripts in a directory in my own user home directory. I wanted to understand how the attacker had gotten in, but I couldn’t find anything beyond normal-looking log entries.

Frustrated, I left it at that and decided to sleep on it. Nothing important was running on that VPS anyway, it was purely a private machine, just to have a Linux server somewhere on the internet (which is surprisingly useful).

It was late, I was tired, and definitely not in the state of mind in which one should operate a root shell on a compromised system. I logged out and went to bed.

The next day, when I wanted to retrieve the collected data, the directory contained exactly one file:

please.txt

Inside it:

Noooo please, don’t throw me out!

My “guest” had been there overnight. He had noticed that I was about to evict him, erased the logs I had collected and politely asked to stay.

I was perplexed - but also curious. So back to the logs.

There it was: an SSH login a few hours after I had changed my password. While I was asleep. The source IP? The company’s production server.

The production server had been compromised.

The previous day, I had investigated the incident from that server. I had SSH’d from there to my VPS when I changed the password.

Someone was sitting on that machine, silently logging every keystroke I made. When I changed my VPS password to ’lock the door,’ I was actually handing the intruder the new key in real-time.

The production server was running a Linux distribution that was already five years past its prime. In 2005, updating a linux server wasn’t a background process; it was a manual, often breaking, ordeal. So, we just didn’t do it.

Hacking into a Linux server and installing a keylogger while avoiding detection requires a respectable level of technical competence. This was not a script-kiddie. Whoever this was had access to better tools - and knew how to use them. I also knew he was polite, the kind who would leave a note instead of trashing the system.

The only mistake my ‘guest’ had made was to log in to my VPS from the compromised server after I’d changed the password. By doing that, he raised a flag that said “here I am, come and find me”. Until today, I like to think that he learned from that mistake as well.

I reported the incident to my boss, the company owner. He told me not to worry about it - he would take care of it.

I knew that probably meant nothing would happen. But I let it go.

Don’t judge me, remember: this was short after the dot com bubble burst. There were no incident response playbooks, no security teams, no mandatory reporting procedures. I was responsible for the software, but not for the server it ran on.

At the time, I was already in the process of leaving Vienna for good. I was moving to another city, had handed in my notice, and didn’t have much time left at that company. I never found out if the production server eventually was replaced, or what happened to the friendly uninvited guest. But I never had another issue on my own VPS.

And I never again used a production server for private purposes.

What I Learned

Key lessons from this episode:

  • Keep your systems updated. Unpatched machines are a breach waiting to happen. It’s like leaving a door unlatched and hoping that no one will notice.
  • Only run production code on production servers. Never use them for personal tasks or as a jump host.
  • Don’t trust - assume breach. Treat every system as if it could be compromised.