Blacklisted

After I migrated to DigitalOcean (Amsterdam) from Vultr (Chicago), I realized something weird that was happening to my domain, Caspershire[dot]net. I tried tweeting a blog link from Caspershire Meta, but Twitter said there was something wrong with the URL. Perplexed, I tried shortening my blog link with goo.gl service, that also did not work. Google and Twitter were not explicit with the error message, they just gave me "oh, URL not found" or "there is a problem with your tweet".

Frustrated, I decided to run some detective works.

Figuring Out The Problem

Let's start with a theory. If my domain was blacklisted, I would suspect it has something to do with email blacklisting as in my email appeared in spam, somewhere. So let's scan email-related database for spam blacklist. I ran a check against the MX Toolbox Blacklist Check and yes, I found my domain in 1 database out of 102 known databases. Caspershire was listed in SURBL multi database.

To double check, let's verify the domain against SURBL multi's database on their website. I then verified that my domain was indeed listed in SURBL multi. I was wondering why my domain was blacklisted because I was curious whether it was a problem with the IP address so I could blame DigitalOcean's (DO) Amsterdam (AMS) server fleet, or probably my current IP address on the DO AMS had been implicated with something not so good in the past (answer: no, the IP address was fine). If it was not because of the IP, I could narrow it down that maybe my domain ended up somewhere that SURBL multi picked it up.

Fixing It - Part I

There are 2 parts for fixing this problem. SUBRL multi only allows for a removal request if the requester uses an email associated with the domain s/he is requesting for removal. In my case, I just deleted my Zoho Mail personal business account so I needed to create it again. So in this first part, I had to sign up for Zoho Mail.

I will not go through step by step on registering the domain. I will just summarize what I did.

Right after registering my own domain to be used with Zoho Mail (and thankfully, they did not block it), Zoho Mail instructed me to add a CNAME record to my Zone file, which was hosted at CloudFlare. So, I went to CloudFlare to add a CNAME record pointing to Zoho Mail's verification server. After adding the CNAME record, I waited for 10-15 minutes for DNS changes to take place. It would take a while for DNS setting to propagate across the DNS universe. If you are impatient and could not wait that long, you won't be able to use CNAME method to verify you domain. Zoho Mail will then ask for other methods to verify, e.g. TXT method. I don't recommend this.

Once your domain has been verified by Zoho Mail, feel free to remove that CNAME from your Zone record. The next step, Zoho Mail will ask you to add two MX records. Add that, and wait for another 10-15 minutes for the DNS change to propagate.

I added the MX records, then waited for 20 minutes. The records were then verified by Zoho Mail. The next step was adding the Sender Policy Framework (SPF) and DomainKeys Identified Mail (DKIM) records to the DNS Zone. These two records are critical to make sure the recipient email system (e.g. Gmail) will not flag emails coming from my domain as a spam. Adding SPF record is as easy as adding a TXT record on the DNS Zone. As for DKIM, there are extra steps to generate the encryption key, which later you will add that to your DNS Zone as a TXT record.

To summarize what I did, I added a CNAME record for verification, which later I removed from my Zone. Then, I proceeded by adding two MX records. After that, I added one TXT record for SPF, and another TXT record for DKIM. In total, now I have 4 records on my DNS Zone file (2 MXs, 2 TXTs) for setting up this email system.

To verify that I now could receive email, I tried sending an email from my other Gmail account. It worked!

Onward to the second part.

Fixing It - Part II

Very convenient. Today, 04 February 2018, the SURBL was down (verified with Down). We will continue this adventure some other day.

Update!! On the very same day around 12:38 PM, SURBL was up & running. I managed to file a removal request and I received the ticket ID via email. Late afternoon of the same day around 6:00 PM, I tweeted a link from Caspershire Meta and it went through. To verify that it was not because of a pure dumb luck, I ran my domain against the SURBL database. Weirdly, my domain was still listed. I checked my Zoho email, still got nothing. Hmm okay. I would like to run my domain against their database against probably the next day.

Update!!, 05 February 2018. I received an email from SURBL Whitelist at 4:30 AM this morning while I was sleeping. My site has been removed from their blacklist. I verified it by running a check against their lookup system on their website, I then confirmed that I was not blacklisted anymore. It is all good, all good.