Unfortunately, you don't provide enough info.
We don't know what OS, where it is hosted, etc.
The only reason that comes to my mind for the error to happen is because there's some misconfigured firewall/port forwarding.
You'll need to provide more info for us to be able to help.
Yes exactly, if you use Let's Encrypt with a configuration like that, and the name of the certificate and the server name match, then you won't get a "certificate failed" error message and all will be good.
Just tried to connect to your server, and it is still serving a self-signed certificate on port 6697, it is not serving the Let's Encrypt certificate on that port.
Have you rehashed? And when you do REHASH, make sure it doesn't print any errors. When you see errors during rehash, the rehash is not successful and the old configuration stays in effect.