The SSL certificate that your TurnKey server is using is a "self signed" certificate. I.e. it was created by, and signed by your server itself (during firstboot). As your server is not a recognised Certificate Authority (CA), warnings about the certificate are expected behaviour.

The generally excepted way of resolving that is to get a certificate signed by a recognised third party CA. Although if you plan to be communicating with known parties, an alternate approach would be to get each of the remote users to import your server as a known CA. I have little experience with that, so I can't really comment on the pros and cons, nor how you would actually do it; but I know it's possible.

Another thing that you may hit, is that the certificate (and server in general) may be using some sub-optimal SSL configuration. You may be able to improve that, although generally the best way to resolve it on TurnKey, is to migrate your data to the v14.x version of the appliance. Unfortunately though, we deprecated the Ejabberd appliance in v13.x. So there is no migration pathway to v14.x (which supports/provides much more secure "proper" SSL aka TLS).

But seeing as TurnKey is Debian under the hood, you could try doing an "in place" upgrade to Debian Jessie base. I can't guarantee that it will go smoothly (or even that it will work 100%) but it could be worth a try?! Personally I would not attempt it on a production server, but if you can create a snapshot of your current appliance first (and do a test run on the snapshot) then it's probably worth a try. Even though you won't be able to use TKLBAM (because there is no v14.x Ejabberd appliance), the doc page on migration, may still provide some valuable info?!