Blog Post

Using Let’s Encrypt Free Certs with your Linux Servers


February 16, 2021

Part 2 of our Blog series on certificates focuses on a practical matter: using the free Let’s Encrypt certificates to secure servers that may not be publicly available, but still need better security than self-signed certs can give you. 

As we explained in our last blog on this subject, to use HTTPS encryption with certificates, you can choose from a number of options:

  • self-signed certificate
  • a cert from a private Certificate Authority (CA), in this case, you or your company run the CA, not a trivial task!
  • a certificate signed by a Root CA you trust

GroundWork supports any of these (or even two at once on the same server). What you choose to use depends on a lot of things, like your tolerance for trust failure reports in your browser from self-signed or private CA certificates. Basically, only root-signed certs are trusted by browsers out-of-the-box, so unless you want to deal with users reporting and complaining about those failures, and explaining how to explicitly trust the certs you use, it’s best to use certs signed by a Root CA.

Which one? What kind?

Ok, so you want to use a root-signed certificate, but which one? What kind? Well, again there’s a lot to choose from. You can buy a cert from one of a number of CA providers, and the type you get determines the trust level the browser reports, usually with the lock symbol in the address line. Have you ever noticed that sometimes it’s green? That’s because the domain owner used enhanced validation.

Most of the time, you can use a “standard” validation cert that is verified to belong to you using a DNS TXT record you add to your domain. This makes the little lock symbol turn white on the address bar. The way this works is that the CA gives you a coded string for you to put in the TXT record, and then verifies it by doing a DNS lookup. Similarly, you can add a coded string to a URL on the domain, validating it is under your control. Fees for standard validation certs range from free (for Let’s Encrypt) to a few tens of dollars a year. 

Some CAs offer extended validation procedures, which allows them to certify to a high degree of trust that you exist, and are in fact the owner of the domain(s) you ask the cert to be valid for. Typically you need to provide extensive documentation, affidavits, business licenses, etc. for this type of cert, and fees can be over $1000 a year. Not for everyone, but you do get a green lock symbol, which should make you confident that the site owners are who they say they are. 

Also, you can get a cert valid for one domain name, several at once, or even all subdomains for a given name at one level of the namespace, a so-called “wildcard” cert. 

Got all that? Ok, let’s start making it easier instead of harder to understand. 

The Let’s Encrypt project is designed to democratize encryption by making the standard validation type of cert free. We like democracy, and encryption, and we like free, so we decided to give you an example of how you can generate and use these free certs with GroundWork, and to automate that process. Here we go:

Automatic DNS cert validation

DNS allows you to map a domain in your external zone file to an internal address, if you want to. So, let’s say you have a GroundWork server which isn’t available publicly. The DNS record for a public name for it can happily resolve to a private IP address. This makes the server accessible to you and your co-workers or anyone on your internal network or VPN using the public name, but not to anyone else. 

Since servers check the name you access the server by against the cert, you can use a cert for the public name on an internally resolved IP address. The GroundWork server never has to be publicly accessible, but it can still use a Root CA signed cert.  

How do you validate the cert? Well, the public nature of DNS means you can use DNS validation. The way you do this will depend on exactly how your DNS provider works. You can do it by hand (see Generating Let’s Encrypt SSL Certificates), but that’s inconvenient for the long term, since the Let’s Encrypt certs expire every 90 days and you need to repeat the process. You will want to automate this. Fortunately, you can.

Use an API

Here at GroundWork, we use Gandi.net (https://gandi.net) for our public DNS, and there’s a convenient plugin for Gandi DNS for certbot. Your provider might be different, so check to see if you can manipulate the DNS with an API. Odds are you can. There’s a list of such providers in the article DNS providers who easily integrate with Let’s Encrypt DNS  validation. For a lot more detailed information about using this method, see this page on the Let’s Encrypt site Challenge Types

1, 2, 3 Go

Here’s the steps to validate your cert with DNS and Gandi’s API. This example works on Ubuntu. The steps might be a little different if you use another Linux distro:

  1. Install certbot and the Gandi API plugin, which needs pip:
    sudo apt update
    sudo apt upgrade
    sudo apt install python3-pip
    sudo pip3 install certbot certbot-plugin-gandi
  2. Get your Gandi API key (see https://api.gandi.net/docs/reference/)
  3. Edit the config file to include your API key:
    sudo vi /etc/letsencrypt/gandi.ini
    
         #Live dns v5 api key
         certbot_plugin_gandi:dns_api_key=YOUR-DNS-API-KEY-GOES-HERE
    
         # optional organization id, remove it if not used
         #certbot_plugin_gandi:dns_sharing_id=SHARINGID
    
    sudo chmod 400 /etc/letsencrypt/gandi.ini
  4. Then just run certbot:
sudo certbot certonly -a certbot-plugin-gandi:dns \ --certbot-plugin-gandi:dns-credentials /etc/letsencrypt/gandi.ini -d \ mygw.example.com

This plugs in the DNS TXT record for this domain that Let’s Encrypt checks for. Once it’s validated, you will have your key and cert files on disk. The following commands will get them installed in your GroundWork server once it’s running:

cd gw8 (or wherever you installed GroundWork 8)
sudo cp /etc/letsencrypt/live/mygw.example.com/fullchain.pem .
sudo cp /etc/letsencrypt/live/mygw.example.com/privkey.pem .
TAG=$(grep '^TAG=' .env | sed 's/^TAG=//');
docker run --rm -t \
     -v ${PWD}:/mnt \
     -v /var/run/docker.sock:/var/run/docker.sock \
     --name gw8 groundworkdevelopment/gw8:${TAG} \
     /src/docker_cmd.sh loadCertificates mygw.example.com \
     privkey.pem fullchain.pem 
rm privkey.pem

Note for the last step, never leave the private key lying around where unprivileged users can see it. 

Make sure to go back into your DNS every so often and clear out the old TXT records this process makes. It’s ok to have a few there, but it might cause problems if there are too many. 

You could easily write a shell script to use the steps above to renew the cert and load it. That can be added to the GroundWork server host crontab and run once a week. Your Let’s Encrypt cert will get renewed as soon as it’s close to expiration, and you’ll never have to touch it again. 

We hope this information is useful and practical. Please let us know if you have any questions about applying these steps to your GroundWork server at GroundWork Support, or email us at support@gwos.com. Happy browsing!

GroundWork Open Source

Other Posts...

7 Ways GroundWork Delivers Bulletproof Infrastructure Monitoring

Lately, security has become top of mind across infrastructure monitoring customers. This is no surprise considering the widespread reports about supply-chain vulnerabilities and embedded compromises rampant in popular network monitoring software. In light of this, we want to underscore how seriously we have always taken our security processes, and how we cultivate a culture based on a foundation of sound security protocols.

We strive to be good stewards of our customer’s data and take great pains to ensure we are always on the bleeding edge of security best practices. A chain is only as strong as its weakest link, which is why we integrate secure processes into the development and deployment of GroundWork, and immediately respond to feedback and suggestions from customers. In this post we outline 7 ways in which our security policies manifest within the platform and our company culture.

Read More

Looking Inside TLS Certificates

The Difficulty of Dealing with Certs

In the last decade, it has become increasingly important to secure websites and applications using HTTPS instead of HTTP. A GroundWork Monitor installation is no exception, so in GroundWork 8, using HTTPS to access the system is the default setup, and you can add TLS certificates to it that you generate or purchase. See Adding Certificates to HTTPS for more information on doing so. TLS (Transport Layer Security) is the successor to the now-obsolete SSL (Secure Sockets Layer), and TLS certificates support the companion protocol that uses modern cryptography to ensure your HTTPS data on the wire cannot be usefully seen by or altered by third parties.

When dealing with certificates, there are many technical questions about how to efficiently and effectively manage the security setup on a web application. While GroundWork does offer several ways to manage certs and system naming, it’s important at the start to make sure you have the right certificates to begin with. To that end, this post describes a small tool we have developed to assist in this process. Future blog posts and documentation pages will cover additional aspects of the security setup on GroundWork systems.

Read More