Configure tunnels

For some Horizon features to work, such as pushing certificates to third-party connectors, it will need to have network access to the third-party. Sometimes, this is not possible due to network restrictions. In these cases, you can configure a tunnel service such as Chisel to allow Horizon to communicate with the third-party service through encrypted tunnels over HTTP.

Horizon will eventually support connecting to on-premises resources, so you won’t need to set up a separate tunnel service. However, this feature is not yet available in Horizon.

Generate a server key

An SSH key will be required to establish a secure connection between the Chisel server and clients. To generate a key, install Chisel on your local machine and run the following command:

$ chisel server --keygen /tmp/chisel_key

Then, start the server with the --keyfile argument pointing to the generated key:

$ chisel server --keyfile /tmp/chisel_key

The server will output a fingerprint that you will need to copy. This fingerprint will be used to verify the connection from the Chisel client. The output will look something like this:

* server: Fingerprint: Nz5NRzt20kNugkeyHDcxEXQ3+D4Noy8lThsPzkiNJc8=

Set up the server

The Chisel server should be set up in the same network location as Horizon. This can be on the same server as Horizon or on a separate server that has access to Horizon. The Chisel server will listen for incoming connections from the Chisel client and forward requests to the third-party service.

RPM
Kubernetes

Fetch the Chisel RPM from their releases page and install it on the server that will run the Chisel server:

$ yum install chisel_1.10.1_linux_amd64.rpm

Copy the generated key to the server, in a path such as /var/chisel/chisel_key. Make sure the key is readable by the user that will run the Chisel server.

When starting the server, add the --keyfile argument with the path to the key:

$ chisel server --keyfile /var/chisel/chisel_key --port 80 --reverse

You’ll need to ensure that the server will run as a service, such as by using systemd. You’ll also need to expose the Chisel server with a reverse proxy, such as Nginx, to make it accessible over the network.

Create a Kubernetes Secret to store the Chisel key. You can do this by running the following command, replacing <base64-encoded-chisel-key> with the base64-encoded content of your chisel_key file:

$ kubectl create secret generic chisel-config --from-file=chisel_key=/tmp/chisel_key

You can deploy the Chisel server to a cluster by applying the following manifests:

chisel.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: chisel
  labels:
    app.kubernetes.io/name: chisel
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: chisel
  template:
    metadata:
      labels:
        app.kubernetes.io/name: chisel
    spec:
      containers:
        - name: chisel
          image: jpillora/chisel:1
          args: ["server", "--port", "80", "--reverse"]
          ports:
            - containerPort: 80
            volumeMounts:
                - name: chisel-config
                    mountPath: /var/chisel
      volumes:
        - name: chisel-config
          secret:
            secretName: chisel-config
---
apiVersion: v1
kind: Service
metadata:
  name: chisel
spec:
  selector:
    app.kubernetes.io/name: chisel
  ports:
    - name: chisel
      protocol: TCP
      port: 80
      targetPort: 80
    - name: server
      protocol: TCP
      port: 9000
      targetPort: 9000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: chisel
spec:
  ingressClassName: nginx
  rules:
    - host: tunnel.example.org
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: chisel
                port:
                  number: 80

$ kubectl apply -f chisel.yaml

By default, Chisel doesn’t come with any authentication, meaning any client will be able to create a tunnel to the server. To secure a Chisel tunnel, many options are available:

Whitelist specific client IPs using a firewall or reverse proxy;
Use Chisel built-in password authentication by generating a username and password, then passing them to the --auth argument on both the server and client;
Use Chisel’s built-in mTLS authentication by generating a CA, server, and client certificates, then passing them to the --tls-cert and --tls-key arguments to the client, and --tls-ca to the server.

Configure the client

Next to the third-party service, you will need to run the Chisel client. This can be done on the same server as the third-party service or on a separate server that has access to the third-party service.

In the example below, we will assume that the third-party service is running on example-third-party.local and that you want to expose it on port 9000 of the Chisel client.

RPM
Kubernetes

Fetch the Chisel RPM from their releases page and install it on the server that will run the Chisel client:

$ yum install chisel_1.10.1_linux_amd64.rpm

When starting the client, add the --fingerprint argument with the value you copied from the server logs:

$ chisel client tunnel.example.org --fingerprint <server-fingerprint> R:9000:example-third-party.local:80

You’ll need to ensure that the client will run as a service, such as by using systemd.

You can deploy the Chisel client to a cluster by applying the following manifests. The fingerprint can be provided as a command-line argument:

chisel-client.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: chisel-client
  labels:
    app.kubernetes.io/name: chisel-client
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: chisel-client
  template:
    metadata:
      labels:
        app.kubernetes.io/name: chisel-client
    spec:
      containers:
        - name: chisel-client
          image: jpillora/chisel:1
          args: ["client", "tunnel.example.org", "--fingerprint", "<server-fingerprint>", "R:9000:example-third-party.local:80"]
          volumeMounts:
            - name: chisel-config
              mountPath: /var/chisel
          ports:
            - containerPort: 9000
      volumes:
        - name: chisel-config
          secret:
            secretName: chisel-config
---
apiVersion: v1
kind: Secret
metadata:
  name: chisel-config
type: Opaque
data:
  chisel_key: <base64-encoded-chisel-key>
---

Apply the manifest to your cluster:

$ kubectl apply -f chisel-client.yaml

Reference tunnel in Horizon

Now, you’ll be able to reference the tunnel in Horizon. For example, if you want to push a certificate to a third-party service that is accessible through the tunnel, you can configure the connector in Horizon to point to the tunnel instead of directly to the third-party service.

For instance, if your server is running on http://chisel:9000 in your Kubernetes cluster, you can use this address in the connector configuration.

Configure a reverse proxy

Having a single Chisel server is useful but limited if Horizon needs to connect to multiple third-party services, such as a fleet of F5 load balancers. In this case, the best way to manage a large number of connections is to couple Chisel with a reverse proxy, such as tinyproxy. In this setup, instead of connecting directly to the Chisel tunnel, Horizon will use it as a proxy to reach the third-party services, and has to be configured as such.

Here’s a simple docker-compose.yml file that can set up tinyproxy to use the Chisel tunnel as an upstream proxy:

services:
  tinyproxy:
    image: monokal/tinyproxy:latest
    container_name: tinyproxy
    command: ["ANY"]
    restart: always
  chisel:
    image: jpillora/chisel:1.11.3
    ...