Skip to content

Implementing IP Whitelisting for Load Balancers with Security Groups on 3Engines Cloud🔗

In this article we describe how to use commands in Horizon, CLI and Terraform to secure load balancers for Kubernetes clusters in 3Engines by implementing IP whitelisting.

What Are We Going To Do🔗

Introduction🔗

Load balancers without proper restrictions are vulnerable to unauthorized access. By implementing IP whitelisting, only specified IP addresses are permitted to access the load balancer. You decide from which IP address it is possible to access the load balancers in particular and the Kubernetes cluster in general.

Prerequisites🔗

No. 1 Account

You need a 3Engines Cloud hosting account with access to the Horizon interface: https://3engine.rootxwire.com/.

No. 2 List of IP addresses/ranges to whitelist

This is the list of IP addresses that you want the load balancer to be able to listen to.

No. 3 A preconfigured load balancer

In 3Engines, each time you create a Kubernetes cluster, the corresponding load balancers are created automatically.

See article How to Create a Kubernetes Cluster Using 3Engines Cloud 3Engines Magnum

No. 4 3Engines command operational

This is a necessary for CLI procedures.

This boils down to sourcing the proper RC file from Horizon. See How To Use Command Line Interface for Kubernetes Clusters On 3Engines Cloud 3Engines Magnum

No. 5 Python Octavia Client

To operate Load Balancers with CLI, the Python Octavia Client (python-octaviaclient) is required. It is a command-line client for the 3Engines Load Balancing service. Install the load-balancer (Octavia) plugin with the following command from the Terminal window, on Ubuntu 22.04:

pip install python-octaviaclient

Or, if you have virtualenvwrapper installed:

mkvirtualenv python-octaviaclient
pip install python-octaviaclient

Depending on the environment, you might need to use variants such as python3, pip3 and so on.

No. 6 Terraform installed

You will need Terraform version 1.50 or higher to be operational.

For complete introduction and installation of Terrafom on 3Engines see article Generating and authorizing Terraform using Keycloak user on 3Engines Cloud

To use Terraform in this capacity, you will need to authenticate to the cloud using application credentials with unrestricted access. Check article How to generate or use Application Credentials via CLI on 3Engines Cloud

Horizon: Whitelisting Load Balancers🔗

We will whitelist load balancers by restricting the relevant ports in their security groups. In Horizon, use command Network –> Load Balancers to see the list of load balancers:

whitelisting_again-4v2.png

Let us use load balancer with the name starting with gitlab. There is no direct connect from load balancer to security groups, so we first have to identify an instance which corresponds to that load balancer. Use commands Project –> Compute –> Instances and search for instances containing gitlab in its name:

whitelisting_again-5v2.png

Edit the security groups of those instances – for each instance, go to the Actions menu and select Edit Security Groups.

whitelisting_again-7v2.png

Filter by gitlab:

whitelisting_again-8v2.png

Use commands Project –> Network –> Security Groups to list security groups with gitlab in its name:

whitelisting_again-9v2.png

Choose which one you are going to edit; alternatively, you can create a new security group. Anyways, be sure to enter the following data:

  • Direction: Ingress
  • Ether Type: IPv4
  • Protocol: TCP
  • Port Range: Specify the port range used by your load balancer.
  • Remote IP Prefix: Enter the IP address or CIDR to whitelist.

Save and apply the changes.

Verification🔗

To confirm the configuration:

  1. Go to the Instances section in Horizon.
  2. View the security groups applied to the load balancers’ associated instances.
  3. Ensure the newly added rule is visible.

CLI: Whitelisting Load Balancers🔗

The 3Engines CLI provides a command-line method for implementing IP whitelisting.

Be sure to work through Prerequisites Nos 4 and 5 in order to have 3Engines command fully operational.

List the security groups associated with the load balancer:

3Engines loadbalancer show <LOAD_BALANCER_NAME_OR_ID>

Identify the pool associated with the load balancer:

3Engines loadbalancer pool list

Show details of the pool to list its members:

3Engines loadbalancer pool show <POOL_NAME_OR_ID>

Note the IP addresses of the pool members and identify the instances hosting them.

Create a security group for IP whitelisting:

3Engines security group create <SECURITY_GROUP_NAME>

Add rules to the security group:

3Engines security group rule create \
--ingress \
--ethertype IPv4 \
--protocol tcp \
--dst-port <PORT_RANGE> \
--remote-ip <IP_OR_CIDR> \
<SECURITY_GROUP_ID>

Apply the security group to the instances hosting the pool members:

3Engines server add security group <INSTANCE_ID> <SECURITY_GROUP_NAME>

Verification🔗

Verify the applied security group rules:

3Engines security group show <SECURITY_GROUP_ID>

Confirm the security group is attached to the appropriate instances:

3Engines server show <INSTANCE_ID>

Terraform: Whitelisting Load Balancers🔗

Terraform is an Infrastructure as Code (IaC) tool that can automate the process of configuring IP whitelisting.

Create a security group and whitelist rule in main.tf:

# main.tf

# Security Group to Whitelist IPs
resource "3Engines_networking_secgroup_v2" "whitelist_secgroup" {
  name        = "loadbalancer_whitelist"
  description = "Security group for load balancer IP whitelisting"
}

# Add Whitelist Rule for Specific IPs
resource "3Engines_networking_secgroup_rule_v2" "allow_whitelist" {
  direction         = "ingress"
  ethertype         = "IPv4"
  protocol          = "tcp"
  port_range_min    = 80             # Replace with actual port range
  port_range_max    = 80
  remote_ip_prefix  = "192.168.1.0/24" # Replace with actual CIDR
  security_group_id = 3Engines_networking_secgroup_v2.whitelist_secgroup.id
}

# Existing Instances Associated with Pool Members
resource "3Engines_compute_instance_v2" "instances" {
  count           = 2 # Adjust to the number of pool member instances
  name            = "pool_member_${count.index + 1}"
  flavor_id       = "m1.small"  # Replace with an appropriate flavor
  image_id        = "image-id"  # Replace with a valid image ID
  key_pair        = "your-key-pair"
  security_groups = [3Engines_networking_secgroup_v2.whitelist_secgroup.name]
  network {
    uuid = "network-uuid" # Replace with the UUID of your network
  }
}

# Associate the Load Balancer with Security Group via Instances
resource "3Engines_lb_loadbalancer_v2" "loadbalancer" {
  name          = "my_loadbalancer"
  vip_subnet_id = "subnet-id" # Replace with the subnet ID
  depends_on    = [3Engines_compute_instance_v2.instances]
}

Initialize and apply the configuration:

terraform init
terraform apply

Verification

Use Terraform to review the applied state:

terraform show
3Engines server show <INSTANCE_ID>
3Engines security group show <SECURITY_GROUP_ID>

State of Security: Before and after whitelisting the balancers🔗

Before implementing IP whitelisting, the load balancer accepts traffic from all sources. After completing the procedure:

  • Only specified IPs can access the load balancer.
  • Unauthorized access attempts are denied.

Verification Tools🔗

Various tools can ensure the protection is installed and active:

livez
Kubernetes monitoring endpoint.
nmap
(free): For port scanning and access verification.
curl
(free): To confirm access control from specific IPs.
Wireshark
(free): For packet-level analysis.

Testing with nmap🔗

nmap -p <PORT> <LOAD_BALANCER_IP>

Testing with http and curl🔗

curl http://<LOAD_BALANCER_IP>

Testing with curl and livez🔗

This would be a typical response before changes:

curl -k https://<KUBE_API_IP>:6443/livez?verbose
[+]ping ok
[+]log ok
[+]etcd ok
[+]poststarthook/start-kube-apiserver-admission-initializer ok
[+]poststarthook/generic-apiserver-start-informers ok
[+]poststarthook/priority-and-fairness-config-consumer ok
[+]poststarthook/priority-and-fairness-filter ok
[+]poststarthook/storage-object-count-tracker-hook ok
[+]poststarthook/start-apiextensions-informers ok
[+]poststarthook/start-apiextensions-controllers ok
[+]poststarthook/crd-informer-synced ok
[+]poststarthook/start-system-namespaces-controller ok
[+]poststarthook/bootstrap-controller ok
[+]poststarthook/rbac/bootstrap-roles ok
[+]poststarthook/scheduling/bootstrap-system-priority-classes ok
[+]poststarthook/priority-and-fairness-config-producer ok
[+]poststarthook/start-cluster-authentication-info-controller ok
[+]poststarthook/start-kube-apiserver-identity-lease-controller ok
[+]poststarthook/start-deprecated-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-kube-apiserver-identity-lease-garbage-collector ok
[+]poststarthook/start-legacy-token-tracking-controller ok
[+]poststarthook/aggregator-reload-proxy-client-cert ok
[+]poststarthook/start-kube-aggregator-informers ok
[+]poststarthook/apiservice-registration-controller ok
[+]poststarthook/apiservice-status-available-controller ok
[+]poststarthook/kube-apiserver-autoregistration ok
[+]autoregister-completion ok
[+]poststarthook/apiservice-openapi-controller ok
[+]poststarthook/apiservice-openapiv3-controller ok
[+]poststarthook/apiservice-discovery-controller ok
livez check passed

And, this would be a typical response after the changes:

curl -k https://<KUBE_API_IP>:6443/livez?verbose -m 5
curl: (28) Connection timed out after 5000 milliseconds

What To Do Next🔗

Compare with articles:

Configuring IP Whitelisting for 3Engines Load Balancer using Horizon and CLI on 3Engines Cloud

Configuring IP Whitelisting for 3Engines Load Balancer using Terraform on 3Engines Cloud