Quick Guide: Multus CNI Setup and Pod-to-Pod Connectivity Testing

Overview

This guide shows you how to quickly set up Multus CNI in Amazon EKS and test connectivity between pods using multiple network interfaces. Perfect for learning and proof-of-concept scenarios.

Time Required: 15-20 minutes
What You'll Do: Install Multus → Create networks → Deploy test pods → Test connectivity

Prerequisites

• EKS cluster running
• kubectl configured
• Basic Kubernetes knowledge

# Quick check
kubectl cluster-info
kubectl get nodes

Step 1: Install Multus CNI (2 minutes)

# Install Multus
kubectl apply -f https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset-thick.yml

# Verify installation
kubectl get pods -n kube-system -l app=multus

Wait until all Multus pods show Running status.

Step 2: Create Test Networks (1 minute)

# Save as test-networks.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-a
spec:
  config: '{
    "cniVersion": "0.3.1",
    "type": "bridge",
    "bridge": "test-br0",
    "isDefaultGateway": false,
    "ipam": {
      "type": "static",
      "addresses": [{"address": "192.168.1.0/24"}]
    }
  }'
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: net-b
spec:
  config: '{
    "cniVersion": "0.3.1",
    "type": "bridge",
    "bridge": "test-br1", 
    "isDefaultGateway": false,
    "ipam": {
      "type": "static",
      "addresses": [{"address": "192.168.2.0/24"}]
    }
  }'

kubectl apply -f test-networks.yaml
kubectl get network-attachment-definitions

Step 3: Deploy Test Pods (2 minutes)

# Save as test-pods.yaml
apiVersion: v1
kind: Pod
metadata:
  name: pod-a
  annotations:
    k8s.v1.cni.cncf.io/networks: net-a
spec:
  containers:
  - name: test
    image: nicolaka/netshoot
    command: ["sleep", "infinity"]
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-b
  annotations:
    k8s.v1.cni.cncf.io/networks: net-a
spec:
  containers:
  - name: test
    image: nicolaka/netshoot
    command: ["sleep", "infinity"]
---
apiVersion: v1
kind: Pod
metadata:
  name: pod-c
  annotations:
    k8s.v1.cni.cncf.io/networks: net-b
spec:
  containers:
  - name: test
    image: nicolaka/netshoot
    command: ["sleep", "infinity"]

kubectl apply -f test-pods.yaml
kubectl wait --for=condition=ready pod pod-a pod-b pod-c --timeout=60s

Step 4: Test Connectivity (5 minutes)

Check Network Interfaces

# Check pod-a interfaces
kubectl exec pod-a -- ip addr show

# You should see:
# eth0 = Primary EKS network
# net1 = Multus network (net-a)

Test Same Network Communication

# Get pod IPs
kubectl get pods -o wide

# Test connectivity between pod-a and pod-b (both on net-a)
kubectl exec pod-a -- ping -c 3 <pod-b-primary-ip>

# Test via Multus interface (if IPs are assigned)
kubectl exec pod-a -- ping -c 3 -I net1 <pod-b-net1-ip>

Test Network Isolation

# pod-c is on net-b, should be isolated from net-a pods
kubectl exec pod-a -- ping -c 3 <pod-c-primary-ip>
# This works (primary network)

kubectl exec pod-a -- ping -c 3 <pod-c-net1-ip>  
# This should fail (different Multus networks)

Verify Network Configuration

# Check routing tables
kubectl exec pod-a -- ip route show
kubectl exec pod-c -- ip route show

# Check bridge interfaces on nodes
kubectl get nodes -o wide
kubectl debug node/<node-name> -it --image=nicolaka/netshoot -- ip link show | grep test-br

Expected Results

✅ Success Indicators:

• All pods show Running status
• Each pod has eth0 (primary) + net1 (Multus) interfaces
• Pods on same Multus network can communicate
• Pods on different Multus networks are isolated
• Bridge interfaces created on worker nodes

❌ Common Issues:

• Pods stuck in ContainerCreating → Check Multus logs
• No net1 interface → Check NetworkAttachmentDefinition
• Connectivity fails → Check security groups/network policies

Quick Troubleshooting

# Check Multus logs
kubectl logs -n kube-system -l app=multus --tail=20

# Check pod events
kubectl describe pod pod-a

# Verify networks exist
kubectl get network-attachment-definitions

# Test basic connectivity
kubectl exec pod-a -- ping -c 1 8.8.8.8

Cleanup

# Remove test resources
kubectl delete pod pod-a pod-b pod-c
kubectl delete network-attachment-definitions net-a net-b
kubectl delete -f https://raw.githubusercontent.com/k8snetworkplumbingwg/multus-cni/master/deployments/multus-daemonset-thick.yml

# Clean up files
rm test-networks.yaml test-pods.yaml

What You Learned

1. Multus Installation - Single command to add multi-networking
2. Network Creation - Bridge plugin for simple pod-to-pod communication
3. Pod Deployment - Using annotations to attach multiple networks
4. Connectivity Testing - Verifying network isolation and communication
5. Troubleshooting - Basic debugging techniques

Next Steps

• Try different CNI plugins (ipvlan, host-device)
• Implement network policies for security
• Test with real applications instead of netshoot
• Scale to multiple nodes and test cross-node communication
• Explore advanced IPAM configurations

---

Total Time: ~15 minutes for setup + testing
Key Takeaway: Multus enables multiple network interfaces per pod with simple YAML configurations