OpenWhisk using Kubernetes and Docker

OpenWhisk components are delivered as a Kubernetes pod by deploying Docker containers. The base images are pulled from OpenWhisk repository on Docker Hub to build clusters and some tips and issues that we experienced are provided in this guide to help other users.

Docker 1.12, not 17.09 ce

Kubernetes needs a container engine, e.g. Docker but the latest one may not work as expected. We find two reasons why Docker 1.12 is chosen, because 1) Kubernetes recommends v1.12 (also here) and 2) `Docker CE 17.09 does not support RHEL7<https://docs.docker.com/engine/installation/linux/docker-ee/rhel/>`_

Note

Docker package is also available in a RHEL extras repository which is recommended to receive support by Red Hat.

Manual Installation

This section provides a list of commands to install Docker manually, instead of using package pamangers i.e. yum to use an exact version of Docker.

Download Docker rpm package:

wget https://packages.docker.com/1.12/yum/repo/main/centos/7/Packages/docker-engine-1.12.6.cs8-1.el7.centos.x86_64.rpm

Note

If this link is broken, please look up substitute the repository.

The following command to install:

sudo yum -y install docker-engine-1.12.6.cs8-1.el7.centos.x86_64.rpm

Note

–installroot=/opt/docker may used to separate installed location, set variable by export DOCKER_ROOT=/opt/docker

Service file from the Moby Project:

wget https://raw.githubusercontent.com/moby/moby/master/contrib/init/systemd/docker.service -P $DOCKER_ROOT/etc/systemd/system
wget https://raw.githubusercontent.com/moby/moby/master/contrib/init/systemd/docker.socket -p $DOCKER_ROOT/etc/systemd/system

(Optional) for /opt/docker:

export DOCKER_ROOT_ESCAPED=`echo $DOCKER_ROOT|sed -e 's/\//\\\\\//g'`
sed -ie "s/ExecStart=/Environment=\"LD_LIBRARY_PATH=$DOCKER_ROOT_ESCAPED\/usr\/lib64\"\nEnvironment=\"PATH=\$PATH:$DOCKER_ROOT_ESCAPED\/bin:$DOCKER_ROOT_ESCAPED\/sbin:\/sbin\"\nExecStart=$DOCKER_ROOT_ESCAPED/" $DOCKER_ROOT/etc/systemd/system/docker.service
sudo mkdir /var/lib/docker
sudo mkdir /etc/docker
sudo ln -s /opt/docker/etc/systemd/system/docker.s* /etc/systemd/system

Devicemapper as storage driver

cat << EOF > /etc/docker/daemon.json
{
  "storage-driver": "devicemapper"
}
EOF

Service reload/restart:

sudo systemctl daemon-reload
sudo systemctl start docker

Kubernetes (WIP)

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

setenforce 0
yum install -y kubelet kubeadm kubectl

sudo sed -i "s/ExecStart=\/usr\/bin/ExecStart=$DOCKER_ROOT_ESCAPED\/bin\/usr\/bin/" $DOCKER_ROOT/etc/systemd/system/kubelet.service
sudo sed -i "s/ExecStart=\/usr\/bin/ExecStart=$DOCKER_ROOT_ESCAPED\/bin\/usr\/bin/" $DOCKER_ROOT/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
sudo sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/" $DOCKER_ROOT/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
sudo sed -i "s/KUBELET_EXTRA_ARGS/KUBELET_EXTRA_ARGS --fail-swap-on=false/" $DOCKER_ROOT/etc/systemd/system/kubelet.service.d/10-kubeadm.conf

# Weave net CNI addon
export kubever=$(kubectl version | base64 | tr -d '\n')
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"


export K8S_ROOT=/opt/kubernetes

# Master to check token
sudo $K8S_ROOT/bin/kubeadm token list

# Node to join
sudo $K8S_ROOT/bin/kubeadm join --token <TOKEN> <MASTER_IP>:<MASTER_PORT i.e. 6443> --skip-preflight-checks --discovery-token-unsafe-skip-ca-verification

# Label for nodes
$K8S_ROOT/bin/kubectl label node  <NODE_HOSTNAME> node-role.kubernetes.io/node=