Zeppelin in Kubernetes

Step to step guide to deploy Zeppelin for multi users in Kubernetes

Objective

Deploy Zeppelin:0.9.0 in K8

Authenticate Zeppelin UI via Apache Shiro JDBC realm

Repo

Code being discussed can be found in Github

Steps to meet the objective

Enable Kubernetes

Follow below link to run K8 on Mac.

Running Kubernetes Dashboard On Docker-On-Mac

Build the docker image

docker build . -t sg/zeppelin:latest

Dockerfile

#https://hub.docker.com/r/apache/zeppelin/dockerfile
ARG ZEPPELIN_IMAGE=apache/zeppelin:0.9.0
FROM ${ZEPPELIN_IMAGE}

WORKDIR ${Z_HOME}

USER root
ADD /shiro.ini ${Z_HOME}/conf/
#Required for local set up. Overriding so as to make imagePullPolicy as Never
ADD /100-interpreter-spec.yaml ${Z_HOME}/k8s/interpreter/

#Once ZEPPELIN-5192 is resolved, we can remove the below command. 
ADD /credentials.json ${Z_HOME}/conf/

#Required for starting the Zeppelin server. Without this Shiro won't be able to set the env configuration.
ADD https://repo1.maven.org/maven2/org/postgresql/postgresql/42.2.5/postgresql-42.2.5.jar  ${Z_HOME}/lib
ENV CLASSPATH=${Z_HOME}/lib/postgresql-42.2.5.jar:${CLASSPATH}
RUN chmod 775 ${Z_HOME}/lib/postgresql-42.2.5.jar

USER 1000

ENTRYPOINT [ "/usr/bin/tini", "--" ]
WORKDIR ${Z_HOME}
CMD ["bin/zeppelin.sh"]

Why we need a custom image?

We need to provide Zeppelin UI Authentication via Apache Shiro. So need to include shiro.ini as part of the docker image.

Why we need to override 100-interpreter-spec.yaml?

Since the docker image sg/zeppelin:latest is not in docker hub, we need to override the file to make imagePolicy as never

Why credentials.json is part of the docker image?

This should get resolved with ZEPPELIN-5192

Build the stack for Kubernetes

This will build the postgresql image with Zeppelin database with following tables:

1. zeppelin.zeppelin_users
2. zeppelin.zeppelin_user_roles

The tables will populated with user zeppelin_ui_user and encrypted password Welcome123

docker-compose -f docker-compose-zeppelin.yaml build

Push the stack to kubernetes

docker stack deploy --compose-file docker-compose-zeppelin.yaml  --orchestrator kubernetes zeppelin

You can connect the database with zeppelin_user/Password123

Pre populated user

PostgreSQL deployed in K8

Apply the kubernetes manifest file

kubectl apply -f zeppelin-server-manifest.yaml

Why custom manifest file?

1. ZEPPELIN-5191: The manifest provided by Zeppelin is pointing to wrong docker image.
2. We need to use a custom docker image.

Open the proxy terminal for Zeppelin UI

Open a terminal and execute

kubectl port-forward deployment/zeppelin-server 8080:8080

Access the Zeppelin UI

Go to http://localhost:8080/ and try zeppelin_ui_user/Welcome123

Creating a new user

Generate Password

Execute `java -jar shiro-tools-hasher-1.5.0-cli.jar -p` and enter the password that you want to hash.

Insert the user in zeppelin.zeppelin_users table

INSERT INTO zeppelin.zeppelin_users (id, username, password, created_at, updated_at) VALUES (DEFAULT, 'test_ui', '$shiro1$SHA-256$500000$8FQTUt1U4IHPv3CZgs2k3A==$n8ZSN4zc6PJ4fMs9el6i1sOVObCez0mryQ5tNjFZ2TA=', DEFAULT, DEFAULT);

Assign a role to the user.

INSERT INTO zeppelin.zeppelin_user_roles (id, username, role_name, created_at, updated_at) VALUES (DEFAULT, 'test_ui', 'admin', DEFAULT, DEFAULT);

Hope this was useful :)