Using the wildfly-cloud-tests project as an example to see how to deploy WildFly based project in cloud based environment
The WildFly community has provided a project called wildfly-cloud-tests:
- https://github.com/wildfly-extras/wildfly-cloud-tests.git
This project can be used as a sample project showing how to deploy WildFly based projects in a cloud based environment. In this article, I’d like to introduce the basic usage of the project and show how to run the project in a local environment.
By default, this project supports running on the local minikube environment or the OpenShift environment. In this article, I’ll use my local minikube environment backed with Docker engine, and I assume you have installed and started minikube together with Docker.
Here is my local minikube running status:
➤ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
And here is my minikube container running on the Docker:
➤ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
cc2a261ce55c kicbase/stable:v0.0.39 "/usr/local/bin/entr…" About a minute ago Up About a minute 127.0.0.1:62255->22/tcp, 127.0.0.1:62256->2376/tcp, 127.0.0.1:62258->5000/tcp, 127.0.0.1:62259->8443/tcp, 127.0.0.1:62257->32443/tcp minikube
Note: In the README file of the project, it says your minikube instance needs at least 4gb to run the example, so please use the following command to setup your minikube instance:
➤ minikube start --memory='4gb'
After minikube and Docker are setup properly, we need to build the project properly. There is a directory called images in the project:
➤ pwd
/Users/weli/works/wildfly-cloud-tests
➤ ls images
cloud-server datasources elytron-oidc-client microprofile-reactive-messaging-kafka pom.xml web-clustering
And in the images directory there are several images that are needed to run the project, so I recommend building them first. Entering the directory and run the following build and install command:
➤ cd images/
➤ mvn install -Pimages
And it will start to build these images. Please note the Docker service must be running during the build process. Because it needs docker to build these images. If everything goes fine, these images are built:
[INFO] wildfly-cloud-test-images .......................... SUCCESS [ 2.517 s]
[INFO] image-cloud-server ................................. SUCCESS [ 8.680 s]
[INFO] image-microprofile-reactive-messaging-kafka ........ SUCCESS [ 8.145 s]
[INFO] image-cloud-server-with-postgresql-datasource ...... SUCCESS [ 7.696 s]
[INFO] image-cloud-server-with-elytron-oidc-client ........ SUCCESS [ 10.779 s]
[INFO] image-cloud-server-with-web-clustering ............. SUCCESS [ 12.823 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 50.867 s
[INFO] Finished at: 2023-06-04T18:09:41+08:00
[INFO] ------------------------------------------------------------------------
And if we check the docker image list, then we can see these images are already in the local image repository:
➤ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
wildfly-cloud-test-image/image-cloud-server-with-web-clustering latest a3082c9de498 3 days ago 545MB
wildfly-cloud-test-image/image-cloud-server-with-elytron-oidc-client latest 7813cc26f915 3 days ago 538MB
wildfly-cloud-test-image/image-cloud-server-with-postgresql-datasource latest ff60c7f09b4a 3 days ago 539MB
wildfly-cloud-test-image/image-microprofile-reactive-messaging-kafka latest 85f450ee812c 3 days ago 550MB
wildfly-cloud-test-image/image-cloud-server latest 56455bc57b9a 3 days ago 538MB
After the above images are built, then we can build the common module in the project:
➤ pwd
/Users/weli/works/wildfly-cloud-tests
➤ ls common/
pom.xml src
The above module contains some basic classes that are needed by the test suites, here is the class diagram of the above module:
From the above class diagram, we can see there are some basic utilities for the project to interoperate the WildFly instance, Kubernetes and Openshift runtime. To build the module, use the following command to do so:
➤ pwd
/Users/weli/works/wildfly-cloud-tests/common
➤ mvn install
If everything goes fine, the module will get built. Until now, we have built the necessary components for the testsuite to run. In the next step, we can play with the tests in the project as an example.
Running the test suite
After we have built the images and the common module, now we can start to run the test suite. Before running the tests, we need to do some preparations. As noted in the README file of the project, the registry add-on need to be added into minikube. Use the following command to do so:
$ minikube addons enable registry
When installing the add-on, minikube will generate some warning message about the used port:
╭──────────────────────────────────────────────────────────────────────────────────────────────────────╮
│ │
│ Registry addon with docker driver uses port 62258 please use that instead of default port 5000 │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────────────╯
You can ignore this message and go ahead with the installation. After the add-on is installed, we can use the following command to verify its running status:
➤ minikube addons list | grep registry
| registry | minikube | enabled ✅ | Google |
| registry-aliases | minikube | disabled | 3rd party (unknown) |
| registry-creds | minikube | disabled | 3rd party (UPMC Enterprises) |
According to the above command output, we can see the registry add-on is started. According to the README file of the project, the next step is to forward the port of the registry service. The method to achieve the goal varies between operating systems, and the README file has introduced the ways to do the port forwarding on different operating systems. Because I’m using the MacOS operating system, so I used the following command introduced in README file, and together with its output:
$ docker run --rm -it --network=host alpine ash -c "apk add socat && socat TCP-LISTEN:5000,reuseaddr,fork TCP:$(minikube ip):5000"
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/main/x86_64/APKINDEX.tar.gz
fetch https://dl-cdn.alpinelinux.org/alpine/v3.18/community/x86_64/APKINDEX.tar.gz
(1/4) Installing ncurses-terminfo-base (6.4_p20230506-r0)
(2/4) Installing libncursesw (6.4_p20230506-r0)
(3/4) Installing readline (8.2.1-r1)
(4/4) Installing socat (1.7.4.4-r1)
Executing busybox-1.36.0-r9.trigger
OK: 9 MiB in 19 packages
After the above container is created, we can see these containers are running:
➤ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c4a4a2d367e0 alpine "ash -c 'apk add soc…" 31 minutes ago Up 31 minutes confident_jang
cc2a261ce55c kicbase/stable:v0.0.39 "/usr/local/bin/entr…" 7 hours ago Up 7 hours 127.0.0.1:62255->22/tcp, 127.0.0.1:62256->2376/tcp, 127.0.0.1:62258->5000/tcp, 127.0.0.1:62259->8443/tcp, 127.0.0.1:62257->32443/tcp minikube
From the above command output, we can see one container is the minikube instance, and the other is the created container used for port forwarding. After all these preparations are done, now we can start to run one of the tests to see if the whole structure is running properly. I will choose the env-vars-override-model as the running example here:
$ pwd
/Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model
To run the above test, entering the directory and run the following command:
➤ mvn integration-test
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for org.wildfly.cloud-tests:wildfly-cloud-tests-core-env-vars-override-model:war:1.0-SNAPSHOT
[WARNING] 'parent.relativePath' of POM org.wildfly.cloud-tests:wildfly-cloud-tests-parent:1.0-SNAPSHOT (/Users/weli/works/wildfly-cloud-tests/pom.xml) points at ge.ol:back-starter instead of org.jboss:jboss-parent, please verify your project structure @ org.wildfly.cloud-tests:wildfly-cloud-tests-parent:1.0-SNAPSHOT, /Users/weli/works/wildfly-cloud-tests/pom.xml, line 26, column 13
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] --< org.wildfly.cloud-tests:wildfly-cloud-tests-core-env-vars-override-model >--
[INFO] Building wildfly-cloud-tests-core-env-vars-override-model 1.0-SNAPSHOT
[INFO] --------------------------------[ war ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-java-version) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M2:enforce (enforce-maven-version) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO]
[INFO] --- buildnumber-maven-plugin:1.4:create (get-scm-revision) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Executing: /bin/sh -c cd '/Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model' && 'git' 'rev-parse' '--verify' 'HEAD'
[INFO] Working directory: /Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model
[INFO] Storing buildNumber: c0b0a6defd404b0dae0e858e1b928c010a3405a9 at timestamp: 1685894386643
[INFO] Storing buildScmBranch: main
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:resources (default-resources) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1-jboss-1:compile (default-compile) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:3.1.0:testResources (default-testResources) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1-jboss-1:testCompile (default-testCompile) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-war-plugin:3.2.3:war (default-war) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Packaging webapp
[INFO] Assembling webapp [wildfly-cloud-tests-core-env-vars-override-model] in [/Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/target/ROOT]
[INFO] Processing war project
[INFO] Copying webapp resources [/Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/src/main/webapp]
[INFO] Webapp assembled in [65 msecs]
[INFO] Building war: /Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/target/ROOT.war
[INFO]
[INFO] --- maven-source-plugin:3.1.0:jar-no-fork (attach-sources) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Building jar: /Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/target/ROOT-sources.jar
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M7:integration-test (default) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
[INFO] Found version:1.26
[INFO] Running org.wildfly.test.cloud.env.vars.override.EnvVarsOverrideTestCaseIT
[INFO] Starting project at /Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model
Apply test resources from:/Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model/target/classes/META-INF/dekorate/kubernetes.yml
[INFO] Performing docker build.
#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s
#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 202B 0.0s done
#2 DONE 0.0s
#3 [internal] load metadata for docker.io/wildfly-cloud-test-image/image-cloud-server:latest
#3 DONE 0.0s
#4 [internal] load build context
#4 transferring context: 9.30kB done
#4 DONE 0.0s
#5 [1/3] FROM docker.io/wildfly-cloud-test-image/image-cloud-server:latest
#5 CACHED
#6 [2/3] COPY --chown=jboss:root target/ROOT.war /opt/server/standalone/deployments
#6 DONE 0.0s
#7 [3/3] RUN chmod -R ug+rwX /opt/server
#7 DONE 2.0s
#8 exporting to image
#8 exporting layers
#8 exporting layers 1.1s done
#8 writing image sha256:4c97962fa77a613c44023781378ccc06174239248af3066cae7945785dfa8865 done
#8 naming to localhost:5000/weli/wildfly-cloud-tests-core-env-vars-override-model:1.0-SNAPSHOT done
#8 DONE 1.1s
[INFO] Performing docker push.
The push refers to repository [localhost:5000/weli/wildfly-cloud-tests-core-env-vars-override-model]
0a125b9a066b: Preparing
051b380e89d8: Preparing
ad254ca50ee7: Preparing
4fc9e93f9cc1: Preparing
55ea6d5a354e: Preparing
55ea6d5a354e: Layer already exists
4fc9e93f9cc1: Layer already exists
ad254ca50ee7: Layer already exists
051b380e89d8: Pushed
0a125b9a066b: Pushed
1.0-SNAPSHOT: digest: sha256:2de080c58831eaa654b875bb2f45b8eacb836087f36b74863059776a53a2aa1a size: 1377
[INFO] Created: Deployment name:wildfly-cloud-tests-core-env-vars-override-model.
[INFO] Created: Service name:wildfly-cloud-tests-core-env-vars-override-model.
[INFO] Waiting until ready (500000 ms)...
[INFO] Waited: 21519 ms.
[INFO] Found version:1.26
[INFO] Found version:1.26
[INFO] Cleaning up...
[INFO] Deleting: Deployment name:wildfly-cloud-tests-core-env-vars-override-model. Deleted:[StatusDetails(causes=[], group=apps, kind=deployments, name=wildfly-cloud-tests-core-env-vars-override-model, retryAfterSeconds=null, uid=1c2cdb62-8636-49a5-ab38-fd40e557d15b, additionalProperties={})]
[INFO] Deleting: Service name:wildfly-cloud-tests-core-env-vars-override-model. Deleted:[StatusDetails(causes=[], group=v1, kind=Service, name=wildfly-cloud-tests-core-env-vars-override-model, retryAfterSeconds=null, uid=57b7e9ce-c12e-48e6-8970-46e653dbcaad, additionalProperties={})]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 65.016 s - in org.wildfly.test.cloud.env.vars.override.EnvVarsOverrideTestCaseIT
[INFO]
[INFO] Results:
[INFO]
[INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0
[INFO]
[INFO]
[INFO] --- maven-failsafe-plugin:3.0.0-M7:verify (default) @ wildfly-cloud-tests-core-env-vars-override-model ---
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:10 min
[INFO] Finished at: 2023-06-05T00:00:55+08:00
[INFO] ------------------------------------------------------------------------
weli@192:~/w/w/t/c/env-vars-override-model|main⚡?
➤ pwd 00:00:56
/Users/weli/works/wildfly-cloud-tests/tests/core/env-vars-override-model
weli@192:~/w/w/t/c/env-vars-override-model|main⚡?
➤
From the above command output, we can see during the test building process, it uses a tool called dekorate1 to provision the docker image build and generates the k8s config file. The tool used the image-cloud-server image as the basic image and copy the test resources into the new image, and then it generated the k8s config file and start pods for us. And then the tests are running inside the k8s pods, and finally the test passed.
Until now, we have completed all the steps to play with the project. In this article I just introduce the basic usages of the project. In the later articles, I will go into details of the design of the project and introduce the tools used in this project.