In this blog I’m going to show you how to create a performance monitoring solution on the go.
Total code can be found in github repo
Create a docker compose file which has jmeter,influxdb and grafana services to automate performance execution and monitor the live results from grafana using influxdb timeseries database storage
With a single docker-compose up -d command we can setup our jmeter execution and monitor the live results in grafana and saves the raw and html report of the jmeter execution in your host machine
Create a folder under which docker-compose file resides and add grafana,influxdb and jmeter folder to store the dockerfile and concerned config and data files accordingly on the github repo
For setting up only jmeter docker image please refer the below links https://performanceengineeringsite.wordpress.com/2020/02/01/setup-jmeter-in-linux-docker-container-with-all-plugins/
https://github.com/manojkumar542/Jmeter-docker
Basic of docker-compose file. Tweak the docker-compose file according to your need and always go with latest versions of images.
<strong>Docker compose file</strong>
version: '3.8'
services:
influxdb:
image: 'influxdb:1.8.9-alpine'
container_name: influxdb
restart: always
ports:
- '8086:8086'
networks:
- monitoring
environment:
- INFLUXDB_DB=jmeter
volumes:
- influxdb-storage:/var/lib/influxdb
grafana:
build:
context: ./grafana
image: 'grafana:8.1.0'
container_name: grafana
ports:
- '3000:3000'
restart: always
networks:
- monitoring
environment:
- GF_SECURITY_ADMIN_USER=admin
- GF_SECURITY_ADMIN_PASSWORD=grafana
- GF_INSTALL_PLUGINS=grafana-clock-panel,briangann-gauge-panel,natel-plotly-panel,grafana-simple-json-datasource
volumes:
- grafana-volume:/etc/grafana/provisioning
- grafana-data:/var/lib/grafana
depends_on:
- influxdb
jmeter:
build:
context: ./jmeter
image: 'jmeter-docker:latest'
container_name: jmeter
restart: always
networks:
- monitoring
volumes:
- ./jmeter:/results
depends_on:
- influxdb
- grafana
networks:
monitoring:
volumes:
influxdb-storage:
grafana-data:
grafana-volume:
Steps to be followed to create Docker Compose Yaml file
1. Create customized docker files for jmeter & grafana and place in each respective folders and for influxdb can be directly pull it from dockerhub.
2. Create a Network to connect multiple containers to establish connection between them and default network is bridge and just define it in the yaml no need to explicitly create it.
3. Create volumes such as named volumes to persist the data whenever there are any unseen container exists and we can use volumes to get the data back just define it in Yaml docker will create for you
4. Create environment variables to pass it to the runnning containers such as ex: creating TSDB with name Jmeter in influxdb before running jmeter executions. INFLUXDB_DB:jmeter
5. Use docker-build to build the docker image from the specific build context folder where dockerfile resides to get the latest changes into docker image.
6. Finally just run docker-compose up -d to run all the services defined in docker-compose.yaml file in detached mode.
Yaml files grafana and automatic creation of Datasource and Dashboard
FROM grafana/grafana:8.1.0
ENV GF_SECURITY_ADMIN_USER=admin
ENV GF_SECURITY_ADMIN_PASSWORD=grafana
ENV GF_INSTALL_PLUGINS=grafana-clock-panel,briangann-gauge-panel,natel-plotly-panel,grafana-simple-json-datasource
# automatically provisions datasources and dashboards in grafana
ADD ./provisioning /etc/grafana/provisioning
# Add configuration file
#ADD ./grafana.ini /etc/grafana/grafana.ini
Grafana Datasource Yaml format
apiVersion: 1
datasources:
- name: 'Influx-Jmeter'
type: influxdb
access: proxy
database: jmeter
url: http://influxdb:8086
isDefault: true
editable: true
Grafana Dashboard format download here
Now go to docker-compose file folder and run docker-compose up -d
Use docker exec -it containerid /bin/sh to connect to containers and check the influxdb to check the database with name jmeter is created or not same way you can do it for grafana to check the dashboards and datasources are referring correctly from the copied files.
Since i have enabled port mapping to expose container port to localhost port i can able to access both grafana and influxdb using httP://localhost:portnumber
But in containers they use internal ips to communicate and internally container name DNS is mapped to ip that’s the reason i have used container names to configure datasource connection in influxdb url in grafana
Using docker-compose build by providing build context we can build the image from the specific docker file folder using docker-compose yaml itself as below
jmeter:
build:
context: ./jmeter
image: 'jmeter-docker:latest'
container_name: jmeter
restart: always
networks:
- monitoring
volumes:
- ./jmeter:/results
depends_on:
- influxdb
- grafana
Context - where exactly the dockerfile resides and build and tag with the image name specified in our case it's <strong>'jmeter-docker:latest'</strong>
depends_on - make sure services in the section specified will be starting and running which are dependent on the current service.
Type influx command and influx shell get's opened show databases - shows list of databases use databasename - connect to specified database show measurements - shows all the tables in the database select * from measurement - list of all data in the table See the above image for reference.
Total code repo can be found here
Performance Engineering & DevOps Guide 