test-infra

Test observability platform

Spec

• ✅ Should be able to list all the tests in the suite
• ✅ Should be able to identify owners for a given test in the suite
• ✅ Should be able to tag each test to a functional area
• ✅ Should be able to get metrics on how many tests belong to a given area
• ✅ Should be able to trigger a slack summary with each individual test failure as a child message in thread
• Should be able to calculate if a test is broken consistently
• Should be able to calculate if a test is flaky consistently

Running tests

Run all tests

./gradlew clean test --info

Run only groups belonging to a certain group

./gradlew test -DincludedGroups=smoke --info

Run tests belonging to a group but excluding few

./gradlew test -DincludedGroups=smoke -DexcludedGroups=slow --info

Run tests belonging to multiple groups

./gradlew test -DincludedGroups=smoke,regression --info

Run tests only when they belong to both the groups specified

./gradlew runTests -Dgroups=smoke,regression --info

Check test listing dashboard

cd build/resources/main

python3 -m http.server 8000

Open the generated html by navigating to:

http://localhost:8000/static/index.html

Report portal (one time setup)

brew install docker-compose

curl -LO https://raw.githubusercontent.com/reportportal/reportportal/master/docker-compose.yml

docker-compose -p reportportal up -d --force-recreate

Setup Jenkins on docker

docker pull jenkins/jenkins:lts

docker run -d -p 8090:8080 -p 50000:50000 --name jenkins -v ~/self/test-infra/jenkins:/var/jenkins_home jenkins/jenkins:lts

docker logs jenkins