Skip to content

Commit

Permalink
Merge branch 'krkn-chaos:main' into main
Browse files Browse the repository at this point in the history
  • Loading branch information
@tsebastiani
tsebastiani authored Jun 12, 2024
2 parents 55a166c + a59f340 commit 5b52346
Show file tree
Hide file tree
Showing 51 changed files with 350 additions and 374 deletions.
5 changes: 4 additions & 1 deletion README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,10 @@ You can take advantage of the [get_docker_params.sh](get_docker_params.sh) scrip
This will take all environment variables and put them in the form "-e <var>=<value>" to make a long string that can get passed to the command

For example:
`docker run $(./get_docker_params.sh) --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:power-outages`
`docker run $(./get_docker_params.sh) --net=host -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/redhat-chaos/krkn-hub:power-outages`

**TIP**: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:
```kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v ~kubeconfig:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:<scenario>```

### Adding New Scenarios/Testing Changes

Expand Down
21 changes: 9 additions & 12 deletions application-outages/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,15 @@

FROM quay.io/krkn-chaos/krkn:latest

ENV KUBECONFIG /root/.kube/config

# Install dependencies
RUN yum install -y which
ENV KUBECONFIG /home/krkn/.kube/config

# Copy configurations
COPY config.yaml.template /root/kraken/config/config.yaml.template
COPY metrics_config.yaml.template /root/kraken/config/kube_burner.yaml.template
COPY application-outages/env.sh /root/env.sh
COPY env.sh /root/main_env.sh
COPY application-outages/run.sh /root/run.sh
COPY common_run.sh /root/common_run.sh
COPY application-outages/app_outages.yaml.template /root/kraken/scenarios/app_outage.yaml.template
COPY config.yaml.template /home/krkn/kraken/config/config.yaml.template
COPY metrics_config.yaml.template /home/krkn/kraken/config/kube_burner.yaml.template
COPY application-outages/env.sh /home/krkn/env.sh
COPY env.sh /home/krkn/main_env.sh
COPY application-outages/run.sh /home/krkn/run.sh
COPY common_run.sh /home/krkn/common_run.sh
COPY application-outages/app_outages.yaml.template /home/krkn/kraken/scenarios/app_outage.yaml.template

ENTRYPOINT /root/run.sh
ENTRYPOINT /home/krkn/run.sh
14 changes: 7 additions & 7 deletions application-outages/run.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,21 @@
set -ex

# Source env.sh to read all the vars
source /root/main_env.sh
source /root/env.sh
source /home/krkn/main_env.sh
source /home/krkn/env.sh

ls -la /root/.kube
ls -la /home/krkn/.kube

source /root/common_run.sh
source /home/krkn/common_run.sh
checks
config_setup

# Substitute config with environment vars defined
envsubst < /root/kraken/scenarios/app_outage.yaml.template > /root/kraken/scenarios/app_outage.yaml
envsubst < /root/kraken/config/config.yaml.template > /root/kraken/config/app_outage_config.yaml
envsubst < /home/krkn/kraken/scenarios/app_outage.yaml.template > /home/krkn/kraken/scenarios/app_outage.yaml
envsubst < /home/krkn/kraken/config/config.yaml.template > /home/krkn/kraken/config/app_outage_config.yaml

# Run Kraken
cd /root/kraken
cd /home/krkn/kraken


python3.9 run_kraken.py --config=config/app_outage_config.yaml
15 changes: 6 additions & 9 deletions chaos-recommender/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,12 @@

FROM quay.io/krkn-chaos/krkn:latest

ENV KUBECONFIG /root/.kube/config
ENV KUBECONFIG /home/krkn/.kube/config

# Install dependencies
RUN yum install -y which wget
RUN wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /usr/bin/yq && chmod +x /usr/bin/yq
# Copy configurations
COPY env.sh /root/main_env.sh
COPY chaos-recommender/run.sh /root/run.sh
COPY chaos-recommender/env.sh /root/env.sh
COPY common_run.sh /root/common_run.sh
COPY env.sh /home/krkn/main_env.sh
COPY chaos-recommender/run.sh /home/krkn/run.sh
COPY chaos-recommender/env.sh /home/krkn/env.sh
COPY common_run.sh /home/krkn/common_run.sh

ENTRYPOINT /root/run.sh
ENTRYPOINT /home/krkn/run.sh
2 changes: 1 addition & 1 deletion chaos-recommender/run.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

set -ex

ROOT_FOLDER="/root"
ROOT_FOLDER="/home/krkn"
KUBECONFIG_PATH="$ROOT_FOLDER/.kube/config"
KRAKEN_FOLDER="$ROOT_FOLDER/kraken"

Expand Down
4 changes: 2 additions & 2 deletions common_run.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ check_cluster_version() {
set_kubernetes_platform() {
if ! kubectl get clusterversion;
then
yq -i '.kraken.distribution="kubernetes"' /root/kraken/config/config.yaml.template
yq -i '.kraken.distribution="kubernetes"' /home/krkn/kraken/config/config.yaml.template
fi
}

Expand All @@ -57,7 +57,7 @@ checks() {

# Config substitutions
config_setup(){
envsubst < /root/kraken/config/kube_burner.yaml.template > /root/kraken/config/kube_burner.yaml
envsubst < /home/krkn/kraken/config/kube_burner.yaml.template > /home/krkn/kraken/config/kube_burner.yaml
}

setup_arcaflow_env(){
Expand Down
23 changes: 10 additions & 13 deletions container-scenarios/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,15 @@

FROM quay.io/krkn-chaos/krkn:latest

ENV KUBECONFIG /root/.kube/config

# Install dependencies
RUN yum install -y which
ENV KUBECONFIG /home/krkn/.kube/config

# Copy configurations
COPY config.yaml.template /root/kraken/config/config.yaml.template
COPY metrics_config.yaml.template /root/kraken/config/kube_burner.yaml.template
COPY container-scenarios/env.sh /root/env.sh
COPY env.sh /root/main_env.sh
COPY container-scenarios/run.sh /root/run.sh
COPY container-scenarios/container_scenario.yaml.template /root/kraken/scenarios/container_scenario.yaml.template
COPY common_run.sh /root/common_run.sh

ENTRYPOINT /root/run.sh
COPY config.yaml.template /home/krkn/kraken/config/config.yaml.template
COPY metrics_config.yaml.template /home/krkn/kraken/config/kube_burner.yaml.template
COPY container-scenarios/env.sh /home/krkn/env.sh
COPY env.sh /home/krkn/main_env.sh
COPY container-scenarios/run.sh /home/krkn/run.sh
COPY container-scenarios/container_scenario.yaml.template /home/krkn/kraken/scenarios/container_scenario.yaml.template
COPY common_run.sh /home/krkn/common_run.sh
USER krkn
ENTRYPOINT /home/krkn/run.sh
12 changes: 6 additions & 6 deletions container-scenarios/run.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,19 +3,19 @@
set -ex

# Source env.sh to read all the vars
source /root/main_env.sh
source /root/env.sh
source /home/krkn/main_env.sh
source /home/krkn/env.sh

source /root/common_run.sh
source /home/krkn/common_run.sh
checks
config_setup

# Substitute config with environment vars defined
envsubst < /root/kraken/scenarios/container_scenario.yaml.template > /root/kraken/scenarios/container_scenario.yaml
envsubst < /root/kraken/config/config.yaml.template > /root/kraken/config/container_scenario_config.yaml
envsubst < /home/krkn/kraken/scenarios/container_scenario.yaml.template > /home/krkn/kraken/scenarios/container_scenario.yaml
envsubst < /home/krkn/kraken/config/config.yaml.template > /home/krkn/kraken/config/container_scenario_config.yaml

# Run Kraken
cd /root/kraken
cd /home/krkn/kraken

cat config/container_scenario_config.yaml

Expand Down
13 changes: 8 additions & 5 deletions docs/application-outages.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,20 +6,23 @@ This scenario disrupts the traffic to the specified application to be able to un
If enabling [Cerberus](https://github.com/krkn-chaos/krkn#kraken-scenario-passfail-criteria-and-report) to monitor the cluster and pass/fail the scenario post chaos, refer [docs](https://github.com/krkn-chaos/krkn-hub/tree/main/docs/cerberus.md). Make sure to start it before injecting the chaos and set `CERBERUS_ENABLED` environment variable for the chaos injection container to autoconnect.

```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
$ podman logs -f <container_name or container_id> # Streams Kraken logs
$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
```

```
$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
OR
$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
$ docker logs -f <container_name or container_id> # Streams Kraken logs
$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
```

**TIP**: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:
```kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v ~kubeconfig:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:<scenario>```

#### Supported parameters

The following environment variables can be set on the host running the container to tweak the scenario/faults being injected:
Expand All @@ -39,9 +42,9 @@ BLOCK_TRAFFIC_TYPE | It can be Ingress or Egress or Ingress, Egress ( needs

**NOTE** Defining the `NAMESPACE` parameter is required for running this scenario while the pod_selector is optional. In case of using pod selector to target a particular application, make sure to define it using the following format with a space between key and value: "{key: value}".

**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/root/kraken/config/metrics-aggregated.yaml` and `/root/kraken/config/alerts`. For example:
**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/home/krkn/kraken/config/metrics-aggregated.yaml` and `/home/krkn/kraken/config/alerts`. For example:
```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/home/krkn/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/home/krkn/kraken/config/alerts -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:application-outages
```

#### Demo
Expand Down
5 changes: 3 additions & 2 deletions docs/chaos-recommender.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,10 @@ This tool profiles an application and gathers telemetry data such as CPU, Memory
#### Run

```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/root/.kube/config:Z -e NAMESPACE=<NAMESPACE> -e PROMETHEUS_ENDPOINT=<PROMETHEUS_ENDPOINT> -e PROMETHEUS_TOKEN=<PROMETHEUS_TOKEN> quay.io/krkn-chaos/krkn-hub:chaos-recommender
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/home/krkn/.kube/config:Z -e NAMESPACE=<NAMESPACE> -e PROMETHEUS_ENDPOINT=<PROMETHEUS_ENDPOINT> -e PROMETHEUS_TOKEN=<PROMETHEUS_TOKEN> quay.io/krkn-chaos/krkn-hub:chaos-recommender
```

**TIP**: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:
```kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v ~kubeconfig:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:<scenario>```
#### Supported parameters

See list of variables that apply to all scenarios [here](all_scenarios_env.md) that can be used/set in addition to these scenario specific variables
Expand Down
12 changes: 7 additions & 5 deletions docs/container-scenarios.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,20 +5,22 @@ This scenario disrupts the containers matching the label in the specified namesp
If enabling [Cerberus](https://github.com/krkn-chaos/krkn#kraken-scenario-passfail-criteria-and-report) to monitor the cluster and pass/fail the scenario post chaos, refer [docs](https://github.com/redhat-chaos/krkn-hub/tree/main/docs/cerberus.md). Make sure to start it before injecting the chaos and set `CERBERUS_ENABLED` environment variable for the chaos injection container to autoconnect.

```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ podman logs -f <container_name or container_id> # Streams Kraken logs
$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
```

```
$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
OR
$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ docker logs -f <container_name or container_id> # Streams Kraken logs
$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
```

**TIP**: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:
```kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v ~kubeconfig:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:<scenario>```
#### Supported parameters

The following environment variables can be set on the host running the container to tweak the scenario/faults being injected:
Expand All @@ -37,9 +39,9 @@ CONTAINER_NAME | Name of the container to disrupt
ACTION | kill signal to run. For example 1 ( hang up ) or 9 | 1 |
EXPECTED_RECOVERY_TIME | Time to wait before checking if all containers that were affected recover properly | 60 |

**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/root/kraken/config/metrics-aggregated.yaml` and `/root/kraken/config/alerts`. For example:
**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/home/krkn/kraken/config/metrics-aggregated.yaml` and `/home/krkn/kraken/config/alerts`. For example:
```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/home/krkn/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/home/krkn/kraken/config/alerts -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
```


Expand Down
11 changes: 6 additions & 5 deletions docs/network-chaos.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,19 @@ This scenario introduces network latency, packet loss, bandwidth restriction in
If enabling [Cerberus](https://github.com/krkn-chaos/krkn#kraken-scenario-passfail-criteria-and-report) to monitor the cluster and pass/fail the scenario post chaos, refer [docs](https://github.com/redhat-chaos/krkn-hub/tree/main/docs/cerberus.md). Make sure to start it before injecting the chaos and set `CERBERUS_ENABLED` environment variable for the chaos injection container to autoconnect.

```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:network-chaos
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:network-chaos
$ podman logs -f <container_name or container_id> # Streams Kraken logs
$ podman inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
```

```
$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:network-chaos
$ docker run -e <VARIABLE>=<value> --net=host -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:network-chaos
$ docker logs -f <container_name or container_id> # Streams Kraken logs
$ docker inspect <container-name or container-id> --format "{{.State.ExitCode}}" # Outputs exit code which can considered as pass/fail for the scenario
```

**TIP**: Because the container runs with a non-root user, ensure the kube config is globally readable before mounting it in the container. You can achieve this with the following commands:
```kubectl config view --flatten > ~/kubeconfig && chmod 444 ~/kubeconfig && docker run $(./get_docker_params.sh) --name=<container_name> --net=host -v ~kubeconfig:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:<scenario>```

#### Supported parameters

Expand Down Expand Up @@ -57,7 +58,7 @@ NETWORK_PARAMS | latency, loss and bandwidth are the three supported network
WAIT_DURATION | Ensure that it is at least about twice of test_duration | 300 |


**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/root/kraken/config/metrics-aggregated.yaml` and `/root/kraken/config/alerts`. For example:
**NOTE** In case of using custom metrics profile or alerts profile when `CAPTURE_METRICS` or `ENABLE_ALERTS` is enabled, mount the metrics profile from the host on which the container is run using podman/docker under `/home/krkn/kraken/config/metrics-aggregated.yaml` and `/home/krkn/kraken/config/alerts`. For example:
```
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/root/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/root/kraken/config/alerts -v <path-to-kube-config>:/root/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
$ podman run --name=<container_name> --net=host --env-host=true -v <path-to-custom-metrics-profile>:/home/krkn/kraken/config/metrics-aggregated.yaml -v <path-to-custom-alerts-profile>:/home/krkn/kraken/config/alerts -v <path-to-kube-config>:/home/krkn/.kube/config:Z -d quay.io/krkn-chaos/krkn-hub:container-scenarios
```
Loading

0 comments on commit 5b52346

Please sign in to comment.