Skip to content

Commit

Permalink
Merge branch 'master' into 451-enable-streaming-of-build-logs
Browse files Browse the repository at this point in the history
  • Loading branch information
munishchouhan authored Jul 22, 2024
2 parents fd9ea32 + 760f217 commit 7dedb5f
Show file tree
Hide file tree
Showing 92 changed files with 1,624 additions and 267 deletions.
1 change: 1 addition & 0 deletions .codespellignore
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
carrer
ser
40 changes: 40 additions & 0 deletions .github/workflows/typespec.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
name: Typespec_Validation

on:
push:
branches:
- '**'
paths :
- 'typespec/**'
pull_request:
types: [opened, reopened, synchronize]
paths:
- 'typespec/**'

permissions:
contents: read

jobs:
typespec_validation:
name: validate typespec files
runs-on: ubuntu-latest

steps:
- name : Checkout
uses : actions/checkout@v4

- name : Setup Node.js environment
uses : actions/setup-node@v4
with :
node-version : '20.9.0'

- name : Install tsp
run : npm install -g @typespec/compiler

- name : Validate tsp files
run : |
cd typespec
tsp install
tsp compile .
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,8 @@ scan-workspace/
.cache
site/
deployment-url.txt

#typespec
tsp-output/
node_modules/
package-lock.json
13 changes: 12 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ images.
* Augment container images i.e. dynamically add one or more container layers to existing images;
* Build container images on-demand for a given container file (aka Dockerfile);
* Build container images on-demand based on one or more Conda packages;
* Build container images on-demand based on one or more Spack packages;
* Build container images on-demand based on one or more Spack packages, Spack support will be removed in future releases;
* Build container images for a specified target platform (currently linux/amd64 and linux/arm64);
* Push and cache built containers to a user-provided container repository;
* Build Singularity native containers both using a Singularity spec file, Conda package(s) and Spack package(s);
Expand Down Expand Up @@ -108,6 +108,17 @@ container registry where the image is stored, while the instrumented layers are
'-Djdk.httpclient.HttpClient.log=requests,headers'
```

## TypeSpec API Specifications

- You can find the API specifications using (typespec)[https://github.com/microsoft/typespec] in typespec directory. Use following command to generate the API specifications.

```bash
'cd typespec'
'tsp install'
'tsp compile .'
```

- Check `typespec/tsp-output` directory for the generated API specifications.

## Related links
* [Wave command line tool](https://github.com/seqeralabs/wave-cli)
Expand Down
2 changes: 1 addition & 1 deletion VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
1.8.0
1.9.0
2 changes: 1 addition & 1 deletion build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ dependencies {
implementation("jakarta.persistence:jakarta.persistence-api:3.0.0")
api 'io.seqera:lib-mail:1.0.0'
api 'io.seqera:wave-api:0.10.0'
api 'io.seqera:wave-utils:0.12.0'
api 'io.seqera:wave-utils:0.13.1'

implementation("io.micronaut:micronaut-http-client")
implementation("io.micronaut:micronaut-jackson-databind")
Expand Down
34 changes: 34 additions & 0 deletions changelog.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,38 @@
# Wave changelog
1.9.1 - 11 Jul 2024
- Prevent hard error when launch credentials cannot be accessed [a318a855]

1.9.0 - 11 Jul 2024
- Add Typespec API definitions (#537) [32f7dd16]
- Add cache record-stats (#534) [229926e2]
- Add http 429 error to auth service retry condition [8282a492]
- Check and delete corrupted blobs cache uploads (#533) [b0c775a3]
- Deprecate the support for Spack and remove the support for it in the codebase (#550) [85a05196]
- Enable ECR authentication via AWS compute env credentials (#303) [ec895222]
- Fix Blob cache config for local dev [0d5413bf]
- Fix client cache deadlock (#547) [cc6012ff]
- Fix multiple s3clients in wave (#554) [45689500]
- Minor change in mail notification [0aba6997]
- Refactored metrics service (#549) [5e0d32ac]
- Remove buildlogs aws config to not initialize AwsS3Operations (#558) [d50e5b7a]
- Update metrics response (#536) [b6b36a97]
- Bump buildkit 0.14.0 (#528) [c54172d1]
- Bump buildkit 0.14.1 (#548) [37afb782]
- Bump trivy 0.53.0 [f27ff527]
- Bump version 1.8.3 [55b473e1]

1.8.3 - 29 Jun 2024
- Fix client cache deadlock (#547) [cc6012ff]
- Fix failing test [263b44d3]
- Add cache record-stats (#534) [229926e2]

1.8.2 - 14 Jun 2024
- Fix get image auth digest (#530) [df8ec041]

1.8.1 - 13 Jun 2024
- Fix digest warning (#526) [6e34ff0a]
- Fix image name strategy (#513) [3bd8ffb5]

1.8.0 - 10 Jun 2024
- Add breakdown per org on stats metrics when a date is specified (#517) [b24894a4]
- Replace Kaniko with Buildkit (#503) [a26f5dfe]
Expand Down
2 changes: 2 additions & 0 deletions configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ Below are the standard format for known registries, but you can change registry

### Spack configuration for wave build process

**Note**: Spack support will be removed in future releases.

Spack configuration consists of the path of its secret file, the mount path for the secret file in the spack container, and the optional S3 bucket name for the spack binary cache.

**Note**: these configuration are mandatory to support Spack in a wave installation.
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
138 changes: 114 additions & 24 deletions docs/api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ This API endpoint is deprecated in current versions of Wave.
| `containerConfig.layers.gzipSize` | The size in bytes of the the provided layer tar gzip file. |
| `containerFile` | Dockerfile used for building a new container encoded in base64 (optional). When provided, the attribute `containerImage` must be omitted. |
| `condaFile` | Conda environment file encoded as base64 string. |
| `spackFile` | Spack recipe file encoded as base64 string. |
| `spackFile` | `Deprecated` Spack recipe file encoded as base64 string. Spack support will be removed in future releases. |
| `containerPlatform` | Target container architecture of the built container, e.g., `linux/amd64` (optional). Currently only supporting amd64 and arm64. |
| `buildRepository` | Container repository where container builds should be pushed, e.g., `docker.io/user/my-image` (optional). |
| `cacheRepository` | Container repository used to cache build layers `docker.io/user/my-cache` (optional). |
Expand Down Expand Up @@ -168,7 +168,7 @@ The endpoint returns the name of the container request made available by Wave.
| `containerConfig.layers.gzipSize` | The size in bytes of the the provided layer tar gzip file. |
| `containerFile` | Dockerfile used for building a new container encoded in base64 (optional). When provided, the attribute `containerImage` must be omitted. |
| `condaFile` | Conda environment file encoded as base64 string. |
| `spackFile` | Spack recipe file encoded as base64 string. |
| `spackFile` | `Deprecated` Spack recipe file encoded as base64 string. Spack support will be removed in future releases. |
| `containerPlatform` | Target container architecture of the built container, e.g., `linux/amd64` (optional). Currently only supporting amd64 and arm64. |
| `buildRepository` | Container repository where container builds should be pushed, e.g., `docker.io/user/my-image` (optional). |
| `cacheRepository` | Container repository used to cache build layers `docker.io/user/my-cache` (optional). |
Expand All @@ -178,9 +178,9 @@ The endpoint returns the name of the container request made available by Wave.
| `towerEndpoint` | Seqera Platform service endpoint from where container registry credentials are retrieved (optional). Default `https://api.cloud.seqera.io`. |
| `towerAccessToken` | Access token of the user account granting access to the Seqera Platform service specified via `towerEndpoint` (optional). |
| `towerWorkspaceId` | ID of the Seqera Platform workspace from where the container registry credentials are retrieved (optional). When omitted the personal workspace is used. |
| `packages` | This object specifies Conda or Spack packages environment information. |
| `packages` | This object specifies Conda packages environment information. |
| `environment` | The package environment file encoded as a base64 string. |
| `type` | This represents the type of package builder. Use `SPACK` or `CONDA`. |
| `type` | This represents the type of package builder. Use `CONDA`. |
| `entries` | List of the packages names. |
| `channels` | List of Conda channels, which will be used to download packages. |
| `mambaImage` | Name of the docker image used to build Conda containers. |
Expand Down Expand Up @@ -285,13 +285,11 @@ Provides status of build against buildId passed as path variable

```json
{
serviceInfo: {
id: string,
status: string,
startTime: string,
duration: string,
succeeded: boolean
}
id: string,
status: string,
startTime: string,
duration: string,
succeeded: boolean
}
```

Expand Down Expand Up @@ -508,7 +506,7 @@ curl --location 'http://localhost:9090/v1alpha1/inspect' \
These APIs provide usage (builds and pulls) metrics of Wave for a specific date and/or a specific organization.
These APIs require basic authentication, so you must provide a username and password while calling these APIs.

All Metrics API endpoints use these query parameters (at least one is required):
All Metrics API endpoints use these query parameters:

| Name | Description | sample Value |
|------|-----------------------------------------------------------------------|--------------|
Expand All @@ -521,63 +519,125 @@ These APIs are used to retrieve metrics about container builds performed by Wave

### GET `/v1alpha2/metrics/builds`

This endpoint is used to retrieve the total number of builds performed by Wave.
This endpoint is used to retrieve the builds performed by Wave.

### Response

```json
{
count: integer
metric: "builds",
count: integer,
orgs: {
String: integer,
String: integer,
...
}
}
```

#### Examples

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/builds"
{
"metric": "builds",
"count": 18,
"orgs": {
"seqera.io": 13,
"gmail.com": 5
}
}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/builds?date=2024-04-08&org=seqera.io"
{"count":4}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/builds?date=2024-04-08"
{"count":6}
{
"metric": "builds",
"count": 8,
"orgs": {
"gmail.com": 4,
"seqera.io": 4
}
}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/builds?org=seqera.io"
{"count":4}
{
"metric": "builds",
"count": 13,
"orgs": {
"seqera.io": 13
}
}
```
### Pull Metrics API

These APIs are used to get the metrics about the container pulls through Wave.

### GET `/v1alpha2/metrics/pulls`

This endpoint is used to get the total numbers of pulls performed through Wave.
This endpoint is used to get the pulls performed through Wave.

### Response

```json
{
count: integer
metric: "pulls",
count: integer,
orgs: {
String: integer,
String: integer,
...
}
}
```

#### Examples

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/pulls"
{
"metric": "pulls",
"count": 11,
"orgs": {
"seqera.io": 7,
"gmail.com": 4
}
}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/pulls?date=2024-04-08&org=seqera.io"
{"count":5}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/pulls?date=2024-04-08"
{"count":7}
{
"metric": "pulls",
"count": 4,
"orgs": {
"seqera.io": 1,
"gmail.com": 3
}
}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/pulls?org=seqera.io"
{"count":5}
{
"metric": "pulls",
"count": 7,
"orgs": {
"seqera.io": 7
}
}
```

### Fusion Pull Metrics API
Expand All @@ -586,29 +646,59 @@ These APIs are used to get the metrics about the Fusion-based container pulls th

### GET `/v1alpha2/metrics/fusion/pulls`

This endpoint is used to get the total numbers of pulls of Fusion-based containers performed through Wave.
This endpoint is used to get the pulls of Fusion-based containers performed through Wave.

### Response

```json
{
count: integer
metric: "fusion",
count: integer,
orgs: {
String: integer,
String: integer,
...
}
}
```

#### Examples

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/fusion/pulls"
{
"metric": "fusion",
"count": 2,
"orgs": {
"seqera.io": 1,
"gmail.com": 1
}
}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/fusion/pulls?date=2024-04-08&org=seqera.io"
{"count":2}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/fusion/pulls?date=2024-04-08"
{"count":2}%
{
"metric": "fusion",
"count": 1,
"orgs": {
"gmail.com": 1
}
}
```

```shell
curl -u foo:bar "http://localhost:9090/v1alpha2/metrics/fusion/pulls?org=seqera.io"
{"count":2}
{
"metric": "fusion",
"count": 1,
"orgs": {
"seqera.io": 1
}
}
```
Loading

0 comments on commit 7dedb5f

Please sign in to comment.