Skip to content

Commit

Permalink
chore(report): Week 9 and 10 project report CI Scanner
Browse files Browse the repository at this point in the history
Signed-off-by: Rajul Jha <rajuljha49@gmail.com>
  • Loading branch information
rajuljha committed Aug 13, 2024
1 parent 78bfb22 commit 6f72d54
Show file tree
Hide file tree
Showing 3 changed files with 90 additions and 0 deletions.
44 changes: 44 additions & 0 deletions docs/2024/ci-scanner/updates/2024-08-01.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: Week 9
author: Rajul Jha
tags: [gsoc24, CI]
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
-->

# Week 9
*(July, 26 2024 - Aug 1, 2024)*

## Meeting 1
*(July 31, 2024)*

## Attendees

* [Rajul Jha](https://github.com/rajuljha)
* [Gaurav](https://github.com/GMishx)
* [Shaheem Azmal](https://github.com/shaheemazmalmmd)
* [Kaushlendra](https://github.com/Kaushl2208)
* [Avinal Kumar](https://github.com/avinal)

## Discussions

* Discussed potential issues that might arise with the approach we had in mind:
* Current package-url python tool doesn’t support download urls for PyPi, PHP, and cocoapods package managers. They are working on providing the support in this [PR](https://github.com/nexB/fetchcode/pull/119) and this [issue](https://github.com/nexB/fetchcode/issues/116)
* Similarly it also does not support golang packages download urls, however the repository urls are available. Since golang packaging is quite complicated. [Here](https://github.com/package-url/packageurl-python/pull/113) is where they are tracking it.
* The GH Actions for some of the cyclone dx-tools uses outdated and unmaintained versions of their binaries.
* All the GH Actions just use their respective cli tools to generate the SBOM’s ultimately. We can remove dependency from it completely by using cli tools directly inside our environment. Since these dependencies might be unnecessary.
* [Gaurav](https://github.com/GMishx) suggested we create our own Github Actions for generating the Software BOMs in a language dependant manner for the packages using cyclonedx tools under the hood.

## Work Done

* Worked out a plan for how the dependencies scanning would be done inside the CI workflow.

![Screenshot](/img/ci/Download-deps-fossology.png)
* Started working on the new Github Action for scanning Python Dependencies using [CycloneDX Python BOM tool](https://cyclonedx-bom-tool.readthedocs.io/en/latest/)

## Planning for next week
* Complete testing the Github Action and successfully generate SBOMs for python projects.
* Need to figure out how to extract the download-urls from the generated BOMs.
46 changes: 46 additions & 0 deletions docs/2024/ci-scanner/updates/2024-08-08.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
title: Week 10
author: Rajul Jha
tags: [gsoc24, CI]
---
<!--
SPDX-License-Identifier: CC-BY-SA-4.0
SPDX-FileCopyrightText: 2024 Rajul Jha <rajuljha49gmail.com>
-->

# Week 10
*(Aug, 02 2024 - Aug 08, 2024)*

## Meeting 1
*(Aug 07, 2024)*

## Attendees

* [Rajul Jha](https://github.com/rajuljha)
* [Gaurav](https://github.com/GMishx)
* Katharina

## Discussions
* To tackle the issue with the python-packageurl tool not supporting `PyPI` and other registries, [Gaurav](https://github.com/GMishx) suggested to try out new tools like [ORT Downloader](https://oss-review-toolkit.org/ort/docs/tools/downloader), [pypi-simple](https://pypi.org/project/pypi-simple/) and REST API for [warehouse](https://warehouse.pypa.io/api-reference/json.html)
* Talked about tagging and releasing the [fossology-action](https://github.com/fossology/fossology-action) after renaming it to something like **FOSSOps**

## Work Done

* Completed the Github Action for generating SBOMs for python packages. Their are four ways in which we can create an SBOM:
* From Python Virtual Environment: `cyclonedx-py environment`
* From Pipfile and Pipfile.lock: `cyclonedx-py pipenv`
* From pyproject.toml and poetry.lock: `cyclonedx-py poetry`
* From requirements.txt file: `cyclonedx-py requirements`
* Currently the actions looks for these files in the root folder of the repo from where the action is triggered. In future releases, new features like custom path for the file, creating BOMs in xml foramt etc. can be provided.
* It creates Software Bill of Materials in [cyclonedx](https://cyclonedx.org/specification/overview/) format.
* BOMs are saves in a directory called `sbom` and for each language, currently python, the bom file is appended with the language specific identifier.
For example, for python files, the bom file will `sbom_py.json`.
* After we have the bom file, we can provide a new cli flag to `fossologyscanner` that will take these sbom files, download the project dependecies listed in them, scan them using fossology scanners (nomos, ojo etc) and spit out the results.
* Started working on the new feature in `fossologyscanner`

## Planning for next week

* Figure out the method for getting the package metadata after trying out different approaches.
* Work on successfully downloading and scanning the packages once finalized.
* Keep an eye on making the code modular in order for easy integration of other language specific actions.
Binary file added static/img/ci/Download-deps-fossology.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 6f72d54

Please sign in to comment.