From 69edfa3d469d29e2834b7652fdbc0e16160447ee Mon Sep 17 00:00:00 2001 From: rajuljha Date: Sat, 17 Aug 2024 10:27:56 +0530 Subject: [PATCH] chore(report): Add week 11 CI Scanner report Signed-off-by: Rajul Jha --- docs/2024/ci-scanner/updates/2024-08-15.md | 53 ++++++++++++++++++++++ 1 file changed, 53 insertions(+) create mode 100644 docs/2024/ci-scanner/updates/2024-08-15.md diff --git a/docs/2024/ci-scanner/updates/2024-08-15.md b/docs/2024/ci-scanner/updates/2024-08-15.md new file mode 100644 index 000000000..a8a5ab89a --- /dev/null +++ b/docs/2024/ci-scanner/updates/2024-08-15.md @@ -0,0 +1,53 @@ +--- +title: Week 11 +author: Rajul Jha +tags: [gsoc24, CI] +--- + + +# Week 11 +*(Aug, 09 2024 - Aug 15, 2024)* + +## Meeting 1 +*(Aug 14, 2024)* + +## Attendees + +* [Rajul Jha](https://github.com/rajuljha) +* [Gaurav](https://github.com/GMishx) +* [Avinal Kumar](https://github.com/avinal) +* Katharina + + +## Discussions + +* Showed the demo of the download dependencies feature to [Gaurav](https://github.com/GMishx) and got feedback for the same. +* Discussed using the `ununpack` agent for unpacking the downloaded packages rather than the current dependency on python libraries. +* Rather than taking an input directory as input to the `--sbom-path`, it would be better to take a single file as input. + * Software projects only have one `sbom` as a convention. So, if the project is using multiple languages or dependencies, then they are combined to a single sbom file, which can be passed to our scanner. + * We want to implement different parsers for different languages. So, we need a parent class of the parsers whose job is to set which parser should handle a particular component. + * We can differentiate between languages and which parser to use based on the `purl` for the component. +* Got a new feature request in order to allow scanning a particular directory by providing the path to it via CLI. + + +## Work Done + +* Implemented the feature of scanning dependencies using `fossologyscanner`. + * Introduced a module `ScanDeps` that has a Parsers file that contains specific parsers for each supported language. + * Currently, it has `PythonParser` only. Working on `NPMParser`. The job of the parsers is to take the sbom file and extract the download urls from it, + save it in a particular format like a tuple of `( , )`. + * These results are then fed to a `Downloader` class that has methods to concurrently download these packages from the urls, and then extract them + from their respective formats to a directory called `pkg_downloads`. + * The `download_concurrently` method improves the speed of downloading and exporting packages significantly by allowing multiple downloads to run on different threads parallelly. +* Started working on the new feature and improving the different parsers. + + +## Planning for next week + +* Complete the PythonParser and raise a PR for the same. +* Complete the directory scan feature and raise a PR. +* Start working on `NPMParser` next.