Merge pull request #79 from dataforgoodfr/gg-documentation

A little bit of documentation with dendron
dataforgoodfr · Feb 21, 2024 · d3f3e01 · d3f3e01
2 parents 4e03976 + 1ec79a0
commit d3f3e01
Show file tree

Hide file tree

Showing 19 changed files with 423 additions and 46 deletions.
diff --git a/README.md b/README.md
@@ -1,15 +1,124 @@
-# Bloom Project
+![banner](images/banner.png)
 
-This project is maintained by D4G in order to gather vessels data. 
-A cron job is launched every 15min, does calls to API and save the data in a Postgresql database.
+## What is Trawl Watch
 
-## About directory architecture
-The domain directory ...
-The infra directory ...
+**[Trawl Watch](https://twitter.com/TrawlWatch)** is an initiative launched by the **[Bloom Association](https://www.bloomassociation.org/en/)** to track and expose the most destructive fishing vessels. Inspired by L’[Avion de Bernard](https://www.instagram.com/laviondebernard/), which monitors the movements of private jets, **Trawl Watch** aims to make visible the impact of these massive trawlers on our oceans. These vessels, often referred to as _mégachalutiers_, deploy gigantic nets that can engulf marine life from the surface down to the ocean floor. The consequences are both ecological—as they devastate crucial nursery and breeding areas for marine animals—and social, as they deprive artisanal fishermen of a healthy marine ecosystem. The solution proposed by **Bloom** is to dismantle these industrial fishing ships and redistribute their quotas to small-scale fishers. A petition has been launched, and **Bloom** continues to track these megatrawlers while awaiting action from European institutions.
 
+**Did you know that, in Europe, the largest fishing vessels, which represent 1% of the fleet, catch half of the fish?** These factory-vessels can measure up to 144 meters in length and catch 400,000 kilos of fish per day! This is as much as 1,000 small-scale fishing vessels in one day at sea.
 
-More information can be found there :
-1. [Database initialisation and versioning](./documentation/database_init.md)
-2. [Development environment](./documentation/dev_env.md)
-3. [Useful SQL examples](./documentation/sql_examples.md)
-4. [Data models](#todo)
+**These veritable sea monsters are devastating Europe’s biodiversity and coastlines.** It is important to measure the scale of the damage: about 20 of these factory-vessels can obliterate hundreds of thousands of marine animals and biodiversity treasures in one day, including in the so-called ‘Marine Protected Areas’ of French territorial waters, which are not protected at all.
+
+## What is Bloom Association
+
+**BLOOM** is a non-profit organization founded in 2005 that works to preserve the marine environment and species from unnecessary destruction and to increase social benefits in the fishing sector. **BLOOM** wages awareness and advocacy campaigns in order to accelerate the adoption of concrete solutions for the ocean, humans and the climate. **BLOOM** carries out scientific research projects, independent studies and evaluations that highlight crucial and unaddressed issues such as the financing mechanisms of the fishing sector. **BLOOM**’s actions are meant for the general public as well as policy-makers and economic stakeholders.
+
+## Installing Trawl Watch with `poetry`
+
+### Prerequisites:
+
+1. Python (≥ `3.10`) installed on your system.
+2. Ensure [Docker](https://docs.docker.com/get-docker/) is installed.
+3. Ensure you have `poetry` installed. If not, you can install them using `pip`.
+
+```bash
+pip install poetry
+```
+
+### Steps:
+
+1. **Clone the GitHub Repository:**
+
+   Clone the GitHub repository you want to install locally using the `git clone` command.
+
+   ```bash
+   git clone https://github.com/dataforgoodfr/12_bloom.git
+   ```
+
+2. **Navigate to the Repository Directory:**
+
+   Use the `cd` command to navigate into the repository directory.
+
+   ```bash
+   cd 12_bloom/
+   ```
+
+3. **Configure `poetry` to create a Virtual Environment inside the project:**
+
+   Ensure that poetry will create a `.venv` directory into the project with the command:
+
+   ```bash
+   poetry config virtualenvs.in-project true
+   ```
+
+4. **Install Project Dependencies using `poetry`:**
+
+   Use `poetry` to install the project dependencies.
+
+   ```bash
+   poetry install
+   ```
+
+   This will read the `pyproject.toml` file in the repository and install all the dependencies specified.
+
+5. **Make sure everything is all right using `poetry env info`:**
+
+   ```bash
+   poetry env info
+   ```
+
+   It should looks something likes:
+
+   ```bash
+   Virtualenv
+   Python:         3.11.2
+   Implementation: CPython
+   Path:           /home/guillaume/12_bloom/.venv
+   Executable:     /home/guillaume/12_bloom/.venv/bin/python
+   Valid:          True
+
+   System
+   Platform:   linux
+   OS:         posix
+   Python:     3.11.2
+   Path:       /usr
+   Executable: /usr/bin/python3.11
+   ```
+
+6. **Activate the Virtual Environment:**
+
+   Activate the virtual environment to work within its isolated environment.
+
+   On Unix or MacOS:
+
+   ```bash
+   poetry shell
+   ```
+
+7.
+
+### Once you're done working with the project, deactivate the virtual environment.
+
+```bash
+deactivate
+```
+
+## Documentation
+
+[Dendron](https://marketplace.visualstudio.com/items?itemName=dendron.dendron) is a powerful Visual Studio Code, or [VSCodium](https://vscodium.com/), extension designed to streamline and enhance the documentation process. With **Dendron**, documenting projects becomes intuitive and efficient, thanks to its hierarchical note-taking system. Users can organize their documentation into a tree-like structure, making it easy to navigate and manage. The extension offers robust features such as bidirectional linking, which allows for seamless navigation between related notes, and support for Markdown formatting, enabling users to create rich and visually appealing documentation. Additionally, **Dendron** provides powerful search functionality, enabling users to quickly locate specific information within their documentation vault. Overall, **Dendron** empowers developers, writers, and teams to create comprehensive and well-organized documentation, facilitating better knowledge management and collaboration. The documentation is locaed inside the `./docs/notes` directory.
+
+Here's some basic Dendron shortcuts:
+
+| Shortcut       | Description  |
+| -------------- | ------------ |
+| `Cmd/Ctrl + L` | Lookup notes |
+
+## More information can be found there
+
+1. [Database initialisation and versioning](./docs/database.initialisation.md)
+2. [Development environment](./docs/development.environment.md)
+3. [Useful SQL examples](./docs/sql.examples.md)
+4. [Data models](#todo)
+
+## FAQ
+
+#todo
diff --git a/data/_info_subset_data.docx b/data/_info_subset_data.docx
diff --git a/data/vessels_subset_02022024.csv b/data/vessels_subset_02022024.csv
@@ -0,0 +1,11 @@
+"id","country_iso3","cfr","IMO","registration_number","external_marking","ship_name","ircs","mmsi","loa","type","mt_activated"
+621,"ESP","27678 ESP","8516043","3VI-5-1-15","3VI-5-1-15","PARADANTA PRIMERO","EAAB",224378000,"58.38",">25m",False
+784,"FRA","FRA000919999","9828936","919999","CC919999","SCOMBRUS","FLVN",226347000,"81.37",">25m",False
+795,"FRA","NLD198300393","8224406","716900","FC716900","PRINS BERNHARD","FKHH",227302000,"88.24",">25m",False
+835,"FRA","FRA000932206","9741097","932206","CC932206","GEVRED","FIUO",228066900,"77",">25m",False
+863,"FRA","FRA000914221","9352913","914221","MA914221","VENT DU NORD II","FMEU",228215800,"41.92",">25m",False
+920,"GBR",NULL,"9877389","GY127",NULL,"HENDRIKA JACOBA",NULL,232031183,NULL,"Demersal seiners",False
+1194,"NLD","FRA000544858","7928835","FRA000544858","SL-9","JOHANNA","PHOE",244938000,"34.3",">25m",False
+1280,"NLD","NLD201001155","9599731","NLD201001155","UK-153","LUB SENIOR","PCGC",246749000,"28.5",">25m",False
+1431,"POL","IRL000I13000","9204556","PL02315","GDY-151","ANNELIES ILENA","SPG4839",261084090,"144.6",">25m",False
+1507,"PRT","PRT000023094","9297694","PTAVE-117345-N","PTAVE-117345-N","FRANÇA MORTE","CUFE7",263581000,"73.8",">25m",False
diff --git a/docs/.gitignore b/docs/.gitignore
@@ -0,0 +1,6 @@
+node_modules
+.dendron.*
+build
+seeds
+.next
+pods/service-connections
diff --git a/docs/dendron.code-workspace b/docs/dendron.code-workspace
@@ -0,0 +1,37 @@
+{
+  "folders": [
+    {
+      "path": "notes",
+      "name": "docs"
+    }
+  ],
+  "settings": {
+    "dendron.rootDir": ".",
+    "files.autoSave": "onFocusChange",
+    "pasteImage.path": "${currentFileDir}/assets/images",
+    "pasteImage.prefix": "/",
+    "markdown-preview-enhanced.enableWikiLinkSyntax": true,
+    "markdown-preview-enhanced.wikiLinkFileExtension": ".md",
+    "editor.snippetSuggestions": "inline",
+    "editor.suggest.snippetsPreventQuickSuggestions": false,
+    "editor.suggest.showSnippets": true,
+    "editor.tabCompletion": "on",
+    "dendron.enableSelfContainedVaultWorkspace": true
+  },
+  "extensions": {
+    "recommendations": [
+      "dendron.dendron",
+      "dendron.dendron-paste-image",
+      "dendron.dendron-markdown-shortcuts",
+      "redhat.vscode-yaml"
+    ],
+    "unwantedRecommendations": [
+      "dendron.dendron-markdown-links",
+      "dendron.dendron-markdown-notes",
+      "dendron.dendron-markdown-preview-enhanced",
+      "shd101wyy.markdown-preview-enhanced",
+      "kortina.vscode-markdown-notes",
+      "mushan.vscode-paste-image"
+    ]
+  }
+}
diff --git a/docs/dendron.yml b/docs/dendron.yml
@@ -0,0 +1,108 @@
+version: 5
+dev:
+    enablePreviewV2: true
+    enableSelfContainedVaults: true
+commands:
+    lookup:
+        note:
+            selectionMode: extract
+            confirmVaultOnCreate: true
+            vaultSelectionModeOnCreate: smart
+            leaveTrace: false
+            bubbleUpCreateNew: true
+            fuzzThreshold: 0.2
+    randomNote: {}
+    insertNoteLink:
+        aliasMode: none
+        enableMultiSelect: false
+    insertNoteIndex:
+        enableMarker: false
+    copyNoteLink:
+        aliasMode: title
+    templateHierarchy: template
+workspace:
+    vaults:
+        -
+            fsPath: .
+            selfContained: true
+            name: docs
+    journal:
+        dailyDomain: daily
+        name: journal
+        dateFormat: y.MM.dd
+        addBehavior: childOfDomain
+    scratch:
+        name: scratch
+        dateFormat: y.MM.dd.HHmmss
+        addBehavior: asOwnDomain
+    task:
+        name: task
+        dateFormat: y.MM.dd
+        addBehavior: asOwnDomain
+        statusSymbols:
+            '': ' '
+            wip: w
+            done: x
+            assigned: a
+            moved: m
+            blocked: b
+            delegated: l
+            dropped: d
+            pending: 'y'
+        taskCompleteStatus:
+            - done
+            - x
+        prioritySymbols:
+            H: high
+            M: medium
+            L: low
+        todoIntegration: false
+        createTaskSelectionType: selection2link
+    graph:
+        zoomSpeed: 1
+        createStub: false
+    enableAutoCreateOnDefinition: false
+    enableXVaultWikiLink: false
+    enableRemoteVaultInit: true
+    enableUserTags: true
+    enableHashTags: true
+    workspaceVaultSyncMode: noCommit
+    enableAutoFoldFrontmatter: false
+    enableEditorDecorations: true
+    maxPreviewsCached: 10
+    maxNoteLength: 204800
+    enableFullHierarchyNoteTitle: false
+    enablePersistentHistory: false
+preview:
+    enableFMTitle: true
+    enableNoteTitleForLink: true
+    enableFrontmatterTags: true
+    enableHashesForFMTags: false
+    enablePrettyRefs: true
+    enableKatex: true
+    automaticallyShowPreview: false
+publishing:
+    enableFMTitle: true
+    enableNoteTitleForLink: true
+    enablePrettyRefs: true
+    enableKatex: true
+    copyAssets: true
+    siteHierarchies:
+        - root
+    writeStubs: false
+    siteRootDir: docs
+    seo:
+        title: Dendron
+        description: Personal Knowledge Space
+    github:
+        enableEditLink: true
+        editLinkText: Edit this page on GitHub
+        editBranch: main
+        editViewMode: tree
+    enableSiteLastModified: true
+    enableFrontmatterTags: true
+    enableHashesForFMTags: false
+    enableRandomlyColoredTags: true
+    enableTaskNotes: true
+    enablePrettyLinks: true
+    searchMode: search
diff --git a/docs/notes/ais.md b/docs/notes/ais.md
@@ -0,0 +1,21 @@
+---
+id: g1i0o9ctmib83n4mydtsqfo
+title: Automatic identification system
+desc: ""
+updated: 1708461719224
+created: 1708461557185
+---
+
+The automatic identification system (AIS) is an automatic tracking system that uses transceivers on ships and is used by vessel traffic services (VTS). When satellites are used to receive AIS signatures, the term Satellite-AIS (S-AIS) is used. AIS information supplements marine radar, which continues to be the primary method of collision avoidance for water transport. Although technically and operationally distinct, the ADS-B system is analogous to AIS and performs a similar function for aircraft.[^1]
+
+## Limits
+
+- out of range
+- intense traffic area
+- expensive
+
+## Data source
+
+https://spire.com/maritime/
+
+[^1]: https://en.wikipedia.org/wiki/Automatic_identification_system
diff --git a/documentation/database_init.md → docs/notes/database.initialisation.md b/documentation/database_init.md → docs/notes/database.initialisation.md
@@ -1,16 +1,18 @@
-# Database initialisation and versioning
-
-
-## Database initialisation
+---
+id: 8vo1vo09y37hwnrkmcy7w77
+title: Database initialisation
+desc: ""
+updated: 1708410470701
+created: 1708410417004
+---
 
 First you need to run scripts which are in alembic/init_script :
+
 - the load_vessels_data.py script will load vessels metadata from the data/chalutier_pelagique.csv file.
 - the load_geometry_data.py file will load shape data from the Nonterrestrial_WDPA_Jan2023.shp file. This file is not included in this github project but you can ask for it. It's only used for the alerting part.
 
 The second step is to load the [distance-from-port-v20201104.tiff](https://globalfishingwatch.org/data-download/datasets/public-distance-from-port-v1) and [distance-from-shore.tif](https://globalfishingwatch.org/data-download/datasets/public-distance-from-shore-v1) files. They are only used for the alerting part.
+
 - install psql and raster2pgsql.
 - install raster type in db with postgis-raster using `create extension postgis_raster`
-- adapt this command for each file :  `raster2pgsql -t auto -I -C -M /PATH_TO/distance-from-shore.tif public.distance_shore | PGPASSWORD='POSTGRES_PASSWORD' psql -h POSTGRES_HOSTNAME -d POSTGRES_DB -U POSTGRES_USER -p POSTGRES_PORT`
-
-## Database versioning
-The command ` alembic upgrade head` can be used in the root of the project in order to update the database schema to the last version.
+- adapt this command for each file : `raster2pgsql -t auto -I -C -M /PATH_TO/distance-from-shore.tif public.distance_shore | PGPASSWORD='POSTGRES_PASSWORD' psql -h POSTGRES_HOSTNAME -d POSTGRES_DB -U POSTGRES_USER -p POSTGRES_PORT`
diff --git a/docs/notes/database.versioning.md b/docs/notes/database.versioning.md
@@ -0,0 +1,9 @@
+---
+id: kofmnl4h6i8e9uacvtd2wkr
+title: Database versioning
+desc: ""
+updated: 1708410507271
+created: 1708410490074
+---
+
+The command ` alembic upgrade head` can be used in the root of the project in order to update the database schema to the last version.