Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

i.eodag: add i.eodag module #1090

Merged
merged 29 commits into from
Jun 4, 2024
Merged

i.eodag: add i.eodag module #1090

merged 29 commits into from
Jun 4, 2024

Conversation

HamedElgizery
Copy link
Contributor

@HamedElgizery HamedElgizery commented May 27, 2024

Adding i.eodag addon module. The module can be used for downloading different imagery datasets e.g. Landsat, Sentinel, and MODIS. EODAG [1] provides a single API interface to request imagery datasets from a number of different providers.

[1] https://eodag.readthedocs.io/en/stable/index.html

@ninsbl ninsbl marked this pull request as draft May 27, 2024 21:21
@ninsbl ninsbl added new addon PR contains a new addon or issue proposes one Python Related code is in Python gsoc Reserved for Google Summer of Code student(s) labels May 27, 2024
Copy link
Member

@ninsbl ninsbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some first comments. Very good start, Hamed.

@wenzeslaus Is it OK if we use a draft PR for technical GSoC discussions, Or should we have such discussions at a different place for now?

src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
Copy link
Contributor

@veroandreo veroandreo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @HamedElgizery! I haven't tested yet, but some general comments:

  • the .gitignore should not be part of the PR
  • the html only includes the following sections: Description, Examples, See also, Author, all the rest is added by the parser from the code (the .py file in this case). Check the html file in i.landsat.download or i.sentinel.download for guidance

@wenzeslaus
Copy link
Member

@wenzeslaus Is it OK if we use a draft PR for technical GSoC discussions, Or should we have such discussions at a different place for now?

Draft PR is appropriate. A great place to have a discussion over specific code!

Copy link
Member

@ninsbl ninsbl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice. It is taking shape. Please run pylint on your code to get some more hints on code formatting...

src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.py Outdated Show resolved Hide resolved
@ninsbl
Copy link
Member

ninsbl commented May 29, 2024

BTW, if searching does not require credentials, it would be good to add a basic unit-test for searching...

@HamedElgizery
Copy link
Contributor Author

HamedElgizery commented May 29, 2024

BTW, if searching does not require credentials, it would be good to add a basic unit-test for searching...

Most of the providers doesn't requirer credentials for searching, while others require credentials... So, if the user set a specific provider for searching (e.g. provider=usgs). If usgs requires credentials, then eodag will attempt to search using a different provider, if it didn't find the usgs credentials...

So, I think we could leave it for eodag to handle it, and just send a warning if the credentials are not there for that specified provider.

Something along the lines of:
"Credentials for {provider} was not found... Searching might be automatically rerouted to a different provider."

What do you think?

@ninsbl
Copy link
Member

ninsbl commented May 29, 2024

(...) So, I think we could leave it for eodag to handle it, and just send a warning if the credentials are not there for that specified provider. (...)
What do you think?

Sounds convinceing. We should then make sure in the module logic that the provider is not a required option (and consider potential consequences, e.g. lines 193, 199, ...)...

@HamedElgizery
Copy link
Contributor Author

HamedElgizery commented May 29, 2024

After a second thought, I think making the provider required by the user would constraint the behavior of the library better, and only searching in that provider. By that I mean, preventing eodag from automatically rerouting the searching process to a different provider, and instead it just terminates with a relevant error message or "0 results found", when the searching fails for the given provider.

That will make it more convenient when using it inside the other modules e.g. i.landsat.download & i.sentinel.download.

We can also provide a method for the user to see which providers have the datasets they are looking for... and then they can use one of these to search for the intended product.

Nevertheless, we could add a search feature without setting the "provider" later on.

@ninsbl
Copy link
Member

ninsbl commented May 29, 2024

After a second thought, I think making the provider required by the user would constraint the behavior of the library better,

Agreed, Also, by requesting a specific provider I would expect results to be more reliable. Anyway, you may also consider allowing multiple input (comma separated list) in the providers option... (which is probably a solution inbetween).

Used to search in one and only one provider, if it doesn't find the
intended product, it doesn't search in other providers
HamedElgizery and others added 3 commits May 31, 2024 13:07
Co-authored-by: Stefan Blumentrath <stefan.blumentrath@gmx.de>
Co-authored-by: Stefan Blumentrath <stefan.blumentrath@gmx.de>
Co-authored-by: Stefan Blumentrath <stefan.blumentrath@gmx.de>
Copy link
Contributor

@veroandreo veroandreo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just left some comments on the html file. I think we can merge after those are solved.

src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Outdated Show resolved Hide resolved
src/imagery/i.eodag/i.eodag.html Show resolved Hide resolved
HamedElgizery and others added 2 commits June 2, 2024 19:43
Co-authored-by: Veronica Andreo <veroandreo@gmail.com>
Co-authored-by: Veronica Andreo <veroandreo@gmail.com>
@HamedElgizery
Copy link
Contributor Author

Thanks for the comments, @veroandreo . I will fix them ASAP.

HamedElgizery and others added 4 commits June 3, 2024 12:55
Co-authored-by: Veronica Andreo <veroandreo@gmail.com>
Co-authored-by: Veronica Andreo <veroandreo@gmail.com>
Co-authored-by: Veronica Andreo <veroandreo@gmail.com>
Copy link
Contributor

@veroandreo veroandreo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this first version is ready to merge

HamedElgizery and others added 2 commits June 3, 2024 15:41
@ninsbl
Copy link
Member

ninsbl commented Jun 4, 2024

Taking the liberty to merge this first version. Further improvements can be discussed in new PRs.

@ninsbl ninsbl merged commit 384f4d5 into OSGeo:grass8 Jun 4, 2024
6 checks passed
@ninsbl
Copy link
Member

ninsbl commented Jun 4, 2024

Congrats to your first contributed module, @HamedElgizery !

@HamedElgizery
Copy link
Contributor Author

Congrats to your first contributed module, @HamedElgizery !

Thanks @ninsbl, and thank you all for the help!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
gsoc Reserved for Google Summer of Code student(s) new addon PR contains a new addon or issue proposes one Python Related code is in Python
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug] i.sentinel.download: Support Copernicus Data Space Ecosystem Service
6 participants