Skip to content

Commit

Permalink
added generic plugins: from-pyfunc, pyfunc-filter, to-pyfunc
Browse files Browse the repository at this point in the history
  • Loading branch information
fracpete committed May 10, 2024
1 parent 0b86892 commit 8d2e6d6
Show file tree
Hide file tree
Showing 2 changed files with 50 additions and 0 deletions.
49 changes: 49 additions & 0 deletions docs/pyfunc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
No library can dream of offering all the required functionality. Especially for one-off tasks, it makes no sense to
develop a whole new plugin library. Hence, there are the following generic plugins that allow the user to utilize
custom Python functions:

* reader: `from-pyfunc` - takes a single string as input and outputs an iterable of image containers (as per specified data type)
* filter: `pyfunc-filter` - takes a single image container or an iterable of them as input and outputs a single container or an iterable of them (as per specified input and output data types)
* writer: `to-pyfunc` - processes a single image container or an iterable of them as per specified data type and an optional split name

In order to use such a custom function, they must be specified in the following format (option: `-f/--function`):

```
module_name:function_name
```

If the code below were available through module `my.code`, then the function specifications would be as follows:

* reader: `my.code:pyfunc_reader`
* filter: `my.code:pyfunc_filter`
* writer: `my.code:pyfunc_writer`


```python
from typing import Iterable
from idc.api import ImageClassificationData, make_list, flatten_list

# reader: generates image classification containers from the path
def pyfunc_reader(path: str) -> Iterable[ImageClassificationData]:
return [ImageClassificationData(source=path)]

# filter: simply adds a note to the meta-data
def pyfunc_filter(data):
result = []
for item in make_list(data):
if not item.has_metadata():
meta = dict()
else:
meta = item.get_metadata()
meta["note"] = "filtered by a python function!"
item.set_metadata(meta)
result.append(item)
return flatten_list(result)

# writer: simply outputs name and meta-data and, if present, also the split
def pyfunc_writer(data: ImageClassificationData, split: str = None):
if split is None:
print("name: ", data.image_name, ", meta:", data.get_metadata())
else:
print("split:", split, ", name:", data.image_name, ", meta:", data.get_metadata())
```
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ nav:
- Image segmentation: image_segmentation.md
- Filter usage: filters.md
- Docker usage: docker.md
- External functions: pyfunc.md
- Additional libraries:
- Image augmentation: imgaug.md
- Image statistics: imgstats.md
Expand Down

0 comments on commit 8d2e6d6

Please sign in to comment.