Skip to content

Commit

Permalink
Merge pull request #74 from timvink/73_filter_tables
Browse files Browse the repository at this point in the history
Additional pandas reader macros
  • Loading branch information
timvink authored Aug 29, 2024
2 parents 47ac5d5 + 14a6c4a commit 6fdf150
Show file tree
Hide file tree
Showing 18 changed files with 524 additions and 108 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,8 @@ In your markdown files you can now use:

Where the path is relative to the location of your project's `mkdocs.yml` file, _or_ your project's `docs/` directory, _or_ the location of your markdown source file (all 3 possible locations will be searched, in that order).

- There are [readers](https://timvink.github.io/mkdocs-table-reader-plugin/readers/) for `.csv`, `.fwf`, `.json`, `.xls`, `.xlsx`, `.yaml`, `.feather` and `.tsv` files. There is also the `read_raw()` reader that will allow you to insert tables (or other content) already in markdown format.
- `table-reader` is compatible with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), which means you can [dynamically insert tables using jinja2 syntax](https://timvink.github.io/mkdocs-table-reader-plugin/howto/use_jinja2/).
- There are [readers](https://timvink.github.io/mkdocs-table-reader-plugin/readers/) available for many common table formats, like `.csv`, `.fwf`, `.json`, `.xls`, `.xlsx`, `.yaml`, `.feather` and `.tsv`. There is also the `read_raw()` reader that will allow you to insert tables (or other content) already in markdown format.
- `table-reader` is compatible with [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/). This enables further automation like filtering tables or inserting directories of tables. See the documentation on [compatibility with macros plugin](howto/use_jinja2.md) for more examples.

## Documentation and how-to guides

Expand Down
2 changes: 1 addition & 1 deletion docs/howto/alternatives.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ Downsides:

## Execute python during build

You could also choose to insert the markdown for tables dynamically, using packages like [markdown-exec]() or [mkdocs-macros-plugin](https://mkdocs-macros-plugin.readthedocs.io/).
You could also choose to insert the markdown for tables dynamically, using packages like [markdown-exec](https://pypi.org/project/markdown-exec/).

For example:

Expand Down
54 changes: 40 additions & 14 deletions docs/howto/customize_tables.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,10 @@ df = pd.read_csv('path_to_table.csv')
df.to_markdown(index=False, tablefmt='pipe')
```

Any keyword arguments you give to <code>\{\{ read_csv('path_to_your_table.csv') \}\}</code> will be matched and passed the corresponding [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) and/or
{% raw %}
Any keyword arguments you give to `{{ read_csv('path_to_your_table.csv') }}` will be matched and passed the corresponding [pandas.read_csv()](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html) and/or
[.to_markdown()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.to_markdown.html) functions.
{% endraw %}

Pandas's `.to_markdown()` uses the [tabulate](https://pypi.org/project/tabulate/) package and any keyword arguments that are passed to it. Tabulate in turn offers many customization options, see [library usage](https://github.com/astanin/python-tabulate#library-usage).

Expand All @@ -23,21 +25,33 @@ Text columns will be aligned to the left [by default](https://github.com/astanin

=== ":arrow_left: left"

<code>\{\{ read_csv('tables/basic_table.csv', colalign=("left",)) \}\}</code>

{% raw %}
```
{{ read_csv('tables/basic_table.csv', colalign=("left",)) }}
```
{% endraw %}

=== ":left_right_arrow: center"
{{ read_csv('tables/basic_table.csv', colalign=("left",)) | add_indentation(spaces=4) }}

<code>\{\{ read_csv('tables/basic_table.csv', colalign=("center",)) \}\}</code>
=== ":left_right_arrow: center"

{% raw %}
```
{{ read_csv('tables/basic_table.csv', colalign=("center",)) }}
```
{% endraw %}

=== ":arrow_right: right"
{{ read_csv('tables/basic_table.csv', colalign=("center",)) | add_indentation(spaces=4) }}

<code>\{\{ read_csv('tables/basic_table.csv', colalign=("right",)) \}\}</code>
=== ":arrow_right: right"

{% raw %}
```
{{ read_csv('tables/basic_table.csv', colalign=("right",)) }}
```
{% endraw %}

{{ read_csv('tables/basic_table.csv', colalign=("right",)) | add_indentation(spaces=4) }}

## Sortable tables

Expand All @@ -47,21 +61,33 @@ If you use [mkdocs-material](https://squidfunk.github.io/mkdocs-material), you c

You can use [tabulate](https://pypi.org/project/tabulate/)'s [number formatting](https://github.com/astanin/python-tabulate#number-formatting). Example:

=== ":zero:"

<code>\{\{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".0f") \}\}</code>
=== "zero points"

{% raw %}
```
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".0f") }}
```
{% endraw %}

=== ":one:"
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".0f") | add_indentation(spaces=4) }}

<code>\{\{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".1f") \}\}</code>
=== "one points"

{% raw %}
```
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".1f") }}
```
{% endraw %}

=== ":two:"
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".1f") | add_indentation(spaces=4) }}

<code>\{\{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".2f") \}\}</code>
=== "two points"

{% raw %}
```
{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".2f") }}
```
{% endraw %}

{{ read_fwf('tables/fixedwidth_table.txt', floatfmt=".2f") | add_indentation(spaces=4) }}

8 changes: 6 additions & 2 deletions docs/howto/preprocess_tables.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Preprocess input tables

{% raw %}

`mkdocs>=1.4` supports [hooks](https://www.mkdocs.org/user-guide/configuration/#hooks), which enable you to run python scripts on `mkdocs serve` or `mkdocs build`.

Here are some example of workflows that use hooks and the `table-reader` plugin:
Expand All @@ -26,7 +28,7 @@ Here are some example of workflows that use hooks and the `table-reader` plugin:

<code>
My table:
\{\{ read_csv("docs/assets/output_table.csv") \}\}
{{ read_csv("docs/assets/output_table.csv") }}
</code>

=== "mkdocs.yml"
Expand Down Expand Up @@ -74,7 +76,7 @@ Here are some example of workflows that use hooks and the `table-reader` plugin:

<code>
My table:
\{\{ read_csv("docs/assets/nyc_data.csv") \}\}
{{ read_csv("docs/assets/nyc_data.csv") }}
</code>

=== "mkdocs.yml"
Expand All @@ -101,3 +103,5 @@ Here are some example of workflows that use hooks and the `table-reader` plugin:
```

Note that during development when you use `mkdocs serve` and autoreload, you might not want to run this hook every time you make a change. You could use an environment variable inside your hook, for example something like `if os.environ['disable_hook'] == 1: return None`.

{% endraw %}
16 changes: 9 additions & 7 deletions docs/howto/project_structure.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Choose a project structure
{% raw %}

You have different possible strategies to store and load your tables. This guide gives some examples.

Expand All @@ -23,14 +24,14 @@ If you only want to include an occasional table in a specific markdown file, jus
```md
Here is the table:

\{\{ read_csv("another_table.csv") \}\}
{{ read_csv("another_table.csv") }}
```

In `page.md`, to read `another_table.csv`, you can choose to use:

- <code>\{\{ read_csv("docs/folder/another_table.csv") \}\}</code> (Path relative to mkdocs.yml)
- <code>\{\{ read_csv("folder/another_table.csv") \}\}</code> (Path relative to docs/ directory)
- <code>\{\{ read_csv("another_table.csv") \}\}</code> (Path relative to page source file)
- `{{ read_csv("docs/folder/another_table.csv") }}` (Path relative to mkdocs.yml)
- `{{ read_csv("folder/another_table.csv") }}` (Path relative to docs/ directory)
- `{{ read_csv("another_table.csv") }}` (Path relative to page source file)

## Re-using tables across markdown files

Expand All @@ -54,7 +55,8 @@ Given the following project structure:

In `page.md`, to read `another_table.csv`, you can choose to use:

- <code>\{\{ read_csv("docs/assets/tables/another_table.csv") \}\}</code> (Path relative to mkdocs.yml)
- <code>\{\{ read_csv("assets/tables/another_table.csv") \}\}</code> (Path relative to docs/ directory)
- <code>\{\{ read_csv("../assets/tables/another_table.csv") \}\}</code> (Path relative to page source file _(note that `..` stands for "one directory up")_)
- `{{ read_csv("docs/assets/tables/another_table.csv") }}` (Path relative to mkdocs.yml)
- `{{ read_csv("assets/tables/another_table.csv") }}` (Path relative to docs/ directory)
- `{{ read_csv("../assets/tables/another_table.csv") }}` (Path relative to page source file _(note that `..` stands for "one directory up")_)

{% endraw %}
47 changes: 22 additions & 25 deletions docs/howto/use_jinja2.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
# Use jinja2 for automation
# Compatibility with mkdocs-macros-plugin to enable further automation

{% raw %}

`table-reader` supports [`mkdocs-macros-plugin`](https://mkdocs-macros-plugin.readthedocs.io/en/latest/), which enables you to use jinja2 syntax inside markdown files (among other things).

Expand All @@ -10,26 +12,19 @@ plugins:
- table-reader
```
Now you can do cool things like:
## Dynamically load a list of tables
Everything will work as before, _except_ indentation will not be retrained. This means components that rely on indentation (like [Admonitions](https://squidfunk.github.io/mkdocs-material/reference/admonitions/) and [Content tabs](https://squidfunk.github.io/mkdocs-material/reference/content-tabs/#usage)) will break.
```markdown
# index.md
The solution is to use the custom _filter_ `add_indendation` (a filter added to `macros` by `table-reader` plugin, see the [readers](../readers.md)). For example:

{% set table_names = ["basic_table.csv","basic_table2.csv"] %}
{% for table_name in table_names %}
```jinja
!!! note "This is a note"
{ { read_csv(table_name) }}

{% endfor %}
{{ read_csv("basic_table.csv") | add_indentation(spaces=4) }}
```

## Insert tables into content tabs

If you inserted content has multiple lines, then indentation will be not be retained beyond the first line. This means things like [content tabs](https://squidfunk.github.io/mkdocs-material/reference/content-tabs/#usage) will not work as expected.
The upside is you now have much more control. A couple of example use cases:

To fix that, you can use the custom _filter_ `add_indendation` (a filter add to `macros` by `table-reader` plugin). For example:
## Dynamically load a specified list of tables into tabs

=== "index.md"

Expand All @@ -39,7 +34,7 @@ To fix that, you can use the custom _filter_ `add_indendation` (a filter add to
=== "{{ table_name }}"
{ { read_csv(table_name) | add_indentation(spaces=4) }}
{{ read_csv(table_name) | add_indentation(spaces=4) }}
{% endfor %}
```
Expand All @@ -64,11 +59,6 @@ To fix that, you can use the custom _filter_ `add_indendation` (a filter add to
alternate_style: true
```

!!! note "Note the space in { {"

To avoid the tables being inserted into the code example, we replaced `{{` with `{ {`.
If you copy this example, make sure to fix.


## Recursively insert an entire directory of tables

Expand Down Expand Up @@ -100,12 +90,19 @@ Now you could do something like:
{% for table_name in listdir('docs/assets/my_tables") %}
{ { read_csv(table_name) }}
{{ read_csv(table_name) }}
{% endfor %}
```

!!! note "Note the space in { {"
## Filter a table before inserting it

When you enable the `macros` plugin in your `mkdocs.yml`, `table-reader` will add additional _macros_ and _filters_ (see the [readers overview](../readers.md)).

You can use the `pd_<reader>` variants to read a file to a pandas dataframe. Then you can use the pandas methods like [`.query()`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.query.html). For example:

```
{{ pd_read_csv("numeric_table.csv").query("column_name >= 3") | convert_to_md_table }}
```

To avoid the tables being inserted into the code example, we replaced `{{` with `{ {`.
If you copy this example, make sure to fix.
{% endraw %}
9 changes: 6 additions & 3 deletions docs/options.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ hide:
---

# Options
{% raw %}

You can customize the plugin by setting options in `mkdocs.yml`. For example:

Expand All @@ -24,7 +25,7 @@ Default is `.`. Set a default path to the searched directories in order to short

Given a file path, `table-reader` will search for that file relative to your your project's `mkdocs.yml` and relative to your `docs/` folder. If you use a folder for all your table files you can shorten the path specification by setting the `data_path`.

For example, if your table is located in `docs/assets/tables/basic_table.csv`, you can set `data_path` to `docs/assets/tables/`. Then you will be able to use <code>\{\{ read_csv("basic_table.csv") \}\}</code> instead of <code>\{\{ read_csv("docs/assets/tables/basic_table.csv") \}\}</code> inside any markdown page.
For example, if your table is located in `docs/assets/tables/basic_table.csv`, you can set `data_path` to `docs/assets/tables/`. Then you will be able to use `{{ read_csv("basic_table.csv") }}` instead of `{{ read_csv("docs/assets/tables/basic_table.csv") }}` inside any markdown page.

!!! info

Expand All @@ -38,7 +39,7 @@ Default: `False`. When enabled, if a filepath is not found, the plugin will rais

## `select_readers`

Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites.
Default: Selects all available readers. Specify a list of readers to improve documentation build times for very large sites. This option is ignored when you use this plugin with `mkdocs-macros-plugin` ([read more](howto/use_jinja2.md))

## `enabled`

Expand All @@ -57,4 +58,6 @@ Which enables you to disable the plugin locally using:
```bash
export ENABLED_TABLE_READER=false
mkdocs serve
```
```

{% endraw %}
Loading

0 comments on commit 6fdf150

Please sign in to comment.