Documentation for using Docker.Run Tasks

[sebastianknopf]

Description

I have an issue with the documentation of Docker.Run tasks. In the current documentation is explained, how to mount a directory from the host to the running docker container using the volumes property. In another issue, I found an information, that a Docker.Run task behaves like running on the host system, even if Kestra is running in a docker container itself.

Howewer, I wonder how to pass an output file of a previous task to the docker container of the Docker.Run task? I have the following scenario:

• A task (e.g. http.Download) produces an output file
• The next task (a Docker.Run) task should work with the previously created file; I would register it as input file
• From now on, I see two options: Either mount the local working directory (using $(PWD) ) to the Docker.Run task, or mount the whole output file to the Docker.Run task

I tested both, but no one of these options works. If I mount the directory of my host system, everything works fine.

Am I missing something here? Or is this not possible at all using a Docker.Run task? I'd be glad if the documentation could be enhanced for this topic.

[sebastianknopf]

After trying around several things, I found a solution at least for some docker containers. See the following flow definition for reference:

id: gtfsvtortest
namespace: company.team
tasks:
  - id: download
    type: io.kestra.plugin.core.http.Download
    uri: https://www.nvbw.de/fileadmin/user_upload/service/open_data/fahrplandaten_ohne_liniennetz/vpe.zip
  - id: dockertest
    type: io.kestra.plugin.docker.Run
    commands:
      - gtfs.zip
      - "-o test.html"
    containerImage: laurentgregoire/gtfsvtor
    entryPoint:
      - /app/gtfsvtor/bin/gtfsvtor
    env:
      TZ: Europe/Berlin
    inputFiles:
      gtfs.zip: "{{ outputs.download.uri }}"
    namespaceFiles:
      enabled: true
    outputFiles:
      - "*.html"

If I register the URI of the previous task as inputFile in the Docker.Run task, this file seems to be available in the docker command. If I remove the second command (-o test.html) everything works fine. With the second command including, the docker container fails, if I put all commands in one line, the docker container fails.

Also, other docker containers don't work this way, they seem to be unable to read from the kestra internal URI.

[Ben8t]

Hello @sebastianknopf can you provide the corresponding docker-compose or Kestra configuration you're using :) ?

I will try to reproduce what you mentioned and clear the doc accordingly