Python runner module

[B1TC0R3]

Is your feature request related to a problem? Please describe.
There is no problem. :)

Describe the solution you'd like
Hello, devs! 👋
I am a German IT student working on EMBA and dynamic firmware analysis as part of my Bachelors Thesis.
Currently, I am implementing new EMBA module that has the ability to execute user-supplied Python scripts during analysis.

It's still needs a bunch of work, but I wanted to clear up some questions in advance:

1. Could such a feature even be considered for integration into EMBA?
2. What module category would this fit in best? For the current development I have added a new experimental category "E".
3. If this is a feature you are interested in, what are requirements I absolutely have to fulfill apart from the contribution requirements?

Priority issue
Are you already a Sponsor? - [N]

Additional context

• The current additions can be found here: https://github.com/B1TC0R3/emba/tree/python-runner

[github-actions]

Thank you for contributing an issue!

Welcome to the EMBA firmware analysis community!

We are glad you are here and appreciate your contribution. Please keep in mind our contributing guidelines here and here.
Also, please check existing open issues and consider to open a discussion in the dedicated discussion area.
Additionally, we have collected a lot of details around EMBA, the installation and the usage of EMBA in our Wiki.

If you like EMBA you have the chance to support us by becoming a Sponsor or buying some beer here.

To show your love for EMBA with nice shirts or other merch you can check our Spreadshop.

This is an automatic message. Allow for time for the EMBA community to be able to read the issue and comment on it.

[m-1-k-3]

Hi @B1TC0R3

good to hear you are working on EMBA stuff. Looking forward to the results.

I would recommend to build a S-module. Looks as you are doing some further python analysis. Probably the easiest way would be to integrate it into the already available python module (S21) as dedicated function. If your module is getting bigger a dedicated module would be another option. The next available number would be S28.

You have already found the contribution docs. We are currently refactoring EMBA to name all local variables in the format lVAR_NAME -> see the l as first letter. You should always ensure your code passes the strict mode (start firmware analysis with -S parameter) and test it with multiple linters via check_project.sh.

If you have further questions feel free to open issues or use this one

[m-1-k-3]

Now I have seen your module and you should use a dedicated module for such a runner.

btw good idea to build such a python runner

[B1TC0R3]

Thank you!

Should I still add it as an "S" module?

There's a lot of bench-marking necessary before I can tell whether Bash or Python is more viable for my use case,
but I plan on adding more emulation/dynamic analysis on top of this, primarily checks with PEASS-ng, Metasploits exploit suggester and some more optional NMAP scans.

In case Bash is marginally faster, I will add these other changes separately in the "L" or "Q" category tho.

[m-1-k-3]

As you are poking with the emulation engine you need to move your modules to the L(ive testing) area (after L10 which is doing the main emulation stuff). If you plan modules which are working on the filesystem (no system emulation) you need to add S modules.

Some of your mentioned checks are already available:

• Metasploit in L35
• Nmap integrated in L10 and in L15

[B1TC0R3]

Makes sense.

In this case I will add the current code for the python runner as "S28" and see whether and how I can add
python scripts to live testing later on.

I still need to verify my code with strict mode and check_project anyways.

[B1TC0R3]

@m-1-k-3
Hello again.

The changes are almost done, I am currently verifying the last few things.
Meanwhile, I have tried to figure out how to contribute to the wiki in order to add documentation for the new feature,
only to find out that this is rather difficult/not supported on Github.

What would be the best option to provide you with my doc files once the patch is fully ready?

[m-1-k-3]

What would be the best option to provide you with my doc files once the patch is fully ready?

Yea ... The Github wiki is a bit ugly in collaborating. Probably the easiest way would be to write the doc into a comment here and I will transfer it to the wiki and add you as author.

[B1TC0R3]

I understand.

I will see that I get the bigger changes and documentation written until the end of the month.

[B1TC0R3]

Here is the documentation I have come up with.

# Python Runner Module

The Python runner module `S28_python_run` allows users to run Python scripts as EMBA modules.
Recommended use cases for this is module code which requires a lot of complex string or
data manipulation, as such operations can get quite difficult to implement and later read
when using Bash.

## Queuing a Python script for execution

Queuing a new Python script to be run during an EMBA scan requires the addition of the scripts name to an
array called `PYTHON_SCRIPTS` in the scan profile. There is an example in the profile `default-scan-python-runner.emba`.

Assuming there are two Python module files called `python_mod_a.py` and `python_mod_b.py`, queueing both of them would look like this:

\```bash
export PYTHON_SCRIPTS=( "python_mod_a" "python_mod_b" )
\```

## Reading the results

The log files created by a Python module have grep-able symbols at the start of each line.

- `[*]` means a log entry that is not a finding.
- `[Fx]` means a finding, where `x` is the number of the finding. E.g. the very first finding is identified by `[F1]`, while the 137th finding would be identified by `[F137]`.
- `[!]` means an error.

## Writing a new Python module

Each Python module needs to be located in `<emba directory>/modules/S28_python_run/`.

In order to ensure proper integration into EMBA, this directory also contains a file called `embamodule.py`,
which needs to be imported.

The methods `setup_module` function should be the first function that is called
and returns an object of type `EmbaModule`.
This object is used to accumulate all findings and also takes care of logging.

The `EmbaModule` object offers multiple functions:

- The function `module.add_finding()` can be used to add new findings for the module.
- The function `module.log()` can be used to write log entries without reporting them as a finding.
- The function `module.panic()` can be used to raise exceptions while ensuring proper logging.

At the very end of the module, this object needs to be passed to the `shutdown_module` function.
This will then pass the results back to EMBA.

An example module us located at `<emba directory>/modules/S28_python_run/example_script.py`.

Here is an example of how a very basic Python module looks:

\```python
from embamodule import setup_module, shutdown_module
from sys import argv
from os import environ


def main():
    # Setup module and logging.
    # This line is required.
    module = setup_module(argv, environ)

    # This is just some example code.
    # The module logic would go here.
    module.log("Received arguments a total of {len(environ)} environment variables.")
    for key in environ.keys():
        module.add_finding(f"Found envvar: {key}={environ[key]}")

    # Shutdown module and report results.
    # This line is required
    shutdown_module(module)

main()
\```

## Function Documentation

\```
setup_module(
    argv: list
        This should be the argv that is also passed to the python module.

    env: _Environ
        The environment variables of the EMBA module. These are required to
        figure out the correct locations to write logs to.

) returns EmbaModule
\```

\```
shutdown_module(
    module: EmbaModule
        The EmbaModule object that was used to accumlate findings.
)
\```

\```
EmbaModule.add_finding(
    description: str
        A description of the finding. This will also create a new log entry.       
)
\```

\```
EmbaModule.log(
    text: str
        Text which will be written to the module log but not reported as a finding.
)
\```

\```
EmbaModule.panic(
    description: str
        A description of the error.

    except_type: type[Exception]
        The exception type that will be thrown. For example, to throw a AtrributeError, this parameter can be set to
        "AttributeError".
)
\```

[B1TC0R3]

Here's what still needs to be done:

• Proper error handling. So far, exceptions raised by a Python module will break the result reporting of the main Bash script. Has to be tested and fixed.
• Validations in check_project.sh for the Python code.

Please let me know if I am missing anything.